Paging a FeatureLayer query past maxRecordCount from Python: what arcgis 2.4 does for you, and what it silently does not

In the ArcGIS API for Python, FeatureLayer.query() pages past a layer’s maxRecordCount for you by default. In JavaScript, when a layer has a maxRecordCount, the FeatureSet sets exceededTransferLimit if a query matches more than that many features, so a JavaScript habit expects the opposite of the Python default. The trouble sits at the edges: what the default does not page, and what it never tells you.

We ran every query below on October 22, 2026, on arcgis 2.4.3, the version bundled with ArcGIS Pro 3.7.1, and on arcgis 2.4.4, the latest release on PyPI, published October 20, 2026. Every request was anonymous, against public Esri Living Atlas layers. Where we describe how the library pages, that is what we read in its source and saw in our request logs. Those internals changed between the two releases: 2.4.4 adds a guard against endless paging loops.

What the default query does

return_all_records defaults to True, and the reference says that when it is True, “the query operation will call the service until all records that satisfy the where_clause are returned.” On USA Major Cities, a layer with a maxRecordCount of 2,000, a plain query returned all 4,186 rows. The first server page held 2,000 rows with exceededTransferLimit set to true, and the library kept going.

Observed in the 2.4.3 and 2.4.4 source and request logs, the pager sends one normal request, then a count request, then a request for the object IDs, then requests for 200 IDs at a time from a pool of five threads. It discards the first page’s features and fetches them again. That came to 24 requests for 4,186 rows: 3, plus one per 200 rows, rounded up. The chunks are appended as they finish, so the rows did not come back in object ID order. Setting order_by_fields switches the library to a single-threaded offset loop that keeps the order; the same query ordered by OBJECTID took 4 requests.

Setting return_all_records=False on its own changes nothing. The reference says so: “If this parameter is set to False but no other limit is specified, the default is True.” Our run with it still returned 4,186 rows in 24 requests.

Where the reference and the code do not agree

Two statements in the query() reference do not match what 2.4.3 and 2.4.4 do. The reference says “result_offset and result_record_count will be ignored if return_all_records is True,” but in those versions a result_record_count is not ignored. Giving one sets return_all_records to False, and the offset is then honored: an offset of 4,000 with a count of 500 returned the last 186 rows, not all 4,186.

The Python and REST references both give the layer’s maximum record count as the maximum for result_record_count, which holds for raw REST but not for the Python wrapper. A raw REST request for 3,000 rows returned 2,000, but the same count through query() was not capped: the library kept paging until it reached the count, and returned 3,000 rows in 18 requests.

What it does not page, and what it does not tell you

Grouped statistics are not paged. Observed in 2.4.3 and 2.4.4, and not mentioned in the Python reference, a query with out_statistics and group_by_fields_for_statistics returns before the paging code runs. Grouping the test layer by NAME returned 2,000 groups of 3,535, in one request, with no warning.

Nothing in the result tells you. The Python FeatureSet has no exceededTransferLimit, because FeatureSet.from_dict drops the key, so a truncated FeatureSet looks complete. Checking exceededTransferLimit is advice from the JavaScript side, where the FeatureSet does carry it.

One more path returns something other than a FeatureSet. If the service fails with “Error performing query operation” or an HTTP 504 gateway timeout, the library retries in halves of the total count and returns a plain dict of raw JSON, even with as_df=True. In a simulated failure, each half was still capped at maxRecordCount, and we got 4,000 of 4,186 rows with no error raised. If query() hands you a dict you did not ask for, this is one source-level path that does it, so check the type before you trust the row count.

Paging it yourself

When you want control of page size and order, page explicitly. This script, in four parts labeled A to D in its comments, ran unchanged on 2.4.3 and 2.4.4, anonymously, against the same public layer:

import math
from arcgis.gis import GIS
from arcgis.features import FeatureLayer

URL = "https://services.arcgis.com/P3ePLMYs2RVChkJx/arcgis/rest/services/USA_Major_Cities_/FeatureServer/0"
gis = GIS()                      # anonymous: public layer only
lyr = FeatureLayer(URL, gis=gis)
props = lyr.properties
page = props.maxRecordCount
paged = props.advancedQueryCapabilities.supportsPagination
print("maxRecordCount:", page, "| supportsPagination:", paged)
if not paged:
    raise SystemExit("this layer does not support pagination; samples B and D need it")

# A. Count first, then let the API page for you (default return_all_records=True).
total = lyr.query(where="1=1", return_count_only=True)
fs = lyr.query(where="1=1", out_fields="OBJECTID,NAME,STATE_ABBR", return_geometry=False)
print("A. count:", total, "| default query returned:", len(fs.features))
if len(fs.features) != total:
    raise SystemExit(f"A. got {len(fs.features)} of {total} rows")

# B. Manual offset paging: you control the page size.
#    order_by_fields gives you control of the row order.
#    OBJECTID is this layer's object ID field; use your layer's own.
rows, offset = [], 0
while True:
    batch = lyr.query(where="1=1", out_fields="OBJECTID,NAME", return_geometry=False,
                      order_by_fields="OBJECTID ASC",
                      result_offset=offset, result_record_count=page)
    rows.extend(batch.features)
    if len(batch.features) < page:
        break
    offset += page
print("B. offset-paged:", len(rows), "| unique OIDs:", len({f.attributes["OBJECTID"] for f in rows}))
if len(rows) != total:
    raise SystemExit(f"B. got {len(rows)} of {total} rows")

# C. Object-ID chunking: the ID list is not limited by maxRecordCount
#    (REST caps it at 1 million IDs); fetch the features in chunks.
#    1000 IDs per request: REST warns that more can be slow on enterprise geodatabase sources.
ids = lyr.query(where="1=1", return_ids_only=True)["objectIds"]
chunk, got = 1000, []
for i in range(0, len(ids), chunk):
    part = lyr.query(object_ids=",".join(map(str, ids[i:i + chunk])),
                     out_fields="OBJECTID,NAME", return_geometry=False)
    got.extend(part.features)
print("C. ids:", len(ids), "| chunks:", math.ceil(len(ids) / chunk), "| features:", len(got))
if len(got) != total:
    raise SystemExit(f"C. got {len(got)} of {total} rows")

# D. Statistics are NOT paged for you: group by NAME has more groups than maxRecordCount.
stats = [{"statisticType": "count", "onStatisticField": "OBJECTID", "outStatisticFieldName": "n"}]
one = lyr.query(where="1=1", group_by_fields_for_statistics="NAME", out_statistics=stats)
groups, offset = [], 0
while True:
    b = lyr.query(where="1=1", group_by_fields_for_statistics="NAME", out_statistics=stats,
                  order_by_fields="NAME ASC", result_offset=offset, result_record_count=page)
    groups.extend(b.features)
    if len(b.features) < page:
        break
    offset += page
distinct = lyr.query(where="1=1", out_fields="NAME", return_distinct_values=True,
                     return_geometry=False)
print("D. single stats call:", len(one.features), "| paged stats:", len(groups),
      "| distinct NAME values:", len(distinct.features))

Its output on 2.4.4:

maxRecordCount: 2000 | supportsPagination: True
A. count: 4186 | default query returned: 4186
B. offset-paged: 4186 | unique OIDs: 4186
C. ids: 4186 | chunks: 5 | features: 4186
D. single stats call: 2000 | paged stats: 3535 | distinct NAME values: 3535

Count first: return_count_only=True returns an int in one request, and Samples A to C stop with an error if their row count does not match it. Sample B is the offset loop the REST reference describes for resultOffset: “You can use this option to fetch records that are beyond maxRecordCount,” and the option applies only when the layer reports supportsPagination as true, so the script stops first if it does not. Because Sample B sets result_record_count to maxRecordCount, each call returns at most one page. It stops on a short page, and it returned all 4,186 unique object IDs.

Sample C is the other documented route. The REST reference says clients can “get all the query conforming object IDs by specifying returnIdsOnly as true and subsequently requesting feature sets for subsets of object IDs.” return_ids_only=True returns a plain dict, and all 4,186 IDs came back in one request. We did not test any route in this post, including the default pager and order_by_fields, on a layer that was being edited between page requests.

Sample D pages grouped statistics by hand, with an offset, a count and an order on the group field, and it recovered all 3,535 groups. We ran it on ArcGIS Online only. For hosted feature services in ArcGIS Enterprise, the REST reference says “the supportsPaginationOnAggregatedQueries property must be true on the layer. Hosted feature services using a spatiotemporal data store do not currently support pagination on aggregated queries.” Sample D’s distinct-values check printed 3,535 on 2.4.4 and 4,145 on 2.4.3, so do not trust return_distinct_values counts on 2.4.3.

A cap per request, not a lock

At the time of writing (October 2026), the REST Query reference says maxRecordCount defaults to 1,000 or 2,000 unless the service owner or an admin changes it, and it can vary by the type of data: polygon, point, polyline or table. It caps each request, not what a caller can read. On a public layer the whole layer is retrievable by paging: the default query pulled all 4,186 rows anonymously, in 24 requests. For a hosted feature layer in ArcGIS Online, sharing governs access, not maxRecordCount.

At the time of writing (October 2026), the ArcGIS Online credits page lists storage, not queries, as the credit cost of a hosted feature layer. As noted above, our requests were anonymous, against Esri’s public layers. What grows with the layer is the request count. At the 3 requests plus one per 200 rows we observed in 2.4.3 and 2.4.4, the default pager would send about 41,000 requests for an 8.2-million-row layer, a figure we derived and did not run. On a large layer, set order_by_fields: the same 4,186-row query ordered by OBJECTID took 4 requests instead of 24.

The trap we found in our own notes

Our own data-harvesting notes carry a Python pattern that does not do what its comment says. It passes result_record_count=2000, return_all=True, chunk_size=2000 under the comment “API for Python handles paging,” but it does not page. return_all and chunk_size are not parameters: query() accepts unknown keyword arguments silently, so a misspelled parameter is ignored, not rejected. With result_record_count=2000 set, the library does not return more than 2,000 rows. On both versions the pattern returned 2,000 of 4,186 rows, with no error. Dropping all three arguments lets the default page everything; Sample B is the explicit alternative.

Before you ship

• Check the type of what query() returns before you count its rows.
• Never look for exceededTransferLimit on a Python FeatureSet; compare the row count with return_count_only=True instead.
• Page grouped statistics yourself; the default query does not.
• Pin your arcgis version: the paging code differs between 2.4.3 and 2.4.4, and on 2.4.3 the return_distinct_values count was wrong, 4,145 against 3,535.

References

• ArcGIS API for Python reference: arcgis.features (FeatureLayer.query, return_all_records, result_offset, result_record_count and the FeatureSet constructor; accessed 2026-10-22)
• ArcGIS REST APIs: Query (Feature Service/Layer) (maxRecordCount, resultOffset, resultRecordCount, returnIdsOnly and aggregated-query pagination)
• ArcGIS Maps SDK for JavaScript: FeatureSet (the JavaScript FeatureSet and its exceededTransferLimit, for contrast)
• ArcGIS Online: Understand credits (what consumes credits, including feature storage)
• ArcGIS Online: Share items (what sharing with everyone exposes)