← Back to Blog
ArcGIS Online · Data Engineering

Stream only what the map needs — Spatially Optimized Parquet feature layers

August 18, 2026 · 10 min read

Stream only what the map needs — Spatially Optimized Parquet feature layers

Every organization with a genuinely large dataset ends up doing the same dance. The source is tens of millions of features in a warehouse or a cloud store. The map needs a fraction of it at any one moment. So you copy it into a hosted feature layer, generalize it by hand into two or three scale-dependent copies, wire up a tile cache for the zoomed-out view, and then spend the next year keeping four derivatives in sync with one source of truth. Parquet feature layers — new in the June 2026 ArcGIS Online release — are Esri’s answer to that pattern, and the June 2026 ArcGIS Maps SDK for JavaScript 5.1 release is what made them genuinely usable in a browser.

The short version: a Parquet feature layer is an open GeoParquet dataset that Esri lays out on disk so that a client can stream only the bytes it needs to draw the current extent, at the current scale. No tile cache, no hand-built generalizations, and the one dataset you do keep is an open file the rest of your stack can read. Here is how the optimization works, what 5.1 changed, how you author one, what it costs, and — the part that decides whether you can use it at all — the constraints that will disqualify some projects outright.

Read this gate first. This is a beta, and five constraints disqualify projects before design even starts. No editing — the layer is read-only. No ArcGIS Pro, and no offline use. Web Mercator or WGS 1984 basemaps only — if your organization standardizes on a State Plane, UTM, or national-grid basemap, this layer cannot sit on it. Symbology and pop-ups cannot be saved to the layer; they persist only in the web map where you configure them. And it is not supported by Esri Technical Support. If any of those is fatal for your project, stop here and use a hosted feature layer.

What a Parquet feature layer actually is

Parquet is a columnar file format built for analytics on large tabular datasets: it stores values by column rather than by row, which lets a reader skip entire columns and entire row groups it does not need. GeoParquet is the open convention that adds geospatial metadata and geometry types — point, line, and polygon — on top of it. That much is standard, and it is why Parquet has become the default interchange format in the modern data stack. Esri describes its layers as “currently” built on GeoParquet, which is worth reading as the hedge it is.

What Esri adds is a layout. Per the ArcGIS Online announcement, a Spatially Optimized Parquet dataset carries two properties a generic GeoParquet file does not:

PropertyWhat it doesWhy it matters in a browser
Geographic clustering “Data that is geographically close is also stored close together within the file(s).” A spatial query touches a contiguous slice of the file instead of scattered reads across the whole thing. Fewer range requests, less transfer, faster first paint.
Multi-scale geometry “Multiple generalized geometries are created and used by clients for optimal visual display.” Zoomed out, the client pulls the coarse geometry. Zoomed in, it pulls the detailed one. The generalization you used to build and maintain by hand is a property of the file.

Together those give you what a multi-scale tile cache gives you — sensible detail at every zoom — without the cache or the rebuild step. The SDK 5.1 release notes put the effect plainly: the layer uses “spatial clustering and a multi-scale index to efficiently stream only the data required for visualization,” which “reduces data transfer and significantly improves rendering performance for large datasets.” Esri publishes no benchmark numbers, and neither will we until we have measured a build of our own.

What 5.1 changed on the client

ParquetLayer existed before 5.1, but it was thin. The 5.1 release is where it became something you would put in front of a client:

That last bullet is the one worth reading twice. Deferring the feature download until after the predicate is evaluated is the difference between “download ten million features, then filter” and “read the column statistics, decide which row groups can possibly match, fetch only those.” It is exactly how a columnar engine is supposed to behave, and it is now happening in the browser.

Security: definitionExpression is not an access boundary. The class reference defines it as the where clause used to filter features on the client. There is no service enforcing it. The file is reachable in full — nothing on the server restricts which rows or columns a client may request — so anyone can drop the expression in devtools and pull the rest. There is also no feature layer view equivalent here: the server-side row and column restriction you would normally reach for does not exist for this layer type. The rule follows directly — filter sensitive data out at pipeline time, so it never enters the file. Do not put owner names, addresses, or anything else you would not publish into a Parquet feature layer you intend to share publicly.

Adding one to a map takes two imports — the layer, and the class that wraps its data source:

import ParquetLayer from "@arcgis/core/layers/ParquetLayer.js";
import ParquetPortalItemData from "@arcgis/core/layers/support/ParquetPortalItemData.js";

// Published as an ArcGIS Online item by Data Pipelines
const parcels = new ParquetLayer({
  data: new ParquetPortalItemData({
    portalItem: { id: "<parquet-feature-layer-item-id>" }
  }),
  definitionExpression: "usetype = 'Commercial'",   // client-side display filter
  popupEnabled: true
});

map.add(parcels);

The data property is the detail to get right. It takes a ParquetPortalItemData for a published ArcGIS Online item, or a ParquetFilesData for files you host yourself — more on that second path below. Passing portalItem directly to the constructor does not work — it is not a property of this layer, and the layer will have no data source.

Two paths, and the second one is more interesting

Most coverage of this release stops at Data Pipelines. But ParquetLayer also reads files directly:

import ParquetFilesData from "@arcgis/core/layers/support/ParquetFilesData.js";

const wells = new ParquetLayer({
  data: new ParquetFilesData({
    urls: ["https://your-bucket.s3.amazonaws.com/wells/part-0.parquet"]
  })
});

No Data Pipelines, no ArcGIS Online item, no credits — files in your own bucket, read straight by the browser. GeoAnalytics Engine can write spatially optimized Parquet for exactly this path. The trade is governance: you give up the portal item, its sharing model, and its place in your organization’s content inventory, and you take on CORS configuration and HTTP range-request support on the bucket yourself.

That is the sharpest version of this release’s promise. The artifact you maintain is a file in object storage that DuckDB, GeoPandas and GDAL can all read, and the map is one more reader of it.

Authoring one: ArcGIS Data Pipelines

For the published-item path, you do not publish from ArcGIS Pro and you do not upload from your desktop. Per Esri’s announcement, Parquet feature layers “can only be created and maintained using ArcGIS Data Pipelines” — the no-code data engineering app — using the Parquet feature layer (Beta) output tool. Data Pipelines reads from URLs and APIs, cloud storage such as Amazon S3 and Microsoft Azure Storage, and cloud databases including Snowflake, Databricks, and Google BigQuery.

ParameterNotes
Input dataset“The dataset that will be written as a file.”
Geometry fieldRequired — per the limitations, “Parquet feature layers must have a geometry. Tables are not supported.”
Output methodCreate (default) or Overwrite. Overwrite is how you refresh a published layer in place.
Overwrite if item already existsEnabled by default on Create. For scheduled refreshes Esri recommends the Overwrite method instead of this option.
Title / FolderStandard item properties; Folder applies to Create only.
Parquet feature layerThe target item, for the Overwrite method.

Note what is not in that list: no clustering setting, no generalization tolerance, no index configuration. The spatial optimization is applied for you. That is a defensible design — one fewer knob to get wrong — but you cannot tune the multi-scale levels to a cartographic requirement in this release.

Entitlement and cost, before you plan around it. Data Pipelines requires a Creator or Professional user type and a Publisher, Facilitator, or Administrator role (or an equivalent custom role). Personal Use, Developer, Trial, and ArcGIS Location Platform subscriptions cannot use Data Pipelines at all. It also consumes credits: 70 credits per hour for scheduled jobs, calculated per minute, and 50 credits per hour for interactive editing with a ten-minute minimum — so authoring the pipeline costs credits too. Separately, an administrator can block beta capabilities org-wide, in which case the output tool will not appear.

Refresh semantics — and what they cost you

Pair the Overwrite method with a scheduled pipeline task and the refresh story is simple: the warehouse changes overnight, the pipeline runs, the published layer is replaced in place, and every map pointing at it resolves the new data. No republish, no broken item IDs. Three caveats that matter more than the convenience:

What consumers see during an Overwrite is not documented. We have not measured it, so we will not tell you it is seamless. Test it with an open map before you put it in front of users.

Parquet feature layer or hosted feature layer?

This is the decision the reader actually came for:

Parquet feature layer (beta)Hosted feature layer
EditingNone — read-onlyFull, with capability control
OfflineNoYes
ArcGIS ProNoYes
Basemap projectionWeb Mercator or WGS 1984 onlyAny
Row / column securityNone — client-side filter onlyViews: server-enforced query and field visibility
RefreshFull Overwrite onlyReplace, and add-and-update
Analysis toolsNot as input to analysis; is valid as a Data Pipelines inputYes
Symbology / pop-upsPer web map; nothing saved on the layerSaved on the item
Underlying artifactOpen GeoParquet you can read anywhereEsri-managed service
Esri Technical SupportNo — betaYes

The rest of the constraints

Nearly every one of those is scoped by Esri to “this release” or “currently.” Treat this list as dated August 2026 and re-check it before you design around any single line of it.

Where this lands in a real architecture

The interesting thing about Parquet feature layers is not the performance claim — it is where they put the boundary. Until now, getting warehouse-scale data onto an Esri map meant an ETL job that produced an Esri-shaped copy, and that copy became a thing you owned, monitored, and reconciled. A Parquet feature layer moves the boundary one step earlier: the pipeline writes an open, standard GeoParquet dataset, and the map reads it directly.

That matters commercially as much as technically. GeoParquet is not an Esri format. The same files are readable by DuckDB, GeoPandas, GDAL, and the rest of the modern spatial stack — so the optimization you pay for to make the map fast is not a lock-in cost, it is an asset the whole data platform can use. This is the first Esri layer type where the artifact you maintain is an open dataset rather than an Esri-shaped derivative.

The honest read: this is a visualization and query layer for large, slow-changing, non-sensitive reference data, in an organization on a Web Mercator or WGS 1984 basemap, with credits to spend on a recurring pipeline. Road networks, building footprints, census geography, sensor histories. It is not a replacement for a hosted feature layer in an editing workflow, it is not a security boundary, and until it leaves beta it is not something to put under an SLA. Within those lines, it removes a derivative you have been maintaining for years.

References

Sitting on warehouse-scale data that never made it onto a map?

We design and build the pipeline — cloud store or warehouse through ArcGIS Data Pipelines to a production web map — and we do it without leaving you a derivative dataset to babysit.

Book a free intro call