A self-updating data dictionary for ArcGIS Online

Every GIS team is asked for a data dictionary, and almost none have a current one. What are all our hosted services? Which fields, which coded-value domains, how many records, what spatial reference? The honest answer is usually “let me click through forty item pages.” You can generate the whole thing in one run instead — a multi-sheet Excel inventory of your entire ArcGIS Online account — with the ArcGIS API for Python.

Here’s how the crawl works, the part everyone gets wrong (domains), and the one design choice that makes the logic testable without a portal at all — plus a free notebook and Pro tool that produce the workbook.

Crawl the account, not the item pages

The ArcGIS API for Python gives you the whole account through your active sign-in. Walk the root plus every folder, keep the feature services, de-dup by item id — no credentials in code, just the session you’re already in:

from arcgis.gis import GIS
gis = GIS("home")              # active ArcGIS sign-in (or GIS("pro") in a Pro notebook)

me = gis.users.me
services = {}
for folder in [None] + me.folders:        # root + each folder
    for item in me.items(folder=folder):
        if item.type in ("Feature Service", "Feature Layer"):
            services[item.id] = item       # de-dup by item id

That covers the “everything I own” case. The tool also takes a single item id or service URL to pinpoint one service (a URL resolves to its item via the layer collection’s serviceItemId), or a search-text scope that runs gis.content.search("<text> owner:me", item_type="Feature *"). Same crawl, four ways in.

The part everyone gets wrong: domains

A field’s domain is the most useful thing in a data dictionary and the most commonly dropped. There are two kinds and they have different shapes — flatten both, or your dictionary is lying by omission:

def domain_text(field):
    d = field.get("domain")
    if not d: return ""
    if d.get("codedValues"):                         # coded-value domain
        return "; ".join(f'{c["code"]}={c["name"]}' for c in d["codedValues"])
    if "range" in d:                                # range domain
        lo, hi = d["range"]
        return f"min={lo}, max={hi}"
    return ""

Coded-value domains become readable code=label pairs; range domains become min/max. Now “status field, 1=Active, 2=Retired” actually appears in the dictionary instead of a bare integer column nobody can interpret (attribute domains overview).

One accessor that makes it all testable

The ArcGIS API hands you layer metadata as a PropertyMap (attribute access), but a plain JSON fixture is a dict (key access). Route every read through one tiny tolerant accessor and the entire describe / serialize / write path runs against ordinary dicts — which means you can unit-test it with no portal connection at all:

def g(obj, key, default=None):
    # tolerate both a dict (fixture) and an arcgis PropertyMap (live)
    if isinstance(obj, dict): return obj.get(key, default)
    return getattr(obj, key, default)

That one helper is the difference between “works on my machine against the live org” and a function you can actually test in CI with synthetic coded- and range-domain fixtures.

The output: three sheets that read like a dictionary

For each service the tool reads every layer and table’s properties, optionally fires one return_count_only query per layer for record counts, and pulls the spatial reference. It writes a multi-sheet workbook with auto-sized columns:

Counting is one query per layer, so it’s the slow part — turn it off for a fast metadata-only pass on large services, and counts that fail degrade to n/a rather than crashing the run.

Grab the tool — free

Same logic, two ways to run it. The notebook has a single CONFIG cell — choose scope and output, Run All. The Pro tool is the geoprocessing-dialog version — drop the .pyt in a toolbox, no code. Both authenticate with your active ArcGIS sign-in; neither stores a credential.

MIT-licensed. Needs the ArcGIS API for Python (ships with Pro); the XLSX writer uses openpyxl (CSV / TXT are dependency-free).

References

• ArcGIS API for Python — Esri Developer documentation
• Accessing and managing content — ArcGIS API for Python
• An overview of attribute domains — ArcGIS Pro