← Back to Blog

Mirroring a weekly CSV drop into an enterprise geodatabase — truncate-and-reload or a differential edit, and the documented rules that choose for you

A stack of spreadsheets on the left, topped by a round sync badge, points by arrow to a tile holding a blue check shield and on to a card where a blue database cylinder sits among four corner tiles showing stacked layers, a chain link, a paperclip and a clock ringed by arrows, and from that card two curves branch right, the upper curve to a pale panel where a gray database cylinder points to a blank table and then to a filled blue table, with an arrow arcing from the blank table to the blue one, and the lower curve to a cyan outlined panel where a blue database cylinder marked with a small check shield fans out to three round badges, a plus beside a cyan row, a pencil beside a row with one darker cell and a trash can beside a dashed empty row, above a plain gray row, while a cyan bracket gathers the three badged rows and forks to two small blue tables on the right

Every week another system hands you a set of CSV files. Each file is the complete, current state of one table, and your job is to make the matching table in an enterprise geodatabase look exactly like it: new rows in, changed rows updated, vanished rows gone. An Esri Community thread describes precisely this job: more than 80 CSV files a week, landing in non-spatial tables that nothing else edits, registered with a SQL Server enterprise geodatabase. When we read it on September 17, 2026, none of its answers had been accepted. Having found raw SQL against the registered tables problematic, its author weighs four options: the ArcGIS API for Python, truncate-and-reload, Append with its upsert, and ArcPy cursors. This post is mostly about choosing between the last three, and the short answer is that documented constraints make that choice, not preference.

Everything below was checked against the ArcGIS Pro 3.7 documentation, the service section against the ArcGIS API for Python reference, and the cost note against the ArcGIS Online credits page. Where we say measured, we ran it on ArcGIS Pro 3.7.1 against file and mobile geodatabases, not against SQL Server, and we label each measured result.

There is no mirror tool, only building blocks

We did not find an Esri page that describes one ArcPy pattern for inserting, updating and deleting against a full-state drop in a single pass. What the reference does document is the parts:

The tools whose names suggest they could compute the delta for you cannot. Table Compare and Feature Compare are only effective at identifying the first addition or deletion they meet. Detect Feature Changes works on line features and requires an Advanced license. The Data Loading toolset migrates data to a new schema; it does not synchronize changes. Working out which rows were added, changed and removed is your script’s job.

The first question: how is the target table versioned?

The thread never says, and no single fact decides more. Ask whoever administers the geodatabase before you write a line of code.

Target tableTruncate-and-reloadDifferential edit with ArcPy cursors
Not registered as versioned Possible, subject to the rules in the next section Possible, inside arcpy.da.Editor with multiuser_mode=False
Traditional versioning Refused: Truncate Table does not accept versioned data. Unregistering first needs an exclusive lock and deletes every edit in named versions that has not been posted to default and compressed Possible, with multiuser_mode=True. The edits go to the version the connection uses, and a load into versioned data is not seen in other versions until a reconcile and post; traditional versioning may also need a compress
Branch versioning Not over a database connection Not over a database connection: branch-versioned data can only be edited through web feature layers

One detail in that table is easy to get backwards. The multiuser_mode argument of arcpy.da.Editor defaults to True, the versioned setting. The reference says to set it to False when editing a nonversioned table, and that with False, editing versioned data raises an error. Pass it explicitly either way.

If the tables are new, Esri’s guidance on the first load is also worth knowing: regardless of the versioning type, it recommends loading all the data before registering the dataset as versioned.

What rules out truncate-and-reload

On a table that is not versioned, both approaches are possible in principle. Then the Truncate Table reference and its neighbors start removing options. Each row below is either a hard refusal or a silent change for something downstream.

ConditionWhat truncate-and-reload doesEvidence
The job does not connect as the data owner Refused. The input must come from a database connection established as the data owner. A data editor account can run Append and cursor edits, but not the truncate Documented
Attachments Refused for feature classes with attachments; Delete Rows is the documented alternative. Esri’s page for the error it returns also names a table with attachments, and Pro 3.7.1 refused one Documented, and measured on 3.7.1
Origin of a composite relationship class Refused. The tool page does not mention relationship classes Measured on 3.7.1
Origin of a simple relationship class Runs, and leaves every child row’s foreign key untouched. Deleting an origin row in a simple relationship is documented to set the child’s foreign key to null; the truncate did not, so keys on GlobalID or ObjectID point at parents that no longer exist Measured on 3.7.1 with GlobalID and ObjectID keys, against the documented delete rule
Anything keyed on ObjectID or GlobalID Every reloaded row gets a new ObjectID and, by default, a new GlobalID, and editor tracking records each one as an insert GlobalIDs documented and measured; ObjectIDs measured on file and mobile geodatabases only; editor tracking follows its documented insert rule
Archiving enabled History rows remain after the truncate. Because deletes and updates retire rows rather than removing them, a weekly reload adds the whole table to history every week; a differential edit archives only what changed History rows documented; the weekly growth follows from the archive rules; measured on a mobile geodatabase
Inside an edit session Refused. Truncate commands do not use database transactions and are unrecoverable, so there is nothing to roll back. This limits how a reload runs, not which tables allow it: the truncate has to run outside arcpy.da.Editor No transactions documented; the refusal measured on 3.7.1

The message does not point at the likely cause. For the attachment-enabled table, the attachment-enabled feature class and the composite relationship origin, Pro 3.7.1 returned the same error, ERROR 160592: Truncation not allowed while editing., with no edit session open in any of the three cases. Esri’s page for that error explains more than the message does: it also covers a table that requires an edit session to make edits, such as a table with attachments.

If you see that from a script that never started editing, look for attachments and relationship classes on the target, not for a stray edit session.

There is also a window to think about. The Truncate Table page says truncate commands do not use database transactions, and the Load data page says a load into nonversioned data commits as a single transaction that other users see on their next refresh. Read together, those two sentences mean that between the truncate and the load’s commit, anyone reading the table can find it empty, and a load that fails after the truncate leaves it empty, or part-loaded if the job loads in more than one call, until the job succeeds. We did not measure this with a concurrent reader; it follows from the two documented behaviors. A differential edit, by contrast, commits its changes when the edit session stops.

Preserve GlobalIDs is not the escape hatch it looks like. First, a CSV from another system carries none of your table’s GlobalIDs, so there is nothing to preserve unless the job looks them up by business key. Second, Append honors the Preserve GlobalIDs environment only when the target’s GlobalID field has a unique index; otherwise the tool may fail. Enterprise geodatabase data that is not registered as versioned can take a unique index on a GlobalID field, a file geodatabase cannot hold one at all, and on 3.7.1 a mobile geodatabase, whose GlobalID index was unique, preserved all three test GlobalIDs. Separately, a dataset with unique indexes defined cannot be registered as versioned, and the documentation does not say whether a GlobalID index is an exception.

When truncate-and-reload is the right call

Truncate Table documents its own use case: “workflows in which all rows will be removed from a table or feature class and there is no requirement to back up the transactions, such as the nightly reloading of data.” That description fits a table that is not versioned or archived, has no attachments, is in no relationship class and is not identified anywhere by ObjectID or GlobalID, loaded by a job that connects as the owner. For that table the reload is simpler, and both Truncate Table and Append run at every ArcGIS Pro license level.

Because the truncate cannot be undone, check the drop before it runs: that the file is there, is not empty, and holds about as many rows as last week. That check is our recommendation, not Esri’s documentation. The reload also has to survive the CSV itself, which is the first of the three traps below.

Three Pro 3.7.1 traps a mirror has to design around

1. ArcGIS decides your key is a number

ArcGIS reads a .csv file as a comma-delimited table and honors a schema.ini file for column types. Without one, Pro 3.7.1 read our test drop’s business key and ZIP code columns as Integer: the key 00101 came back as 101, the ZIP 01234 as 1234, and no schema.ini was written.

Quoting the values did not help. The text-file page says that values bracketed with double quotes are interpreted as text fields, but on 3.7.1, with no schema.ini present, a file whose bytes were "00101","01234",10.5 still produced Integer fields holding 101 and 1234. What did work was a schema.ini beside the CSV with a section for that file declaring all of its columns, the keys as text (for example Col1=BIZ_KEY Text). Reading the file with Python’s own csv module also worked: it returns every value as a string and kept 00101 intact.

2. Append’s upsert, fed a CSV, duplicates rows

On Pro 3.7.1, Append with match_fields reading a CSV directly updated the matching rows and inserted them again. It reported appended_row_count 3 and updated_row_count 0, and left duplicate keys behind. We reproduced it with and without a field map, with and without a schema.ini, and with skip_options="NO_SKIP". The same rows staged into a file geodatabase table first gave the documented result: 1 appended, 2 updated, no duplicates. We found no documentation either way and did not test other Pro builds, so treat it as a 3.7.1 observation, and stage the CSV into a geodatabase table before any upsert.

Even working as documented, the upsert has two properties that matter for a mirror. It never deletes, so you still need a delete pass. And it rewrites every matched row: in our 3.7.1 test on file and mobile geodatabases, an upsert of an identical copy of the data updated 3 of 3 rows and moved last_edited_date on all three. That matches the editor tracking rule that new values are recorded when a row is “modified in any way.” If anything downstream uses last_edited_date to find what changed this week, a weekly upsert tells it everything did.

3. Cursor edits outside an edit session

Three measured results, all on Pro 3.7.1 file and mobile geodatabases:

The rule that falls out is short: every cursor edit a mirror makes goes inside arcpy.da.Editor. Truncate Table is the exception: on 3.7.1 it refused to run inside one.

A differential edit that designs around all three traps

This is the script we ran on Pro 3.7.1. It reads the drop with Python’s csv module, compares it with the table key by key, and makes every change inside one edit session.

"""Mirror a full-state CSV drop into a geodatabase table: inserts, updates and deletes in one edit session."""
import csv
import datetime

import arcpy


def _text(value):
    return value if value != "" else None


def read_drop(csv_path, key, fields, converters):
    """Read the drop with Python's csv module, so text keys keep their leading zeros."""
    rows = {}
    with open(csv_path, newline="", encoding="utf-8-sig") as fh:
        for record in csv.DictReader(fh):
            k = record[key]
            if k in rows:
                raise ValueError(f"duplicate key {k!r} in {csv_path}")
            rows[k] = tuple(converters.get(f, _text)(record[f]) for f in fields)
    return rows


def mirror(csv_path, target, key, fields, converters, workspace, multiuser_mode=False):
    incoming = read_drop(csv_path, key, fields, converters)
    inserted = updated = deleted = 0
    with arcpy.da.Editor(workspace, multiuser_mode=multiuser_mode):
        with arcpy.da.UpdateCursor(target, [key, *fields]) as cursor:
            for row in cursor:
                new_values = incoming.pop(row[0], None)
                if new_values is None:
                    cursor.deleteRow()
                    deleted += 1
                elif tuple(row[1:]) != new_values:
                    cursor.updateRow([row[0], *new_values])
                    updated += 1
        with arcpy.da.InsertCursor(target, [key, *fields]) as cursor:
            for k, values in incoming.items():
                cursor.insertRow([k, *values])
                inserted += 1
    return inserted, updated, deleted


CONVERTERS = {
    "AMOUNT": lambda v: float(v) if v != "" else None,
    "ASOF": lambda v: datetime.datetime.fromisoformat(v) if v != "" else None,
}

How it works, in order:

  1. Read the drop as text. read_drop() reads every value as a string with Python’s csv module, so 00101 stays 00101, and keys a dictionary on the business key. A key that appears twice in the drop stops the run before anything is edited. Every other column stays text unless you give it a converter; the example converts an amount and a date. The key itself is never converted.
  2. Open one edit session. multiuser_mode=False is the documented setting for a nonversioned table. For traditional versioning, pass True. Used as a with block, the Editor saves the session when the block succeeds and stops editing without saving if an exception is raised. A nonversioned edit session has no undo or redo, so that exit is your rollback, and it only rolls back a run that raises.
  3. Walk the table once. For each row, pop() the key out of the drop. No match means the row has left the source, so it is deleted. A match with different values is updated. A match with identical values is left alone, which is what keeps last_edited_date honest.
  4. Insert what is left. Anything still in the dictionary is new. The insert cursor opens only after the update cursor’s block has closed, so the two are never open at once, and neither is used after its with block. For how cursor lifetime changed in Pro 3.7, see the cursor scope post.

We ran it against a table with GlobalIDs, UTC editor tracking and a simple relationship class keyed on GlobalID. The second-week drop added one row, changed one, left one unchanged and dropped one, and the run returned (1, 1, 1). The changed row kept its ObjectID, its GlobalID and its created_date, and only its last_edited_date moved. The unchanged row kept all four values. The child rows of the two surviving parents still resolved to them; the child of the removed parent was left with a null foreign key, which is the documented simple-relationship delete rule applied inside the edit session. In a composite relationship the same delete removes the child rows. Running the same drop again returned (0, 0, 0).

Know its limits before you schedule it:

On speed: Esri documents that edit sessions batch updates, which it describes as offering significant performance advantages when editing enterprise geodatabases. We found no Esri throughput figures comparing cursors with Append, and the timings in the thread’s replies could not be reproduced without an enterprise geodatabase, so we are not repeating them.

Why not go through a feature service?

The first alternative the thread lists is the ArcGIS API for Python. Its edit_features method does add, update and delete in a single call, but against a feature layer or table in a service, not over a database connection, and the reference says that for 250 or more records at once, append should be used instead. The same reference describes append as updating hosted feature layers, available in ArcGIS Online and ArcGIS Enterprise 10.8.1 and later. And the Append tool page is explicit about services that sit on a geodatabase: use a database connection to perform large data loading operations directly to the underlying database. For a registered table you can reach over a database connection, the service is the long way around; for a branch-versioned table, a web feature layer is the only way in.

Running it every week

Scheduling. ArcGIS Pro schedules geoprocessing tools, at every license level, through Windows Task Scheduler, so the mirror has to be wrapped in a script tool to be scheduled from Pro. For enterprise geodatabase data, the connection has to be made with Save User/Password checked. With a Named User license, check Sign in automatically, which Esri says is required for the tool to run outside ArcGIS Pro. That license also brings a token with a maximum expiration set by your administrator, and if that interrupts unattended runs, Esri suggests a single-use license where possible. Running a task whether or not the user is logged on requires the run-as account to be in the local administrators group, and Esri advises considering a generic user account for critical scheduled tasks.

Credentials. A saved password in a connection file is a credential at rest. Our recommendation, not Esri’s documentation: where your DBMS allows operating system authentication, use it for this account instead.

Least privilege. This is where the choice of approach and the security model meet. A differential edit can run as a data editor rather than the owner. On SQL Server, Esri’s minimum for a data editor is SELECT, INSERT, UPDATE and DELETE on the datasets plus EXECUTE on the stored procedures associated with the data, and in a relationship class the privileges go on both the origin and the destination. A truncate-and-reload needs a connection as the data owner, and only the owner can alter the schema. Editor tracking records the name the job connects as: the database user, or the operating system login under operating system authentication. That name is what appears as the last editor on every row the job inserts or changes. The grants are not the whole boundary, though: anyone who can write to the drop location can rewrite or empty every mirrored table through this job, so restrict who can write there. That last point is our recommendation.

After the run. Both Analyze Datasets and Rebuild Indexes say that after data loading, deleting and updating it is important to run them. Analyze Datasets has to connect as the owner of the datasets, so it cannot run under the mirror’s data editor account: schedule it as a separate job that connects as the owner, or leave statistics to the database administrator’s own maintenance. Updating statistics is I/O-intensive, so schedule it for when database traffic is at its lightest.

Licensing. Truncate Table, Append and Delete Rows run at every Pro license level. Analyze Datasets, Rebuild Indexes and Register With Geodatabase need Standard or Advanced, as does creating tables in an enterprise geodatabase if the job bootstraps missing targets, and that step needs an account allowed to create tables, not the mirror’s data editor. On traditional-versioned tables, check the license for the reconcile, post and compress steps too. If your tables participate in relationship classes, check your seat: Pro’s help message for editing data in relationship classes says it requires additional licensing and carries a Standard or Advanced banner.

Cost. ArcGIS Online credits are consumed by specific transactions and types of storage, and this job does not call ArcGIS Online: it edits the database over a connection. What it costs is database I/O and an ArcGIS Pro license for the account that runs it, at Standard or Advanced if the same job runs Analyze Datasets or Rebuild Indexes.

Before the first scheduled run

  1. Ask how each target table is versioned, and whether archiving is on. Branch-versioned rules out both approaches over a database connection; traditional rules out the truncate.
  2. List what hangs off the table. Attachments, relationship classes (we tested GlobalID and ObjectID keys), and anything else that remembers ObjectIDs or GlobalIDs each rule out the reload.
  3. Pick the accounts. Owner for a reload; a data editor is enough for a differential edit, with statistics and index maintenance run separately as the owner.
  4. Pin the CSV types. Read with Python’s csv module, or ship a schema.ini that declares keys as text. Quoting the values was not enough on 3.7.1.
  5. Never point an Append upsert at the CSV itself on 3.7.1. Stage it into a geodatabase table first.
  6. Put every cursor edit inside arcpy.da.Editor, with multiuser_mode set for how the table is versioned. A reload’s truncate is the exception: it runs outside, and it cannot be undone.
  7. Guard against a bad drop. Refuse a missing or empty file, and cap deletes for each table.
  8. Prove it on a copy. For a differential edit, run a week, run the same drop again and expect (0, 0, 0), then feed it an empty file and expect a refusal. For a reload, check that the row count and keys match the drop and that GlobalIDs changed, and do not use ObjectIDs to tell one week’s rows from the last.
  9. Refresh statistics and indexes afterwards, in the quiet window, under the owner connection.
If you are not the one writing the scripts. The choice between reloading a table and editing it is not a matter of style. It decides whether the weekly job can run at all on tables with attachments or composite relationships, whether simple relationships keep pointing at real parents, whether the service account has to own the data, whether change dates mean anything, and how fast archive history grows. Discovering those constraints after go-live means child rows keyed on ObjectID or GlobalID pointing at parents that no longer exist, or a history table that grew by the whole table every week. Discovering them before costs two conversations: one with whoever administers the geodatabase, about versioning, archiving, attachments, relationships and ownership, and one with whoever produces the files, about what the job should do when a file is missing, empty or late.

Esri does not hand you a mirror, and the thread asking for one had no accepted answer when we read it. Esri does hand you documented rules, and together with the 3.7.1 behavior we measured, they leave the reload standing only for the plainest tables. Find out which kind of table you have before the first weekly run, design around the three traps and guard against an empty drop. Then the job becomes the boring kind: one edit session per table, three counts you log, and a second run that changes nothing.

References

Have data that has to land in a geodatabase every week?

We build ArcPy data pipelines that are designed around the geodatabase they write to — its versioning, its relationships and who is allowed to touch it. Book a free intro call and we will look at your tables and tell you which design they leave you.

Book a free intro call