File lifecycle
Every state a file passes through in the file ingestion flow, and which of them reach you as a notification.
The unit tracked here is the batch file Simetrik builds from the records you post — not the individual record. One file covers every record that landed in the same batch, so one state change speaks for all of them. If you registered a notification_config, some of these states are also pushed to your endpoint.
The happy path
| State | What just happened |
|---|---|
CAPTURED | The batch file was picked up for processing. |
SOURCE_CAPTURED | Its source was resolved. |
JOB_DEFINED | A processing job was prepared for that source's batch. |
DISPATCHED | The job was handed off for execution. |
MESSAGE_DELETED | The capture record was cleared — the file won't be picked up twice. |
DOWNLOADED | The file content was read. |
PARSED | JSON was converted to CSV using the source's columns. |
UPLOADED | The CSV was delivered to the source. |
NOTIFIED | Reconciliation was told there's new data. This is the end state. |
Failures
Each step has its failure counterpart, and it's the last state the file keeps:
| State | What it tells you |
|---|---|
SOURCE_CAPTURED_FAILED | The source couldn't be resolved. |
JOB_DEFINED_FAILED | The job couldn't be prepared. |
DISPATCH_FAILED | The job couldn't be handed off. |
MESSAGE_DELETION_FAILED | Processing worked, but the capture record couldn't be cleared. |
DOWNLOAD_FAILED | The file content couldn't be read. |
PARSE_FAILED | A record in the batch couldn't be read as JSON. |
UPLOAD_FAILED | The CSV couldn't be delivered. |
NOTIFICATION_FAILED | The file was delivered, but reconciliation couldn't be notified. |
A file that fails doesn't take the others down with it: every other batch for that source carries on and finishes normally. Within a single file, though, a malformed record fails the whole file — every record batched alongside it is affected.
What reaches your endpoint
Internal states are collapsed into a smaller set of events, so you get progress without having to model the pipeline:
| State | Event you receive |
|---|---|
DOWNLOADED, PARSED, UPLOADED | download_status.processing |
NOTIFIED | download_status.completed |
DOWNLOAD_FAILED | download_status.download_failed |
PARSE_FAILED | download_status.validation_failed |
UPLOAD_FAILED, NOTIFICATION_FAILED | download_status.processing_failed |
States before the download — capture, job definition, dispatch — don't produce events. The first thing you hear about a file is download_status.processing.
You only receive the events you subscribed to in notification_config.events. Subscribing to the three failure events and to download_status.completed is usually enough: you learn when a file is done and when it isn't, without the noise in between.
Correlating them takes two steps, because you never named the file. data.filename is generated by Simetrik, so it can't be matched to a record you posted — but every event for the same batch carries the same one, which makes it the right key for grouping events into a batch. To go from a batch back to the records you sent, use data.source_id and the batch's timestamp against the window you posted in.