Storage backends

Tide persists durable state through a storage abstraction so workflows survive crashes, restarts, and redeployments. This page covers what gets stored, the three built-in backends, and the DurableStore trait for custom ones.

What gets stored

Every invocation has three pieces of persisted state:

• Journal — the ordered sequence of entries recording every durable operation. On replay it lets Tide skip completed work. See Durable execution.
• Frozen timestamp — the Date.now() value captured at startup, so time is identical across runs.
• Workflow input — the validated JSON input, saved on first run so replays reuse it. See input persistence.

A backend also implements the effect-intent reservation behind exactly-once external effects (see below).

Choosing a backend

Select one with --store on run, resume, test, or serve:

tide run main.ts                    # fs (default)
tide run main.ts --store sqlite
tide run main.ts --store postgres   # requires DATABASE_URL

All three implement the same DurableStore trait, so switching backends changes nothing about workflow behavior — only where state lives.

Filesystem backend (fs)

The default. Each invocation's state is plain JSON under .tide/:

.tide/
  invocations/
    <invocation-id>/
      journal.json       # operation journal (array of entries)
      timestamp.json     # frozen timestamp (a number, ms since epoch)
      input.json         # persisted workflow input (raw JSON)

A journal entry has three fields:

{ "op": "op_write_file", "result": null, "is_error": false }

Appending rewrites the full journal.json. This is maximally transparent and ideal for development — inspect, edit, or reset state with standard tools:

cat .tide/invocations/my-run/journal.json | jq .
rm -rf .tide/invocations/my-run/        # reset one invocation

For workflows with thousands of operations the repeated full-file writes become a bottleneck; use SQLite or Postgres there. The fs backend has no built-in atomicity for multi-entry writes — the transactional backends do.

SQLite backend (sqlite)

All invocations share one .tide/tide.db file. Uses a bundled SQLite build — no system SQLite needed; the file and schema are created on first use.

CREATE TABLE journal (
    invocation_id TEXT NOT NULL,
    position INTEGER NOT NULL,
    op TEXT NOT NULL,
    result TEXT NOT NULL,         -- JSON-serialized
    is_error INTEGER NOT NULL,
    PRIMARY KEY (invocation_id, position)
);
CREATE TABLE invocations ( id TEXT PRIMARY KEY, frozen_timestamp REAL );
CREATE TABLE inputs ( invocation_id TEXT PRIMARY KEY, input TEXT NOT NULL );

Appending an entry is a single insert; a step commit inserts all buffered entries in one SQL transaction, so it is atomic — if the process crashes mid-commit SQLite rolls back, and on retry Tide re-executes the step from scratch. Inspect with the sqlite3 CLI:

sqlite3 .tide/tide.db \
  "SELECT position, op FROM journal WHERE invocation_id = 'my-run' ORDER BY position;"

PostgreSQL backend (postgres)

Persists into a self-contained tide schema in a Postgres database. Set DATABASE_URL and select the backend:

export DATABASE_URL=postgres://user:pass@host/db
tide run main.ts --store postgres

Like SQLite it gives incremental inserts and transactional step commits, and it implements the effect-intent reservation in the database — suitable for shared, multi-process deployments.

Exactly-once effect intents

External effects (a human-approval request, an LLM call, an ontology mutation) reserve a deterministic effect intent before the host call runs, keyed by (invocation_id, effect_ordinal). The reservation is idempotent: a crash-and-replay re-reads the existing reservation instead of re-invoking the host. All three backends implement it, so the at-most-once guarantee holds even on the local fs default — not only in a database.

The DurableStore trait

Both built-in backends implement DurableStore (in runtime/src/store.rs). Implement it for any backing store — a remote database, object storage, etc.

#[async_trait]
pub trait DurableStore: Send + Sync {
    async fn load_journal(&self, invocation_id: &str) -> Result<Vec<JournalEntry>, StoreError>;
    async fn append_journal_entry(&self, invocation_id: &str, entry: &JournalEntry) -> Result<(), StoreError>;
    async fn append_journal_entries(&self, invocation_id: &str, entries: &[JournalEntry]) -> Result<(), StoreError>;

    async fn load_timestamp(&self, invocation_id: &str) -> Result<Option<f64>, StoreError>;
    async fn save_timestamp(&self, invocation_id: &str, ts: f64) -> Result<(), StoreError>;

    async fn load_input(&self, invocation_id: &str) -> Result<Option<String>, StoreError>;
    async fn save_input(&self, invocation_id: &str, input: &str) -> Result<(), StoreError>;

    // Exactly-once effect reservation; defaults to unsupported.
    async fn reserve_effect_intent(
        &self,
        invocation_id: &str,
        effect_ordinal: i64,
        op: &str,
        request_json: &serde_json::Value,
    ) -> Result<(), StoreError> { /* default: unsupported */ }
}

Every method is scoped to an invocation_id — each invocation's state is fully independent.

Implementation requirements

• append_journal_entries must be atomic. Either all entries commit or none do — this is what makes step crash recovery reliable. Use your store's transactions; if it has none, build your own atomicity.
• load_journal preserves order. Entries must come back in append order; replay is position-indexed.
• Isolation by invocation_id. One invocation's state must never affect another.
• Idempotent saves. save_timestamp / save_input may be re-called with the same value on retry; treat them as upserts.

Related

• Durable execution — how the journal drives replay.
• Steps — atomic commits through append_journal_entries.
• CLI reference — the --store flag.
• Architecture — how storage fits the runtime.