Local-First Architecture

Offline-first data management with real-time synchronization

Local-First Architecture

The platform uses a local-first architecture built on SignalDB with IndexedDB persistence. All data interactions happen locally first, with bi-directional synchronization to the server via REST APIs.

Overview

  • Immediate Responsiveness: All reads and writes target local IndexedDB-backed collections. UI updates are instant.
  • Offline Functionality: Collections persist in IndexedDB across page reloads. When offline, users can browse cached data and make edits; changes are queued and pushed when connectivity returns.
  • Automatic Synchronization: The SyncManager handles incremental pull (via updatedAt timestamps) and push (local mutations).
  • Conflict Resolution: Server-side baseRevision checking detects concurrent edits. A Conflict Resolution UI lets users choose "Accept Theirs" or "Keep Mine".
  • Per-Entity Sync State: An EntitySyncStateService derives per-entity status (synced, localChanges, conflict).
graph TB
    subgraph "Client"
        UI[Svelte Components] --> Collections[SignalDB Collections]
        Collections --> IDB[IndexedDB Persistence]
        SM[SmallstackSyncManager] --> Collections
        SM --> API[REST API calls]
        NS[NetworkStatusService] --> SM
        ESS[EntitySyncStateService] --> SM
    end

    subgraph "Server"
        API --> Auth[Auth.js]
        Auth --> Endpoints[Generic CRUD Endpoints]
        Endpoints --> MongoDB
    end

Core Services

SignalDbService

Located at packages/client/src/services/signal-db.service.svelte.ts. The central service for data management.

Key responsibilities:

  • Creates and manages SignalDB collections with IndexedDB persistence adapters
  • Configures the SmallstackSyncManager for bi-directional sync
  • Tracks initial sync state per collection (initiallySynced)
  • Tracks last sync timestamps per collection (lastSyncedAt)
  • Manages sync conflicts and their resolution
  • Handles configurable polling for opted-in collections (default 5s; drops to 30s heartbeat when Ably is active)
// Getting a collection (creates it on first access, starts sync)
const collection = signalDbService.getCollection<MyEntity>("tenant-abc-types");

// With real-time polling enabled
const collection = signalDbService.getCollection<MyEntity>("tenant-abc-types", { realtime: true });

// Wait for initial data to be available
await signalDbService.isInitiallySynced("tenant-abc-types");

// Adjust polling interval (e.g., reduce to 30s heartbeat while Ably is active)
signalDbService.setPollingInterval(30_000);

// Check whether a collection has been registered
signalDbService.hasCollection("tenant-abc-types"); // boolean

// Check whether a collection has realtime polling enabled
signalDbService.isRealtimeCollection("tenant-abc-types"); // boolean

// Manually trigger a pull sync for a collection
await signalDbService.refreshCollection("tenant-abc-types");

SmallstackSyncManager

Located at packages/client/src/services/smallstack-sync-manager.ts. A subclass of SignalDB's SyncManager that exposes pending change information:

// Get IDs of entities with un-pushed local changes
const pendingIds: Set<string> = syncManager.getPendingChangeIds("my-collection");

// Get detailed pending changes for a specific entity
const changes = syncManager.getPendingChanges("my-collection", "entity-123");

// Get total pending change count across all collections
const count: number = syncManager.getPendingChangeCount();

NetworkStatusService

Located at packages/client/src/services/network-status.service.svelte.ts. Reactive online/offline detection using browser APIs:

import { networkStatusService } from "@smallstack/client";

// Reactive boolean ($state)
networkStatusService.isOnline; // true or false

// Listen for status changes
const unsubscribe = networkStatusService.onStatusChange((isOnline) => {
    console.log("Network status changed:", isOnline);
});

EntitySyncStateService

Located at packages/client/src/services/entity-sync-state.service.svelte.ts. Derives per-entity sync state:

import { entitySyncStateService } from "@smallstack/client";

// Returns: "synced" | "localChanges" | "conflict"
const state = entitySyncStateService.getState("entity-id", "collection-name");

// Total pending count
const count = entitySyncStateService.getTotalPendingCount();

Synchronization

How Pull Works

The SyncManager pull handler uses incremental sync:

  1. Initial sync (lastFinishedSyncEnd is null): Fetches all entities via GET /api/...?updatedAt=0, returns { items: [...] }
  2. Subsequent syncs: Fetches only entities modified since the last sync via GET /api/...?updatedAt=<timestamp>, categorizes into added/modified/removed based on revision and deletedAt
  3. Persistence: The SyncManager itself uses an IndexedDB persistence adapter (__signaldb_sync__), so sync metadata (snapshots, change tracking, sync operations) survive page reloads

How Push Works

Local mutations are captured by SignalDB's change tracking and pushed via:

  • Inserts: POST /api/... with the full entity
  • Updates: PATCH /api/.../entityId with { modifier, baseRevision }
  • Deletes: DELETE /api/.../entityId

Conflict Detection

The server checks baseRevision against the current document revision:

  1. Client sends { modifier, baseRevision: N } in PATCH request
  2. Server validates that the document's current revision === N
  3. If mismatch, server returns HTTP 409 with conflict details
  4. Client stores the conflict in signalDbService.conflicts
  5. User resolves via Conflict Resolution UI

Reconnect Behavior

When the network comes back online, networkStatusService.onStatusChange triggers syncManager.syncAll() to:

  • Push any queued local changes
  • Pull any server-side updates missed while offline

UI Components

OfflineBannerComponent

Fixed banner that appears when offline, auto-dismissing "Back online" notification on reconnect.

ConflictResolutionContainer / ConflictResolutionDialog

Renders active conflicts from signalDbService.conflicts as side-by-side cards with "Accept Theirs" / "Keep Mine" buttons.

EntitySyncBadge

Inline badge showing conflict or pending-changes status for a specific entity. Used in the entity editor.

PendingChangesCounter

Displays total pending change count. Placed in both app layouts.

StaleBadgeComponent

Shows per-collection staleness indicators based on lastSyncedAt timestamps.

Collection Naming & API Resolution

Collections are named using a convention that maps to API paths:

tenant-{projectId}-{typeId}     → /api/projects/{projectId}/types/{typeId}/data
tenant-{projectId}-types        → /api/projects/{projectId}/types
tenant-{projectId}-applications → /api/projects/{projectId}/applications

The describeCollection() utility from @smallstack/shared handles this mapping.

Offline Auth

When offline, server-side auth hooks (hooks.server.ts) catch network errors and allow the request to proceed with the locally cached session. Auth.js uses JWT strategy with 6-month expiry, so offline sessions remain valid for extended periods.

Real-Time Updates

When an ABLY_API_KEY is configured server-side, the platform uses Ably-driven push notifications to trigger collection syncs instantly when data changes elsewhere — replacing the continuous 5s polling loop with a 30s safety-net heartbeat.

Without Ably (dev environments or missing key), polling remains at the default 5s interval.

Ably-driven sync behaviour

  • Realtime collections ({ realtime: true }) auto-sync silently within ~1.5 s of a remote change.
  • Manual collections (no realtime flag) show a badge on the refresh button so users decide when to reload.
  • Self-notifications are filtered: changes made in the current browser window never trigger a redundant re-fetch.
  • If Ably is unavailable (e.g. token endpoint returns 503), the system continues on polling with no errors.

CollectionSyncNotificationService

Located at packages/client/src/services/collection-sync-notification.service.svelte.ts. Subscribes to the Ably project:{projectId}:sync channel and coordinates with SignalDbService.

// Subscribe when entering a project context (called from layout)
const connected = await syncNotificationService.subscribeToProject(projectId, userId);
if (connected) signalDbService.setPollingInterval(30_000); // reduce heartbeat while Ably is active

// Unsubscribe and restore polling on teardown
await syncNotificationService.unsubscribeFromProject();
signalDbService.setPollingInterval(5000);

// Check whether a manual collection has unseen remote changes
syncNotificationService.hasRemoteChanges("tenant-abc-contacts"); // boolean ($state)

// Clear the badge after a manual refresh
syncNotificationService.clearRemoteChanges("tenant-abc-contacts");

Collections opt in to the polling-based fallback as before:

signalDbService.getCollection("my-collection", { realtime: true });
// or
signalDbService.startRealtime("my-collection");

Best Practices

  1. Use getCollection() lazily: Collections sync on first access. Don't pre-register collections you don't need yet.
  2. Wait for initial sync: Use await signalDbService.isInitiallySynced(name) before rendering data-dependent UI.
  3. Soft deletes: Entities use deletedAt field. The sync system tracks deletions via this field.
  4. baseRevision: Always include baseRevision in PATCH requests for conflict detection.
  5. Avoid raw fetch for synced data: Use collection.find() / collection.findOne() for reads. The sync layer handles server communication.

Debugging

In development, you can inspect:

  • IndexedDB: Open Browser DevTools → Application → IndexedDB to see collections and the __signaldb_sync__ metadata stores
  • Console logs: Collection creation and sync operations are logged at debug level
  • SignalDB DevTools: Uncomment the devtools import in signal-db.service.svelte.ts
Markdown llm.txt