# Local-First Architecture

The platform uses a local-first architecture built on **SignalDB** with IndexedDB persistence. All data interactions happen locally first, with bi-directional synchronization to the server via REST APIs.

## Overview

- **Immediate Responsiveness**: All reads and writes target local IndexedDB-backed collections. UI updates are instant.
- **Offline Functionality**: Collections persist in IndexedDB across page reloads. When offline, users can browse cached data and make edits; changes are queued and pushed when connectivity returns.
- **Automatic Synchronization**: The `SyncManager` handles incremental pull (via `updatedAt` timestamps) and push (local mutations).
- **Conflict Resolution**: Server-side `baseRevision` checking detects concurrent edits. A Conflict Resolution UI lets users choose "Accept Theirs" or "Keep Mine".
- **Per-Entity Sync State**: An `EntitySyncStateService` derives per-entity status (`synced`, `localChanges`, `conflict`).

```mermaid
graph TB
    subgraph "Client"
        UI[Svelte Components] --> Collections[SignalDB Collections]
        Collections --> IDB[IndexedDB Persistence]
        SM[SmallstackSyncManager] --> Collections
        SM --> API[REST API calls]
        NS[NetworkStatusService] --> SM
        ESS[EntitySyncStateService] --> SM
    end

    subgraph "Server"
        API --> Auth[Auth.js]
        Auth --> Endpoints[Generic CRUD Endpoints]
        Endpoints --> MongoDB
    end
```

## Core Services

### SignalDbService

Located at `packages/client/src/services/signal-db.service.svelte.ts`. The central service for data management.

**Key responsibilities:**
- Creates and manages SignalDB collections with IndexedDB persistence adapters
- Configures the `SmallstackSyncManager` for bi-directional sync
- Tracks initial sync state per collection (`initiallySynced`)
- Tracks last sync timestamps per collection (`lastSyncedAt`)
- Manages sync conflicts and their resolution
- Handles configurable polling for opted-in collections (default 5s; drops to 30s heartbeat when Ably is active)

```typescript
// Getting a collection (creates it on first access, starts sync)
const collection = signalDbService.getCollection<MyEntity>("tenant-abc-types");

// With real-time polling enabled
const collection = signalDbService.getCollection<MyEntity>("tenant-abc-types", { realtime: true });

// Wait for initial data to be available
await signalDbService.isInitiallySynced("tenant-abc-types");

// Adjust polling interval (e.g., reduce to 30s heartbeat while Ably is active)
signalDbService.setPollingInterval(30_000);

// Check whether a collection has been registered
signalDbService.hasCollection("tenant-abc-types"); // boolean

// Check whether a collection has realtime polling enabled
signalDbService.isRealtimeCollection("tenant-abc-types"); // boolean

// Manually trigger a pull sync for a collection
await signalDbService.refreshCollection("tenant-abc-types");
```

### SmallstackSyncManager

Located at `packages/client/src/services/smallstack-sync-manager.ts`. A subclass of SignalDB's `SyncManager` that exposes pending change information:

```typescript
// Get IDs of entities with un-pushed local changes
const pendingIds: Set<string> = syncManager.getPendingChangeIds("my-collection");

// Get detailed pending changes for a specific entity
const changes = syncManager.getPendingChanges("my-collection", "entity-123");

// Get total pending change count across all collections
const count: number = syncManager.getPendingChangeCount();
```

### NetworkStatusService

Located at `packages/client/src/services/network-status.service.svelte.ts`. Reactive online/offline detection using browser APIs:

```typescript
import { networkStatusService } from "@smallstack/client";

// Reactive boolean ($state)
networkStatusService.isOnline; // true or false

// Listen for status changes
const unsubscribe = networkStatusService.onStatusChange((isOnline) => {
    console.log("Network status changed:", isOnline);
});
```

### EntitySyncStateService

Located at `packages/client/src/services/entity-sync-state.service.svelte.ts`. Derives per-entity sync state:

```typescript
import { entitySyncStateService } from "@smallstack/client";

// Returns: "synced" | "localChanges" | "conflict"
const state = entitySyncStateService.getState("entity-id", "collection-name");

// Total pending count
const count = entitySyncStateService.getTotalPendingCount();
```

## Synchronization

### How Pull Works

The `SyncManager` pull handler uses **incremental sync**:

1. **Initial sync** (`lastFinishedSyncEnd` is null): Fetches all entities via `GET /api/...?updatedAt=0`, returns `{ items: [...] }`
2. **Subsequent syncs**: Fetches only entities modified since the last sync via `GET /api/...?updatedAt=<timestamp>`, categorizes into `added`/`modified`/`removed` based on `revision` and `deletedAt`
3. **Persistence**: The SyncManager itself uses an IndexedDB persistence adapter (`__signaldb_sync__`), so sync metadata (snapshots, change tracking, sync operations) survive page reloads

### How Push Works

Local mutations are captured by SignalDB's change tracking and pushed via:

- **Inserts**: `POST /api/...` with the full entity
- **Updates**: `PATCH /api/.../entityId` with `{ modifier, baseRevision }`
- **Deletes**: `DELETE /api/.../entityId`

### Conflict Detection

The server checks `baseRevision` against the current document revision:

1. Client sends `{ modifier, baseRevision: N }` in PATCH request
2. Server validates that the document's current `revision === N`
3. If mismatch, server returns HTTP 409 with conflict details
4. Client stores the conflict in `signalDbService.conflicts`
5. User resolves via Conflict Resolution UI

### Reconnect Behavior

When the network comes back online, `networkStatusService.onStatusChange` triggers `syncManager.syncAll()` to:
- Push any queued local changes
- Pull any server-side updates missed while offline

## UI Components

### OfflineBannerComponent

Fixed banner that appears when offline, auto-dismissing "Back online" notification on reconnect.

### ConflictResolutionContainer / ConflictResolutionDialog

Renders active conflicts from `signalDbService.conflicts` as side-by-side cards with "Accept Theirs" / "Keep Mine" buttons.

### EntitySyncBadge

Inline badge showing conflict or pending-changes status for a specific entity. Used in the entity editor.

### PendingChangesCounter

Displays total pending change count. Placed in both app layouts.

### StaleBadgeComponent

Shows per-collection staleness indicators based on `lastSyncedAt` timestamps.

## Collection Naming & API Resolution

Collections are named using a convention that maps to API paths:

```
tenant-{projectId}-{typeId}     → /api/projects/{projectId}/types/{typeId}/data
tenant-{projectId}-types        → /api/projects/{projectId}/types
tenant-{projectId}-applications → /api/projects/{projectId}/applications
```

The `describeCollection()` utility from `@smallstack/shared` handles this mapping.

## Offline Auth

When offline, server-side auth hooks (`hooks.server.ts`) catch network errors and allow the request to proceed with the locally cached session. Auth.js uses JWT strategy with 6-month expiry, so offline sessions remain valid for extended periods.

## Real-Time Updates

When an `ABLY_API_KEY` is configured server-side, the platform uses **Ably-driven push notifications** to trigger collection syncs instantly when data changes elsewhere — replacing the continuous 5s polling loop with a 30s safety-net heartbeat.

Without Ably (dev environments or missing key), polling remains at the default 5s interval.

### Ably-driven sync behaviour

- **Realtime collections** (`{ realtime: true }`) auto-sync silently within ~1.5 s of a remote change.
- **Manual collections** (no `realtime` flag) show a badge on the refresh button so users decide when to reload.
- Self-notifications are filtered: changes made in the current browser window never trigger a redundant re-fetch.
- If Ably is unavailable (e.g. token endpoint returns 503), the system continues on polling with no errors.

### CollectionSyncNotificationService

Located at `packages/client/src/services/collection-sync-notification.service.svelte.ts`. Subscribes to the Ably `project:{projectId}:sync` channel and coordinates with `SignalDbService`.

```typescript
// Subscribe when entering a project context (called from layout)
const connected = await syncNotificationService.subscribeToProject(projectId, userId);
if (connected) signalDbService.setPollingInterval(30_000); // reduce heartbeat while Ably is active

// Unsubscribe and restore polling on teardown
await syncNotificationService.unsubscribeFromProject();
signalDbService.setPollingInterval(5000);

// Check whether a manual collection has unseen remote changes
syncNotificationService.hasRemoteChanges("tenant-abc-contacts"); // boolean ($state)

// Clear the badge after a manual refresh
syncNotificationService.clearRemoteChanges("tenant-abc-contacts");
```

Collections opt in to the polling-based fallback as before:

```typescript
signalDbService.getCollection("my-collection", { realtime: true });
// or
signalDbService.startRealtime("my-collection");
```

## Best Practices

1. **Use `getCollection()` lazily**: Collections sync on first access. Don't pre-register collections you don't need yet.
2. **Wait for initial sync**: Use `await signalDbService.isInitiallySynced(name)` before rendering data-dependent UI.
3. **Soft deletes**: Entities use `deletedAt` field. The sync system tracks deletions via this field.
4. **baseRevision**: Always include `baseRevision` in PATCH requests for conflict detection.
5. **Avoid raw fetch for synced data**: Use `collection.find()` / `collection.findOne()` for reads. The sync layer handles server communication.

## Debugging

In development, you can inspect:

- **IndexedDB**: Open Browser DevTools → Application → IndexedDB to see collections and the `__signaldb_sync__` metadata stores
- **Console logs**: Collection creation and sync operations are logged at `debug` level
- **SignalDB DevTools**: Uncomment the devtools import in `signal-db.service.svelte.ts`