cms/tibi-docs
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial: Tibi-Doku auf Starlight-Basis (Nova-Theme, Docker-Stack, Sync-Skripte)

Browse Source
This commit is contained in:
apairon committed 2026-10-16 11:43:04 +00:00
commit 58502aa3c8
78 files changed
+22860
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

src/assets/favicon.svg
+40
View File
@@ -0,0 +1,40 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512">
<defs>
<!-- Core gradient: warm center to hot edge -->
<radialGradient id="core" cx="50%" cy="50%" r="50%">
<stop offset="0%" stop-color="#fde68a"/>
<stop offset="30%" stop-color="#fbbf24"/>
<stop offset="60%" stop-color="#f59e0b"/>
<stop offset="85%" stop-color="#ea580c"/>
<stop offset="100%" stop-color="#c2410c"/>
</radialGradient>
<!-- Soft ambient glow -->
<radialGradient id="ambientGlow" cx="50%" cy="50%" r="50%">
<stop offset="0%" stop-color="#fbbf24" stop-opacity="0.25"/>
<stop offset="50%" stop-color="#f59e0b" stop-opacity="0.1"/>
<stop offset="100%" stop-color="#ea580c" stop-opacity="0"/>
</radialGradient>
<!-- Glow filter for the core -->
<filter id="coreGlow" x="-60%" y="-60%" width="220%" height="220%">
<feGaussianBlur in="SourceGraphic" stdDeviation="18" result="blur"/>
<feMerge>
<feMergeNode in="blur"/>
<feMergeNode in="blur"/>
<feMergeNode in="SourceGraphic"/>
</feMerge>
</filter>
</defs>
<!-- Dark background -->
<circle cx="256" cy="256" r="256" fill="#111827"/>
<!-- Ambient warm glow -->
<circle cx="256" cy="256" r="230" fill="url(#ambientGlow)"/>
<!-- Glowing core sphere -->
<circle cx="256" cy="256" r="130" fill="url(#core)" filter="url(#coreGlow)"/>
<!-- Hot white center -->
<circle cx="256" cy="256" r="50" fill="#fff" opacity="0.6"/>
<circle cx="256" cy="256" r="28" fill="#fff" opacity="0.85"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 1.5 KiB

src/assets/houston.webp
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 96 KiB

src/content.config.ts
+7
View File
@@ -0,0 +1,7 @@
import { defineCollection } from 'astro:content';
import { docsLoader } from '@astrojs/starlight/loaders';
import { docsSchema } from '@astrojs/starlight/schema';
export const collections = {
docs: defineCollection({ loader: docsLoader(), schema: docsSchema() }),
};
src/content/docs/adr/0000-vorlage.md
+12
View File
@@ -0,0 +1,12 @@
---
title: 'ADR-0000: Vorlage'
description: Vorlage für Architektur-Entscheidungen.
---
# ADR-0000: Titel
- **Status:** vorgeschlagen
- **Datum:** 2026-10-16
- **Kontext:** Warum musste entschieden werden?
- **Entscheidung:** Was wurde beschlossen?
- **Konsequenzen:** Was folgt daraus (gut/schlecht)?
src/content/docs/adr/0001-starlight-entscheidung.md
+28
View File
@@ -0,0 +1,28 @@
---
title: 'ADR-0001: Starlight statt Svelte-Starter'
description: Warum die Doku auf Astro Starlight basiert.
---
# ADR-0001: Starlight statt Svelte-Starter als Doku-Basis
- **Status:** beschlossen
- **Datum:** 2026-10-16
## Kontext
Eigene Doku-Seite für Entwickler (später Anwender) zu Tibi-Server und Tibi-Admin-Nova. Anforderungen: Markdown-Quellen, gerenderte durchsuchbare Webseite, statisch hostbar, agentenpflegbar. `tibi-svelte-starter` stand als Basis im Raum.
## Entscheidung
Eigenes Repo `cms/tibi-docs` auf **Astro Starlight**-Basis, statischer Build, Pagefind-Suche, Traefik-Docker-Stack wie üblich, Theme an Nova angelehnt.
## Begründung
- Starter/Nova-Stack (SPA + SSR-Hook + Admin-Bundle + Docker-Verbund) ist für Doku Overkill; Suche/Sidebar/TOC/Versionierung müssten selbst gebaut werden.
- Starlight liefert das alles gratis, Markdown bleibt Single Source, Build ist ohne Backend deploybar.
- Altes `tibi-docs` (Demo+Docs vermischt) wird zu `tibi-docs-legacy` archiviert.
## Konsequenzen
- Doku-Repo braucht Node/Astro-Know-how statt nur Svelte/esbuild.
- Server-Docs werden per `npm run sync` übernommen, nicht geforkt.
src/content/docs/anwender/uebersicht.md
+14
View File
@@ -0,0 +1,14 @@
---
title: Übersicht für Anwender
description: Platzhalter für die spätere Redakteurs-Dokumentation.
---
> **Noch im Aufbau.** Hier entsteht die Anwender-Doku für Redakteure: Anmelden, Inhalte pflegen, Medien hochladen, Seiten bauen, freigeben.
Geplant:
- Anmelden und Rollen
- Sammlungen und Dokumente anlegen/bearbeiten
- Medien und Bilder
- Exporte
- Häufige Fragen
src/content/docs/einstieg/begriffe.md
+17
View File
@@ -0,0 +1,17 @@
---
title: Begriffe
description: Die wichtigsten Tibi-Begriffe in einem Glossar.
---
| Begriff | Bedeutung |
|---|---|
| **Namespace / Projekt** | Ein Mandant im Tibi-Server (`namespace:` in der Projekt-YAML), eigene MongoDB-Datenbank (`DB_PREFIX_Namespace`). |
| **Collection** | Eine Dokument-Sammlung mit Feldschema, Validatoren und Permissions (z. B. `articles`). |
| **Dokument** | Ein Datensatz in einer Collection (JSON, mit `_id`). |
| **Hook** | JavaScript-Lebenszyklus (`bind → validate → create/update/delete → return`) pro Collection/Route. |
| **Permission-Set** | `public` / `user` / Rollen: welche Methoden (`get/post/put/delete`) mit welchem Vorfilter erlaubt sind. |
| **PAT** | Personal Access Token für API-Zugriff ohne Login. |
| **SSE-Kanal** | Server-Sent-Events-Kanal für Echtzeit (pub/sub mit Replay und TTL). |
| **SSR-Hook** | Server-Side-Rendering für Kunden-SPAs (`api/hooks/ssr/`), HTML-Cache in der `ssr`-Collection. |
| **Nova** | Das Svelte-Admin-Frontend (`tibi-admin-nova`). |
| **Mock-Modus** | Nova läuft lokal gegen gemockte Daten statt gegen den echten Server. |
src/content/docs/einstieg/schnellstart.md
+48
View File
@@ -0,0 +1,48 @@
---
title: Schnellstart
description: Tibi-Doku lokal lesen und Tibi-Server plus Admin lokal starten.
---
## Diese Doku lokal lesen
```sh
cd /WM_Dev/src/gitbase.de/cms/tibi-docs
npm install
npm run dev
# → http://localhost:4321
```
Oder über den Docker-Stack (Traefik):
```sh
make docker-up
# → https://tibi-docs.code.testversion.online/
make docker-down
```
## Tibi-Server lokal starten
```sh
cd ../tibi-server
cp config.yml.example config.yml # falls nötig
docker compose -f docker-compose-local.yml up -d
go run cmd/server/main.go
```
## Tibi-Admin-Nova lokal starten
```sh
cd ../tibi-admin-nova
yarn install
make docker-up
# → https://tibi-admin-nova.code.testversion.online/
```
## Doku pflegen
Server-Docs nach Änderungen synchronisieren:
```sh
npm run sync
npm run dev # prüfen
```
src/content/docs/einstieg/was-ist-tibi.md
+28
View File
@@ -0,0 +1,28 @@
---
title: Was ist Tibi?
description: Überblick über Tibi-Server, Tibi-Admin-Nova und Tibi-Projekte.
---
**Tibi** ist unser Headless-CMS-Baukasten:
- **Tibi-Server** (Go + MongoDB): konfigurationsgetriebenes API-Backend. Collections werden in YAML definiert, Logik läuft über JavaScript-Hooks, Echtzeit über SSE-Kanäle. Dazu LLM-Anbindung, Dateiupload/Bildverarbeitung, PDF, Audit-Log, Cron-Jobs, OpenAPI und Prometheus-Metriken.
- **Tibi-Admin-Nova** (Svelte 5 + esbuild): das Admin-Frontend. Läuft lokal im Mock-Modus und in Staging/Live statisch hinter einem Reverse Proxy gegen den echten Tibi-Server.
- **Tibi-Projekt** (z. B. aus `tibi-svelte-starter`): Kunden-Website als Svelte-SPA mit SSR-Hook im Tibi-Server.
## Wie hängt das zusammen?
```text
Browser (Nova-Admin / Kunden-SPA)
│ REST /api/v1, SSE
▼
Tibi-Server (Go, config.yml + collections/*.yml + hooks/*.js)
│ MongoDB, Mail, Dateien, LLM-Upstream
▼
MongoDB / Maildev / Storage
```
## Wo geht es weiter?
- Schnellstart: [Einstieg/Schnellstart](/einstieg/schnellstart/)
- Server-Details: [Server/Übersicht](/server/uebersicht/)
- Admin-Details: [Nova/Konzept](/nova/konzept/)
src/content/docs/index.mdx
+40
View File
@@ -0,0 +1,40 @@
---
title: Tibi Doku
description: Entwickler- und Anwender-Dokumentation für Tibi-Server und Tibi-Admin-Nova.
template: splash
hero:
tagline: Headless-CMS mit Go-Backend und Svelte-Frontends — hier steht, wie man damit baut.
image:
file: ../../assets/favicon.svg
actions:
- text: Schnellstart
link: /einstieg/schnellstart/
icon: right-arrow
- text: Tibi-Server
link: /server/uebersicht/
icon: open-book
variant: minimal
- text: Tibi-Admin-Nova
link: /nova/konzept/
icon: open-book
variant: minimal
---
import { Card, CardGrid } from '@astrojs/starlight/components';
## Wo finde ich was?
<CardGrid stagger>
<Card title="Einstieg" icon="rocket">
Was ist Tibi, Schnellstart mit Docker, Begriffe von Namespace bis Collection.
</Card>
<Card title="Tibi-Server" icon="server">
Go-Backend: Collections-YAML, Hooks, Auth, Jobs, Realtime, LLM, Deploy. Wird aus dem Server-Repo synchronisiert.
</Card>
<Card title="Tibi-Admin-Nova" icon="pencil">
Svelte-Admin: Sammlungen pflegen, Exporte, Rollen, Testpläne.
</Card>
<Card title="Referenz" icon="document">
Autogeneriert: OpenAPI-Spec, Hook-JS-API, Collection-Schema. Nicht von Hand pflegen.
</Card>
</CardGrid>
src/content/docs/nova/collection-config.md
+2297
View File
File diff suppressed because it is too large. Load diff
src/content/docs/nova/collection-export.md
+331
View File
@@ -0,0 +1,331 @@
---
title: 'Export-Funktion für Listenansichten (CSV / XLSX / JSON)'
---
# Export-Funktion für Listenansichten (CSV / XLSX / JSON)
> Status: **implementiert** (2026-08-17) — alle Phasen abgeschlossen, 10/10 Playwright-Tests grün, `yarn validate` grün.
> Ziel: Export der aktuell gefilterten Liste (alle Seiten, nicht nur die aktuelle Seite) als CSV, XLSX oder JSON, mit vorgeschalteter Feldauswahl.
---
## 1. Ziel & Umfang
- Export der **aktuell gefilterten Liste** (Suchquery, Feld-Filter, aktiver SubNavigation) über **alle Seiten** hinweg.
- **Drei Formate:** JSON (Default), CSV, XLSX.
- **Vorgeschaltete Feldauswahl** im Dialog — nicht nur die aktuell sichtbaren Spalten, sondern (optional) alle Felder.
- **Batch-Laden** mit Fortschrittsanzeige, da der Server `limit` begrenzen kann.
- **Count-Warnung** bei sehr großen Sammlungen.
- **Per-Feld konfigurierbare `object[]`-Behandlung** (nur im CSV/XLSX-Modus relevant).
- Wiederverwendung bestehender UI-Patterns und Utilities (DRY).
**Nicht im Scope:**
- Relationaler „Pivot"-Export (eine Zeile pro Array-Item mit Wiederholung der Elternfelder).
- Server-seitige Aggregation (max. Array-Länge per Query) — kein passender Endpoint vorhanden.
- Styling/mehrere Sheets in XLSX — abgedeckt via `xlsx-js-style` (SheetJS-Fork mit `cell.s`-Styles); kein `exceljs` nötig.
---
## 2. Ablauf (Flow)
```
[Export-Button in Toolbar]
│
▼
[Export-Dialog: Format + Feldauswahl]
│
▼
[Count-Check (getCount)] ──> wenn total > Schwellwert ──> [Count-Warnung (ConfirmDialog)]
│
▼
[Batch-Laden aller Datensätze mit Fortschritts-Bar]
│
▼
┌─ nur CSV/XLSX ─────────────────────────────┐
│ [Post-Load-Check: max. Array-Länge je │
│ "Dynamische Spalten"-Feld berechnen] │
│ └─> wenn Spaltenzahl > Warnschwellwert ──>│
│ [2. Dialog mit echten Zahlen] │
└─────────────────────────────────────────────┘
│
▼
[Rows bauen + formatieren] ──> [Datei erzeugen] ──> [Download]
```
Der Export läuft **komplett clientseitig** über die bestehende `getEntries`-API. Kein neuer Server-Endpoint nötig.
---
## 3. UI
### 3.1 Button-Platzierung
- In **Toolbar-Zeile 2** von `CollectionView.svelte` (die Zeile mit Filter / Sort / Bulk / View-Toggle / New Entry, ~Zeile 1976).
- Konsistent neben den anderen Listen-Aktionen, in **allen View-Modi** verfügbar (Tabelle, Cards, Media, Navigation) — Export betrifft die Daten, nicht die Darstellung.
- Öffnet den Export-Dialog (Modal, Muster wie `ConfirmDialog` / Create-Language-Dialog, mit `layerStack` für Z-Index).
### 3.2 Export-Dialog (1. Dialog)
**Format-Radio (3 Optionen):**
1. **JSON** — Default, sichere erste Wahl
2. **CSV**
3. **XLSX**
**Feldauswahl:**
- Default = **aktuell sichtbare Tabellenspalten** (`tableColumns`).
- **„Alle auswählen"-Toggle** → alle geflatteneten Felder.
- Checkboxen pro Feld.
**Format-abhängiges Verhalten:**
| | JSON | CSV / XLSX |
|---|---|---|
| Feldauswahl-Granularität | Top-Level-Felder (rohe Struktur) | Geflattenete Blatt-Spalten |
| `object[]`-Strategie-Auswahl | **entfällt** | Strategie-Dropdown pro `object[]`-Feld |
| Post-Load-Check | **entfällt** (keine Spaltenexplosion) | ja (s. 3.3) |
**`object[]`-Strategie-Dropdown (nur CSV/XLSX, nur wenn Feld angehakt):**
- **JSON pro Spalte** (Default — verlustfrei)
- **Dynamische Spalten** (`items.0.name`, `items.1.name`, …)
### 3.3 Zweiter Dialog (nur CSV/XLSX, nach dem Laden)
- Ausgelöst, wenn ein „Dynamische Spalten"-Feld mehr als den **Warnschwellwert** (Default 50) Spalten erzeugen würde.
- Zeigt **echte Zahlen** (max. Array-Länge × Subfeld-Anzahl = resultierende Spaltenzahl).
- **Kein hartes Limit** — „Trotzdem dynamisch" erzeugt die Spalten ohne Einschränkung.
> „Feld `items` hat max. 120 Einträge → 120 × 3 Subfelder = **360 Spalten**. Auf JSON umschalten?"
> **[Auf JSON umschalten (empfohlen)]** **[Trotzdem dynamisch]** **[Abbrechen]**
---
## 4. Daten laden: Batch + Fortschritt + Count-Warnung
Da der Server `limit` begrenzen kann, **nicht** auf `_limit: 9999` verlassen, sondern **paginiert laden**:
1. **`getCount(ns, col, filter)`** → Gesamtzahl (mit aktuellem Filterzustand).
2. **Count-Warnung:** wenn `total > THRESHOLD` (z.B. 500, analog `BULK_EDIT_COUNT_THRESHOLD`) → `ConfirmDialog` „Diese Sammlung enthält X Einträge. Export kann dauern. Fortfahren?".
3. **Batch-Loop:** `for offset = 0; offset < total; offset += BATCH` (BATCH = 500, konfigurierbar) → `getEntries({ _offset, _limit: BATCH, _select, _filter, _q, _lookup })`.
- Query-Parameter werden **beim Exportstart gesnapshotet** (Filter/Suchzustand einfrieren), damit Änderungen während des Ladens keine Auswirkung haben.
- **Fortschritt:** inline Progress-Bar im Dialog (`geladen / total`), aktualisiert pro Batch.
4. Nach dem letzten Batch: Rows bauen → Datei erzeugen → Download.
**Fehlerbehandlung & Abbruch:**
- **Cancel-Button** im Dialog während des Ladens — bricht den laufenden Batch ab (via `AbortSignal`, das `getEntries` über `GetEntriesOptions.signal` unterstützt).
- **Batch-Fehler:** schlägt ein Batch fehl (Netzwerk/Timeout), wird er **einmal retried**; schlägt er erneut fehl, wird der Export abgebrochen, die Progress-Bar in einen Fehlerzustand versetzt und ein Fehler-Toast gezeigt. Bereits geladene Batches werden verworfen (kein teilweiser Export).
**DRY:** Die Query-Bau-Logik (Filter + SubNav + `_q` + `_lookup`) aus `CollectionView` in eine geteilte Helper-Funktion `buildListQueryParams()` extrahieren — Liste & Export filtern damit identisch.
---
## 5. Feld-Flattening (Objekte → Spalten)
`flattenExportColumns(fields)` rekursiv durch `subFields`:
- Jedes **Blatt** = eine Spalte, Pfad `parent.subfield` (z.B. `address.street`, `address.zip`).
- Header = `resolveLabel(field.meta.label)` (i18n) oder der Dotted-Path.
- Wert = `deepGet(record, path)` (existiert bereits in `preview-resolver.ts`).
Damit ist `prop1.unterprop1`, `prop1.unterprop2`, … direkt gelöst: rekursives Flatten des Schemas.
**Pfad → Feld-Auflösung für Formatierung:** `formatValue` benötigt ein `AdminCollectionField`-Objekt. Für geflattenete Dotted-Pfade (`address.street`) muss das Schema traversiert werden, um das SubField an diesem Pfad zu finden (analog `resolveFieldByPath` in preview-resolver.ts). Nur so funktionieren Datums-/Zahlen-/FK-Formatierung auch bei verschachtelten Feldern.
**FK-Lookups für den geflatteneten Spaltensatz:** `getTableFkLookupEntries` arbeitet auf `TableColumn[]`. Für den Export muss der geflattenete Spaltensatz in eine parallele `TableColumn[]`-Struktur überführt (oder manuell Lookup-Specs für FK-Felder im geflatteneten Satz gebaut) werden, damit Fremdschlüssel als lesbare Werte exportiert werden.
**i18n-Felder:** Felder mit `subFields` keyed by language code (i18n-Objekte) werden beim Flatten **nicht** in einzelne Sprach-Spalten aufgeteilt, sondern als **eine Spalte** mit dem Wert der aktuell gewählten Sprache exportiert (via `formatValue`/`resolveI18nObjectValue`).
---
## 6. Array-Behandlung
| Typ | Behandlung |
|-----|-----------|
| `string[]`, `number[]`, `file[]` | **Eine Spalte**, Werte mit **`,`** gejoint. Bei `file[]` die Pfade/URLs. |
| `object[]` | **Per-Feld konfigurierbar** (nur CSV/XLSX):<br>**(a) JSON pro Spalte** — das ganze Array als JSON-String (verlustfrei).<br>**(b) Dynamische Spalten** — `items.0.name`, `items.1.name`, … (nur 1. Ebene, tiefer = JSON). |
| `object[]` **innerhalb** eines `object[]`-Items | Komplett als JSON (bei Strategie „Dynamische Spalten"). |
**Dynamische Spalten — Spaltenanzahl:**
- Da alle Datensätze ohnehin geladen werden, wird die **maximale Array-Länge über alle Datensätze** berechnet und daraus `items.0…items.(max-1)` erzeugt (datengetrieben, korrekt).
- Überschreitet die resultierende Spaltenzahl den **Warnschwellwert** (Default 50), greift der **zweite Dialog** (s. 3.3) — aber **ohne hartes Limit**.
**Tradeoff (bewusst):** Dynamische Spalten machen den Export **nicht-deterministisch** — ändert sich die Datenlage (längeres Array), ändert sich die Spaltenanzahl zwischen zwei Exports. Für CSV/XLSX ok und erwartet.
---
## 7. JSON-Export (Default)
- Feldauswahl auf **Top-Level-Feldern** (rohe Struktur bleibt erhalten, verlustfrei).
- Export = **Array von Objekten** mit den ausgewählten Feldern.
- **Keine** `object[]`-Strategie-Auswahl und **kein** Post-Load-Check nötig.
- Batch-Laden + Fortschritt + Count-Warnung gelten trotzdem.
**Format:** rohe (verschachtelte) Struktur der ausgewählten Top-Level-Felder — entspricht „einfacher JSON-Export".
---
## 8. Packages
**`xlsx-js-style`** (SheetJS-0.18.5-Fork mit Zell-Styles) als einziges neues Paket — deckt beide Tabellenformate ab:
- `XLSX.utils.json_to_sheet(rows)` + `XLSX.writeFile(wb, "export.xlsx")`
- `XLSX.utils.sheet_to_csv(ws)` für CSV
→ **DRY:** ein Paket, ein Code-Pfad, eine Row-Struktur. CSV-Quoting/Escaping übernimmt die Lib (Kommas in Werten werden korrekt gequotet).
---
## 9. DRY / Wiederverwendung
Bestehende Bausteine werden wiederverwendet statt neu gebaut:
- **`deepGet`** (preview-resolver) → Wertextraktion
- **`formatValue`** (format-value.ts) → Datums-/Zahlen-/Bool-/FK-Formatierung (gleiche Darstellung wie in der Tabelle)
- **`resolveLabel`** → Spalten-Header (i18n)
- **`buildListQueryParams()`** (neu extrahiert aus `CollectionView`) → identische Filterung für Liste & Export
- **`getTableFkLookupEntries`** → FK-Lookups, damit Fremdschlüssel als lesbare Werte exportiert werden
- **UI-Pattern:** Dialog wie `ConfirmDialog`/Create-Language-Dialog, Toast-Feedback über `toast-store`, `layerStack` für Overlay-Z-Index
- **i18n:** neue Keys unter `admin.export.*` in `de.json`/`en.json`
---
## 10. Neue Dateien & Änderungen
### Neue Dateien
1. **`frontend/src/lib/utils/export.ts`**
- `flattenExportColumns(fields)` → `ExportColumn[]` (Pfad, Label, Feld, Array-Strategie)
- `buildExportRows(records, columns)` → `(string | number)[][]` (mit `deepGet` + `formatValue` + Array-Join / JSON / Dynamische Spalten)
- `generateCsv(rows)` / `generateXlsx(rows, sheetName)` / `generateJson(records, fields)` + `triggerDownload(blob, filename)`
- `loadAllRecordsInBatches(params, onProgress)` → Batch-Loop mit Fortschritts-Callback
- `computeMaxArrayLengths(records, fields)` → für den Post-Load-Check
2. **`frontend/src/lib/components/ui/ExportDialog.svelte`**
- Format-Radio (JSON / CSV / XLSX)
- Feld-Checkboxen (Default = sichtbare Spalten + „Alle"-Toggle)
- `object[]`-Strategie-Dropdown (nur CSV/XLSX)
- Progress-Bar
- Count-Warnung + zweiter Dialog (Post-Load-Check)
### Änderungen
3. **`CollectionView.svelte`**
- Export-Button in Toolbar-Zeile 2
- Dialog-Verdrahtung
- `buildListQueryParams()`-Refactor (Query-Bau extrahieren)
4. **i18n** — Keys unter `admin.export.*` in `de.json`/`en.json`
---
## 11. Konfigurierbare Schwellwerte
| Schwellwert | Default | Zweck |
|---|---|---|
| `BATCH_SIZE` | 500 | Batch-Größe beim Laden |
| `COUNT_WARNING_THRESHOLD` | 500 | Count-Warnung vor dem Laden |
| `COLUMN_WARNING_THRESHOLD` | 50 | Auslöser für den 2. Dialog (kein hartes Limit) |
---
## 12. Entscheidungen (alle geklärt)
- [x] **JSON-Export-Format:** **rohe (verschachtelte) Struktur** der ausgewählten Top-Level-Felder (verlustfrei, entspricht „einfacher JSON-Export").
- [x] **Schwellwerte:** feste, sinnvolle Konstanten (Batch 500 / Count 500 / Spalten 50) — **nicht** im Dialog konfigurierbar.
- [x] **Einfache Arrays** (`string[]`, `number[]`, `file[]`): bleiben beim **Komma-Join** — kein Strategie-Dropdown dafür. Das `object[]`-Strategie-Dropdown existiert nur für `object[]`-Felder.
---
## 13. Teststrategie (Playwright-only)
**Wichtig:** Das Repo hat **keine Unit-Test-Infrastruktur** (kein Vitest/Jest) — nur Playwright (e2e / e2e-mobile / e2e-visual). Es wird **kein neues Test-Framework** eingeführt. Die pure Logik in `export.ts` wird über Playwright getestet, indem der **Download abgefangen und der Dateiinhalt validiert** wird:
- **Download-Capture:** `page.waitForEvent('download')` → `download.saveAs(tempPath)` → Datei lesen.
- **CSV:** Text lesen, Zeilen parsen, Header/Quoting prüfen.
- **XLSX:** mit der `xlsx-js-style`-Lib im Test (`XLSX.read`, Styles via `{ cellStyles: true }`) das Sheet laden und Zellen prüfen.
- **JSON:** `JSON.parse` und Struktur/Felder prüfen.
- Die Tests laufen gegen den bestehenden **Mock-Server** (`frontend/src/lib/mock/`), mit geseedeten Fixtures (inkl. `object[]`-Feldern mit unterschiedlich langen Arrays).
**Playwright-Testplan (UI-Flow + Download-Inhalt):**
- [ ] Export-Button in Toolbar sichtbar, öffnet Dialog.
- [ ] Format-Radio (JSON/CSV/XLSX), Default = JSON.
- [ ] Feldauswahl: Default = sichtbare Spalten, „Alle"-Toggle.
- [ ] `object[]`-Strategie-Dropdown nur im CSV/XLSX-Modus sichtbar.
- [ ] JSON-Export: alle gefilterten Datensätze, rohe Struktur, korrekte Feldauswahl (Download-Inhalt prüfen).
- [ ] CSV-Export: Header, Zeilen, FK-Werte, Quoting bei Kommas/Anführungszeichen (Download-Inhalt prüfen).
- [ ] XLSX-Export: Sheet-Zellen prüfen (Download-Inhalt prüfen).
- [ ] `object[]` Strategie „JSON pro Spalte" und „Dynamische Spalten" (Spaltennamen `items.0.name`, … prüfen).
- [ ] Dynamische Spalten mit max. Länge > Schwellwert → 2. Dialog, alle 3 Optionen.
- [ ] Batch-Laden: Fortschritts-Bar, korrekte Gesamtzahl, keine Duplikate/Lücken.
- [ ] Cancel-Button bricht Laden ab; Batch-Fehler → Retry, dann Abbruch + Fehler-Toast.
- [ ] Count-Warnung bei großer Sammlung (Abbrechen / Fortfahren).
- [ ] Filterzustand (Suchquery, Feld-Filter, SubNav) wird korrekt übernommen.
**Hinweis:** Der Export respektiert den aktuellen Filterzustand. Soft-Delete-/`published`-Zustände werden **nicht** gesondert gefiltert — es wird exportiert, was die Liste zeigt (gleiche Semantik wie die Listenansicht).
---
## 14. Schrittweiser Implementierungsplan
> **Strenge Regeln für jeden Schritt:**
> 1. **Nach jedem Code-Change:** `LocalTester` mit dem engsten ausführbaren Check (`yarn validate` bzw. betroffene Playwright-Spec) **bevor** weitere Edits folgen.
> 2. **Nach jedem Code-Change:** `LocalReviewer` über den Diff — **vor** der finalen Antwort.
> 3. **Nach jedem Code-Change:** `index_repository` (Codebase-Memory) neu laufen lassen.
> 4. **Keine neuen Abhängigkeiten** außer `xlsx-js-style`.
> 5. **DRY:** bestehende Utilities wiederverwenden (`deepGet`, `formatValue`, `resolveLabel`, `layerStack`, `toast-store`, `ConfirmDialog`) — nichts duplizieren.
> 6. **Keine Server-Änderungen** — alles clientseitig.
### Phase 0 — Vorbereitung
- [ ] `xlsx-js-style` als Dependency installieren (`yarn add xlsx-js-style`)
- [ ] i18n-Keys unter `admin.export.*` in `de.json`/`en.json` anlegen
### Phase 1 — Pure Logik `frontend/src/lib/utils/export.ts`
- [ ] `flattenExportColumns(fields)` → rekursives Flatten, Dotted-Pfade, Header-Labels, i18n-Felder als eine Spalte
- [ ] Pfad→Feld-Auflösung (Dotted-Path → `AdminCollectionField`) für `formatValue` (analog `resolveFieldByPath`)
- [ ] `buildExportRows(records, columns)` → `deepGet` + `formatValue`, Komma-Join einfacher Arrays, `object[]`-Strategien (JSON pro Spalte / Dynamische Spalten), JSON-Fallback für tiefe Verschachtelung
- [ ] `computeMaxArrayLengths(records, fields)` → für Post-Load-Check
- [ ] `generateCsv` / `generateXlsx` / `generateJson` + `triggerDownload(blob, filename)`
- [ ] `loadAllRecordsInBatches(params, onProgress, signal)` → Batch-Loop (500), Retry (1×), AbortSignal
- [ ] FK-Lookup-Specs für den geflatteneten Spaltensatz bauen (Übersetzung von `TableColumn[]`)
- [ ] **Regel:** `yarn validate` grün
### Phase 2 — Dialog `frontend/src/lib/components/ui/ExportDialog.svelte`
- [ ] Grundgerüst: Format-Radio (JSON/CSV/XLSX, Default JSON), Feld-Checkboxen (Default = `tableColumns`), „Alle auswählen"-Toggle
- [ ] `object[]`-Strategie-Dropdown (nur CSV/XLSX, nur für angehakte `object[]`-Felder)
- [ ] Progress-Bar + Cancel-Button
- [ ] Count-Warnung via `ConfirmDialog` (> 500)
- [ ] Post-Load-Check + 2. Dialog (Spaltenzahl > 50 → [Auf JSON umschalten] [Trotzdem dynamisch] [Abbrechen])
- [ ] **Regel:** `yarn validate` grün
### Phase 3 — Integration `CollectionView.svelte`
- [ ] `buildListQueryParams()` aus `CollectionView` extrahieren (DRY-Refactor, Filter + SubNav + `_q` + `_lookup`)
- [ ] Export-Button in Toolbar-Zeile 2
- [ ] Dialog-Verdrahtung (State, Aufräumen bei Navigation/Component-Destroy)
- [ ] **Regel:** `yarn validate` grün
### Phase 4 — Playwright-Tests (`tests/e2e/export.spec.ts`)
- [ ] Dialog-UI-Test (Button, Format-Radio, Default, Alle-Toggle, Strategie-Dropdown-Sichtbarkeit)
- [ ] JSON-Export-Test (Download-Inhalt: gefilterte Datensätze, rohe Struktur)
- [ ] CSV-Export-Test (Download-Inhalt: Header, Zeilen, FK-Werte, Quoting)
- [ ] XLSX-Export-Test (Download-Inhalt: Sheet-Zellen via `xlsx-js-style`-`read`, Header-Style prüfen)
- [ ] `object[]`-Strategie-Tests (JSON pro Spalte / Dynamische Spalten)
- [ ] 2. Dialog-Test (Spaltenzahl > Schwellwert, alle 3 Optionen)
- [ ] Count-Warnung-Test
- [ ] Cancel-/Retry-Test
- [ ] **Regel:** betroffene Playwright-Specs grün (`yarn test:e2e tests/e2e/export.spec.ts`)
### Phase 5 — Abschluss
- [x] `LocalReviewer` über den kompletten Diff (Bugs, Regressions, Validierungslücken)
- [x] `yarn validate` + `yarn test:e2e` komplett grün
- [x] `index_repository` (Codebase-Memory-Sync)
- [x] Plan-Datei als „implementiert" markieren bzw. Erfahrungen zurückschreiben
## 15. Umsetzungs-Notizen (Erfahrungen)
- **Header für verschachtelte Felder:** Dotted-Path (`seo.metaTitle`) statt nur Leaf-Label — Leaf-Labels sind mehrdeutig (zwei Objekte können gleiche Subfeld-Labels haben).
- **`rows`-Feld in pages.yml hat Label „Seiteninhalt"** — Tests müssen Labels statt Feldnamen prüfen.
- **`content` (richtext) wird ausgeschlossen** — gleiche Semantik wie die Listenansicht (`getDisplayFields`).
- **Pagination-Bound:** `getCount` unterstützt kein `_q` (Suche) → der echte Gesamtwert kommt aus dem `count`-Header der ersten Batch-Antwort (`loadAllRecordsInBatches`).
- **CSV-Formula-Injection:** Zellen mit `=`, `+`, `-`, `@` werden mit `'`-Präfix sanitized (`sanitizeCsvCell`).
- **Abbrechen im Spalten-Warn-Dialog** schließt den Export-Dialog nicht (nur den aktuellen Export-Versuch) — UX-Fix aus Test EXP-10.
- **Dev-Server-Port:** `esbuild.config.js` liest jetzt `PORT`-Env (Default 3000) — nötig, weil Port 3000 in der Dev-Umgebung von openchamber belegt ist. Tests laufen mit `CODING_URL=http://localhost:3001`.
- **Vorbestehende Test-Failures:** `COLL-01`/`COLL-10` (collection-list) erwarten 20 Collections, Mock liefert 21 — Daten-Drift, unabhängig vom Export-Feature.
src/content/docs/nova/custom-export-script.md
+121
View File
@@ -0,0 +1,121 @@
---
title: 'Custom Export Scripts (`collection.meta.exports`)'
---
# Custom Export Scripts (`collection.meta.exports`)
Custom exports are ES modules that run **client-side** in the ExportDialog.
One module may serve several exports — `ctx.exportName` tells them apart.
## Config
```yaml
# collections/<name>.yml
meta:
exports:
- name: bestellreport
label: { de: "Bestellreport", en: "Order report" }
file: "_/assets/exports/reports.mjs" # _/... project-relative (recommended);
# also: /_/... (alias), /assets/... (host-absolute), https://..., data:, blob:, { eval }
filename: "bestellreport-{date}.xlsx" # optional, {date} → YYYY-MM-DD
```
No `select` — the script loads via `db.find` / `db.loadAll` itself (server syntax, see below).
Server schema: `tibi-types/schemas/config/collection-meta.schema.json` (`meta.exports`,
`additionalProperties: false` — `name` + `file` required).
In the ExportDialog, collections with `meta.exports` show two tabs: **Predefined**
(custom scripts, active by default — no format/field selection) and **Generic**
(the standard JSON/CSV/XLSX export). Without `meta.exports` the dialog is
unchanged (generic only, no tabs).
## Module contract
```js
// _/assets/exports/reports.mjs
export default async function (ctx) {
if (ctx.exportName === "bestellreport") { /* … */ }
return {
workbook: ctx.xlsx.utils.book_new(), // OR blob | text | bytes (exactly one)
filename: "report-{date}.xlsx", // {date} supported
contentType: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
}
}
```
Return: `{ filename, contentType }` + exactly one payload —
`workbook` (XLSX workbook, needs Excel contentType), `blob`, `text` (CSV/JSON/TXT),
`bytes` (`Uint8Array`/`ArrayBuffer`, e.g. PDF). Filenames are sanitized
(`/:*?"<>|` → `_`). See `frontend/src/lib/utils/custom-export-result.ts`.
## `ctx` (mirrors tibi-server hook context where sensible)
| Key | Notes |
|---|---|
| `api`, `project`, `collection` | GetMap-shaped JSON copies (safe to read, mutating is pointless) |
| `namespace`, `$project`, `$namespace`, `$method="export"`, `context` (self) | like `jsData` |
| `export` / `exportName` | own `meta.exports` entry / its `name` |
| `view` | current list state: `search`, `searchMode`, `sortField`, `sortOrder`, `manualSort`, `fieldFilters`, `subNavFilter`, `filter` (merged) |
| `db.find(coll, opts)` | server syntax: `{ filter, select, selector{…:1}, sort["-a","b"], offset, limit, lookup }`. Only `sort[0]` is used (REST limit). |
| `db.count(coll, { filter })` | server syntax |
| `db.loadAll(coll, opts, onProgress?)` | client helper: count → 500-batches (retry-once) + abort + progress. Or loop `find` manually. |
| `xlsx` | gebundelte `xlsx-js-style`-Lib (SheetJS + Zell-Styles, siehe unten) — kein Eigen-Import nötig |
| `helpers` | `deepGet`, `formatValue`, `csvCell` (formula-injection guard), `csvWithBom` (Excel umlauts) |
| `lang`, `languageCodes`, `defaultLanguage` | i18n |
| `progress(done, total, label?)` | dialog progress bar + label |
| `signal` | `AbortSignal` — "Abbrechen" button; `fetch` throws `AbortError` |
No `$auth`: requests run as the logged-in admin (`fetchWithAuth`), the server
enforces permissions. Read-only: no create/update/delete.
## Full example (multi-sheet Excel)
```js
export default async function (ctx) {
ctx.progress(0, 0, "Lade Bestellungen…")
const orders = await ctx.db.loadAll(
"orders",
{ select: ["orderNo", "totals"], filter: ctx.view.filter },
(done, total) => ctx.progress(done, total, "Lade Bestellungen…")
)
const wb = ctx.xlsx.utils.book_new()
ctx.xlsx.utils.book_append_sheet(
wb,
ctx.xlsx.utils.aoa_to_sheet([["Order", "Sum"], ...orders.map((o) => [o.orderNo, o.totals?.grandTotal ?? ""])]),
"Übersicht"
)
const lines = orders.flatMap((o) => (o.items ?? []).map((it) => [o.orderNo, it.sku, it.qty]))
ctx.xlsx.utils.book_append_sheet(wb, ctx.xlsx.utils.aoa_to_sheet([["Order", "SKU", "Qty"], ...lines]), "Positionen")
return { workbook: wb, filename: "bestellreport-{date}.xlsx", contentType: "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" }
}
```
Demo: `frontend/assets/mock/demo-exports/reports.mjs` (mock `pages` collection).
Types: `frontend/src/lib/utils/custom-export.types.ts` (single source of truth).
## Styling (`cell.s` — nur via `xlsx-js-style` möglich)
`ctx.xlsx` ist `xlsx-js-style` (SheetJS-API + schreibbare Zell-Styles).
Unterstützte `cell.s`-Keys: `fill`, `font`, `alignment`, `border`, `numFmt`
(Farben als ARGB, z.B. `"FF2F5496"`).
```js
const HEADER_FILL = "FF2F5496"
function styleHeaderRow(ws, colCount) {
for (let c = 0; c < colCount; c++) {
const addr = ctx.xlsx.utils.encode_cell({ r: 0, c })
if (ws[addr]) ws[addr].s = {
fill: { patternType: "solid", fgColor: { rgb: HEADER_FILL } },
font: { bold: true, color: { rgb: "FFFFFFFF" } },
}
}
}
const ws = ctx.xlsx.utils.aoa_to_sheet([["Order", "Sum"], ...orders.map((o) => [o.orderNo, o.totals?.grandTotal ?? ""])])
styleHeaderRow(ws, 2)
```
Spaltenbreiten + Autofilter gehen ebenfalls ohne Zusatz-Lib:
`ws["!cols"] = [{ wch: 18 }, { wch: 14 }]` und
`ws["!autofilter"] = { ref: "A1:B10" }`.
Hinweis für Tests: `XLSX.read(buf, { cellStyles: true })` — ohne die Option
werden Styles beim Lesen ignoriert.
src/content/docs/nova/konzept.md
+24
View File
@@ -0,0 +1,24 @@
---
title: Nova Konzept
description: Was Tibi-Admin-Nova ist und wie es betrieben wird.
---
**Tibi-Admin-Nova** ist das Svelte-5-Admin-Frontend für Tibi.
- SPA als statisches Bundle, clientseitiges Routing, Daten über die Tibi-API.
- Lokal mit Mock-Modus entwickeln, in Staging/Live statisch hinter Reverse Proxy gegen den echten Tibi-Server.
- Laufzeitkonfiguration (nicht geheim) in `frontend/assets/system.config.json`, Proxy-Upstreams/Modelle in `config/ai-proxy.config.json`.
- Toolchain: `yarn install`, `make docker-start`, URLs unter `https://tibi-admin-nova.code.testversion.online/`.
## Detailseiten
Unter `nova/` liegen übernommene bzw. gepflegte Seiten:
- `sammlungen.md` — Collection-Konfiguration im Admin
- `export.md` — Collection-Export
- `export-skript.md` — eigenes Export-Skript
- `testplaene/` — manuelle Testpläne
## Konvention für Agenten
- Neue Admin-Funktion → `tibi-admin-nova/docs/` aktualisieren, dann hierher synchronisieren (`npm run sync`), Screenshots bei UX-Änderungen.
src/content/docs/nova/test-plans/02-navigation-routing.md
+35
View File
@@ -0,0 +1,35 @@
---
title: 'Navigation & Routing'
---
# Navigation & Routing
## Scope
- Authenticated SPA routing between project list, collection list, collection view, entry view, and action view.
- Browser history, breadcrumb navigation, and navigation-guard behavior.
## Scenarios
| ID | Scenario | Expected outcome | Priority |
| --- | --- | --- | --- |
| NAV-01 | Open /projects after auth | Project list renders without crashing | High |
| NAV-02 | Open /projects/:id | Collection list for the project renders | High |
| NAV-03 | Open /projects/:id/collections/:name | Collection view renders with entry actions | High |
| NAV-04 | Open /projects/:id/collections/:name/entries/:entryId | Entry view renders with editable shell | High |
| NAV-05 | Navigate via project card | SPA navigation happens without a full reload | High |
| NAV-07 | Use browser back after entering a project | Previous view is restored | High |
| NAV-08 | Use browser forward after going back | Forward history restores the collection list | Medium |
| NAV-09 | Open a deep collection URL directly | Correct route component loads | High |
| NAV-10 | Navigate to an unknown route | App shell stays usable and can recover | Medium |
| NAV-11 | Trigger navigation away from a dirty entry | Navigation guard blocks the route change | High |
| NAV-12 | Confirm discard in the guard dialog | Pending navigation continues | High |
| NAV-13 | Render breadcrumb on nested routes | Breadcrumb reflects the current hierarchy | Medium |
| NAV-14 | Click breadcrumb home | Navigation returns to the project list | High |
| NAV-14b | Click a parent breadcrumb segment from entry view | Navigation exits the entry route | High |
| NAV-14c | Click the collection breadcrumb from entry view | Navigation returns to the collection list and preserves remembered query state | High |
## Regression Focus
- Breadcrumb links from entry routes must use the same guarded SPA navigation path as other in-app links.
- The collection breadcrumb should reuse the remembered collection return URL when one exists so list filters and pagination survive the round-trip.
src/content/docs/nova/test-plans/05-collection-view.md
+48
View File
@@ -0,0 +1,48 @@
---
title: 'Test Plan: Collection View'
---
# Test Plan: Collection View
## Scope
- Collection list rendering for table, card, media, and navigation-capable collection views
- Shared collection search, sort, filter, pagination, selection, and URL-state behavior
- Entry navigation from the collection list, including actions triggered while debounced search is pending
## Prerequisites
- Playwright runs in mock mode against project `t1`
- Collections `pages`, `articles`, `blog`, `downloads`, `people`, and `products` are available in mock data
- The shared search input uses its standard debounce timing
## Test Cases
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| CV-01 | Default collection view renders the table with rows and headers | Rows are visible and the list is interactive | High |
| CV-03 | View toggle switches between table and cards | The chosen view renders without a full-page reload | Medium |
| CV-04 | Search filters entries | URL updates with `q`, and only matching rows remain visible | High |
| CV-04b | Successive debounced searches abort stale in-flight list loads | Only the newest search request completes; earlier ones abort and do not overwrite results | High |
| CV-05 | Sorting updates the active order | Column sort changes the URL and list order state | Medium |
| CV-08 | Filter popover opens and applies field filters | Active filters narrow the result list and show visible state | Medium |
| CV-12 | Pagination appears for large collections and page changes work | The next page loads and URL state stays in sync | Medium |
| CV-18 | Clicking a row opens the entry editor | Navigation lands on the selected entry route | High |
| CV-19a | Opening a record during pending debounced search remains stable | Navigation to the entry editor is not reverted by a late search response | High |
| CV-19b | Creating a new record during pending debounced search remains stable | Navigation to `/entries/new` is not reverted by a late search response | High |
| CV-20 | Search state persists in the URL | Reloadable list state includes the active search term | Medium |
## Edge Cases
- Repeated typing across debounce boundaries must not allow stale search responses to flash old rows back into the table
- Silent background refetches must reuse the same latest-query rule and avoid overwriting newer list state
- Search-triggered navigation to entry routes must stay stable even if an older list request resolves later
## Accessibility
- The search input must remain keyboard-focusable and usable with shortcut-driven focus
- Table navigation and entry activation must continue to work after search, sort, and filter changes
## Spec Mapping
- `tests/e2e/collection-view.spec.ts`
src/content/docs/nova/test-plans/07-entry-editor-advanced.md
+61
View File
@@ -0,0 +1,61 @@
---
title: 'Entry Editor Advanced — Test Plan'
---
# Entry Editor Advanced — Test Plan
Tests: undo/redo, diff dialog, drilldown navigation, field search (Ctrl+F), jump targets (j chord).
See `tests/e2e/entry-editor-advanced.spec.ts` for the Playwright implementation.
## Field Search (Ctrl+F)
| ID | Test Case | Steps | Expected Outcome |
| ----- | ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| EA-15 | Ctrl+F opens field search overlay | 1. Open an entry. 2. Press Ctrl+F. | Overlay with search input appears, focused. |
| EA-16 | Field search finds field by name | 1. Open field search. 2. Type a field name (e.g. "titel"). | Matches appear in the result list. |
| EA-17 | Field search navigates to field on selection | 1. Open field search. 2. Type field name. 3. Press Enter. | Overlay closes, target field is scrolled into view and flash-highlighted. |
| EA-20 | Field search expands accordion and navigates into flat `object[]` | 1. Open editor_lab/lab-base.<br>2. Open field search. 3. Search for unique value `#ffffff` (in `variants[0].variantValue`). 4. Press Enter. | Overlay closes. The collapsed `variants[0]` accordion item is expanded. The field `variants.0.variantName` is in the DOM. |
| EA-21 | Field search navigates through drilldown into accordion-mode `object[]` | 1. Open editor_lab/lab-base.<br>2. Open field search. 3. Search for "regionsname" (inside `campaign.channels[0].targeting.regions[0]`). 4. Press Enter. | Overlay closes. Breadcrumb shows "Kanaele #1". The `regions[0]` accordion is expanded, `regionName` field is visible. |
### Bug Baseline (regression safeguard)
Before the `expandAccordionAncestors` + `resolveFieldSegments` fixes:
- EA-20 failed: `object[]` with only primitive subfields was never expanded → field not in DOM.
- EA-21 failed: `object[]` inside drilldown added an extra drill segment for the flat array → landed one level too deep, empty content.
## Jump Targets (j chord)
| ID | Test Case | Steps | Expected Outcome |
| ----- | ---------------------------- | --------------------------------------------------------------- | ----------------------------------------------------------------- |
| EA-18 | `j` key opens jump list | 1. Open an entry. 2. Press `j`, wait 600ms. | Command palette opens, filtered to "Sektion". |
| EA-19 | `j` + digit jumps to section | 1. Open an entry. 2. Press `j` then `1` within 500ms. | Chord consumed, page scrolls to the first section. No overlay. |
## Undo / Redo
| ID | Test Case | Steps | Expected Outcome |
| ------ | --------------------------------------- | ----------------------------------------------------------------------------- | ---------------------------------------- |
| EA-01 | Undo reverts last change | 1. Edit title. 2. Press Ctrl+Z. | Title reverts to original value. |
| EA-02 | Redo restores undone change | 1. Edit title. 2. Ctrl+Z. 3. Ctrl+Shift+Z. | Title re-applies the edit. |
| EA-03 | Undo availability indicator | 1. Edit title. 2. Check undo button. | Undo button becomes enabled. |
| EA-04 | Redo availability indicator | 1. Edit title. 2. Ctrl+Z. 3. Check redo button. | Redo button becomes enabled. |
| EA-05 | Undo button reverts last change | 1. Edit title. 2. Click undo button. | Title reverts. |
| EA-06 | Redo button restores undone change | 1. Edit title. 2. Undo. 3. Click redo button. | Title re-applies. |
| EA-07 | Multiple undos stepped | 1. Edit title twice. 2. Ctrl+Z twice. | Both changes reverted. |
| EA-08 | Undo after save preserves history | 1. Edit title. 2. Save. 3. Ctrl+Z. | Undo still works after save. |
| EA-09 | Undo updates modified badge | 1. Edit title. 2. Ctrl+Z. | Field "Geändert" badge disappears. |
| EA-10 | Dirty state toggles with undo/redo | 1. Edit title. 2. Ctrl+Z. 3. Ctrl+Shift+Z. | Dirty state reflects actual changes. |
| EA-11 | Redo button disabled after new edit | 1. Edit. 2. Undo. 3. Make new edit. | Redo button disabled (history branch). |
| EA-12 | Ctrl+Z at root closes drilldown first | 1. Drill into nested object. 2. Edit inside. 3. Ctrl+Z. | Undo inside drilldown works first. |
| EA-13 | Ctrl+Z at root closes drilldown after | 1. Drill and edit. 2. Ctrl+Z until drilldown pop. 3. Ctrl+Z at root. | Drilldown pops first, then undo at root. |
| EA-14 | Ctrl+Shift+Z redo after drilldown pop | 1. Drill, edit, pop. 2. Ctrl+Shift+Z. | Redo restores the drilled edit. |
## Diff Dialog
| EA-22 | Diff dialog shows field changes | 1. Edit several fields. 2. Click diff button. | Modal shows before/after comparison. |
## Drilldown
| EA-23 | Click drilldown card navigates nested | 1. Click a drilldown card. | Breadcrumb shows new level, sub-fields rendered. |
| EA-24 | Breadcrumb navigation pops level | 1. Drill in twice. 2. Click level 1 breadcrumb. | Returns to level 1, fields rendered. |
| EA-25 | ESC pops drilldown level | 1. Drill in. 2. Press ESC. | Returns to previous level. |
src/content/docs/nova/test-plans/11-foreign-keys.md
+58
View File
@@ -0,0 +1,58 @@
---
title: 'Foreign Keys'
---
# Foreign Keys
## Scope
- Single `foreignKey` widgets in entry forms
- Multi-row `foreignKey` widgets and compact `ForeignKeyChipArrayWidget` chips
- Foreign key drawer browse/select flow and inline create flow
- Direct editing of already linked entries through nested drawers
## Prerequisites
- Run in mock mode only
- Use project `t1`
- Primary route: `/projects/t1/collections/blog/entries/cb1`
- Covered widgets:
- `author` for single FK editing/browsing
- `coauthors` for multi FK row rendering/browse flows
- `relatedArticles` for chip-array behavior
## Test Cases
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| FK-01 | Single FK renders a resolved selected preview | The linked entry preview is visible and human-readable | High |
| FK-02 | Browse button opens the FK drawer | The drawer opens as a modal layer and lists target entries | High |
| FK-03 | Drawer search filters records | Result count changes and matching entries remain selectable | High |
| FK-04 | Selecting a record from the drawer updates the widget | The chosen record becomes the linked value and the single drawer closes | High |
| FK-05 | Removing a selected FK clears the value | The selected preview disappears and the widget returns to add/browse mode | High |
| FK-05b | Editing a selected single FK entry opens a nested edit drawer | The drawer shows the form plus a prominent referenced-by panel, saving refreshes the preview text | High |
| FK-05c | Object drilldowns inside the nested linked-entry drawer stay scoped to that drawer | Clicking a drilldown card renders breadcrumb plus nested fields inside the same edit drawer and persists nested edits | High |
| FK-06 | Multi FK widgets render multiple linked entries | Existing linked rows stay visible and removable/reorderable | High |
| FK-10 | Chip-array FK widgets render linked chips and support drawer pagination for large target collections | Chips stay interactive and the drawer pagination works across pages | Medium |
| FK-10a | Chip body tap/click opens the linked-entry editor | Saving updates the chip label while the chip remove button remains separate | High |
| FK-12 | Inline create is available from the FK drawer | The drawer can switch from browse mode into the embedded create form | High |
| FK-14d | Nested FK drawers keep browse shortcuts scoped to the topmost drawer | Sort/filter/view shortcuts in the inner FK drawer must not leak hidden state into the outer FK drawer's browse view | High |
| FK-15 | Create-and-select in single mode closes back to the field | The new entry is linked and visible in the widget preview | High |
| FK-16 | Create-and-select in multi mode keeps the drawer usable | The new entry is added without closing the overall FK drawer | High |
| FK-20 | Already selected records are highlighted in the drawer | Checked rows remain distinguishable and toggleable | Medium |
| FK-23 | FK drawers with sub-navigation expose category chips and respect them for inline create | The active filter drives create defaults and post-create visibility | Medium |
## Edge Cases
- The chip-array variant must not rely on hover; chip body opens edit, close button removes
- Direct linked-entry editing only targets already selected entries, not drawer search results
- The nested edit drawer must not replace the parent entry editor context or main route entry
- Object drilldowns inside the nested edit drawer must stay inside that drawer instead of hijacking the host editor
- Nested FK drawers must keep global browse shortcuts bound to the visible top drawer only
- Referenced-by information needs to stay visible high in the edit drawer, not buried below the form
## Accessibility
- Selected-entry edit actions remain keyboard reachable in list and card layouts
- Chip-array editing works on tap/click without hover-only affordances
- Nested edit drawers stay dismissible via ESC and reuse the existing drawer focus management
src/content/docs/nova/test-plans/16-ai-integration.md
+45
View File
@@ -0,0 +1,45 @@
---
title: 'Test Plan: AI Integration'
---
# Test Plan: AI Integration
## Scope
- AI chat panel open/close and message flow
- Tool-call recovery and retry behavior
- AI tool resolution for form-field choices and select inputs
- AI pagebuilder context for nested and flat block schemas
- Browser-native voice input in AI chat with API transcription as fallback
## Core checks
1. Open and close the AI chat via FAB and keyboard shortcuts.
2. Send a mocked chat message and verify the user turn appears in the panel.
3. Abort an active stream and verify the chat returns to an idle state.
4. Retry after a failed turn and preserve completed tool context.
5. For select/selectArray and foreignKey/foreignFile fields, AI tool input should resolve valid choice IDs, labels, and numeric 0-based indices to the actual stored choice ID, and reject out-of-range indices.
6. For pagebuilder context, `get_entry_context.pagebuilderFields` must expose the correct path shape for the active schema: nested paths like `rows.N.hero.title` only when the schema defines per-block objects, flat sibling paths like `rows.N.title` when fields are gated directly by `blockType` conditions.
7. For single-item pagebuilder arrays, FK/foreignFile drawer tools should auto-resolve omitted indices like `rows.mediaRef` to `rows.0.mediaRef` before opening/searching the drawer, so visible media options are not reported as empty.
8. When `SpeechRecognition` or `webkitSpeechRecognition` is available, start voice input through the browser API and do not call `/api/ai/transcribe`.
9. While native browser speech recognition is active, interim transcript updates should already appear in the chat textarea before stopping the recording.
10. After a longer pause without new browser speech-recognition events, the chat should auto-stop voice capture and keep the recognized text.
11. Overlapping browser speech-recognition result chunks must be merged without duplicating already recognized words in the final chat textarea, covering mobile Chrome style result streams.
12. When browser speech recognition is unavailable, keep the recorded-audio transcription fallback working via `/api/ai/transcribe`.
13. The microphone button should visualize the shared 3-second pause timeout by fading from active red back toward its normal state and reset to fully active when speech resumes.
14. Persisted AI conversation summaries must be merged into the single leading system message so follow-up turns never send a second `system` role later in the request.
15. On coarse-pointer mobile browser-STT, a short breathing pause must not end dictation at the desktop 3-second mark; the browser-specific pause grace should remain active until after that window.
16. If mobile browser-STT fires an unexpected browser `onend` before the pause timeout expires, dictation should resume automatically instead of finalizing the draft.
17. After voice capture enters pending-send, the UI must expose a visible cancel button for touch devices while keeping the ESC hint only for keyboard-capable devices; cancelling must preserve the text and prevent auto-send.
18. When the AI stream includes reasoning/thinking chunks before `delta.content`, the chat must show them in a subtle expandable panel during streaming and collapse that panel automatically once the final answer text arrives.
19. After `save_entry` creates a new record, the AI must land on the saved `/entries/{id}` route before a later `create_entry`; otherwise repeated create flows would keep editing the first record instead of opening a fresh form.
20. When a second `create_entry` opens directly from a just-saved entry, the `/entries/new` editor must be freshly mounted so prior widget-local UI state is gone and the form shows only defaults/empty values.
21. For date fields (type `date`), `fill_field` and `fill_form` must normalize flexible date inputs: date-only strings like `"2025-06-15"` → full ISO `"2025-06-15T00:00:00.000Z"`, full ISO strings pass through unchanged, and invalid date strings are passed through for server-side validation.
22. Recursive tool-call turns must be capped at a hard limit (25); exceeding it auto-stops with a user-friendly message. The counter resets on new user messages, new conversations, or final completion (no more tool calls).
## Target specs
- `tests/e2e/ai-chat.spec.ts`
- `tests/e2e/ai-tools.spec.ts`
- `tests/e2e/ai-advanced.spec.ts`
- `tests/e2e/ai-pagebuilder.spec.ts`
src/content/docs/nova/test-plans/17-ai-tts.md
+28
View File
@@ -0,0 +1,28 @@
---
title: 'Test Plan: AI TTS'
---
# Test Plan: AI TTS
## Scope
- AI chat TTS for streamed assistant messages and manual playback
- Browser-native SpeechSynthesis as preferred engine
- OpenAI TTS API as fallback engine
## Core checks
1. Sentence splitting keeps incomplete trailing text until a real sentence boundary arrives.
2. Streaming chunks are batched into sensible TTS segments instead of per-token playback.
3. Parallel fetch completion order does not reorder spoken output.
4. TTS fetch errors are logged and do not crash the chat flow.
5. After cancel or a new prompt, later TTS playback still works.
6. When a matching local browser voice exists, the browser SpeechSynthesis engine is preferred and `/api/ai/tts` is not called.
7. When no matching local browser voice exists, playback falls back to `/api/ai/tts`.
8. When browser TTS is active with one matching browser voice, the chat should still show that browser voice in the menu together with the speed control.
9. When TTS falls back to the API, the chat should continue to offer the API voice selection list.
10. When multiple browser voices match the current frontend language, the chat should offer those browser voices and use the selected one for playback.
## Target specs
- `tests/e2e/ai-tts.spec.ts`
src/content/docs/nova/test-plans/25-file-upload.md
+47
View File
@@ -0,0 +1,47 @@
---
title: 'File Upload'
---
# File Upload
## Scope
- FileWidget uploads in entry editors
- Drag-and-drop, preview, image editor and downscale behavior
- Automatic media AI autofill after direct image selection in entry view
## Prerequisites
- Run in mock mode only
- Use project t1
- Fake every `/api/ai/vision` call inside the test; never depend on a live AI endpoint
- Collections:
- editor_lab for generic file/image widget coverage
- media_library for image downscale, palette extraction and media AI autofill
## Test Cases
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| F-01 | Upload image via FileWidget | Preview and metadata appear after selection | High |
| F-12 | Large image upload triggers configured downscale | Notice reflects resized dimensions and preview remains usable | High |
| F-15e | Direct media file selection triggers media AI autofill | The app posts a data URI to the vision endpoint and fills the configured target field | High |
| F-15f | Media AI autofill does not overwrite existing target content | Already filled target values stay unchanged; only missing values are written | High |
| F-15g | Media AI pending state is visible and cancelable | Inline AI status appears on the file widget and cancel prevents autofill | High |
| F-15h | Manual media AI rerun overwrites existing image descriptions | Existing images expose a manual AI trigger in the action slot and replace already filled target values | High |
| F-16 | Duplicate an entry with direct file fields | The new draft keeps normal field values, clears direct file/file[] values, and shows a hint that uploads must be repeated | High |
## Edge Cases
- Vision request uses a resized data URI when media AI image sizing is configured
- Duplicating an entry must not carry direct file objects over to the new entry unless a real server-side copy flow exists
- i18n object targets receive structured JSON and fill only missing language keys
- If the AI endpoint fails, the file upload still succeeds and the widget keeps the selected file
- Canceling the inline media AI run keeps the selected file and leaves target fields untouched
- Existing images can rerun media AI manually from the action slot and intentionally overwrite current target values
## Accessibility
- File input remains keyboard reachable through the widget trigger
- i18n language tabs remain operable after automatic autofill
- AI autofill must not block normal save or focus flow in the entry editor
src/content/docs/nova/test-plans/27-media-grid.md
+44
View File
@@ -0,0 +1,44 @@
---
title: 'Media Grid Test Plan'
---
# Media Grid Test Plan
## Scope
- Media grid rendering and view switching for collections with `meta.viewHint.media`
- Upload queue behavior, including `Edit & upload` and inline editor drawer flows
- Regression coverage for i18n-aware fields rendered inside the shared upload-queue `InlineEntryForm`
## Prerequisites
- Mock mode with project `t1`
- Collection `media_library` available under `/projects/t1/collections/media_library`
- Playwright spec: `tests/e2e/media-grid.spec.ts`
## Test Cases
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| MG-01 | Media grid opens as default view for `media_library` | Grid is visible and URL resolves to `?view=media` | High |
| MG-05b | Wechsel von einer anderen Collection nach `media_library` setzt den Media-Default erneut | Ein zuvor gewaehlter Tabellen-/Kartenmodus aus der Quell-Collection wird nicht uebernommen; `media_library` oeffnet wieder im Grid mit `?view=media` | High |
| MG-11a | Upload queue `Edit & upload` opens the inline editor drawer | A dialog with the shared `InlineEntryForm` opens | High |
| MG-11b | Inline editor shows media collection fields | Editable collection fields render; auto-filled non-editable metadata stays hidden | High |
| MG-11e | Inline editor renders i18n-shaped fields with language tabs, toolbar switcher, and `Ctrl+L` cycling | The drawer shows the global language toggle plus per-field i18n tabs; clicking the toolbar toggle or pressing `Ctrl+L` updates the active field tab instead of exposing raw `de`/`en` object editors | High |
| MG-16 | Upload queue titles use filename-derived auto-fill | Queue rows show the normalized filename title without extension | Medium |
| MG-18 | Quick upload waits for the current queue item's media AI result | The create request is sent only after the item's AI caption is ready, and the request body already contains the generated i18n caption | High |
| MG-19 | Earlier queue items can upload while later queue items still run media AI | The first finished item uploads immediately while later rows keep their running AI status until their own caption is ready | High |
| MG-20 | Stopping a running queue AI task pauses it without surfacing an error and allows a manual restart | After pressing stop the row shows a restart action instead of the error chip, and restarting completes the upload with the generated caption | High |
## Edge Cases
- Multiple queued files keep independent pending states while one drawer is open
- Upload drawer keeps i18n state scoped to the current inline form instance
- Optional i18n fields remain optional and must not block upload when left empty
- All media AI requests are faked in Playwright; upload-queue coverage must not depend on a live `/api/ai/vision` backend
## Accessibility
- Upload queue actions remain keyboard reachable
- The inline editor drawer exposes a proper dialog role and focus management
- Language toggles and i18n tabs stay operable via keyboard in the drawer
src/content/docs/nova/test-plans/34-i18n-fields.md
+38
View File
@@ -0,0 +1,38 @@
---
title: '34 i18n Fields'
---
# 34 i18n Fields
## Scope
Field-level i18n editing in project t1, including language tabs, active-language switching, and AI-assisted subtree translation for supported text widgets.
## Prerequisites
- Mock mode enabled
- Project t1
- Collection i18n_content
- Entry ic2 for missing EN translations
## Test Cases
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| I18N-01 | i18n field shows language tabs | Tabs for configured languages are visible on detected i18n fields | High |
| I18N-10 | Switching tabs changes visible content | Active language content changes without losing the other language value | High |
| I18N-30 | Editing one language marks the entry dirty | Unsaved badge appears after editing an i18n value | High |
| I18N-33 | AI translation fills a missing simple target value | Switching to the target tab and clicking translate inserts the translated target value and marks the entry dirty | High |
| I18N-34 | AI translation fills a nested subtree | Switching to the target tab and clicking translate on a nested i18n object fills supported target leaves while preserving the field shape | High |
## Edge Cases
- Translation button is hidden for readonly i18n fields
- Empty source language content does not overwrite the target
- Invalid AI JSON response shows an error toast and does not mutate the field
- Only supported string widgets are translated; other values remain local copies from the source subtree
## Accessibility
- Language tabs remain keyboard-focusable buttons with correct tab semantics
- Translate actions expose a descriptive aria-label for translating into the active target language from the auto-selected best source language
src/content/docs/nova/test-plans/35-entry-i18n.md
+58
View File
@@ -0,0 +1,58 @@
---
title: 'Entry I18n'
---
# Entry I18n
## Scope
- Entry-level i18n collections in project t1
- Create flow for collections that store one language per entry
- Draft language selection before the first save
## Prerequisites
- Run in mock mode only
- Use project t1
- Collections:
- articles_i18n for default-language create flow
- articles_i18n_picklang for explicit language picker flow
- articles_i18n_nested for nested languageField/groupField tests
## Test Cases
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| EI18N-06 | Open /entries/new for articles_i18n | Draft language switcher is visible, default language is preselected, hidden system fields stay hidden | High |
| EI18N-07 | Change draft language before first save | Selected draft language becomes active and remains active after save | High |
| EI18N-08 | Create from collection without usable default language | Language picker appears before navigation and selected language is prefilled in the draft | High |
| EI18N-08b | Duplicate an existing translated entry from EntryView | New draft keeps the source language active and visible translatable fields are prefilled from the source entry | High |
| EI18N-09 | Run AI translate on an existing target-language entry | Best sibling source is picked by content score and language order, translatable fields update, non-translatable fields stay unchanged | High |
| EI18N-24 | Use the Nova icon inside the create-language menu | The normal row still creates a copy, the Nova action creates the target language and immediately starts translation from the currently open source entry | High |
| EI18N-13 | Edit a normal field in an existing translated entry | TranslationBar state stays stable and no extra sibling list GET is triggered while typing | High |
| EI18N-12 | Press Ctrl+L on an existing translated entry | Entry view navigates to the next sibling language instead of only flashing the shortcut highlight | High |
| EI18N-25 | Create translation from existing entry navigates to create mode with pre-filled fields | After clicking a target language in the create menu, the browser navigates to /entries/new?lang=<target>; required fields from the source entry are preserved, non-required clearFields are emptied | High |
| EI18N-26 | Create translation with unsaved changes shows dirty guard first | If the source entry has unsaved changes, a discard dialog appears before navigating to create mode | Medium |
| EI18N-60 | Nested i18n: details.lang and details.translationGroup are hidden from form | The object field "details" is visible but the sub-fields "lang" and "translationGroup" are not rendered | High |
| EI18N-61 | Nested i18n: regular fields are visible | title, slug, summary, body are visible and editable | High |
| EI18N-62 | Nested i18n: TranslationBar shows language tabs | TranslationBar renders with DE active, EN/FR siblings visible | High |
| EI18N-63 | Nested i18n: create translation from existing entry | Creating a translation navigates to create mode with pre-filled fields, details.lang is set to the target language | High |
## Edge Cases
- Direct deeplink to /entries/new should still get a usable default if one exists
- Collections without a default language must not silently create the wrong language
- Duplicating an EN or FR entry must not reopen the draft in the collection default language and hide the copied values in another language tab
- When multiple sibling source entries have equal content, the earlier configured language should win as deterministic fallback
- The Nova action in the create menu must keep the source pinned to the currently open language entry, even if another sibling would score higher globally
- Ctrl+L must not be swallowed by field-i18n logic when the collection only uses entry-level i18n
- Unsaved typing in non-language fields must not re-request sibling translations for the current translation group
- Nested i18n config (languageField inside an object field): the parent object stays visible but system sub-fields (lang, translationGroup) are filtered out in FieldRenderer
- Creating a translation in nested config: clone must correctly handle dotted clearField paths (e.g. empty nested title/body)
- Unterbrochener Create-Mode (Benutzer navigiert weg vor dem Speichern) muss keine verwaisten POSTs hinterlassen — der create-mode Clone-Flow erzeugt erst beim expliziten Speichern einen POST
## Accessibility
- Language picker must be a modal dialog with focus trap and ESC close
- Draft language controls must stay keyboard accessible
- Existing-entry language switching via Ctrl+L must remain keyboard accessible and trigger real navigation
src/content/docs/nova/test-plans/36-voice-input-fields.md
+35
View File
@@ -0,0 +1,35 @@
---
title: 'Test Plan: Voice Input Fields'
---
# Test Plan: Voice Input Fields
## Scope
- Shared STT behavior for plain-text admin inputs
- Search inputs using the shared search component
- Standard single-line and multiline text widgets in entry forms
- Richtext widgets in WYSIWYG and source-view mode
- Foreign-key inline search fields used in filters and forms
- Free-text inputs inside the bulk edit panel
## Core checks
1. Mic control for plain-text fields appears only when the field is focused, unless recording/transcribing is active.
2. Search inputs can be filled through browser voice input and still apply their normal debounced filtering behavior.
3. Text widgets can be filled through browser voice input and keep the recognized text editable afterwards.
4. Richtext widgets can insert recognized text at the current ProseMirror selection.
5. Richtext source mode can be filled through the same focused mic control pattern as plain-text textareas.
6. Foreign-key inline search fields can be filled through voice input and keep their dropdown search behavior intact.
7. Foreign-key filter fields, including multi-select `string[]` variants, expose the same right-aligned browse button as the regular FK widget.
8. Voice errors surface as toasts instead of chat-only inline messages.
9. Bulk-edit free-text inputs, including string operation values and search/replace text fields, can be filled through the shared mic control without breaking the panel flow.
10. Leaving a debounced search input and immediately triggering route navigation (open record / new record) must not bounce the user back to the collection view.
## Target specs
- `tests/e2e/collection-list.spec.ts`
- `tests/e2e/collection-view.spec.ts`
- `tests/e2e/foreign-key-filter.spec.ts`
- `tests/e2e/widgets.spec.ts`
- `tests/e2e/bulk-operations.spec.ts`
src/content/docs/nova/test-plans/37-pagebuilder.md
+39
View File
@@ -0,0 +1,39 @@
---
title: 'Pagebuilder'
---
# Pagebuilder
## Scope
- Pagebuilder thumbnail and live-preview rendering in project t1
- FK lookup hydration for block registries and mock/demo blocks
- URL normalization for linked media in pagebuilder rows
## Prerequisites
- Run in mock mode only
- Use project t1
- Relevant spec: tests/e2e/pagebuilder.spec.ts
## Test Cases
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| PB-32 | Hero preview without explicit bgImage uses linked mediaRef lookup | Thumbnail/live preview renders the linked media image instead of an empty background | High |
| PB-36b | Direct image-widget block in editor_lab pageContent | Before save, the live thumbnail renders the direct block image source in live mode and the mock placeholder variant in mock mode; after save, the generated screenshot replaces the raw image source | High |
| PB-40b | Lookup normalization receives a data: or blob: media URL | Inline media URI stays unchanged and is not prefixed with /api/_/<namespace>/ | High |
| PB-40c | Existing server-style nested `_lookup` entries in pagebuilder rows are hydrated for preview | Pagebuilder hydration rewrites nested lookup `file.src` values to absolute admin URLs without changing the underlying mock/API payload shape | High |
| PB-35b | Nested object[] items inside a pagebuilder block use FK/foreignFile lookups | Thumbnail/live preview resolves indexed paths like slides.0.image into the correct child item instead of leaving nested media unresolved | High |
## Edge Cases
- Newly uploaded media may exist as data URI before backend persistence; pagebuilder previews must still render it.
- Direct file/image widgets inside screenshot-enabled pagebuilder blocks must be included in both live thumbnail rendering and saved screenshot generation.
- Blob URLs from client-side editors or upload flows must remain browser-local and must not be rewritten.
- Lookup embedding must honor numeric path segments for nested object[] descendants inside pagebuilder blocks.
- Existing server-style `_lookup` payloads inside pagebuilder rows must stay server-style in mock/API data, but preview hydration still has to absolutize nested `file.src` values for admin rendering.
## Accessibility
- Thumbnail previews remain decorative only; no additional focus targets are introduced by the media rendering path.
src/content/docs/nova/test-plans/38-foreign-file.md
+53
View File
@@ -0,0 +1,53 @@
---
title: 'ForeignFile / ForeignMedia'
---
# ForeignFile / ForeignMedia
## Scope
- Single and multi `foreignFile` / `foreignMedia` widgets in the admin editor
- Direct upload via the shared `FileUploadQueue`
- Shared media-AI queue behavior for direct uploads into `media_library`
- Direct editing of already linked media entries via nested drawers
## Prerequisites
- Run in mock mode only
- Use project `t1`
- Fake every `/api/ai/vision` call inside the test; never depend on a live AI endpoint
- Entry route: `/projects/t1/collections/blog/entries/cb1`
- Widget targets:
- `headerImage` for single foreign media
- `relatedMedia` for multi foreign media
## Test Cases
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| FF-07a | Mock-mode thumbnails for preselected related media stay browser-loadable | Existing `cm1`/`cm2`/`cm3` image cards render mock-safe thumbnail URLs instead of broken `/api/_/.../file/...` requests | High |
| FF-05b | Editing the selected single media entry opens the nested edit drawer | The drawer shows media fields plus a prominent referenced-by panel, saving refreshes the widget preview | High |
| FF-05c | Object drilldowns inside the nested media edit drawer stay scoped to that drawer | Clicking a drilldown card renders breadcrumb plus nested fields inside the same edit drawer and persists nested edits | High |
| FF-10a | Editing a selected media card in multi mode updates only that linked card | The card title/preview refreshes in place and the rest of the selection stays intact | High |
| FF-12 | Uploading a file opens the shared queue/editor flow in single mode | The inline editor opens and the queue item remains editable/cancelable | High |
| FF-17 | Single-mode quick upload creates and selects the new media entry | The widget immediately resolves the created media entry as the selected value | High |
| FF-17b | Single-mode quick upload waits for media AI before create | The create payload contains the generated i18n caption and starts only after the item's AI finished | High |
| FF-17c | Multi-mode quick upload appends a newly uploaded media entry | The new created media entry is added to the existing selection without replacing older items | High |
| FF-17d | Multi-mode quick upload all respects per-item AI completion | Earlier queue items upload as soon as their own AI is ready while later items may still show running AI status | High |
| FF-17e | Nested drilldown foreign media queues survive leaving the drilldown | A prepared queue item is preserved when the drilldown closes and is still flushed before entry save | High |
## Edge Cases
- Single mode hides the upload zone while a queue item is pending; users operate on the queue entry instead
- Multi mode keeps the upload zone visible while selected entries and queue items coexist
- Selected media cards expose their edit action without requiring hover so touch devices can reach it
- Nested object drilldowns inside the media edit drawer must stay scoped to that drawer instead of the host editor
- The shared queue behavior must match the `viewHint.media` upload queue semantics for AI timing and payload content
- Drilldown-mounted foreign media widgets must preserve prepared queue items across unmount/remount and still flush them on save
## Accessibility
- Upload zone actions remain keyboard reachable in both single and multi mode
- Inline editor dialog remains dismissible via keyboard after adding queued files
- Direct upload controls stay operable without relying on hover-only affordances
- Linked media edit actions stay visible and keyboard reachable in both single and multi card layouts
src/content/docs/nova/test-plans/39-preview-resolver.md
+28
View File
@@ -0,0 +1,28 @@
---
title: 'Preview Resolver'
---
# Preview Resolver
Scope: Preview semantics in t1 for collection cards/tables, FK mini previews, and object or object[] previews inside entries.
Fixtures:
- Project: t1
- Entry: /projects/t1/collections/editor_lab/entries/lab-base
- Collections: blog, people, downloads, pages, media_library, editor_lab
Regression focus:
- Without explicit preview config, preview auto-detection must still pick meaningful visible fields.
- With explicit preview config, preview stays strict and must not auto-fill missing title, meta, image, or extra object fields.
- Explicit slot paths in dot notation such as profile.name or profile.photo must resolve in collection cards, tables, media previews, and FK-driven target previews.
Cases:
- PR-01 to PR-17: Existing coverage for FK fallback, collection tables/cards, eval previews, and object[] eval labels.
- PR-18: readonly object[] without meta.preview still shows meaningful auto-detected values instead of collapsing to ids or empty output.
- PR-19: object[] with explicit meta.preview eval shows only configured preview text and does not append other values such as variantValue or boolean fields.
- PR-20: people collection uses nested preview slot paths profile.name, profile.position, and profile.photo for card/table rendering and related FK previews.
- PR-21: blog coauthors FK chips resolve people slot previews with label plus secondary text and do not fall back to raw ids.
- PR-22: list/card badge slots that point to foreign keys resolve the target collection preview text instead of showing the stored FK id.
- PR-23: structured FK previews keep badge slots foreign-aware as well, so chips and inline mini previews do not regress to raw ids.
- PR-24: collection tables keep foreign-key cell previews foreign-aware even when the target preview badge itself points to another foreign key.
- PR-25: FK drawer tables keep the same nested badge resolution, so browse rows do not regress to raw foreign ids.
src/content/docs/nova/test-plans/40-actions.md
+25
View File
@@ -0,0 +1,25 @@
---
title: 'Test Plan: Actions'
---
# Test Plan: Actions
## Scope
- Project action discovery in the project overview
- Action detail routing and method-tab state
- Runnable vs. locked action methods
- Action execution result and progress rendering
## Core checks
1. Action cards in the project overview open the expected action detail route.
2. Method badges and method tabs keep the selected method in the query string while the action route is active.
3. Leaving the action detail route after switching methods must not carry the `method` query parameter onto the next route.
4. Back navigation from an action detail returns to the project overview without reopening the action.
5. Runnable actions show inputs, can execute, and render output or progress consistently.
6. Locked or non-runnable actions expose the correct disabled state and hint messaging.
## Target specs
- `tests/e2e/actions.spec.ts`
src/content/docs/nova/test-plans/41-search-index-status.md
+28
View File
@@ -0,0 +1,28 @@
---
title: 'Test Plan: Search Index Status Notice'
---
# Test Plan: Search Index Status Notice
## Scope
- Project-level search-index status notice in the collection overview header
- Collection-level search-index status notice in the collection header
- Live polling and automatic notice removal when the queue drains
## Mock anchor
- Mock project `t1`, collection `products`
- `products` exposes one `ngram` search mode and two records with `_queuePending: true`
## Core checks
1. The project overview aggregates pending `_queue` counts across collections that expose `ngram` or `vector` search modes.
2. The collection header shows the notice only for the current collection when its `_queue` count is greater than zero.
3. The notice disappears automatically after the next polling cycle once `_queue` returns zero.
4. Collections without relevant search modes do not trigger the queue check and do not show the notice.
## Target specs
- `tests/e2e/project-list.spec.ts`
- `tests/e2e/collection-view.spec.ts`
src/content/docs/nova/test-plans/42-impersonation.md
+51
View File
@@ -0,0 +1,51 @@
---
title: 'Impersonation (Admin → User Session)'
---
# Impersonation (Admin → User Session)
## Scope
- Impersonation einer Benutzersession durch einen Admin
- `POST /login/impersonate` Endpoint im Mock-Interceptor
- UI-Trigger im UserDetailDrawer (Button + ConfirmDialog)
- Amber-Banner im AdminLayout mit Stop-Button
- Rollback zur Admin-Session via `stopImpersonation()`
- Berechtigungen: SystemAdmin (role=0) kann alle User impersonaten; OrgAdmin nur User in eigenen Orgs (Serverseitig geprüft)
## Prerequisites
- Run in mock mode only (real impersonation requires live tibi-server)
- Admin-Benutzer `admin` (role=0) und Ziel-User `maria.schmidt` (u2) existieren
- Zweiter Admin `peter.gross` (u7) existiert für Admin-auf-Admin-Tests
- Relevant spec: `tests/e2e/impersonation.spec.ts`
## Test Cases
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| IMP-01 | Admin sieht "Als Benutzer einloggen"-Button im UserDetailDrawer | Button mit `mdiLogin` Icon ist sichtbar für role=0 | High |
| IMP-02 | Admin klickt Impersonate → ConfirmDialog erscheint | ConfirmDialog variant="info" mit Titel und Nachricht | High |
| IMP-03 | Admin bestätigt Impersonation → Weiterleitung zu /projects | Amber-Banner "Eingeloggt als @maria.schmidt" sichtbar; User-Menü zeigt Maria Schmidt | High |
| IMP-04 | Impersonation-Banner zeigt korrekte Impersonation-Info | Banner enthält "(Impersonation SystemAdmin)" | High |
| IMP-05 | Stop-Button im Banner → Admin-Session wiederhergestellt | Zurück zu /projects mit Admin-User im User-Menü; Banner verschwunden | High |
| IMP-06 | Nicht-Admin (role=1) sieht KEINEN Impersonate-Button | Button ist nicht sichtbar für maria.schmidt | High |
| IMP-07 | Admin sieht Impersonate-Button NICHT für sich selbst | admin (u1) kann sich nicht selbst impersonaten | Medium |
| IMP-08 | Admin-zu-Admin Impersonation (admin → peter.gross) funktioniert | Impersonation auf peter.gross möglich; Banner zeigt "@peter.gross" | Medium |
| IMP-09 | Impersonation-Banner verschwindet nach page.reload() | sessionStorage stellt impersonatedBy-State wieder her | Medium |
| IMP-10 | Admin sieht Impersonate-Button auch für User ohne Org/Team | solo.user (u8) hat Impersonate-Button | Low |
## Edge Cases
- Impersonation des eigenen Users: Button muss versteckt sein (canImpersonate = isAdmin && user.id !== currentUser.id)
- Mock-Interceptor lehnt Impersonation ab, wenn currentUser nicht role=0 ist (403)
- Mock-Interceptor gibt 404 bei unbekannter userId
- Nach Stop-Impersonation muss die Admin-Original-Session wiederhergestellt sein (via singleRefresh)
- Bei fehlendem Original-Token (sessionStorage gelöscht) → full logout (Fallback)
## Accessibility
- Amber-Banner hat `data-ai-id="impersonation-banner"` für Test-Selektion
- Stop-Button hat `data-ai-id="impersonation-stop-btn"`
- Impersonate-Button hat `data-ai-id="impersonate-btn"`
- Farbkontrast: Amber auf hell/dunkel erfüllt WCAG AA
src/content/docs/nova/test-plans/43-backup-restore.md
+93
View File
@@ -0,0 +1,93 @@
---
title: 'Backup & Restore'
---
# Backup & Restore
## Scope
- Admin UI: Backup export (SSE progress, download) and import (upload, analyze, restore)
- API communication via `backup-api.ts` (SSE parser, 6 endpoints)
- Mock interceptor coverage for all backup routes
- Access control: project admin permission required
## Prerequisites
- Run in mock mode (default)
- Use project t1
- Admin token must have `AllowBackup: true`
## Test Cases
### Export (Create Backup)
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| BCK-01 | Admin menu "Backup & Restore" button visible on project page | Button appears in admin dropdown; clicking navigates to `/projects/:id/backup` | High |
| BCK-02 | Admin menu opens dropdown with backup entry and navigates | Dropdown shows "Backup & Restore" link; clicking navigates to backup route | High |
| BCK-03 | Direct deep-link to backup route renders | Visiting `/projects/:id/backup` shows the backup page with export tab active | High |
| BCK-04 | Export tab shows collections with header checkbox and pre-selected state | Table lists all non-hidden collections; header checkbox is checked (all selected) | High |
| BCK-05 | Backup export SSE flow completes | Clicking "Start backup" shows progress section; SSE events arrive; final done state shows download button with size and countdown | High |
| BCK-06 | Import tab shows upload zone | Switching to import tab shows drag-and-drop zone for ZIP files | High |
| BCK-07 | Backup back button navigates to project | Clicking back arrow returns to project page | High |
| BCK-08 | Existing action routes still work (no regression) | Other admin routes (e.g. search, inspect) remain functional after backup route registration | High |
| BCK-09 | Unknown action deep-link still shows not-found (no regression) | Visiting `/projects/:id/backup/nonexistent` shows no route match | High |
### Export — Analyze-Download Estimates
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| BCK-10 | Export table shows media file counts from analyze-download | Each collection row has media count and media size; grand summary shown below table | Medium |
| BCK-11 | Analyze-download handles server error gracefully | If endpoint fails, table shows "—" placeholders; no crash | Medium |
### Export — Progress Display
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| BCK-12 | Step progress bar shows per-collection documents/media | During backup, blue step bar displays current collection name, docs count, media count | Medium |
| BCK-13 | Grand progress bar shows overall collections progress | Green grand bar displays "Collection X of Y (Z%)" with cumulative docs/media | Medium |
| BCK-14 | Phase labels appear during processing | Text shows phase: "Reading documents…", "Packing collection…", etc. | Low |
| BCK-15 | Progress bar handles events without step/grand gracefully | If server sends minimal events, fallback pulse bar shows instead of broken UI | Low |
### Import — Upload
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| BCK-16 | Upload with real progress tracking (live mode) | XHR upload fires progress events; progress bar fills during upload | Medium |
| BCK-17 | Invalid ZIP file shows error | Uploading a non-ZIP or corrupt file shows error message | High |
### Import — Analysis & Restore
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| BCK-18 | Analysis table shows field warnings and non-restorable collections | Collections missing in target are greyed out; field diffs shown as warnings | High |
| BCK-19 | Merge toggle affects restore mode | Toggle shows correct description (merge on/off) | Medium |
| BCK-20 | Restore SSE flow completes and shows result | Restoring progress bar appears; done state shows restored docs/media count | High |
### Auth & Permissions
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| BCK-21 | Non-admin user sees permission error | Visiting backup route shows "No administrator permission" message | High |
| BCK-22 | API responds 403 for tokens without `AllowBackup` | Server rejects requests from tokens missing the backup permission | High |
### Edge Cases
- Countdown timer expires → "Download link expired" message with retry button
- Abort during SSE (navigate away or reset) → clean state, no hanging request
- Expired restore token → server returns 404; frontend shows error
- Download token consumed → second attempt returns 410 (expired)
- Very large ZIP upload → progress bar shows accurate percentage
## Accessibility
- Progress bars have `role="progressbar"` with `aria-valuenow/min/max`
- Progress container uses `role="status"` and `aria-live="polite"`
- Tab buttons use `role="tab"` / `role="tablist"` with `aria-selected`
- Checkboxes in collection tables have visible labels and keyboard support
## Test Automation
- E2E tests: `tests/e2e/backup.spec.ts` (BCK-01 through BCK-09)
- Mock routes: `frontend/src/lib/mock/interceptor.ts` (lines 3022–3280)
- API mock data includes edge cases: expired tokens, schema warnings, non-existent collections
src/content/docs/nova/test-plans/44-snapshots.md
+93
View File
@@ -0,0 +1,93 @@
---
title: 'Snapshots (Admin)'
---
# Snapshots (Admin)
## Scope
- Admin UI: Snapshot overview (configs, entries, stats), snapshot trigger (SSE), download (SSE + ZIP blob), restore (SSE), delete, prune
- API communication via `snapshot-api.ts` (SSE parser, 10 endpoints)
- Mock interceptor coverage for all snapshot routes
- Access control: project admin permission required
## Prerequisites
- Run in mock mode (default)
- Use project t1
- Admin token must have `AllowSnapshot: true`
## Test Cases
### Overview & Navigation
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| SNP-01 | Admin menu "Snapshots" button visible on project page | Button appears in admin dropdown; clicking navigates to `/projects/:id/snapshots` | High |
| SNP-02 | Direct deep-link to snapshots renders | Visiting `/projects/:id/snapshots` shows the snapshot view with configs tab active | High |
| SNP-03 | Three tabs exist (Configs, Entries, Stats) | Tabs are visible and switchable by click and by hash fragment | High |
| SNP-04 | Snapshots back button navigates to project | Clicking back arrow returns to project page | High |
### Configs Tab
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| SNP-05 | Config list shows snapshot configurations | List renders with name, description, schedule, collections, media flag, retention info | High |
| SNP-06 | Trigger button starts snapshot SSE flow | Clicking trigger shows progress bar; SSE events arrive; final done state shows entry ID | High |
| SNP-07 | Trigger button is disabled for disabled configs | Config with `enabled: false` has disabled trigger button | Medium |
### Entries Tab
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| SNP-08 | Entries table shows snapshot history | Table renders with rows per entry; columns show config name, status, date, collections, size | High |
| SNP-09 | Clicking a row expands detail view | Expanded detail shows per-collection breakdown (docs, media, size) | Medium |
| SNP-10 | Detail view data matches entry row summary | Aggregate totals in detail match the row-level display | Low |
### Download — SSE + ZIP
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| SNP-11 | Download dialog opens with entry info | Clicking download icon on an entry row opens a dialog showing config name, date, size | High |
| SNP-12 | Download SSE flow shows progress | Clicking "Start download" shows spinner and phase messages (`packing` collections) | High |
| SNP-13 | Download SSE completes with download URL | After SSE finishes, dialog shows "Ready for download" with file size and countdown | High |
| SNP-14 | Download ZIP blob triggers browser download | Clicking "Download" button in done state triggers a file download with expected filename | High |
| SNP-15 | Expired download link shows expired state | Server returns 410; dialog shows "Download link expired" with retry button | Medium |
### Restore
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| SNP-16 | Restore dialog opens with collection selection | Clicking restore icon opens a dialog with merge toggle and checkboxes for collections | High |
| SNP-17 | Restore SSE flow completes | Selecting collections and starting restore shows progress bar; done state shows restored doc/media count | High |
| SNP-18 | Merge toggle description changes | Toggling merge updates description text | Medium |
### Delete, Stats & Prune
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| SNP-19 | Delete confirmation dialog appears | Clicking delete icon opens a ConfirmDialog with entry info | High |
| SNP-20 | Delete removes entry from list | Confirming delete removes row from table; API returns 200 | High |
| SNP-21 | Stats tab shows storage metrics | Stats view renders with entry count, total bytes, media bytes, config breakdown | Medium |
| SNP-22 | Prune button triggers retention cleanup | Clicking prune button sends API request; success message shown | Medium |
### Auth & Permissions
| ID | Description | Expected Result | Priority |
| --- | --- | --- | --- |
| SNP-23 | Non-admin user sees permission error | Visiting snapshots route shows "No administrator permission" message | High |
### Edge Cases
- SSE abort during download (navigate away or close dialog) → clean state, no hanging request
- SSE abort during trigger (navigate away) → clean state
- Expired download token → server returns 410; "Download link expired" message with retry
- Empty entries list → "No snapshots yet" placeholder shown
- Empty configs list → "No configurations" placeholder shown
## Accessibility
- Dialog has `role="dialog"` with `aria-label`
- Progress bars have `role="progressbar"` with `aria-valuenow/min/max`
- Tab list uses `role="tablist"` with `aria-selected`
- Buttons have descriptive titles or aria-labels
src/content/docs/reference/index.md
+12
View File
@@ -0,0 +1,12 @@
---
title: Referenz-Übersicht
description: Autogenerierte API- und Schema-Referenz.
---
> Diese Seiten werden generiert — **nicht von Hand editieren**.
- **OpenAPI**: wird per `npm run sync:openapi` aus dem laufenden Tibi-Server (`/api/openapi.json`) hierher kopiert.
- **Hook-JS-API**: Typdefinitionen aus `tibi-server/tibi-types/` bzw. `tibi-types`.
- **Collection-Schema**: YAML-Schema-Referenz für Sammlungs-Konfiguration.
Solange die Generatoren leer sind, steht hier nur diese Übersicht.
src/content/docs/server/01-architecture.md
+261
View File
@@ -0,0 +1,261 @@
---
title: 'Architecture'
---
# Architecture
## System Overview
Tibi-Server is a Go-based headless CMS that provides dynamically configured REST APIs backed by MongoDB. Projects are defined via YAML configuration files that specify collections, hooks, jobs, and assets. Each project gets its own MongoDB database (prefixed with the server DB prefix).
```text
┌─────────────────────────────────────────────────────────┐
│ HTTP Clients │
└─────────────┬───────────────────────────────┬───────────┘
│ │
▼ ▼
┌─────────────────────────┐ ┌──────────────────────────┐
│ System API │ │ Collection API │
│ /api/v1/login │ │ /api/v1/_/:project/... │
│ /api/v1/user │ │ │
│ /api/v1/project │ │ Dynamic CRUD per │
│ /api/v1/llm/chat │ │ project config │
│ /api/v1/audit │ │ │
│ /api/v1/version │ │ Admin: │
│ Admin: │ │ /reload │
│ /shutdown │ │ /regenerate-search │
│ /backup, /restore │ │ /backup, /restore │
└─────────┬───────────────┘ └─────────┬────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────────────────┐
│ Gin Router / Middleware │
│ Sentry → Gzip → Prometheus → RouterContext → CORS │
└─────────────────────────┬───────────────────────────────┘
│
┌───────────┴───────────┐
▼ ▼
┌──────────────────────┐ ┌─────────────────────────────┐
│ dbapi (System) │ │ Controller (Collections) │
│ Generic CRUD for │ │ CollectionGet/Post/Put/ │
│ User, Project │ │ Delete + BulkUpdate/ │
│ │ │ BulkDelete with Hooks │
└──────────┬───────────┘ └──────────┬──────────────────┘
│ │
│ ┌────────┴────────┐
│ ▼ ▼
│ ┌──────────────────┐ ┌────────────┐
│ │ Hook Engine │ │ Validator │
│ │ (goja JS) │ │ │
│ │ 20+ packages │ └────────────┘
│ └────────┬─────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────────────────┐
│ dbcrud Layer │
│ Abstract DB Interface │
└─────────────────────────┬───────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────┐
│ MongoDB Driver │
│ mongocrud implementation │
└─────────────────────────────────────────────────────────┘
```
## Directory Structure
```text
tibi-server/
├── cmd/server/main.go # Entry point: config loading, DB connection, HTTP server
├── internal/
│ ├── config/server.go # ServerConfig struct, YAML + env variable loading
│ ├── controller/ # HTTP handlers
│ │ ├── setup.go # Router setup, route registration, middleware
│ │ ├── collection_get.go # GET handler with filter, projection, pagination
│ │ ├── collection_count.go # COUNT handler for /_count endpoint
│ │ ├── collection_post.go # POST handler with file upload
│ │ ├── collection_put.go # PUT handler with partial update
│ │ ├── collection_delete.go# DELETE handler with file cleanup
│ │ ├── collection_file.go # File serving with image filter support
│ │ ├── collection.go # bindCollection, handleFileInJSON (file upload logic)
│ │ ├── subst.go # $subst expression resolution for permission filters
│ │ ├── validator.go # Unified validateData (3 modes: Create/Update/Changeset)
│ │ ├── admin_reload.go # Hot-reload project config
│ │ ├── assets_get.go # Static asset serving
│ │ ├── audit_get.go # Audit log query endpoint
│ │ ├── llm_chat.go # LLM chat handler (streaming + non-streaming)
│ │ ├── metric.go # Prometheus middleware
│ │ ├── openapi_project_get.go # OpenAPI spec generation
│ │ ├── collection_bulk_update.go # Bulk update (PUT without ID, filter-based)
│ │ ├── collection_bulk_delete.go # Bulk delete (DELETE without ID, filter-based)
│ │ └── operators.go # Computed update operators ($inc, $mul, $unset)
│ ├── dbapi/ # Generic system-level CRUD framework
│ │ ├── handler.go # Generic GET/POST/PUT/DELETE for registered models
│ │ ├── filter.go # MongoDB filter validation and type conversion
│ │ ├── filter_valid_functions.go # Whitelist of allowed MongoDB operators
│ │ ├── login.go # JWT token creation and validation
│ │ └── register.go # Model registration (User, Project)
│ ├── dbcrud/ # Database abstraction layer
│ │ ├── interface.go # Database, Collection, Query interfaces
│ │ ├── model.go # Base Model (ID, InsertTime, UpdateTime, Ident)
│ │ ├── collection.go # Collection name resolution
│ │ └── validator.go # Struct-tag based validation
│ ├── hook/ # JavaScript hook context packages
│ │ ├── compatibility.go # Legacy function name mapping
│ │ ├── context_db.go # db.find, db.create, db.update, db.delete, db.updateMany, ...
│ │ ├── context_http.go # http.fetch, http.fetchStream
│ │ ├── context_fs.go # fs.readFile, fs.writeFile, fs.stat, ...
│ │ ├── context_smtp.go # smtp.sendMail
│ │ ├── context_response.go # response.write, response.writeSSE, response.flush
│ │ ├── context_channel.go # channel.subscribe, channel.send
│ │ └── ... # 20+ context packages total
│ ├── embedding/ # Embedding provider architecture (ONNX Runtime, tokenizer)
│ ├── image/ # Image processing (resize, effects, WebP)
│ ├── llm/ # LLM provider abstraction, quota management
│ ├── models/ # Domain models and business logic
│ │ ├── api.go # API struct: collections, hooks, jobs, CORS, hub
│ │ ├── project.go # Project model, config loading, setup
│ │ ├── collection.go # Collection model, CollectionConfig, fields, permissions
│ │ ├── permission.go # Permission checking logic
│ │ ├── hook.go # Hook execution (run, exception handling)
│ │ ├── eval_context.go # BuildEvalContext / PopulateJsData — central JS context builder
│ │ ├── field_permissions.go # Eval-based readonly/hidden field checks
│ │ ├── auth.go # AuthData extraction from JWT
│ │ ├── user.go # User model with MD5 password hashing
│ │ └── login.go # Login flow, refresh tokens
│ ├── rankfusion/ # Mode-independent Reciprocal Rank Fusion (N-way weighted RRF)
│ ├── realtime/ # SSE hub with channels, subscriptions, ring buffer
│ ├── suexec/ # Thread-based UID/GID isolation for multi-tenancy
│ ├── utils/ # Helpers: error formatting, search, file paths
│ ├── vectorindex/ # In-memory vector index (dense cosine + sparse inverted)
│ └── version/ # Build version info
├── tibi-types/ # TypeScript type definitions for hook context
├── examples/ # Example project configurations
├── vendor/ # Vendored Go dependencies
├── config.yml # Server configuration
├── Dockerfile # Production Docker image
├── Dockerfile.air # Development Docker image (hot-reload)
├── docker-compose.yml # Full stack: server + mongo + maildev + chisel
└── docker-compose-local.yml # MongoDB only for local development
```
## Request Lifecycle
### Collection Request (e.g., GET /api/v1/_/myproject/articles)
```text
1. HTTP Request
│
2. Gin Middleware Chain
├── Sentry (if configured): trace + error tracking
├── Gzip compression (excludes images, videos, PDFs, /metrics)
├── Prometheus: start timing
├── RouterContext: inject global project API context
└── CORS: resolve config (collection → project → server level)
│
3. Route Handler
├── Create new DB session
├── Resolve project namespace from URL path
├── Resolve collection config from URL path
│
4. Permission Check (MethodIsAllowed)
├── Check token-based permissions (header/query)
├── Check public permissions
├── Check admin role (role=0 → always allowed)
├── Check user permissions (role=1 + project assignment)
└── Check custom permissions (namespace:permName)
│
5. Pre-Filter (getCollectionPreFilter)
├── Apply permission filter (e.g., {published: true})
└── Resolve $subst expressions (auth.id, auth.role, etc.)
│
6. Hook Pipeline (per method)
│ GET: request → read → [db query] → return
│ COUNT: read → [db count] (no return hook)
│ POST: bind → validate → create → [db insert] → return
│ PUT: bind → validate → update → [db update] → return
│ DELETE: delete → [db delete] → return
│
├── Each hook step:
│ ├── Load compiled JS program
│ ├── Build hook context (api, project, collection, request, ...)
│ ├── Inject context packages (db, http, fs, smtp, ...)
│ ├── Execute JS with timeout
│ └── Process return value (data override, results override)
│
├── Hook can throw HookException:
│ ├── {status, json} → JSON response
│ ├── {status, html} → HTML response
│ ├── {status, string} → text response
│ ├── {status, bytes} → binary response
│ └── {status, file} → file download
│
7. Database Operation
├── ReadCollection / CreateDocument / UpdateDocument / DeleteDocument
└── File system operations (upload, delete)
│
8. Audit Log (if enabled)
├── Record action, user, document snapshot, changes
└── Write async to audit collection with TTL index
│
9. Response
├── NaN/Inf values converted to null
├── JSON serialization
└── Prometheus: record duration
```
## Data Model
### Base Model (dbcrud.Model)
Every document in the system inherits from the base model:
```go
type Model struct {
ID interface{} // MongoDB ObjectID
InsertTime *time.Time // Set automatically on create
UpdateTime *time.Time // Set automatically on update
Ident []string // Display identifier (e.g., username, document title)
}
```
### System Models
| Model | Database | Collection | Description |
| -------------- | ---------- | ------------- | ---------------------------------------------- |
| `User` | `{prefix}` | `user` | Admin/user accounts with roles and permissions |
| `Project` | `{prefix}` | `project` | Project definitions with config file paths |
| `AuditLog` | `{prefix}` | `auditlog` | System-wide audit trail |
| `LLMUsageLog` | `{prefix}` | `llmusagelog` | LLM token usage tracking |
### Project Models
| Model | Database | Collection | Description |
| -------------- | ---------------------- | ------------------- | ------------------------------ |
| `Collection` | `{prefix}_{namespace}` | `{collectionName}` | Dynamic, schema-less documents |
Each project gets its own database named `{prefix}_{namespace}`, where `prefix` is the server DB prefix (default: `tibi`) and `namespace` is defined in the project's YAML config.
## Middleware Chain
| Order | Middleware | Source | Purpose |
| ----- | ------------- | --------------------------- | ------------------------------------- |
| 1 | Sentry | `sentrygin.New()` | Error tracking, transaction tracing |
| 2 | Sentry Scope | Custom | Set user IP, start spans |
| 3 | Gzip | `gzip.Gzip()` | Compression (excludes binary formats) |
| 4 | Prometheus | `measureResponseDuration()` | Request timing metrics |
| 5 | RouterContext | Custom | Inject project API context |
| 6 | CORS | `cors.New()` | Cross-origin configuration |
## Key Design Decisions
1. **Schema-less collections**: Collection documents use `map[string]interface{}` (BSON inline) rather than fixed Go structs. Field definitions in YAML config are used for validation and type checking but don't restrict storage.
2. **Per-project databases**: Each project gets its own MongoDB database, providing data isolation between projects.
3. **JavaScript hooks via goja**: Hooks run in a Go-embedded JavaScript engine (goja), not Node.js. This means ECMAScript 5.1 compatibility with some ES6+ features. No npm modules available — all functionality comes from built-in context packages.
4. **Suexec isolation**: On Linux, hook execution and file operations run under the UID/GID of the project's config file owner, providing OS-level multi-tenant isolation.
5. **Session-per-request**: Each HTTP request creates a new MongoDB session that is closed after the request completes.
src/content/docs/server/02-configuration.md
+622
View File
@@ -0,0 +1,622 @@
---
title: 'Configuration'
---
# Configuration
Tibi-Server uses a layered configuration system: a YAML config file with environment variable overrides.
## Server Configuration (`config.yml`)
The server configuration file is loaded from the working directory at startup.
### Complete Reference
```yaml
db:
dial: "mongodb://localhost:27017" # MongoDB connection string
prefix: "tibi" # Database name prefix
logging: "" # Enable DB query logging (any non-empty value)
api:
port: 8080 # HTTP server port
maxBodySize: "100MB" # Max request body size (default: 100MB). Supports B, KB, MB, GB, TB.
# Cascade: Server (hard cap) → Project (default) → Collection (override)
# Project-level: set maxBodySize in project config.yml (can only reduce)
# Collection-level: set maxBodySize in collection config.yml (can override project, but not server)
jwtSecret: "your-jwt-secret" # HMAC-SHA256 secret for JWT signing
secureCookies: true # Set Secure flag on cookies (default: true, set false for local HTTP dev)
accessTokenLifetime: "60m" # JWT access token lifetime (Go duration, default: 60m)
refreshTokenLifetime: "10h" # Refresh token / cookie lifetime (Go duration, default: 10h)
adminTokens: # Server-level admin tokens with granular permissions
- token: "agent-token-xyz" # Token string (sent via X-Admin-Token header or Authorization: Bearer)
label: "ci-deploy" # Optional label for audit logging (identifies the token purpose)
permissions: # List of granted permissions
- project # Shorthand for project.* (all CRUD)
- project.reload # Reload project config
- user # Shorthand for user.* (all CRUD)
- server.shutdown # Server shutdown
- namespace.myproject # All collections in myproject (all actions)
- namespace.*.articles.read # Read articles in any project (wildcard)
- token: "readonly-token"
label: "monitoring" # Shows up in audit logs as tokenLabel
permissions:
- project.read # Only read projects
- user.read # Only read users
- namespace.*.*.read # Readonly on all collections in all projects
mail:
host: "smtp.example.com:587" # SMTP server host:port
username: "user@example.com" # SMTP username
password: "password" # SMTP password
security:
allowAbsolutePaths: false # Allow absolute file paths in hooks/config
allowUpperPaths: false # Allow path traversal above config directory
sentry:
dsn: "" # Sentry DSN for error tracking
environment: "production" # Sentry environment tag
serverName: "tibi-1" # Sentry server name
enableTracing: true # Enable performance tracing
debug: false # Sentry debug mode
cors:
allowOrigins: # Allowed origins (server-level default)
- "https://example.com"
allowCredentials: false # Allow credentials (cookies, auth headers)
allowHeaders: # Allowed request headers
- "Content-Type"
- "X-Auth-Token"
allowMethods: # Allowed HTTP methods
- "GET"
- "POST"
- "PUT"
- "DELETE"
exposeHeaders: [] # Headers exposed to the browser
maxAge: 0 # Preflight cache duration (seconds)
response:
error:
message: "" # Fixed error message for clients (hides details)
stack: false # Include stack traces in error responses
llm:
providers:
- name: "openai" # Provider identifier
type: "openai" # "openai" or "openai-compatible"
baseURL: "https://api.openai.com/v1" # API base URL (auto-set for "openai" type)
apiKey: "sk-..." # API key (prefer env variable)
defaultModel: "gpt-4" # Default model
models: # Available models
- "gpt-4"
- "gpt-3.5-turbo"
maxTokensPerRequest: 4096 # Max tokens per request (optional)
audit:
enabled: false # Enable audit logging
defaultTTL: "720h" # Default audit log retention (30 days)
defaultLimit: 50 # Default query limit for audit GET (fallback: 50)
maxLimit: 10000 # Maximum query limit for audit GET (fallback: 10000)
ratelimit:
enabled: true # Enable login rate limiting (default: true)
loginInitialDelay: "1s" # Initial delay after first failed login attempt
loginMaxDelay: "60s" # Maximum delay cap (exponential backoff)
loginResetAfter: "15m" # Reset attempt counter after inactivity period
embedding:
providers:
- name: "bge-m3" # Provider identifier (referenced in collection search config)
type: "native" # "native" (ONNX), "openai", "openai-compatible", "ollama"
modelPath: "/models/bge-m3" # Path to ONNX model directory (type: native)
dimensions: 1024 # Embedding dimensions (required for native)
- name: "e5-small"
type: "native"
modelPath: "/models/multilingual-e5-small"
dimensions: 384
- name: "openai-embed" # Example: external OpenAI provider
type: "openai"
model: "text-embedding-3-small" # Model name for API calls
apiKey: "sk-..." # API key (prefer env variable)
baseURL: "https://api.openai.com/v1" # API base URL
dimensions: 1536
hook:
cache:
maxEntries: 10000 # Max cache entries per project before LRU eviction (default: 10000)
defaultTTL: "0s" # Default TTL for cache.set() when ttl=0 (default: no expiry)
ratelimit:
backoffInitialDelay: "1s" # Backoff initial delay after first failure (default: 1s)
backoffMaxDelay: "60s" # Backoff maximum delay cap (default: 60s)
backoffResetAfter: "15m" # Backoff reset after inactivity (default: 15m)
windowSize: "60s" # Sliding window duration (default: 60s)
windowMaxHits: 60 # Max hits per sliding window (default: 60)
```
### Default Values (no config file)
When no config file is found and `defaultFallback` is true:
| Setting | Default |
| ------------------------ | --------- |
| `api.port` | `8080` |
| `db.prefix` | `"tibi"` |
| `api.maxBodySize` | `"100MB"` |
| `response.error.stack` | `true` |
## Environment Variable Overrides
Environment variables take precedence over config file values.
| Environment Variable | Config Path | Type | Notes |
| ----------------------------- | ----------------------------- | ------ | ----------------------------------- |
| `DB_DIAL` | `db.dial` | string | MongoDB connection string |
| `MGO_DIAL` | `db.dial` | string | Legacy alias for `DB_DIAL` |
| `DB_PREFIX` | `db.prefix` | string | Database name prefix |
| `DB_LOGGING` | `db.logging` | string | Any non-empty value enables logging |
| `API_UPLOAD_MAX_BODY_SIZE` | `api.upload.maxBodySize` | string | e.g. `"100MB"` (B, KB, MB, GB, TB) |
| `API_PORT` | `api.port` | int | HTTP port (panics if ≤ 0) |
| `PORT` | `api.port` | int | Fallback for `API_PORT` |
| `API_JWTSECRET` | `api.jwtSecret` | string | JWT signing secret |
| `API_SECURE_COOKIES` | `api.secureCookies` | bool | Secure flag on cookies (default: true) |
| `API_ACCESS_TOKEN_LIFETIME` | `api.accessTokenLifetime` | string | Go duration, e.g. `"60m"` (default) |
| `API_REFRESH_TOKEN_LIFETIME` | `api.refreshTokenLifetime` | string | Go duration, e.g. `"10h"` (default) |
| `MAIL_HOST` | `mail.host` | string | SMTP host:port |
| `SECURITY_ALLOWABSOLUTEPATHS` | `security.allowAbsolutePaths` | bool | `"true"` or `"false"` |
| `SECURITY_ALLOWUPPERPATHS` | `security.allowUpperPaths` | bool | `"true"` or `"false"` |
| `RESPONSE_ERROR_STACK` | `response.error.stack` | bool | Show stack traces |
| `RESPONSE_ERROR_MESSAGE` | `response.error.message` | string | Fixed error message |
| `AUDIT_ENABLED` | `audit.enabled` | bool | Enable audit logging |
| `AUDIT_DEFAULT_TTL` | `audit.defaultTTL` | string | Go duration (e.g., `"720h"`) |
| `AUDIT_DEFAULT_LIMIT` | `audit.defaultLimit` | int | Audit GET default page size |
| `AUDIT_MAX_LIMIT` | `audit.maxLimit` | int | Audit GET max page size |
| `RATELIMIT_ENABLED` | `ratelimit.enabled` | bool | Enable login rate limiting |
| `RATELIMIT_LOGIN_INITIAL_DELAY` | `ratelimit.loginInitialDelay` | string | Go duration (e.g., `"1s"`) |
| `RATELIMIT_LOGIN_MAX_DELAY` | `ratelimit.loginMaxDelay` | string | Go duration (e.g., `"60s"`) |
| `RATELIMIT_LOGIN_RESET_AFTER` | `ratelimit.loginResetAfter` | string | Go duration (e.g., `"15m"`) |
| `HOOK_CACHE_MAX_ENTRIES` | `hook.cache.maxEntries` | int | Max cache entries per project |
| `HOOK_CACHE_DEFAULT_TTL` | `hook.cache.defaultTTL` | string | Go duration (e.g., `"5m"`) |
| `HOOK_RATELIMIT_BACKOFF_INITIAL_DELAY` | `hook.ratelimit.backoffInitialDelay` | string | Go duration (e.g., `"1s"`) |
| `HOOK_RATELIMIT_BACKOFF_MAX_DELAY` | `hook.ratelimit.backoffMaxDelay` | string | Go duration (e.g., `"60s"`) |
| `HOOK_RATELIMIT_BACKOFF_RESET_AFTER` | `hook.ratelimit.backoffResetAfter` | string | Go duration (e.g., `"15m"`) |
| `HOOK_RATELIMIT_WINDOW_SIZE` | `hook.ratelimit.windowSize` | string | Go duration (e.g., `"60s"`) |
| `HOOK_RATELIMIT_WINDOW_MAX_HITS` | `hook.ratelimit.windowMaxHits` | int | Max hits per sliding window |
| `LLM_<NAME>_APIKEY` | `llm.providers[name].apiKey` | string | Provider API key (uppercase name) |
| `EMBEDDING_<NAME>_APIKEY` | `embedding.providers[name].apiKey` | string | Embedding provider API key (uppercase name) |
| `EMBEDDING_<NAME>_MODELPATH` | `embedding.providers[name].modelPath` | string | ONNX model directory path (uppercase name) |
| `EMBEDDING_<NAME>_DOCUMENTPREFIX` | `embedding.providers[name].documentPrefix` | string | Global prefix applied to document embeddings (uppercase name) |
| `EMBEDDING_<NAME>_QUERYPREFIX` | `embedding.providers[name].queryPrefix` | string | Global prefix applied to query embeddings (uppercase name) |
### LLM API Key Convention
LLM providers can be configured completely via environment variables (without changing `config.yml`) using `LLM_PROVIDERS`:
```bash
# Register two providers
export LLM_PROVIDERS=openai,local-llama
# Configure the "openai" provider
export LLM_OPENAI_APIKEY=sk-...
# Configure the "local-llama" provider
export LLM_LOCAL-LLAMA_TYPE=openai-compatible
export LLM_LOCAL-LLAMA_BASEURL=http://localhost:11434/v1
export LLM_LOCAL-LLAMA_MODELS=llama3,mistral
export LLM_LOCAL-LLAMA_DEFAULTMODEL=llama3
```
All available config fields (`APIKEY`, `TYPE`, `BASEURL`, `DEFAULTMODEL`, `MODELS`) can be overridden via `LLM_<UPPERCASENAME>_<FIELD>`.
### Embedding Provider Environment Convention
Embedding providers are configured in exactly the same way as LLM providers. Rather than `llm:`, you provide an `embedding:` block in your server's `config.yml`.
Example in `config.yml`:
```yaml
embedding:
providers:
- name: bge-m3
type: native
model: bge-m3
dimensions: 1024
# modelPath: /assets/models/bge-m3
# documentPrefix: ""
# queryPrefix: ""
- name: text-embedding-ada-002
type: openai
model: text-embedding-ada-002
dimensions: 1536
# apiKey: sk-...
```
Just like LLMs, you can provision Embedding providers **purely via environment variables** (very useful for Docker setups):
```bash
# Register two providers via comma-separated list
export EMBEDDING_PROVIDERS=openai-embed,bge-m3
# Configure "openai-embed"
export EMBEDDING_OPENAI-EMBED_TYPE=openai
export EMBEDDING_OPENAI-EMBED_MODEL=text-embedding-ada-002
export EMBEDDING_OPENAI-EMBED_DIMENSIONS=1536
export EMBEDDING_OPENAI-EMBED_APIKEY=sk-...
# Configure "bge-m3"
export EMBEDDING_BGE-M3_TYPE=native
export EMBEDDING_BGE-M3_MODELPATH=/models/bge-m3
export EMBEDDING_BGE-M3_DIMENSIONS=1024
export EMBEDDING_BGE-M3_DOCUMENTPREFIX="search_document: "
export EMBEDDING_BGE-M3_QUERYPREFIX="search_query: "
```
Available config fields: `TYPE`, `MODEL`, `MODELPATH`, `DIMENSIONS`, `BASEURL`, `APIKEY`, `DOCUMENTPREFIX`, `QUERYPREFIX`.
**Note:** The collections within your projects will reference these providers by their `name`.
Default type for embedding providers (if omitted in config) is `"native"` (ONNX).
---
## Project Configuration
Each project has a YAML config file referenced in the project's `configFile` field in the database. The config file defines all collections, hooks, jobs, and assets for that project.
### Complete Project Reference
```yaml
# Project namespace (used as database suffix and URL path segment)
# Must match: [a-zA-Z0-9_]+
namespace: myproject
# Arbitrary metadata accessible in hooks via context.project().meta
meta:
title: "My Project"
version: "1.0"
# Collection definitions
collections:
- name: articles # Collection name (URL path segment, MongoDB collection name)
defaultLanguage: "german" # Default language for text indexes
uploadPath: "./media" # File upload directory (relative to config dir)
fields: # Field definitions (see Collections docs)
- name: title
type: string
validator:
required: true
- name: image
type: file
validator:
maxFileSize: "5MB" # Max file size for this field (supports B, KB, MB, GB, TB)
permissions: # Permission sets (see Authentication docs)
public:
methods:
get: true
filter:
published: true
user:
methods:
get: true
post: true
put: true
delete: true
projections: # Named projections (field subsets)
default:
select:
title: 1
image: 1
detailed:
select:
title: 1
image: 1
content: 1
imageFilter: # Named image filter presets
thumbnail:
- width: 200
height: 200
fit: true
quality: 80
hero:
- width: 1200
height: 600
fill: true
anchor: center
hooks: # Lifecycle hooks (see Hooks docs)
get:
request:
type: javascript
file: "hooks/get_request.js"
timeout: 5000
post:
create:
type: javascript
file: "hooks/post_create.js"
indexes: # Custom MongoDB indexes
- name: title_text
key:
- "$text:title"
defaultLanguage: german
- name: date_sort
key:
- "-publishDate"
background: true
strictFields: true # Reject unknown fields not in schema (default: inherit from project)
cors: # Collection-level CORS override
merge: true
allowOrigins:
- "https://special.example.com"
audit: # Collection-level audit config
enabled: true
actions:
- create
- update
- delete
queryLimits: # Query result limits for GET list endpoint
defaultLimit: 100 # Default limit when no ?limit= param (fallback: 10000)
maxLimit: 500 # Maximum allowed limit (fallback: 10000)
audit: # Project-level audit (opt-in, default false)
enabled: true # required in addition to server audit.enabled
# Cron jobs
jobs:
- cron: "0 */5 * * *" # Every 5 minutes
type: javascript
file: "jobs/cleanup.js"
timeout: 30000 # Milliseconds (30 s)
meta:
description: "Cleanup old drafts"
# Static asset directories
assets:
- name: static
path: "./public"
- name: uploads
path: "./uploads"
# Reject unknown fields for all collections by default (default: false)
strictFields: true
# Project-level CORS configuration
cors:
merge: true # Merge with server CORS (true) or replace (false)
allowOrigins:
- "https://myapp.example.com"
allowCredentials: true
allowHeaders:
- "X-Custom-Header"
# Admin configuration
admin:
tokens:
- token: "deploy-token-abc123"
allowReload: true
# Hook settings (project-level overrides)
hook:
cache:
maxEntries: 5000 # Override server default for this project
defaultTTL: "5m" # Override default TTL for this project
ratelimit:
backoffInitialDelay: "2s" # Override backoff initial delay
backoffMaxDelay: "120s" # Override backoff max delay
backoffResetAfter: "30m" # Override backoff reset period
windowSize: "30s" # Override sliding window duration
windowMaxHits: 30 # Override max hits per window
# Legacy compatibility settings
legacy:
httpCodes: true # true (default): use original status codes for backward compatibility
# false: use RFC-compliant HTTP status codes (201, 204, 401, 409, 422)
```
### Environment Variable Substitution in Project Config
Project config files support a companion `.env` file. If `config.yml.env` exists alongside `config.yml`, its variables are substituted into the YAML before parsing:
```bash
# config.yml.env
API_KEY=secret-key-123
WEBHOOK_URL=https://example.com/webhook
```
```yaml
# config.yml - use ${VAR} syntax
meta:
apiKey: "${API_KEY}"
webhookUrl: "${WEBHOOK_URL}"
```
### YAML Include Support
Project configs support including other YAML files for modular collection definitions:
```yaml
# config.yml
namespace: myproject
collections: !include collections/*.yml
```
This allows splitting collection configs into separate files under a `collections/` directory.
---
## Legacy HTTP Status Codes
By default, Tibi-Server uses legacy HTTP status codes for backward compatibility. To opt into RFC-compliant status codes, set `legacy.httpCodes: false` in the project config:
```yaml
legacy:
httpCodes: false
```
### Status Code Mapping
| Scenario | Legacy (`true`, default) | Modern (`false`) |
|---|---|---|
| POST/create success | 200 OK | **201** Created |
| Single DELETE success | 200 OK + body | **204** No Content |
| Authentication missing | 403 Forbidden | **401** Unauthorized |
| Validation error | 400 Bad Request | **422** Unprocessable Entity |
| Duplicate key conflict | 500 Internal Server Error | **409** Conflict |
| Field permission denied (readonly/hidden) | 400 Bad Request | **403** Forbidden |
**Notes:**
- Bulk DELETE always returns 200 with `{"deleted": N}` regardless of the setting.
- Bulk POST returns 201 in modern mode (with `{"created": N, "ids": [...]}`).
- The default is `true` — existing projects are unaffected unless explicitly opted in.
- The setting is per-project, allowing gradual migration.
---
## CORS Configuration Hierarchy
CORS is configured at three levels, with each level optionally merging with or replacing the parent:
```text
Server CORS (config.yml → cors)
│
├── merge: false → Replace entirely
│
▼
Project CORS (project config → cors)
│
├── merge: true → Append to server origins/headers/methods
├── merge: false → Replace entirely
│
▼
Collection CORS (collection config → cors)
│
└── merge: true → Append to project+server origins/headers/methods
```
### Resolution Order (per request)
1. Check if there's a collection-specific CORS config → `CORSMap["/collectionName"]`
2. Fall back to project-level CORS → `CORSMap["/"]`
3. Fall back to server-level CORS → `CORSMap[""]`
### Merge Behavior
When `merge: true`:
- `allowOrigins` are **appended** to the parent config
- `allowHeaders` are **appended** to the parent config
- `allowMethods` are **appended** to the parent config
- `exposeHeaders` are **replaced** (not appended — see [ISSUES.md](ISSUES.md))
- `allowCredentials` is **replaced**
- `maxAge` is **set to 0** (always)
- Duplicates are automatically removed
### CORSConfig Struct
```go
type CORSConfig struct {
Merge bool `yaml:"merge"`
AllowOrigins []string `yaml:"allowOrigins"`
AllowCredentials bool `yaml:"allowCredentials"`
AllowHeaders []string `yaml:"allowHeaders"`
AllowMethods []string `yaml:"allowMethods"`
ExposeHeaders []string `yaml:"exposeHeaders"`
MaxAge int `yaml:"maxAge"`
}
```
---
## Collection Configuration Reference
### CollectionConfig Struct
```go
type CollectionConfig struct {
Name string // Collection/MongoDB collection name
DefaultLanguage *string // Default language for text indexes
UploadPath *string // File upload directory (relative or absolute)
Permissions map[string]PermissionSet // Permission sets (public, user, token:*, custom)
Projections map[string]ProjectionConfig // Named field projections
Meta map[string]interface{} // Arbitrary metadata
Fields []CollectionField // Field definitions
ImageFilter map[string][]image.FilterParams // Named image filter presets
Hooks map[string]map[string]HookConfig // Lifecycle hooks [method][step]
Indexes []CollectionIndex // Custom MongoDB indexes
CORS *CORSConfig // Collection-level CORS
Audit *AuditCollectionConfig // Collection-level audit config
Bulk *BulkConfig // Bulk operation config
QueryLimits *QueryLimitsConfig // Query result limits (defaultLimit, maxLimit)
}
```
### CollectionIndex
```yaml
indexes:
- name: "unique_email" # Index name
key: # Index keys (prefix with - for descending, $text: for text)
- "email"
unique: true # Unique constraint
sparse: true # Skip documents without the indexed field
background: true # Build in background
dropDups: false # Drop duplicate entries (deprecated in MongoDB 3.0+)
defaultLanguage: "english" # Language for text indexes
languageOverride: "lang" # Field that overrides the language
```
### Field-Level Indexes
Fields can define indexes directly via the `index` property:
```yaml
fields:
- name: email
type: string
index:
- single # Regular index on this field
- unique # Unique index
- text # Text search index
- sparse # Sparse index
```
### Index Auto-Management
On project setup/reload:
1. All configured indexes are created via `EnsureIndex`
2. Indexes that exist in MongoDB but are no longer in the config are automatically dropped (`fixIndexes`)
3. The `_id_` index is never dropped
---
## Admin Configuration
### Project-Level Admin Tokens
Each project can define admin tokens for project-specific operations:
```yaml
admin:
tokens:
- token: "deploy-token-abc123"
allowReload: true
allowRegenerateSearch: true
```
Available permissions:
- `allowReload` – allows reloading the project configuration
- `allowRegenerateSearch` – allows regenerating search indexes (`_search` data)
Usage:
```bash
curl -X POST http://localhost:8080/api/v1/_/myproject/_/admin/reload \
-H "Authorization: Bearer deploy-token-abc123"
```
src/content/docs/server/03-api-endpoints.md
+2410
View File
File diff suppressed because it is too large. Load diff
src/content/docs/server/04-collections.md
+1749
View File
File diff suppressed because it is too large. Load diff
src/content/docs/server/05-authentication.md
+606
View File
@@ -0,0 +1,606 @@
---
title: 'Authentication & Authorization'
---
# Authentication & Authorization
## Overview
Tibi-Server uses JWT-based authentication with a role and permission system. Access control is configured per collection via permission sets that support public access, role-based access, token-based access, and custom permissions.
## Login Flow
### Username/Password Login
```
POST /api/v1/login
Content-Type: application/json
{"username": "admin", "password": "admin"}
```
1. Password is hashed with MD5 and compared against the stored hash
2. If matched, a JWT token and refresh token are generated
3. JWT token is returned in the response body
4. Refresh token is set as an HttpOnly cookie
### Refresh Token Login
```
POST /api/v1/login
Cookie: tibi-refresh-token=eyJ...
```
When no username/password is provided but a refresh token cookie exists:
1. The refresh token JWT is validated (HMAC-SHA256, expiration required)
2. User ID is extracted from the token claims
3. User is looked up by ID
4. New JWT token is issued
### Logout
```
POST /api/v1/login?logout=1
```
Clears the refresh token cookie (sets expiry to -1). Any non-empty `logout` query parameter value triggers logout.
### Impersonation (Admin/OrgAdmin → User)
```
POST /api/v1/login/impersonate
X-Auth-Token: <admin-jwt>
Content-Type: application/json
{"userId": "<target-user-id>"}
```
Allows a SystemAdmin (role=0) or OrgAdmin to obtain a JWT token acting as another user.
This is useful for debugging permission issues ("why can User X not see document Y?").
**Security rules:**
| Role | Can impersonate | Reason |
|---|---|---|
| SystemAdmin (0) | Any user | Admin already has full access |
| OrgAdmin | Users in same org(s) | OrgAdmin can already grant themselves the same team permissions |
| User (1) | Nobody | Would break the permission model |
**Response (200):**
```json
{
"token": "<jwt-acting-as-target-user>",
"user": { "id": "...", "username": "targetuser", "role": 1, ... },
"impersonatedBy": {
"id": "<original-admin-id>",
"username": "admin",
"role": 0
}
}
```
**Important:** The JWT token contains an `impersonatedBy` field in the `object` claim,
which is used by the audit system to record who actually performed each action.
Every audit entry during an impersonated session will include:
- `userId`/`username`/`userRole` → the impersonated (target) user
- `impersonatedBy` → the original admin who initiated the impersonation
- `authMethod` → `"impersonation"`
**Logout from impersonation:** Same as normal logout (`POST /api/v1/login?logout=1`).
The refresh token cookie is cleared, ending the impersonated session. The admin
must re-authenticate with their own credentials.
**Chained impersonation:** If an admin impersonates an org-admin, and that
org-admin then impersonates a user, the `impersonatedBy` field always points to
the **original admin** who started the chain — not the intermediate impersonator.
**Errors:**
| Status | Description |
|---|---|
| 400 | `userId is required` or invalid userId format |
| 403 | `impersonation not allowed: insufficient role` |
| 403 | `impersonation not allowed: not admin of target user's org` |
| 404 | target user not found |
## JWT Token Structure
### Access Token
| Field | Value | Description |
|-------|-------|-------------|
| Algorithm | HS256 | HMAC-SHA256 |
| `object` | User data | `{id, username, role, permissions, orgs, teams, primaryOrgId, meta, ...}` |
| `type` | `"User"` | Token type identifier |
| `exp` | now + 60min | Expiration timestamp |
The `password` field is explicitly excluded from the token (via custom JSON marshaling).
The `object` includes org and team membership if the user has any:
```json
{
"object": {
"id": "60a1b2c3d4...",
"username": "alice",
"role": 1,
"permissions": [],
"orgs": [{"orgId": "...", "role": "admin"}],
"teams": [{"teamId": "...", "role": "member"}],
"primaryOrgId": "..."
},
"type": "User",
"exp": 1700000000
}
```
See [Orgs & Teams](18-orgs-teams.md) for details on org/team-based access control.
**Auth Header:**
```
X-Auth-Token: eyJhbGciOiJIUzI1NiIs...
```
### Refresh Token
| Field | Value | Description |
|-------|-------|-------------|
| Algorithm | HS256 | HMAC-SHA256 |
| `object` | `{id: "userId"}` | Minimal user reference |
| `type` | `"RefreshTokenData"` | Token type identifier |
| `exp` | now + 10h | Expiration timestamp |
**Cookie:**
| Property | Value |
|----------|-------|
| Name | `tibi-refresh-token` |
| MaxAge | 36000 (10 hours) |
| Path | `/` |
| HttpOnly | `true` |
| SameSite | `Strict` |
| Secure | `false` |
### Token Validation
Token validation includes caching — subsequent calls within the same request reuse the validated result:
```go
// Cached in gin.Context:
// "validate-auth-token-object" → user data map
// "validate-auth-token-type" → token type string
// "validate-auth-token-err" → validation error
```
---
## Admin Tokens (Server-Level)
Admin tokens provide token-based access for agents, CI/CD pipelines, and other automated systems without requiring the JWT login flow.
### Configuration
```yaml
api:
adminTokens:
- token: "agent-token-xyz"
label: "ci-deploy" # Optional: human-readable label for audit logs
permissions:
- project # Shorthand: all project CRUD operations
- project.reload # Reload project config
- user # Shorthand: all user CRUD operations
- server.shutdown
- namespace.myproject # All collections in myproject
- token: "readonly-token"
label: "monitoring" # Appears as tokenLabel in audit entries
permissions:
- project.read
- user.read
- namespace.*.*.read # Read-only across all namespaces and collections
- token: "importer-token"
label: "data-import"
permissions:
- namespace.myproject.articles.create # Only create in myproject/articles
- namespace.myproject.articles.update # Only update in myproject/articles
```
### Available Permissions
| Permission | Description |
|-----------|-------------|
| `project` | Shorthand for all project operations (`project.*`) |
| `project.read` | GET projects |
| `project.create` | POST (create) projects |
| `project.update` | PUT (update) projects |
| `project.delete` | DELETE projects |
| `project.reload` | Reload project configuration |
| `user` | Shorthand for all user operations (`user.*`) |
| `user.read` | GET users |
| `user.create` | POST (create) users |
| `user.update` | PUT (update) users |
| `user.delete` | DELETE users |
| `server.shutdown` | Shutdown the server |
| `namespace` | Shorthand: all collections in all projects |
| `namespace.{project}` | Shorthand: all collections in a project |
| `namespace.{project}.{collection}` | Shorthand: all actions on a collection |
| `namespace.{project}.{collection}.read` | GET documents |
| `namespace.{project}.{collection}.create` | POST (create) documents |
| `namespace.{project}.{collection}.update` | PUT (update) documents |
| `namespace.{project}.{collection}.delete` | DELETE documents |
### Wildcards
Use `*` as a wildcard for any single segment in a permission string:
| Permission | Description |
|-----------|-------------|
| `namespace.*` | All collections in any project |
| `namespace.*.articles.read` | Read articles across all projects |
| `namespace.myproject.*.read` | Read all collections in myproject |
| `namespace.*.*.read` | Read-only across everything |
Shorthand works at any level — a permission with fewer segments implies all descendants:
- `namespace` = `namespace.*.*.*` (everything)
- `namespace.myproject` = `namespace.myproject.*.*` (all collections, all actions)
- `namespace.myproject.articles` = `namespace.myproject.articles.*` (all actions)
### Usage
Send the token via `X-Admin-Token` header or `Authorization: Bearer` header:
```
# Via X-Admin-Token header — project management
POST /api/v1/project
X-Admin-Token: agent-token-xyz
Content-Type: application/json
{"name": "my-project", "description": "A new project", "configFile": "/path/to/config.yml"}
```
```
# Via Authorization: Bearer header — collection access
GET /api/v1/_/myproject/articles
Authorization: Bearer readonly-token
```
```
# Create document in a collection
POST /api/v1/_/myproject/articles
X-Admin-Token: importer-token
Content-Type: application/json
{"title": "New Article", "body": "Content..."}
```
### Token Priority
For project/user endpoints:
1. **Admin Token** (`X-Admin-Token` / `Authorization: Bearer`): Checked first. If present and valid, access is granted immediately. If present but invalid, access is denied (no fallthrough).
2. **JWT Auth** (`X-Auth-Token`): Checked as fallback if no admin token is provided.
For collection endpoints (`/_/:project/:collection`):
1. **Admin Token**: If present, checked for `namespace.{project}.{collection}.{action}` permission. If present but lacking permission, access is denied (no fallthrough to JWT or public permissions).
2. **Collection Token Permissions**: Header/query-based tokens defined in collection config.
3. **Public / User / JWT**: Normal permission chain if no admin token is present.
### Project Admin Endpoints
For project admin endpoints (`reload`, `regenerate-search`, `backup`, `restore`), three auth methods are supported (checked in order):
1. Server-level admin tokens with the corresponding permission (e.g. `project.reload`, `project.regenerate-search`, `project.backup`)
2. Project-level tokens configured in the project's `config.yml` (`admin.tokens[].allowReload`, `admin.tokens[].allowRegenerateSearch`, `admin.tokens[].allowBackup`)
3. JWT login with admin role (`X-Auth-Token` header)
### Audit Trail
All admin token actions are logged in the audit log with additional context:
| Field | Description | Example |
|-------|-------------|---------|
| `authMethod` | Authentication method used | `admin-token`, `jwt` |
| `tokenLabel` | Label from token config (if set) | `ci-deploy` |
| `tokenPrefix` | First 8 characters of the token | `agent-to` |
| `ip` | Client IP address | `192.168.1.100` |
The full token is **never** logged for security reasons. Use descriptive `label` values to identify tokens in audit logs.
---
## Roles
| Role | Value | Description |
|------|-------|-------------|
| Admin | `0` | Full access to all system and collection endpoints |
| User | `1` | Access limited to assigned projects and permitted methods |
### Admin Role Behavior
- Can access all projects regardless of assignment
- Can manage all users (create, update, delete)
- Can manage all projects
- Bypasses all collection permission checks
- No projection restrictions
- No pre-filter application
- Only admins can change `role`, `permissions`, and `llmBudgets` on users
### User Role Behavior
- Can only see projects belonging to their orgs (via `project.orgId` ∈ user's org IDs)
- Cannot create/update/delete projects (OrgAdmins can modify `teams[]`)
- Can only GET and PUT their own user profile (OrgAdmins have broader scope)
- Cannot change `username`, `role`, `permissions`, or `llmBudgets`
- Subject to collection permission checks (via team permissions)
- See [Orgs & Teams](18-orgs-teams.md) for the full role hierarchy
---
## Permission System
Each collection defines a `permissions` map with named permission sets:
```yaml
permissions:
public:
methods:
get: true
filter:
published: true
user:
methods:
get: true
post: true
put:
allow: true
bulk: true # allow bulk PUT
delete:
allow: true
bulk: true # allow bulk DELETE
"token:apikey":
methods:
get: true
post: true
editor:
methods:
get: true
put: true # shorthand for {allow: true}
```
### PermissionSet Structure
```go
type PermissionSet struct {
Methods map[string]MethodPermission // Allowed methods (get, post, put, delete)
Filter dbcrud.M // Auto-applied filter for queries
ValidProjections []string // Allowed projection names
CompiledFilterSubs []*CompiledSubst // Pre-compiled $subst expressions
}
// MethodPermission represents a method permission.
// Can be unmarshalled from a boolean (true/false) or an object {allow: bool, bulk: bool}.
type MethodPermission struct {
Allow bool // Allow the method
Bulk bool // Allow bulk operations (POST array, PUT/DELETE without ID)
}
```
### Permission Check Priority
When a request comes in, permissions are checked in this order:
```
1. Token-based permissions (header/query parameter match)
│
├── Token name found, value matches → Use token's permission set
├── Token name found, value WRONG → ACCESS DENIED (no fallback!)
└── No token name found → Continue
│
2. Public permissions
│
├── Method allowed → ACCESS GRANTED (no auth required)
└── Method not allowed → Continue
│
3. Auth token validation (X-Auth-Token header)
│
├── No token / invalid → ACCESS DENIED
└── Valid token → Continue
│
4. Admin role check (role = 0)
│
├── Is admin → ACCESS GRANTED
└── Not admin → Continue
│
5. Team "user" permissions (role = 1, team-assigned to project)
│
├── User is member of a team assigned to this project
│ AND collection has "user" permission set → Use "user" set
└── Otherwise → Continue
│
6. Custom user permissions
│
├── Auth permissions contain "namespace:permName" → Use that permission set
└── No match → Continue
│
7. Custom team permissions
│
├── User's teams (assigned to project) have permissions matching a collection key
│ → Use that team permission set (union of all matching teams)
└── No match → ACCESS DENIED
```
See [Orgs & Teams — Permission System](18-orgs-teams.md#permission-system) for details on team-based permission resolution.
### Token-Based Permissions
Token permissions are matched against HTTP headers or query parameters:
```yaml
permissions:
"token:secretkey123": # Matches ?token=secretkey123 OR header token: secretkey123
methods:
get: true
"x-api-key:my-api-key": # Matches ?x-api-key=my-api-key OR header x-api-key: my-api-key
methods:
get: true
post: true
```
**Important:** The permission key format is `name:value`. The `name` part is checked against both query parameters and headers. If the `name` is found but the `value` doesn't match, access is **denied immediately** with no fallback to public/user permissions.
### Custom Permissions
Users can have custom permission strings in their `permissions` array:
```json
{
"username": "editor1",
"role": 1,
"permissions": ["myblog:editor", "myblog:moderator"]
}
```
These are matched against collection permission keys using the format `{namespace}:{permissionKey}`:
```yaml
# In project config (namespace: myblog)
permissions:
editor: # Matched by "myblog:editor"
methods:
get: true
put: true
moderator: # Matched by "myblog:moderator"
methods:
get: true
delete: true
```
---
## Pre-Filters
Permission sets can define filters that are automatically applied to all queries:
```yaml
permissions:
public:
methods:
get: true
filter:
published: true
deletedAt: null
user:
methods:
get: true
post: true
put: true
filter:
author:
$subst:
expr: "auth.id" # Only see own documents
```
### How Pre-Filters Work
1. The permission set's filter is deep-copied (to avoid mutation)
2. `$subst` expressions are resolved with the current request context
3. The filter is combined with any user-provided filter query via `$and`
4. Admin role (0) never has pre-filters applied
### Filter Combination
```
Final filter = $and: [
{ _id: <from URL if single document> },
{ <pre-filter from permissions> },
{ <user-provided filter query> }
]
```
---
## Projection Restrictions
Permission sets can restrict which projections are available:
```yaml
permissions:
public:
methods:
get: true
validProjections:
- summary # Public can only use "summary" projection
user:
methods:
get: true
validProjections:
- summary
- detailed # Users can also use "detailed"
```
**Behavior:**
- If `validProjections` is empty/not set → no restrictions (any projection or none)
- If `validProjections` is set and no `projection` parameter → first entry is used as default
- If `validProjections` is set and invalid projection requested → `400 Bad Request`
- Admin role (0) is never restricted
---
## AuthData Model
Extracted from the JWT token on each authenticated request:
```go
type AuthData struct {
ID string // User ObjectID (hex)
Username string // Username
Role int // 0=Admin, 1=User
Permissions []string // Custom permission strings
}
```
---
## User Model
```go
type User struct {
Model // ID, InsertTime, UpdateTime
Username *string // Required, alphanumeric, unique
Password *string // Required, min 3 chars (stored as MD5 hash)
Role *int // 0=Admin, 1=User (validated min=0, max=2)
Permissions []string // Custom permission strings
Meta map[string]interface{} // Arbitrary user metadata
LLMBudgets []LLMBudget // Per-provider LLM token budgets
}
```
### User Search
The `_search` parameter on `/api/v1/user` searches across:
- `username`
- `meta.company`
- `meta.prename`
- `meta.surname`
- `meta.email`
- `meta.phone`
---
## Server-Level Admin Authentication
Server-level admin endpoints (e.g. shutdown) are protected by admin tokens with the appropriate permission:
```bash
curl -X POST http://localhost:8080/api/v1/admin/shutdown \
-H "X-Admin-Token: your-admin-token"
```
Admin tokens are configured via `api.adminTokens` in `config.yml`. See [Configuration](02-configuration.md) for details.
## Personal Access Tokens (PAT)
Users can create and manage their own API tokens. A PAT uses the same
`X-Auth-Token` header as the JWT and is always checked live against the owner's
current rights. Tokens are a core feature and always enabled.
See [22-personal-access-tokens.md](22-personal-access-tokens.md) for format,
scope grammar, endpoints, examples and security properties.
src/content/docs/server/06-hooks.md
+1721
View File
File diff suppressed because it is too large. Load diff
src/content/docs/server/07-realtime.md
+190
View File
@@ -0,0 +1,190 @@
---
title: 'Realtime Channels (SSE Pub/Sub)'
---
# Realtime Channels (SSE Pub/Sub)
## Overview
Tibi-Server includes an in-memory Pub/Sub hub for real-time communication via **Server-Sent Events (SSE)**. Each project has its own isolated `Hub` instance. Channels are created on-demand when the first subscriber connects, and automatically garbage-collected when the last subscriber leaves.
## Architecture
```
Client A ─── SSE ──→ [Subscription] ──→ [Channel "chat"] ←── [Subscription] ←── SSE ──── Client B
│
[Ring Buffer]
│
Client C ─── SSE ──→ [Subscription] ──→ [Channel "chat"]
│
Hook/Job ────────→ hub.Send("chat", data)
```
### Components
| Component | Description |
|-----------|-------------|
| **Hub** | Per-project container for all channels. Thread-safe via `sync.RWMutex`. |
| **Channel** | Named pub/sub channel with a fixed-size ring buffer. Created on first subscribe. |
| **Subscription** | Per-client connection to a channel. Each gets independent read position. |
| **Message** | Wrapper for data with `CreatedAt` timestamp. |
## Data Structures
### Message
```go
type Message struct {
Data interface{}
CreatedAt time.Time
}
```
### SubscribeOptions
```go
type SubscribeOptions struct {
BufferSize int // Ring buffer capacity (default: 256, set by first subscriber)
OnFull OnFullStrategy // What to do when subscriber is slow
MessageTTL time.Duration // Skip messages older than this (0 = no TTL)
LastN int // Replay last N messages on connect (0 = none)
MaxAge time.Duration // Replay messages newer than this (0 = none)
}
```
### OnFullStrategy
| Strategy | Value | Behavior |
|----------|-------|----------|
| `DropOldest` | `0` (default) | Slow subscriber's read pointer jumps forward (may miss messages) |
| `DropNewest` | `1` | New message is silently discarded for this subscriber |
## Ring Buffer
The channel uses a fixed-size ring buffer for message storage:
```
Buffer size: 8
Write pointer: writeSeq (monotonically increasing)
Position = writeSeq % size
[0] [1] [2] [3] [4] [5] [6] [7]
↑ ↑
oldest writeSeq
```
- **First subscriber** determines the buffer size (subsequent subscribers use the existing size)
- Messages are deep-copied to each subscriber (independent mutation)
- `oldest = max(0, writeSeq - size)` — positions older than this have been overwritten
## Replay Logic
When a new subscriber connects with `LastN` or `MaxAge`:
1. `readSeq` starts at `writeSeq` (only new messages by default)
2. If `LastN > 0`: candidate = `writeSeq - LastN`, clamped to `oldest`
3. If `MaxAge > 0`: backward scan from `writeSeq` until a message older than `MaxAge` is found
4. The lower value of both is used, but never below `oldest`
This means subscribers immediately receive matching historical messages.
## Message TTL
If `MessageTTL` is set on a subscription, messages older than the TTL are silently skipped during `tryRead()`. This prevents slow subscribers from receiving stale data after reconnection.
## Usage from Hooks
### Subscribe (Blocking SSE Endpoint)
```js
// hooks/events/get.before
context.response.header("Content-Type", "text/event-stream");
context.response.header("Cache-Control", "no-cache");
context.response.header("Connection", "keep-alive");
context.response.writeHeader(200);
var sub = channel.subscribe("notifications", {
bufferSize: 100, // Ring buffer size
onFull: "drop-newest", // "drop-oldest" (default) or "drop-newest"
messageTTL: 60000, // 60 seconds
lastN: 10, // Replay last 10 messages
maxAge: 30000 // Only replay messages from last 30 seconds
});
var msg;
while ((msg = sub.receive()) !== null) {
context.response.writeSSE("message", context.json.stringify(msg));
}
sub.close();
throw { status: 200 };
```
### Send (From Any Hook or Job)
```js
// hooks/posts/post.after
context.channel.send("notifications", {
type: "new_post",
id: context.data.id,
title: context.data.title,
timestamp: new Date().toISOString()
});
```
## Receive Behavior
| Return Value | Meaning |
|-------------|---------|
| `(data, nil)` | Message received (deep-copied) |
| `null` | Client disconnected (request context cancelled) |
| Error | Subscription, channel, or hub was closed |
`receive()` blocks until:
- A new message arrives
- The client disconnects
- The subscription/channel/hub is closed
## Automatic Cleanup
- `channel.subscribe()` calls `DisableTimeout()` — the hook timeout is disabled
- A cleanup function is registered via `RegisterCleanup()` — the subscription is closed when the hook ends
- When the last subscriber leaves a channel, the channel is removed from the hub
## Client-Side SSE
```javascript
const eventSource = new EventSource('/api/v1/myproject/events');
eventSource.addEventListener('message', function(e) {
const data = JSON.parse(e.data);
console.log('Received:', data);
});
eventSource.onerror = function(e) {
console.log('Connection lost, reconnecting...');
};
```
## SSE Wire Format
Messages sent via `response.writeSSE()` follow the standard SSE format:
```
event: message
data: {"type":"new_post","id":"abc123"}
event: message
data: {"type":"update","id":"def456"}
```
Each message is terminated by `\n\n` and flushed immediately.
## Limitations
- **In-memory only** — messages are lost on server restart
- **Single-instance** — no cross-server channel synchronization
- **No persistence** — no message durability guarantees
- **Buffer overflow** — slow subscribers may lose messages (ring buffer wrap-around)
- **No authentication on channels** — access is controlled by the collection's permission system that the hook runs in
src/content/docs/server/08-file-upload-images.md
+292
View File
@@ -0,0 +1,292 @@
---
title: 'File Uploads & Image Processing'
---
# File Uploads & Image Processing
## File Upload Mechanism
Files are uploaded as **Base64 data URIs** embedded in JSON document fields of type `file` or `file[]`.
### Upload Format
```json
{
"avatar": "data:image/jpeg;base64,/9j/4AAQSkZJRg...",
"documents": [
"data:application/pdf;base64,JVBERi0xLjQ...",
"data:text/plain;base64,SGVsbG8gV29ybGQ="
]
}
```
### How It Works
1. **POST/PUT request** contains a `file` field with a Base64 data URI
2. The server detects the `data:` prefix in `handleFileInJSON()`
3. The Base64 content is decoded
4. A file is written to: `{uploadPath}/{documentId}/{fieldName}/{filename}`
5. The data URI in the document is replaced with the relative file path
6. The stored document contains only the path reference, not the binary data
### File Path Structure
```
{uploadPath}/
{documentId}/
{fieldName}/
{originalFilename}
```
- `uploadPath` is configured per collection (default: derived from project config)
- File permissions: `0755`
- Operations run under the project's configured UID (suexec)
### Retrieving Files
```
GET /api/v1/{namespace}/{collection}/{id}/{field}/{filename}
```
Example:
```
GET /api/v1/blog/posts/60a1b2c3d4e5f6/avatar/photo.jpg
```
### File Deletion
When a document is deleted, all associated files in `{uploadPath}/{documentId}/` are removed automatically via `os.RemoveAll`.
### Size Limits
**Request Body Size** (Middleware — checked before any processing):
- Server-level: `api.maxBodySize` (default: 100MB) — absolute hard cap
- Project-level: `maxBodySize` in project config — default for all collections
- Collection-level: `maxBodySize` in collection config — overrides project (can increase, but never above server)
- Returns **413** if exceeded
**Per-Field File Size** (checked during Base64 decode):
```yaml
fields:
- name: avatar
type: file
validator:
maxFileSize: "500KB" # Supports B, KB, MB, GB, TB
```
- Uses `io.LimitReader` for memory-efficient streaming check
- Returns **400** if exceeded (file is deleted from disk)
- Works for both `file` and `file[]` fields
- Enforced on POST and PUT
**Per-Field Image Dimensions** (checked from the decoded image header):
```yaml
fields:
- name: heroImage
type: file
validator:
accept: ["image/jpeg", "image/png", "image/webp"]
image:
minWidth: 1200
maxWidth: 3200
minHeight: 600
maxHeight: 2400
```
- Works for both `file` and `file[]` fields
- Applies to newly uploaded Data-URI files on POST and PUT
- Validates dimensions before the file is kept on disk
- Currently supported for dimension checks: PNG, JPEG, GIF, BMP, TIFF, WebP, and SVG
- `allowNonImages: true` lets a mixed-media field accept non-image uploads unchanged; image uploads on the same field are still checked against `minWidth`, `maxWidth`, `minHeight`, and `maxHeight`
- `allowUnknownDimensions: true` turns the validator into a best-effort check for `image/*`: uploads with unknown dimensions are accepted and min/max checks are skipped for that file
- HEIF and HEIC are currently detected as image MIME types, but are not yet supported by strict dimension checks; with `allowUnknownDimensions: true` they can still pass if the field accepts them
---
## Image Filters
Image filters allow on-the-fly image processing when serving files. Filters are configured per collection and applied via query parameter.
### Configuration
```yaml
collections:
- name: posts
imageFilter:
thumbnail:
- width: 200
height: 200
fit: true
quality: 80
outputType: webp
hero:
- width: 1200
height: 600
fill: true
anchor: center
quality: 85
```
### Usage
```
GET /api/v1/blog/posts/{id}/image/photo.jpg?filter=thumbnail
```
### Caching
Filtered images are cached on disk as:
```
{originalPath}.{filterName}.{extension}
```
Example: `photo.jpg` with filter `thumbnail` → `photo.thumbnail.webp`
Use `?no-cache=1` to force regeneration.
---
## FilterParams Reference
Each filter is an array of processing steps. All parameters are optional.
### Resize Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `width` | int | Target width in pixels |
| `height` | int | Target height in pixels |
| `fit` | bool | Resize to fit within dimensions (preserves aspect ratio, no crop) |
| `fill` | bool | Resize to fill dimensions (crops to fit, uses `anchor`) |
| `resampling` | string | Resampling algorithm (see below) |
| `anchor` | string | Crop anchor point for `fill` mode (see below) |
If neither `fit` nor `fill` is set, `imaging.Resize` is used (stretches to exact dimensions).
### Effect Parameters
| Parameter | Type | Range | Description |
|-----------|------|-------|-------------|
| `brightness` | float64 | -100 to 100 | Adjust brightness |
| `saturation` | float64 | -100 to 100 | Adjust saturation |
| `contrast` | float64 | -100 to 100 | Adjust contrast |
| `gamma` | float64 | > 0 | Gamma correction |
| `blur` | float64 | ≥ 0 | Gaussian blur sigma |
| `sharpen` | float64 | ≥ 0 | Unsharp mask sigma |
| `invert` | bool | — | Invert colors |
| `grayscale` | bool | — | Convert to grayscale |
### Output Parameters
| Parameter | Type | Description |
|-----------|------|-------------|
| `quality` | int | JPEG/WebP quality (1-100) |
| `lossless` | bool | WebP lossless mode |
| `outputType` | string | Output format: `jpeg`, `png`, `webp`, `gif`, `bmp`, `tiff` |
| `skipLargerDimension` | bool | Skip processing if source is smaller than target |
| `skipLargerFilesize` | bool | Keep original if processed file would be larger |
### Resampling Algorithms
| Value | Algorithm | Quality/Speed |
|-------|-----------|---------------|
| `nearestNeighbor` | Nearest Neighbor | Fastest, lowest quality |
| `box` | Box | Fast, low quality |
| `linear` | Bilinear | Good balance |
| `hermite` | Hermite | Good quality |
| `mitchellNetravali` | Mitchell-Netravali | High quality |
| `catmullRom` | Catmull-Rom | High quality |
| `bSpline` | B-Spline | High quality, smooth |
| `gaussian` | Gaussian | High quality |
| `lanczos` | Lanczos | Highest quality, slowest |
| `bartlett` | Bartlett | Good quality |
| `hann` | Hann | Good quality |
| `hamming` | Hamming | Good quality |
| `blackman` | Blackman | High quality |
| `welch` | Welch | Good quality |
| `cosine` | Cosine | Good quality |
### Anchor Positions
Used with `fill: true` to determine crop position:
| Value | Position |
|-------|----------|
| `center` | Center (default) |
| `topLeft` | Top-left |
| `top` | Top-center |
| `topRight` | Top-right |
| `left` | Center-left |
| `right` | Center-right |
| `bottomLeft` | Bottom-left |
| `bottom` | Bottom-center |
| `bottomRight` | Bottom-right |
---
## Supported Image Formats
### Input Formats
| Format | Library |
|--------|---------|
| JPEG | `imaging.Open` |
| PNG | `imaging.Open` |
| GIF | `imaging.Open` |
| BMP | `imaging.Open` |
| TIFF | `imaging.Open` |
| WebP | `golang.org/x/image/webp` decoder |
### Output Formats
| Format | Library |
|--------|---------|
| JPEG | `imaging` |
| PNG | `imaging` |
| WebP | `chai2010/webp` encoder |
Auto-detect: If `outputType` is not set, format is determined from the target file extension.
---
## Processing Pipeline
When multiple filter steps are defined, they are applied **sequentially**:
```yaml
imageFilter:
optimized:
- width: 1200 # Step 1: Resize
height: 800
fit: true
sharpen: 0.5
- quality: 80 # Step 2: Compress
outputType: webp
```
Per step:
1. Skip check (if `skipLargerDimension` and source is smaller)
2. Resize (`fit`, `fill`, or plain resize)
3. Apply effects (blur, brightness, contrast, gamma, saturation, sharpen, invert, grayscale)
4. Encode to output format with quality settings
5. Skip check (if `skipLargerFilesize` and result is larger)
If the final file would be larger than the source and `skipLargerFilesize` is set, the original file is copied to the target path instead.
---
## Image Processing in Hooks
The `image.filter()` function provides the same processing capabilities in hooks:
```js
context.image.filter("/path/to/source.jpg", "/path/to/output.webp", [
{
width: 800,
height: 600,
fit: true,
quality: 85,
outputType: "webp"
}
]);
```
See [06-hooks.md](06-hooks.md#image--image-processing) for full API reference.
src/content/docs/server/09-llm-integration.md
+260
View File
@@ -0,0 +1,260 @@
---
title: 'LLM Integration'
---
# LLM Integration
## Overview
Tibi-Server provides a built-in LLM proxy that supports OpenAI-compatible APIs with streaming, token budgets, and usage logging. Users can interact with LLMs through a unified API that handles provider resolution, model validation, and quota management.
## Configuration
### Server Config
```yaml
llm:
providers:
- name: openai
type: openai # "openai" or "openai-compatible"
apiKey: sk-... # API key
baseURL: https://api.openai.com/v1 # Optional, defaults to OpenAI
defaultModel: gpt-4 # Fallback model
models: # Allowed models (whitelist)
- gpt-4
- gpt-4-turbo
- gpt-3.5-turbo
maxTokensPerRequest: 4096 # Token limit per request
- name: local-llama
type: openai-compatible
apiKey: ""
baseURL: http://localhost:11434/v1
defaultModel: llama3
models:
- llama3
- codellama
```
### API Key via Environment Variable
```yaml
llm:
providers:
- name: openai
type: openai
apiKey: ${OPENAI_API_KEY} # Resolved from environment
```
Convention: `LLM_<UPPERCASE_PROVIDER_NAME>_APIKEY` (e.g., `LLM_OPENAI_APIKEY`).
## API Endpoints
### Chat Completion
```text
POST /api/v1/llm/chat
X-Auth-Token: <jwt-token>
Content-Type: application/json
```
**Request:**
```json
{
"provider": "openai",
"model": "gpt-4",
"messages": [
{ "role": "system", "content": "You are a helpful assistant." },
{ "role": "user", "content": "Hello!" }
],
"stream": false,
"temperature": 0.7,
"maxTokens": 1000
}
```
**Response (non-streaming):**
```json
{
"id": "chatcmpl-abc123",
"model": "gpt-4",
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "Hello! How can I help?" },
"finishReason": "stop"
}
],
"usage": {
"promptTokens": 25,
"completionTokens": 10,
"totalTokens": 35
}
}
```
### Streaming
```json
{
"provider": "openai",
"model": "gpt-4",
"messages": [{ "role": "user", "content": "Tell me a story." }],
"stream": true
}
```
**Response:** SSE stream with `Content-Type: text/event-stream`
```text
data: {"id":"chatcmpl-abc","model":"gpt-4","choices":[{"index":0,"delta":{"role":"assistant","content":"Once"},"finishReason":""}]}
data: {"id":"chatcmpl-abc","model":"gpt-4","choices":[{"index":0,"delta":{"content":" upon"},"finishReason":""}]}
data: {"id":"chatcmpl-abc","model":"gpt-4","choices":[{"index":0,"delta":{"content":" a"},"finishReason":""}]}
data: [DONE]
```
### List Providers
```text
GET /api/v1/llm/providers
X-Auth-Token: <jwt-token>
```
**Response:**
```json
[
{
"name": "openai",
"type": "openai",
"defaultModel": "gpt-4",
"models": ["gpt-4", "gpt-4-turbo", "gpt-3.5-turbo"]
},
{
"name": "local-llama",
"type": "openai-compatible",
"defaultModel": "llama3",
"models": ["llama3", "codellama"]
}
]
```
## Model Resolution
1. If `model` is provided in the request → validated against provider's `models` whitelist
2. If `model` is empty → provider's `defaultModel` is used
3. If the model is not in the whitelist → `400 Bad Request`
## Token Budget System
Administrators can assign per-provider token budgets to users:
```json
{
"username": "editor1",
"llmBudgets": [
{
"provider": "openai",
"tokenBudget": 100000,
"tokensUsed": 45230,
"periodStart": "2024-01-01T00:00:00Z",
"periodEnd": "2024-02-01T00:00:00Z"
}
]
}
```
### LLMBudget Structure
| Field | Type | Description |
| ------------- | ------ | ---------------------------------- |
| `provider` | string | Provider name |
| `tokenBudget` | int | Maximum tokens allowed in period |
| `tokensUsed` | int | Tokens consumed so far |
| `periodStart` | time | Budget period start |
| `periodEnd` | time | Budget period end |
### Budget Check Flow
1. Before each LLM request, **user budgets** are read from the database
2. `CheckBudget()` verifies:
- Budget period has not expired (`PeriodEnd` > now)
- Tokens used < token budget
3. If budget exceeded → `429 Too Many Requests`
4. If the user has a `primaryOrgId`, the **org's budget** is checked as well (soft check — only fails if the org has a budget for this provider and it's exhausted)
5. After completion, `RecordUsage()` increments `tokensUsed` on both the user document and the org document (if applicable)
### Org-Level Budget Cascade
Orgs can have their own `llmBudgets` (see [Orgs & Teams](18-orgs-teams.md#llm-budget-cascade)). When a user with a `primaryOrgId` makes an LLM request:
- **User budget** is checked first (hard check — must exist and have quota)
- **Org budget** is checked second via `user.primaryOrgId` (soft check — only enforced if configured)
- Usage is deducted from both budgets
- The `llm_usage` log entry includes both `userId` and `orgId`
This allows multiple users to share an org-wide token pool while each maintaining individual quotas.
### Token Estimation
When the upstream provider does not return usage information, tokens are estimated:
```go
// ~4 characters per token
func EstimateTokens(text string) int {
return len(text) / 4
}
// +4 tokens per message for framing overhead
func EstimateUsage(messages []ChatMessage, completionContent string) UsageInfo
```
## Usage Logging
Every LLM request is logged to the `llm_usage` collection in the system database:
```json
{
"id": "...",
"userId": "60a1b2c3d4...",
"orgId": "507f1f77bcf86cd799439011",
"provider": "openai",
"model": "gpt-4",
"promptTokens": 25,
"completionTokens": 10,
"totalTokens": 35,
"timestamp": "2024-01-15T10:30:00Z",
"insertTime": "2024-01-15T10:30:00Z"
}
```
The `orgId` field is populated from the user's `primaryOrgId` when present, linking usage to the billed org.
## Authentication
All LLM endpoints require a valid JWT token (`X-Auth-Token` header). Both admin and regular users can use LLM endpoints, but only admins can modify user `llmBudgets`.
## Provider Implementation
Tibi-Server uses the **OpenAI API format** as its universal protocol. All providers (including `openai-compatible`) implement the same interface:
```go
type Provider interface {
Name() string
ChatCompletion(ctx context.Context, req ChatRequest) (*ChatResponse, error)
ChatCompletionStream(ctx context.Context, req ChatRequest) (*ChatStream, error)
}
```
This means any OpenAI-compatible API (Ollama, vLLM, Azure OpenAI, LiteLLM, etc.) can be used by setting `type: openai-compatible` with the appropriate `baseURL`.
## Streaming Implementation Details
- Streaming uses `bufio.Scanner` for SSE parsing
- The `stream_options.include_usage = true` flag is set for OpenAI to get usage info in the final chunk
- `ChatStream.Read()` returns `io.EOF` when `data: [DONE]` is received
- Usage information from the final chunk is captured via `GetUsage()`
src/content/docs/server/10-audit.md
+277
View File
@@ -0,0 +1,277 @@
---
title: 'Audit Log'
---
# Audit Log
## Overview
Tibi-Server provides an audit logging system that records data changes (create, update, delete) with full snapshots. Audit logs are stored in the system database and queryable via API with role-based visibility.
## Configuration
### Server-Level Configuration
```yaml
audit:
enabled: true
defaultTTL: "720h" # Go duration (30 days). "0" = keep forever.
```
### Project-Level Configuration
```yaml
# in project config.yml (opt-in, default false for gradual migration)
audit:
enabled: true
```
> **Migration:** Audit is only active when **both** `server.audit.enabled` **and** `project.audit.enabled` are `true`. Existing projects without `audit.enabled` remain dark even when the server flag is enabled - add `audit.enabled: true` per project you want to migrate. Project level only supports `enabled` (no `actions`).
### Collection-Level Configuration
```yaml
collections:
- name: posts
audit:
enabled: true
actions:
- create
- update
- delete
```
**Default audited actions:** `create`, `update`, `delete`
**Available collection-level actions:** `create`, `update`, `delete`, `get`
**Available action-level actions (for `actions/*.yml`):** `post`, `get` (default `[post, get]`, `enabled` defaults to `true` if omitted)
> **Note:** `db.deleteMany` and `db.updateMany` are audited as `delete` and `update` respectively — all covered by the default actions.
>
> **Note:** Bulk API operations (`POST /collection` with array body, `PUT /collection?filter=...` and `DELETE /collection?filter=...`) are audited as `bulkCreate`, `bulkUpdate` and `bulkDelete` with the filter/count logged.
>
> **Note:** `login`, `reload`, and `shutdown` are system-level actions that are always logged when `audit.enabled` is `true`. They cannot be controlled via collection-level `audit.actions`.
### ShouldAudit Logic
An action is audited if:
1. Server-level `audit.enabled` is `true`
2. **Project-level `audit.enabled` is `true`** (opt-in, `nil`/`false` = off - `internal/models/auditlog.go:IsProjectAuditEnabled`)
3. Collection/Action-level `audit.enabled` is `true` (or not explicitly `false`)
3. The action is in the collection's `actions` list (or defaults are used)
## AuditLog Model
```json
{
"id": "...",
"insertTime": "2024-01-15T10:30:00Z",
"userId": "60a1b2c3d4...",
"username": "admin",
"userRole": 0,
"projectNamespace": "myblog",
"projectName": "My Blog",
"collection": "posts",
"action": "update",
"documentId": "60a1b2c3d4e5f6...",
"source": {
"type": "controller",
"collection": "",
"method": "PUT",
"step": "",
"file": "",
"line": 0
},
"snapshot": {
"title": "Old Title",
"body": "Old content..."
},
"changes": {
"title": "New Title"
},
"filter": null,
"count": null,
"ip": "192.168.1.1",
"authMethod": "jwt",
"tokenLabel": "",
"tokenPrefix": "",
"channel": "mcp"
}
```
### Authentication Context
Each audit entry includes authentication metadata:
| Field | Description | Values |
|-------|-------------|--------|
| `authMethod` | How the request was authenticated | `jwt`, `admin-token`, or empty |
| `tokenLabel` | Human-readable label from admin token config | e.g. `ci-deploy`, `monitoring` |
| `tokenPrefix` | First 8 characters of the admin token | e.g. `agent-to` |
| `userId` | User ID (only for JWT auth) | MongoDB ObjectId |
| `username` | Username (only for JWT auth) | e.g. `admin` |
| `ip` | Client IP address | e.g. `192.168.1.1` |
For JWT-authenticated requests, `userId`/`username`/`userRole` are populated and token fields are empty.
For admin token requests, `authMethod`/`tokenLabel`/`tokenPrefix` are populated and user fields are empty.
For PAT-authenticated requests (incl. MCP), `authMethod` is `access-token` with the token's name/prefix.
### Channel (REST vs. MCP)
| Field | Description | Values |
|-------|-------------|--------|
| `channel` | Through which interface the action came in | `mcp` for MCP tool calls, empty for REST (and older entries) |
Filter MCP-originated entries with `?filter={"channel":"mcp"}`. The concrete
tool follows from `collection` + `action` (`documentId` set = single, `filter`
set = bulk). See [MCP-Server](23-mcp.md).
### Source Types
| Type | Description |
| ------------ | ------------------------------------------ |
| `controller` | Direct API request (CRUD endpoint) |
| `hook` | Database operation from a hook script |
| `system` | Internal system operation |
| `job` | Database operation from a scheduled job |
For hook sources, additional fields are populated:
- `collection` — the hook's parent collection
- `method` — the HTTP method that triggered the hook
- `step` — hook step (e.g. `read`, `return`, `bind`, `validate`, `create`, `update`, `delete`)
- `file` — JavaScript source file
- `line` — line number in the JS file (from goja callstack)
## API
### Query Audit Logs
```text
GET /api/v1/audit
X-Auth-Token: <jwt-token>
```
**Query Parameters:**
| Parameter | Type | Default | Description |
| --------- | ----------- | -------------- | -------------------------------------------------------------------- |
| `filter` | JSON string | — | MongoDB-style filter (validated against AuditLog fields) |
| `limit` | int | 50 | Max results (capped at `maxLimit`, default max: 10,000) |
| `offset` | int | 0 | Pagination offset |
| `sort` | string | `-insertTime` | Sort field(s), prefix `-` for descending |
| `count` | string | — | `"1"` or `"true"` to include total count in `X-Results-Count` header |
**Configurable Limits:**
The audit endpoint enforces query result limits:
- **Default limit:** 50 (configurable via `audit.defaultLimit` in server config)
- **Maximum limit:** 10,000 (configurable via `audit.maxLimit` in server config)
- If `defaultLimit` exceeds `maxLimit`, it is automatically capped to `maxLimit`.
- See [Configuration](02-configuration.md) for YAML and environment variable options.
**Example:**
```bash
curl -H "X-Auth-Token: $TOKEN" \
'http://localhost:8080/api/v1/audit?filter={"collection":"posts","action":"update"}&limit=20&sort=-insertTime'
```
### Role-Based Visibility
**Admin (role=0):**
- Sees all audit entries across all projects
**User (role=1):**
- Only sees entries for projects and collections where they have GET permission
- System-sourced entries (`source.type: "system"`) are hidden
- Snapshot data is filtered through the collection's projection rules (same include/exclude logic as MongoDB projections)
### Sensitive Fields & audit.return Hooks
Audit snapshots store the **full document** at the time of the operation. Fields that are normally stripped by `get.return` hooks (e.g. passwords, API keys) are still present in audit snapshots.
To prevent sensitive data from being exposed through the audit endpoint, use the `audit.return` hook on the collection:
```yaml
hooks:
audit:
return:
type: javascript
file: hooks/users/audit_return.js
```
```js
// hooks/users/audit_return.js
;(function () {
var results = context.results
if (results && results.length > 0) {
if (results[0].snapshot) {
delete results[0].snapshot.password
delete results[0].snapshot.apiKey
}
if (results[0].changes) {
delete results[0].changes.password
delete results[0].changes.apiKey
}
}
return { results: results }
})()
```
Key properties:
- Runs for **all roles** including admin
- Affects existing snapshots retroactively (filtering happens at read-time)
- Each entry is processed individually (`context.one = true`, `context.results` has exactly one entry)
- Returning `{ results: [] }` suppresses the entry from the response
- Throwing an exception aborts the entire audit request
## Database Indexes
Four indexes are automatically created on the audit collection:
1. **TTL Index:** On `insertTime` — auto-deletes entries after configured TTL
2. **Project + Time:** `{projectNamespace, insertTime}`
3. **User + Time:** `{userId, insertTime}`
4. **Document:** `{collection, documentId, insertTime}`
## Write Behavior
- Audit log writes are **asynchronous by default** (fire-and-forget goroutine with own database session)
- Synchronous mode available via `AuditWriteSync = true` (used in tests)
- Each write uses its own MongoDB session to avoid blocking the main request
## What Gets Logged
### Controller Actions
| Action | Snapshot | Changes | Filter |
| -------- | ------------------------ | ---------------- | ------ |
| `create` | — | Created document | — |
| `update` | Document before update | Update data | — |
| `delete` | Document before deletion | — | — |
### Hook Database Actions
| Action | Snapshot | Changes | Filter |
| --------------------- | ------------------------ | ---------------- | ------------------- |
| `db.create` | — | Created document | — |
| `db.update` | Document before update | Update data | — |
| `db.delete` | Document before deletion | — | — |
| `db.deleteMany` | — | — | Filter used + count |
| `db.updateMany` | — | Update data | Filter used + count |
### Bulk API Actions
| Action | Snapshot | Changes | Filter |
| ------------ | -------- | ----------- | ------------------- |
| `bulkCreate` | — | — | Count + IDs |
| `bulkUpdate` | — | Update data | Filter used + count |
| `bulkDelete` | — | — | Filter used + count |
> `db.deleteMany` and `db.updateMany` are audited under the actions `delete` and `update` respectively. Bulk API actions (`bulkCreate`, `bulkUpdate`, `bulkDelete`) are also audited as `create`, `update` and `delete`.
src/content/docs/server/11-jobs.md
+154
View File
@@ -0,0 +1,154 @@
---
title: 'Jobs (Scheduled Tasks)'
---
# Jobs (Scheduled Tasks)
## Overview
Jobs are scheduled JavaScript programs that run on a cron schedule. They use the same goja runtime as hooks but execute independently of HTTP requests. Jobs are defined per project in the `config.yml`.
## Configuration
```yaml
jobs:
- cron: "0 */6 * * *" # Every 6 hours
type: javascript
file: jobs/cleanup.js
timeout: 30000 # Milliseconds (optional)
meta: # Arbitrary metadata (optional)
maxAge: 86400
- cron: "0 8 * * *" # Every day at 8:00
type: javascript
file: jobs/daily_report.js
timeout: 60000
```
### JobConfig Fields
| Field | Type | Required | Description |
| --------- | ------ | -------- | ------------------------------------------------------ |
| `cron` | string | yes | Cron expression (standard 5-field format) |
| `type` | string | yes | Job type (currently only `"javascript"`) |
| `file` | string | yes | Path to JavaScript file (relative to config directory) |
| `timeout` | int | no | Execution timeout in **milliseconds** |
| `meta` | object | no | Arbitrary metadata accessible via `config.project()` |
### Cron Expression Format
```text
┌───────────── minute (0-59)
│ ┌───────────── hour (0-23)
│ │ ┌───────────── day of month (1-31)
│ │ │ ┌───────────── month (1-12)
│ │ │ │ ┌───────────── day of week (0-6, 0=Sunday)
│ │ │ │ │
* * * * *
```
Examples:
- `*/5 * * * *` — Every 5 minutes
- `0 * * * *` — Every hour
- `0 0 * * *` — Daily at midnight
- `0 8 * * 1-5` — Weekdays at 8:00
- `0 0 1 * *` — First of each month
## Available Context
Jobs have access to most hook context packages, with the following **exceptions**:
| Package | Available in Jobs? | Reason |
| ------------------- | ------------------ | ------------------------ |
| `db.*` | Yes | |
| `http.*` | Yes | |
| `fs.*` | Yes | |
| `smtp.*` | Yes | |
| `jwt.*` | Yes | |
| `bcrypt.*` | Yes | |
| `crypto.*` | Yes | |
| `base64.*` | Yes | |
| `json.*` | Yes | |
| `xml.*` | Yes | |
| `tpl.*` | Yes | |
| `image.*` | Yes | |
| `pdf.*` | Yes | |
| `exec.*` | Yes | |
| `config.*` | Yes | |
| `debug.*` | Yes | |
| `charset.*` | Yes | |
| `channel.send` | Yes | |
| `cookie.*` | **No** | No HTTP request/response |
| `user.*` | **No** | No HTTP request |
| `response.*` | **No** | No HTTP response |
| `request` | **No** | No HTTP request |
| `channel.subscribe` | **No** | No HTTP response for SSE |
## Job Examples
### Cleanup Old Documents
```js
// jobs/cleanup.js
var threshold = new Date();
threshold.setDate(threshold.getDate() - 30);
var result = db.deleteMany("temp_data", {
filter: {
createdAt: { "$lt": { "$date": threshold.toISOString() } }
}
});
context.debug.dump("Cleaned up " + result.removed + " documents");
```
### Daily Report Email
```js
// jobs/daily_report.js
var yesterday = new Date();
yesterday.setDate(yesterday.getDate() - 1);
var newPosts = db.count("posts", {
filter: {
insertTime: { "$gte": { "$date": yesterday.toISOString() } }
}
});
var newUsers = db.count("_users", {
filter: {
insertTime: { "$gte": { "$date": yesterday.toISOString() } }
}
});
var html = tpl.execute(
context.fs.readFile(context.fs.configDir() + "/templates/daily_report.html"),
{ newPosts: newPosts, newUsers: newUsers, date: yesterday.toISOString() }
);
context.smtp.sendMail({
to: "admin@example.com",
from: "reports@example.com",
subject: "Daily Report - " + yesterday.toISOString().split("T")[0],
html: html
});
```
### Notify via Realtime Channel
```js
// jobs/notify.js
context.channel.send("system", {
type: "scheduled_maintenance",
message: "Scheduled maintenance in 30 minutes",
timestamp: new Date().toISOString()
});
```
## Execution
- Jobs run in a `JOB_CONTEXT` (FunctionContextType = 2)
- Each job execution creates a fresh goja runtime
- Jobs execute under the project's configured UID (suexec)
- Job errors are logged server-side and reported to Sentry
- Audit log entries from job database operations have `source.type: "job"`
src/content/docs/server/12-deployment.md
+152
View File
@@ -0,0 +1,152 @@
---
title: 'Deployment'
---
# Deployment
## System Requirements
| Component | Requirement |
|-----------|-------------|
| **Go** | 1.23+ (toolchain 1.24) |
| **MongoDB** | 4.4+ (with replica set for transactions) |
| **OS** | Linux (required for suexec UID isolation) |
| **wkhtmltopdf** | Required for PDF generation (optional) |
## Docker Deployment
### Production Dockerfile
```dockerfile
# Multi-stage build
FROM golang:1.24 AS builder
WORKDIR /app
COPY . .
RUN go build -o server ./cmd/server
FROM debian:bookworm-slim
RUN apt-get update && apt-get install -y ca-certificates wkhtmltopdf && rm -rf /var/lib/apt/lists/*
COPY --from=builder /app/server /usr/local/bin/server
CMD ["server"]
```
### Docker Compose
```yaml
version: '3.8'
services:
tibi:
build: .
ports:
- "8080:8080"
volumes:
- ./config.yml:/app/config.yml
- ./projects:/app/projects
environment:
- API_MONGOURI=mongodb://mongo:27017
- API_JWTSECRET=your-jwt-secret-here
depends_on:
- mongo
mongo:
image: mongo:7
command: ["--replSet", "rs0"]
volumes:
- mongo-data:/data/db
ports:
- "27017:27017"
volumes:
mongo-data:
```
### Development with Air (Hot Reload)
A `Dockerfile.air` is provided for development with [Air](https://github.com/cosmtrek/air) hot-reloading:
```bash
docker compose -f docker-compose-local.yml up
```
## Configuration File
See [02-configuration.md](02-configuration.md) for the complete configuration reference.
Minimal `config.yml`:
```yaml
api:
port: 8080
mongoUri: mongodb://localhost:27017
jwtSecret: change-me-to-a-long-random-string
dbPrefix: tibi
projectsDir: ./projects
```
## UID/GID Isolation (suexec)
When running as **root**, tibi-server uses thread-level UID/GID isolation for multi-tenant security:
### How It Works
1. The main goroutine locks its OS thread at startup (`runtime.LockOSThread()`)
2. For each unique UID/GID pair, a dedicated goroutine is spawned that:
- Locks its OS thread
- Changes thread credentials via `SYS_SETGID` / `SYS_SETUID` syscalls
- Waits for jobs in an event loop
3. File operations and `exec.command` calls are dispatched to the appropriate UID thread
4. Thread pool scales up to `runtime.NumCPU()` per UID/GID pair
### Thread Pool Behavior
- New threads are spawned on 1-second send timeout (pool is busy)
- Maximum threads per UID/GID = `runtime.NumCPU()`
- Threads persist for the lifetime of the server
### File-Based UID Detection
`RunInFilesUID(filename)` determines the UID/GID from `os.Stat()` of the given file. This is used for project data directories — the filesystem owner determines the execution context.
### When Not Root
When running as a non-root user, suexec is not available. All file operations run under the server process's UID.
## Graceful Shutdown
The server supports graceful shutdown via:
1. **OS Signal:** `SIGINT` or `SIGTERM`
2. **Admin API:** `POST /api/v1/admin/shutdown` with admin token (requires `server.shutdown` permission)
Shutdown process:
1. HTTP server stops accepting new connections
2. In-flight requests are allowed to complete (with timeout)
3. All realtime hub instances are closed (wakes blocking subscriptions)
4. MongoDB connections are closed
## Admin Reload
```
POST /api/v1/_/{project}/_/admin/reload
X-Admin-Token: your-admin-token
```
Reloads a specific project's configuration from disk without restarting the server. This:
- Re-reads all `config.yml` files from `projectsDir`
- Recompiles all hooks and jobs
- Rebuilds collection indexes
- Does **not** drop existing realtime connections (they continue on the old config until they reconnect)
## Health Check
```
GET /api/v1/version
```
Returns server version information. No authentication required. Suitable for load balancer health checks.
## Monitoring
- **Prometheus Metrics:** `GET /metrics` — see [13-openapi-metrics.md](13-openapi-metrics.md)
- **Sentry Integration:** Configure `api.sentry` in `config.yml` for error tracking
- **Audit Logs:** Enable `audit.enabled: true` for data change tracking
src/content/docs/server/13-openapi-metrics.md
+151
View File
@@ -0,0 +1,151 @@
---
title: 'OpenAPI & Metrics'
---
# OpenAPI & Metrics
## OpenAPI Specification
Tibi-Server auto-generates an OpenAPI 3.0.0 specification for each project based on its collection configuration.
### Endpoint
```
GET /api/v1/{namespace}/openapi
```
No authentication required.
### Query Parameters
| Parameter | Default | Description |
|-----------|---------|-------------|
| `lang` | `en` | Language for i18n fields (resolves `{en: "...", de: "..."}` maps) |
### Generated Content
The specification includes:
1. **Info:** Project name/namespace as title, version `"0.0.0"`
2. **Paths:** Two paths per collection:
- `/{collection}` — List endpoint (GET)
- `/{collection}/{id}` — Single document endpoint (GET)
3. **Schemas:** Generated from collection field definitions (recursive for nested objects)
4. **Security Schemes:** API key schemes derived from token-based permissions
### Collection Metadata for OpenAPI
Collections can customize their OpenAPI representation via `meta.openapi`:
```yaml
collections:
- name: posts
meta:
openapi:
disabled: false # Exclude from spec entirely
get:
disabled: false # Exclude list endpoint
summary: "List posts"
description: "Returns all published posts"
getOne:
disabled: false # Exclude single endpoint
summary: "Get post"
description: "Returns a single post by ID"
```
### Type Mapping
| Collection Field Type | OpenAPI Type |
|----------------------|--------------|
| `string` | `string` |
| `int` | `integer` |
| `float` | `number` |
| `bool` | `boolean` |
| `date` | `string` (format: date-time) |
| `object` | `object` (with nested properties) |
| `file` | `object` with `{ src: string }` |
| `string[]` | `array` of `string` |
| (other arrays) | `array` of corresponding type |
### i18n Support
Field names, descriptions, and summaries can be i18n maps:
```yaml
fields:
- name:
en: Title
de: Titel
key: title
type: string
```
The `lang` query parameter determines which language string is used in the generated spec.
---
## Prometheus Metrics
### Endpoint
```
GET /metrics
```
Registered on the root router (not under `/api/v1/`). Standard Prometheus text format. No authentication required.
### Exposed Metrics
#### `tibi_request_duration_seconds` (Summary)
Tracks response time for all HTTP handlers.
| Label | Description | Example |
|-------|-------------|---------|
| `method` | HTTP method | `GET`, `POST`, `PUT`, `DELETE` |
| `status` | HTTP status code | `200`, `404`, `500` |
| `base` | System collection name (non-project routes) | `user`, `project`, `login` |
| `ns` | Project namespace (project routes) | `myblog`, `myshop` |
| `col` | Collection name | `posts`, `products` |
| `id` | Document ID indicator | `1` (with ID) or `""` (list) |
**Quantiles:**
| Quantile | Error Margin |
|----------|-------------|
| p50 | ±0.05 |
| p75 | ±0.025 |
| p90 | ±0.01 |
| p99 | ±0.001 |
### Example Output
```
# HELP tibi_request_duration_seconds Histogram of response time for handler in seconds
# TYPE tibi_request_duration_seconds summary
tibi_request_duration_seconds{method="GET",status="200",base="",ns="myblog",col="posts",id="",quantile="0.5"} 0.002
tibi_request_duration_seconds{method="GET",status="200",base="",ns="myblog",col="posts",id="",quantile="0.9"} 0.005
tibi_request_duration_seconds{method="GET",status="200",base="",ns="myblog",col="posts",id="",quantile="0.99"} 0.012
tibi_request_duration_seconds_sum{method="GET",status="200",base="",ns="myblog",col="posts",id=""} 1.234
tibi_request_duration_seconds_count{method="GET",status="200",base="",ns="myblog",col="posts",id=""} 500
```
### Grafana Dashboard Example
Query for average request duration per endpoint:
```promql
rate(tibi_request_duration_seconds_sum[5m]) / rate(tibi_request_duration_seconds_count[5m])
```
Query for p99 latency per collection:
```promql
tibi_request_duration_seconds{quantile="0.99", ns="myblog"}
```
### Integration
The metrics middleware (`measureResponseDuration`) is registered globally on all routes. It measures the duration of each request and records it with the appropriate labels.
Standard Go runtime metrics are also available via the default Prometheus registry (`go_*` metrics).
src/content/docs/server/14-security.md
+636
View File
@@ -0,0 +1,636 @@
---
title: 'Security Audit — Tibi-Server'
---
# Security Audit — Tibi-Server
> **Audit-Datum:** 27. Februar 2026
> **Scope:** Vollständiges Security-Audit des tibi-server Quellcodes
> **Methode:** Statische Code-Analyse aller sicherheitsrelevanten Komponenten
---
## Zusammenfassung
| Schweregrad | Anzahl | Status |
|-------------|--------|--------|
| **KRITISCH** | 3 | Offen |
| **HOCH** | 6 | Offen |
| **MITTEL** | 7 | Offen |
| **NIEDRIG** | 5 | Offen / Teilweise gelöst |
| **Gelöst** | 1 | ✅ |
---
## Security Architecture
### Authentication Layer
| Mechanism | Scope | Header/Cookie |
|-----------|-------|---------------|
| JWT (HMAC-SHA256) | User authentication | `X-Auth-Token` header |
| Refresh Token | Token renewal | `tibi-refresh-token` cookie (HttpOnly, SameSite=Strict) |
| Admin Tokens | Server-level CRUD, management & namespace collection access | `X-Admin-Token` header or `Authorization: Bearer` |
| Token Permissions | API key access | Configurable header/query parameter |
### Authorization Layers
1. **Role-Based:** Admin (role=0) bypasses all checks
2. **Project-Based:** Users assigned to projects via `project.users[]`
3. **Collection-Based:** Per-collection permission sets with method restrictions
4. **Pre-Filters:** Automatic query filters based on permission context
5. **Projection Restrictions:** Limit visible fields per permission level
### Data Isolation
- **Database:** Each project gets its own MongoDB database (`{prefix}_{namespace}`)
- **Filesystem:** suexec UID/GID isolation (Linux, requires root)
- **Realtime:** Each project has its own Hub instance
### Input Validation
- Collection field validation (type, required, allowZero, in, eval)
- MongoDB filter operator whitelist (dangerous operators blocked)
- File path sanitization (`CleanFilePath`)
- User input validation (username alphanumeric, password min 3 chars)
### Bulk Operation Security
Bulk operations (`POST` with JSON array body, `PUT`/`DELETE` without `:id`) require **two** permission checks:
1. The standard method permission must be granted (`post`, `put` or `delete` with `allow: true`)
2. The `bulk` flag must be set to `true` in the object form of the permission
Method permissions support two formats:
- **Boolean shorthand:** `post: true` — allows single-document POST only (no bulk)
- **Object form:** `post: {allow: true, bulk: true}` — allows both single and bulk POST
This applies equally to PUT and DELETE. Without explicit `bulk: true`, bulk endpoints return **403 Forbidden**. This ensures that existing projects are not automatically exposed to bulk operations when upgrading.
---
## Findings
### CRITICAL
#### ~~C1. MD5 Password Hashing~~ ✅ BEHOBEN (2025-02-27)
**Location:** `internal/models/login.go`, `internal/models/user.go`
**CVSS-Schätzung:** 9.1 (Kritisch) → **behoben**
**Description:** User passwords wurden zuvor mit MD5 gehasht, was kryptographisch gebrochen ist.
**Fix:** Passwort-Hashing wurde auf bcrypt (`golang.org/x/crypto/bcrypt`, DefaultCost) umgestellt mit abwärtskompatibler MD5-Migration:
- `user.go:ValidateData()` — neue Passwörter werden mit bcrypt gehasht
- `login.go:LoginCheck()` — Login prüft per `CheckPassword()` sowohl bcrypt als auch legacy-MD5
- Transparente Migration: bei erfolgreichem Login mit MD5-Hash wird automatisch auf bcrypt re-gehasht und in der DB aktualisiert
- Neue Hilfsfunktionen: `IsBcryptHash()`, `CheckPassword()` (exportiert für Tests)
- Unit-Tests: `TestIsBcryptHash`, `TestCheckPassword` + alle bestehenden Tests grün
---
#### C2. No Rate Limiting on Login Endpoint — ✅ BEHOBEN
**Location:** `internal/ratelimit/ratelimit.go`, `internal/controller/ratelimit_middleware.go`, `internal/controller/setup.go`
**CVSS-Schätzung:** 8.8 (Hoch–Kritisch)
**Description:** The `POST /api/v1/login` endpoint has no rate limiting or account lockout mechanism. Combined with MD5 password hashing and a minimum password length of only 3 characters, this makes the system highly vulnerable to brute-force attacks.
**Impact:** An attacker can attempt unlimited login attempts without any delay, lockout, or CAPTCHA. With a 3-character minimum password, the search space is trivially small.
**Lösung implementiert:**
- In-Memory Rate Limiter mit exponentiellem Backoff (1s → 2s → 4s → ... → 60s Cap)
- Duales Tracking: per IP und per Username (getrennt)
- Automatischer Reset nach 15 Minuten Inaktivität
- HTTP 429 + `Retry-After` Header bei Blockierung
- Konfigurierbar via `config.yml` (`rateLimit`-Sektion) und Environment-Variablen
- Standardmäßig aktiviert, kann via `rateLimit.enabled: false` deaktiviert werden
- Hintergrund-Cleanup-Goroutine für abgelaufene Einträge
---
#### C3. Admin Token Comparison Vulnerable to Timing Attacks — ✅ BEHOBEN
**Location:** `internal/config/admin_token.go`, `internal/controller/admin_reload.go`
**CVSS-Schätzung:** 7.5 (Hoch) → **behoben**
**Description:** Admin tokens wurden mit Go's `==` Operator verglichen, was nicht constant-time ist und Timing-Angriffe ermöglicht.
**Lösung implementiert:**
- Alle 3 Token-Vergleichsstellen auf `crypto/subtle.ConstantTimeCompare()` umgestellt:
- `config/admin_token.go:CheckAdminToken()` — Server-Admin-Token-Prüfung
- `config/admin_token.go:GetAdminTokenInfo()` — Admin-Token-Metadaten-Abfrage
- `controller/admin_reload.go:ProjectReload()` — Projekt-Admin-Token-Prüfung
- Constant-Time-Vergleich verhindert Rückschlüsse auf Token-Inhalte durch Zeitmessung
---
### HIGH
#### H1. Secrets in Plain Text Configuration
**Location:** `config.yml`, `internal/config/server.go`
**CVSS-Schätzung:** 7.5
**Description:** JWT secrets, SMTP passwords, and LLM API keys are stored in plain text in the configuration file. The shipped `config.yml` contains a hardcoded JWT secret:
```yaml
api:
jwtSecret: 237uiejakymxeou3zhakN # Hardcoded production secret in repo!
```
**Impact:** Anyone with read access to the repository or config file can forge JWT tokens, access SMTP credentials, and use LLM API keys.
**Mitigation:** Environment variable substitution is available (`${ENV_VAR}` syntax).
**Recommendation:**
- Replace all hardcoded secrets in `config.yml` with env-var references
- Warn on startup if `jwtSecret` does not use `${...}` syntax
- Add `.gitignore` entry for production configs
- Consider adding a vault integration
---
#### H2. Token Authentication via Query Parameters — ℹ️ AKZEPTIERTES RISIKO
**Location:** `internal/models/permission.go`
**CVSS-Schätzung:** 7.0
**Status:** Akzeptiertes Risiko (bewusste Design-Entscheidung) + Timing-Attack-Fix
**Description:** Token-based permissions check both headers and query parameters. Query parameters appear in server logs, browser history, referrer headers, and proxy logs, exposing authentication tokens.
**Bewertung:** Die Query-Parameter-Unterstützung ist ein bewusstes Feature für Anwendungsfälle, in denen Header nicht gesetzt werden können (z.B. direkte URL-Aufrufe, Webhooks). Das Risiko ist dokumentiert und dem Betreiber bekannt.
**Timing-Attack-Fix (2025-02-27):** Token-Vergleiche in `GetTokenPermissionSet()` wurden von `==` auf `subtle.ConstantTimeCompare()` umgestellt, um Timing-Angriffe auf Token-Werte zu verhindern.
**Empfehlung für Betreiber:**
- Token-Werte über Environment-Variablen konfigurieren (`"x-api-key:${API_KEY}"`)
- Proxy/Loadbalancer so konfigurieren, dass Query-Parameter nicht geloggt werden
- Wo möglich, Header-basierte Token-Übergabe bevorzugen
---
#### H3. Refresh Token Cookie Missing Secure Flag — ✅ BEHOBEN
**Location:** `internal/models/login.go`, `internal/config/server.go`
**CVSS-Schätzung:** 7.0 → **behoben**
**Description:** The refresh token cookie had `Secure: false` hardcoded.
**Lösung implementiert:**
- Neuer Config-Eintrag `api.secureCookies` (bool, Default: `true`)
- Environment-Variable: `API_SECURE_COOKIES`
- Beide `SetCookie`-Aufrufe (Login + Logout) verwenden jetzt `config.Server.API.IsSecureCookies()`
- Default `true` → Cookies werden nur über HTTPS gesendet
- Für lokale Entwicklung (HTTP) kann `secureCookies: false` gesetzt werden
---
#### H5. SSRF via http.fetch in Hooks
**Location:** `internal/hook/context_http.go`
**CVSS-Schätzung:** 6.5
**Description:** The `http.fetch` and `http.fetchStream` functions in hooks allow requests to arbitrary URLs without restriction. A hook can fetch internal network resources (e.g. `http://169.254.169.254/` for cloud metadata, `http://localhost:27017/` for MongoDB).
```go
request, err := http.NewRequest(method, url, reqBody)
// No URL validation — any scheme, any host, any port
```
**Impact:** Server-Side Request Forgery (SSRF) allowing access to internal services, cloud metadata endpoints, and potentially the MongoDB instance.
**Recommendation:**
- Implement an optional URL whitelist/blocklist at config level
- Block requests to private IP ranges (`10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `169.254.0.0/16`, `127.0.0.0/8`) by default
- Document SSRF risks for project administrators
---
#### H6. Unrestricted Shell Command Execution in Hooks
**Location:** `internal/hook/context_exec.go`
**CVSS-Schätzung:** 6.5
**Description:** The `exec.command` function executes any binary without path or name restrictions. While arguments are passed as separate `args[]` (safe from shell injection), the command name is controlled by hook code:
```go
cmd := exec.CommandContext(timeoutCtx, command, cmdArgs...)
```
The `dir` option also allows setting arbitrary working directories:
```go
if dir, ok := options["dir"].(string); ok {
cmd.Dir = dir
}
```
**Mitigation:** Commands execute under project UID via suexec isolation. Default dir is project config directory.
**Impact:** Arbitrary code execution on the host with project-level privileges. Without suexec (non-root deployment), commands run as the server process user.
**Recommendation:**
- Implement a command whitelist at the project config level
- Restrict `dir` option through `CleanFilePath` validation
- Log all command executions for audit purposes
---
### MEDIUM
#### M1. Error Message Information Disclosure
**Location:** `internal/utils/error.go`, `internal/dbapi/helper.go`
**CVSS-Schätzung:** 5.3
**Description:** By default (`config.response.error.message` not set), the raw error message is included in API responses:
```go
errorInResponse := errorString
if config.Server.Response.Error.Message != "" {
errorInResponse = config.Server.Response.Error.Message
}
```
When `response.error.stack: true` is set, Go stack traces are exposed:
```go
if config.Server.Response.Error.Stack {
o["errorStack"] = errorStack
}
```
**Impact:** Detailed error messages can reveal internal paths, database structure, Go type names, and stack traces.
**Recommendation:**
- Default to generic error messages in production
- Only enable detailed errors when explicitly configured for development
- Never expose stack traces in production
---
#### M2. File Upload: No Size Limit or Type Validation
**Location:** `internal/controller/collection.go:75-140`
**CVSS-Schätzung:** 5.3
**Description:** File uploads via base64-encoded data in JSON have no size limit or content type validation:
```go
if i := strings.Index(srcPrefix, ","); i > 0 &&
strings.HasPrefix(srcPrefix, "data:") &&
strings.Index(srcPrefix, "base64") > 0 { // validate as data
```
The MIME type is extracted from the `data:` URL but not validated against an allowlist. Any file type can be uploaded.
**Impact:**
- Denial of service via extremely large base64 payloads (decoded in memory)
- Storage exhaustion attacks
- Upload of executable files or malicious content
**Recommendation:**
- Implement configurable maximum file size limits
- Add optional MIME type allowlists per collection field
- Consider limiting request body size at the middleware level (Gin's `MaxMultipartMemory`)
---
#### M3. File Upload: Filename Not Sanitized
**Location:** `internal/controller/collection.go:119-121`
**CVSS-Schätzung:** 5.3
**Description:** The original filename from the upload payload is used directly in the file path:
```go
if filename, ok := fileObj["path"].(string); ok {
path += "/" + filename
}
```
While the `path` is within the upload directory tree, the `filename` value is not sanitized. A filename like `../../etc/cron.d/malicious` combined with upload path could potentially escape the intended directory.
**Mitigation:** The upload path structure `uploadPath + "/" + id + "/" + field` provides some containment. The ID is server-generated.
**Recommendation:**
- Sanitize filenames: strip directory separators, use `filepath.Base()`, restrict to alphanumeric + allowed extensions
- Apply `CleanFilePath` validation to constructed upload paths
---
#### M4. ReDoS Risk via $regex Filter Operator
**Location:** `internal/dbapi/filter_valid_functions.go:28`
**CVSS-Schätzung:** 5.3
**Description:** The `$regex` operator is allowed in user-provided query filters:
```go
"$regex": true,
```
MongoDB's `$regex` uses PCRE, which is vulnerable to catastrophic backtracking with malicious patterns. An attacker can craft regex patterns that cause exponential CPU usage on the database server.
**Impact:** Denial of service via CPU exhaustion on MongoDB.
**Recommendation:**
- Set a timeout for MongoDB read operations
- Consider restricting `$regex` to authenticated users only
- Add regex complexity validation (max length, disallow nested quantifiers)
- Document ReDoS risks
---
#### M5. Prometheus /metrics Endpoint Unauthenticated
**Location:** `internal/controller/setup.go:135`
**CVSS-Schätzung:** 5.0
**Description:** The Prometheus metrics endpoint is publicly accessible without authentication:
```go
r.GET("/metrics", gin.WrapH(promhttp.Handler()))
```
**Impact:** Exposes internal metrics including request rates, error rates, latency distributions, and potentially Go runtime metrics, providing reconnaissance data to attackers.
**Recommendation:**
- Add authentication (admin token or basic auth) to `/metrics`
- Or restrict access via network-level controls and document this requirement
---
#### M6. No Request Timeout Middleware
**Location:** `internal/controller/setup.go`
**CVSS-Schätzung:** 5.0
**Description:** A TODO exists for request timeout middleware that was never implemented:
```go
// TODO: timeout middleware
```
**Impact:** Long-running requests (e.g. complex queries, large file uploads, slow hooks) can tie up server goroutines indefinitely, enabling resource exhaustion attacks.
**Recommendation:** Implement `http.TimeoutHandler` or Gin timeout middleware with configurable duration (e.g. 30s default, configurable per collection).
---
#### M7. JWT Token Contains Full User Object
**Location:** `internal/models/login.go:95-103`
**CVSS-Schätzung:** 4.3
**Description:** The JWT access token includes the full user object (minus password), including arbitrary `meta` data:
```go
userCopy := *user
userCopy.InsertTime = nil
userCopy.UpdateTime = nil
userCopy.Meta = nil
tokenData = &userCopy
```
The `Meta` is set to `nil`, but `Permissions`, `LLMBudgets`, and other fields are included. The token is not encrypted, only signed.
**Impact:** Token leakage exposes user metadata and internal permission structures. Tokens are also long-lived (60 minutes) without revocation support.
**Recommendation:**
- Minimize token claims to `{id, username, role}`
- Load full user data from DB when needed
- Consider shorter token lifetime with more frequent refresh
- Implement token revocation (e.g. via token blacklist or `iat` validation)
---
### LOW
#### L1. Typo in Log Output (Unchanged)
**Location:** `internal/controller/setup.go:62`
**Description:** Log message reads `"senty gin activated"` instead of `"sentry gin activated"`.
---
#### L2. Weak Password Policy
**Location:** `internal/models/user.go:38`
**CVSS-Schätzung:** 3.7
**Description:** Password minimum length is 3 characters:
```go
Password *string `json:"password,omitempty" bson:"password" validator:"required,min=3"`
```
**Impact:** Extremely weak passwords are permitted, compounding the brute-force risk.
**Recommendation:** Increase minimum to 8+ characters. Consider complexity requirements.
---
#### L3. `allowUpperPaths: true` in Default Config
**Location:** `config.yml:15`
**Description:** The default config allows traversal above config directory:
```yaml
security:
allowAbsolutePaths: false
allowUpperPaths: true
```
This enables `fs.readFile("../../etc/passwd")` from hooks (though constrained by suexec UID).
**Recommendation:** Default to `false` for `allowUpperPaths` and only enable when explicitly needed.
---
#### L4. Hook Cookie Functions Without Scope Restrictions
**Location:** `internal/hook/context_cookie.go`
**Description:** The `cookie.set` function in hooks can set cookies with arbitrary names, paths, domains, and security flags. A malicious hook could overwrite authentication cookies or set cookies for different domains.
**Recommendation:** Document this as a trust boundary — hook code has the same trust level as server-side code. Consider restricting cookie domain to the current request's domain.
---
#### L5. LLM Quota Race Condition
**Location:** `internal/llm/quota.go:61-104`
**Description:** LLM budget checking and recording happens in two separate operations (read, then write). Under concurrent requests, a user could exceed their token budget before the write propagates:
```go
// 1. Check budget (read)
if err := llm.CheckBudget(budgets, req.Provider); err != nil { ... }
// 2. Use tokens (API call happens here)
// 3. Record usage (write)
_ = qm.RecordUsage(auth.ID, req.Provider, resp.Model, usage, budgets)
```
**Impact:** Minor over-budget usage under high concurrency; not exploitable for significant financial impact.
**Recommendation:** Accept current behavior with documentation, or use MongoDB atomic `$inc` for budget tracking.
---
### RESOLVED
#### ~~R1. Commented-Out Collection Count Endpoint~~
**Location:** `internal/controller/setup.go`, `internal/controller/collection_count.go`
The dedicated `GET /_/:project/:collection/_count` endpoint is now implemented with full permission pipeline.
---
## Security Posture Assessment
### Strengths ✅
1. **MongoDB Operator Whitelist:** Filter operators are validated against an explicit allowlist (`filter_valid_functions.go`), blocking dangerous operators like `$where`
2. **Suexec Process Isolation:** Projects run with dedicated UID/GID on Linux, providing filesystem-level sandboxing
3. **Permission System Depth:** Multi-layered permission model (public → token → user → custom) with pre-filters and projection restrictions
4. **Hook Timeout Support:** JS hooks have configurable timeouts to prevent infinite loops
5. **Audit Logging:** Comprehensive audit trail for admin actions with token identification
6. **Bulk Operation Guard:** Requires explicit `bulk: true` config, preventing accidental mass operations
7. **File Path Sanitization:** `CleanFilePath` provides directory traversal protection with configurable strictness
8. **Error Code Tracking:** Each error gets a unique code for correlation without exposing internals (when `response.error.message` is configured)
9. **Env-Var Substitution:** Secrets can be externalized via `${ENV_VAR}` syntax
10. **Admin Token Granularity:** Fine-grained permission system with wildcards for server admin tokens
### Weaknesses ❌
1. **Cryptographic:** ~~MD5 password hashing~~ ✅ behoben (bcrypt), no salt ✅ behoben (bcrypt nutzt eingebauten Salt), weak password policy
2. **Network Security:** No TLS termination (expected to be behind proxy), ~~no rate limiting~~ ✅ behoben, ~~Cookie Secure=false~~ ✅ behoben (konfigurierbar, Default: true)
3. **Token Security:** ~~Timing-attack-vulnerable comparisons~~ ✅ behoben (constant-time compare), query parameter leakage ℹ️ akzeptiertes Risiko, no revocation
4. **Input Validation Gaps:** No file upload size limits, filename not sanitized, ReDoS via `$regex`
5. **Execution Safety:** Unrestricted shell commands and HTTP fetch in hooks (SSRF)
6. **Missing Middleware:** No request timeout, ~~no rate limiting~~ ✅ behoben (Login-Rate-Limiting), no body size limits
---
## Prioritized Remediation Plan
### Phase 1 — Critical (Immediate, 1-2 Weeks)
| # | Finding | Effort | Impact |
|---|---------|--------|--------|
| ~~C1~~ | ~~Replace MD5 with bcrypt + migration path~~ | ~~Medium~~ | ✅ **ERLEDIGT** — bcrypt + MD5-Migration implementiert |
| ~~C2~~ | ~~Add login rate limiting~~ | ~~Low~~ | ✅ **ERLEDIGT** — Exponentielles Backoff Rate Limiting (IP + Username) implementiert |
| ~~C3~~ | ~~Use `subtle.ConstantTimeCompare` for tokens~~ | ~~Low~~ | ✅ **ERLEDIGT** — Alle Token-Vergleiche auf constant-time umgestellt |
| H1 | Remove hardcoded JWT secret from config.yml | Low | Prevents token forgery from source |
### Phase 2 — High (2-4 Weeks)
| # | Finding | Effort | Impact |
|---|---------|--------|--------|
| H3 | Make cookie `Secure` flag configurable | Low | ✅ **ERLEDIGT** — `api.secureCookies` Config-Eintrag (Default: true) |
| L2 | Increase minimum password length to 8 | Low | Reduces brute-force surface |
| M1 | Default to generic error messages | Low | Prevents info disclosure |
| M6 | Implement request timeout middleware | Medium | Prevents resource exhaustion |
### Phase 3 — Medium (1-2 Months)
| # | Finding | Effort | Impact |
|---|---------|--------|--------|
| ~~H2~~ | ~~Deprecate query-parameter tokens~~ | ~~Medium~~ | ℹ️ **AKZEPTIERTES RISIKO** — bewusste Design-Entscheidung, Timing-Fix angewendet |
| H5 | Add SSRF protection for http.fetch | Medium | Blocks internal network access |
| H6 | Add exec command whitelist config | Medium | Limits command execution |
| M2 | Add file upload size/type validation | Medium | Prevents resource exhaustion |
| M3 | Sanitize uploaded filenames | Low | Prevents path manipulation |
| M4 | Add regex complexity limits | Medium | Prevents ReDoS on MongoDB |
| M5 | Protect /metrics endpoint | Low | Hides internal metrics |
| M7 | Minimize JWT token claims | Medium | Reduces exposure surface |
### Phase 4 — Hardening (Ongoing)
| # | Finding | Effort | Impact |
|---|---------|--------|--------|
| L3 | Default `allowUpperPaths` to false | Low | Tighter default security |
| L4 | Document hook trust boundaries | Low | Operational awareness |
| L5 | Atomic LLM budget tracking | Low | Prevents minor over-use |
---
## Security Best Practices
### Configuration
```yaml
api:
jwtSecret: ${JWT_SECRET} # Always use env vars for secrets
cors:
allowOrigins:
- https://yourdomain.com # Never use "*" in production
allowCredentials: true
security:
allowAbsolutePaths: false
allowUpperPaths: false # Restrict to config directory
response:
error:
message: "An error occurred" # Generic error for production
stack: false # Never in production
```
### Permissions
```yaml
permissions:
public:
methods:
get: true # Minimal public access
filter:
published: true # Always pre-filter public data
user:
methods:
get: true
post: true
put: true
filter:
author:
$subst:
expr: "auth.id" # Users only see own documents
validProjections:
- summary # Restrict visible fields
```
### Token Permissions
```yaml
permissions:
"x-api-key:${API_KEY}": # Use env vars for token values
methods:
get: true
filter:
public: true
```
### Hooks Security
```js
// Always validate input
var req = request();
var body = json.parse(req.body());
if (!body.email || body.email.indexOf("@") === -1) {
throw { status: 400, json: { error: "invalid_email" } };
}
// Use bcrypt for password hashing in custom auth flows
var hash = bcrypt.hash(body.password, { cost: 12 });
// Sanitize exec commands
var allowedCommands = ["convert", "ffmpeg", "python3"];
if (allowedCommands.indexOf(command) === -1) {
throw { status: 400, json: { error: "command_not_allowed" } };
}
```
src/content/docs/server/15-troubleshooting.md
+232
View File
@@ -0,0 +1,232 @@
---
title: 'Troubleshooting & FAQ'
---
# Troubleshooting & FAQ
## Common Errors
### Connection Errors
**Error:** `failed to connect to MongoDB`
- Check that MongoDB is running and reachable
- Verify `api.mongoUri` in `config.yml` or `API_MONGOURI` env var
- For Docker: ensure the MongoDB container is on the same network
- For replica sets: ensure all members are reachable
**Error:** `server selection timeout`
- MongoDB replica set may not be initialized
- Run: `rs.initiate()` in the MongoDB shell
---
### Authentication Errors
**Error:** `401 Unauthorized` on all requests
- Check that `X-Auth-Token` header is set correctly
- Verify JWT is not expired (60 min default)
- Ensure `api.jwtSecret` matches between token generation and validation
**Error:** `403 Forbidden` on collection access
- Check collection permissions in project `config.yml`
- Verify user is assigned to the project (`project.users[]`)
- Check if token-based permission has correct value (wrong value = immediate deny)
- Admin role (0) bypasses all permission checks
**Error:** Login returns wrong password even though it's correct
- Passwords are stored as MD5 hashes
- Check if password was stored in plaintext vs. hashed
- Use `crypto.md5("password")` in a hook to verify the expected hash
---
### Hook Errors
**Error:** `ReferenceError: require is not defined`
- Hooks run in goja (ES5.1+), not Node.js
- There is no `require()`, `import`, or npm
- Use the built-in context packages (`db`, `http`, `fs`, etc.)
**Error:** `ReferenceError: async is not defined`
- goja does not support `async/await`
- All hook functions are synchronous
- Use `http.fetchStream` for streaming, not async patterns
**Error:** `TypeError: X is not a function`
- Check the function name — it may have been renamed
- See the [legacy compatibility table](06-hooks.md#legacy-compatibility)
- Common: `readCollection` → `db.find`, `mail` → `smtp.sendMail`
**Error:** JS hook timeout
- Increase timeout in `hookConfig.timeout` (milliseconds)
- For streaming hooks, use `channel.subscribe()` which auto-disables timeout
- Check for infinite loops in hook code
**Error:** Hook stopped after client disconnected
- By default, hooks follow the incoming HTTP request lifecycle
- Set `ignoreRequestCancel: true` on the hook config only for critical hooks that must continue after client disconnect
- `ignoreRequestCancel` does not disable the hook timeout; increase `timeout` separately if needed
- `DisableTimeout()` only removes the execution timeout for specific built-in long-running operations; it does not, by itself, ignore request cancellation
**Error:** Hook exception not returning expected format
- Ensure the thrown object has the right keys: `status`, `json`, `html`, `string`, `bytes`, `file`
- If `response.write()` was already called, exceptions can only abort — no new response body
---
### File Upload Errors
**Error:** Files not being saved
- Check that the upload path exists and is writable
- File fields must use Base64 data URI format: `data:mime/type;base64,...`
- Field type must be `file` or `file[]` in collection config
**Error:** Image filter returns 404
- Verify the filter name exists in `imageFilter` config
- Check file path: `/{namespace}/{collection}/{id}/{field}/{filename}?filter={name}`
- Use `?no-cache=1` to force regeneration
---
### Configuration Errors
**Error:** Project not loading
- Check YAML syntax in project `config.yml`
- Verify the project directory is under `api.projectsDir`
- Check file permissions (readable by server process)
- Use admin reload: `POST /api/v1/_/{project}/_/admin/reload`
**Error:** Environment variable not resolved
- Format must be `${VAR_NAME}` (with dollar and braces)
- Variable must be set in the server's environment
- Env file path must be correct in `config.yml`
**Error:** CORS not working
- CORS is configured at three levels: server → project → collection
- Check if `merge: true` is set for inheriting parent CORS
- `allowOrigins: ["*"]` does not work with `allowCredentials: true`
- Note: `exposeHeaders` always replaces (not appends) even with `merge: true` (known bug)
---
## Debugging Tools
### Server-Side Debugging
```js
// Print to stdout
context.debug.dump(context.data, "some label");
// Get Sentry trace ID for error tracking
var traceId = debug.sentryTraceId();
// Pause execution
context.debug.sleep(5000); // 5 seconds
```
### Request Inspection
```js
var req = request();
context.debug.dump({
method: req.method,
path: req.path,
url: req.url,
host: req.host,
clientIp: req.clientIp,
authHeader: req.header("Authorization"),
body: req.body()
});
```
### Database Debugging
```js
// Check what's in a collection
var all = db.find("posts", { limit: 5 });
context.debug.dump(all);
// Count documents
var count = db.count("posts", {
filter: { status: "published" }
});
context.debug.dump("Published posts:", count);
```
### HTTP Debugging
Use the `test.http` file in the project root for quick API testing with VS Code's REST Client extension.
---
## FAQ
### Can I use TypeScript in hooks?
No. Hooks run in goja (ECMAScript 5.1+). However, the `tibi-types` package provides TypeScript definitions for IDE autocompletion during development.
### Can I use async/await?
No. The goja runtime is synchronous. Use `http.fetchStream` for streaming and `channel.subscribe` for real-time patterns.
### How do I handle file uploads?
Embed files as Base64 data URIs in the JSON body. Set the field type to `file` or `file[]`. The server automatically extracts and saves the file, replacing the URI with a file path.
### How do I serve static files?
Configure `assets` in your project config with a name and path, then access via `/api/v1/{namespace}/assets/{name}/{path}`.
### Can I use WebSockets?
No. Tibi-Server uses Server-Sent Events (SSE) for real-time communication. Use `channel.subscribe` in a hook for the server side and `EventSource` on the client.
### How do I run database migrations?
Use hooks or jobs. Create a `post.before` hook on a migration collection, or a cron job that checks and applies necessary schema changes.
### How do I reset a user's password?
As admin, update the user via PUT. The password will be automatically MD5-hashed on save.
### How do I add custom endpoints?
Use hooks with custom response writing. A `get.before` hook that calls `response.write()` and throws effectively creates a custom endpoint.
### What happens if a hook throws without a status?
Default status is `500`. The error message is included in a JSON error response.
### How do I keep a critical hook running after the client disconnects?
Set `ignoreRequestCancel: true` on that hook and configure a suitable `timeout`.
Use this only for truly critical side effects such as order submission or writing to an outbox. For larger workflows, prefer persisting work and continuing asynchronously in a job instead of holding the HTTP request open.
### How do I debug CORS issues?
1. Check browser DevTools → Network → look for preflight (OPTIONS) requests
2. Verify `allowOrigins` includes your domain (exact match, not regex)
3. Check if `allowCredentials` is needed (cookies, auth headers)
4. Verify `allowMethods` includes the HTTP method you're using
5. Remember: wildcard origin `*` does not work with credentials
### How do I scale horizontally?
Currently, tibi-server is designed for single-instance deployment. Realtime channels are in-memory and not synchronized across instances. For multi-instance setups, put a load balancer in front and ensure sticky sessions for SSE connections.
src/content/docs/server/16-cookbook.md
+686
View File
@@ -0,0 +1,686 @@
---
title: 'Collection Config Cookbook'
---
# Collection Config Cookbook
Ready-to-use collection configuration recipes for common use cases.
---
## Blog with Drafts and Publishing
```yaml
collections:
- name: posts
fields:
- key: title
type: string
validation:
required: true
- key: slug
type: string
validation:
required: true
- key: body
type: string
- key: excerpt
type: string
- key: coverImage
type: file
- key: tags
type: string[]
- key: author
type: string
- key: authorName
type: string
- key: status
type: string
validation:
required: true
in:
- draft
- published
- archived
- key: publishedAt
type: date
permissions:
public:
methods:
get: true
filter:
status: published
user:
methods:
get: true
post: true
put: true
delete: true
projections:
summary:
title: 1
slug: 1
excerpt: 1
coverImage: 1
tags: 1
authorName: 1
publishedAt: 1
full:
title: 1
slug: 1
body: 1
excerpt: 1
coverImage: 1
tags: 1
author: 1
authorName: 1
status: 1
publishedAt: 1
hooks:
post.before: hooks/posts/before_create.js
imageFilter:
thumbnail:
- width: 400
height: 300
fit: true
quality: 80
outputType: webp
hero:
- width: 1200
height: 600
fill: true
anchor: center
quality: 85
outputType: webp
indexes:
- keys:
slug: 1
options:
unique: true
- keys:
status: 1
publishedAt: -1
audit:
enabled: true
```
**Hook: `hooks/posts/before_create.js`**
```js
var auth = user.auth();
context.data.author = auth.id;
context.data.authorName = auth.username;
if (!context.data.status) {
context.data.status = "draft";
}
if (context.data.status === "published" && !context.data.publishedAt) {
context.data.publishedAt = new Date().toISOString();
}
```
---
## User Profiles (Self-Managed)
```yaml
collections:
- name: profiles
fields:
- key: userId
type: string
validation:
required: true
- key: displayName
type: string
- key: bio
type: string
- key: avatar
type: file
- key: website
type: string
- key: social
type: object
fields:
- key: twitter
type: string
- key: github
type: string
permissions:
public:
methods:
get: true
user:
methods:
get: true
post: true
put: true
filter:
userId:
$subst:
expr: "auth.id"
projections:
public:
displayName: 1
bio: 1
avatar: 1
website: 1
social: 1
indexes:
- keys:
userId: 1
options:
unique: true
imageFilter:
small:
- width: 100
height: 100
fill: true
anchor: center
quality: 80
outputType: webp
```
---
## E-Commerce Products with Variants
```yaml
collections:
- name: products
fields:
- key: name
type: string
validation:
required: true
- key: description
type: string
- key: price
type: float
validation:
required: true
- key: currency
type: string
validation:
in: [EUR, USD, GBP]
- key: images
type: file[]
- key: category
type: string
- key: tags
type: string[]
- key: variants
type: object[]
fields:
- key: name
type: string
- key: sku
type: string
- key: price
type: float
- key: stock
type: int
- key: active
type: bool
permissions:
public:
methods:
get: true
filter:
active: true
"x-api-key:${SHOP_API_KEY}":
methods:
get: true
post: true
put: true
delete: true
projections:
catalog:
name: 1
price: 1
currency: 1
images: 1
category: 1
tags: 1
indexes:
- keys:
category: 1
- keys:
tags: 1
- keys:
active: 1
category: 1
imageFilter:
thumb:
- width: 300
height: 300
fit: true
quality: 85
outputType: webp
large:
- width: 1200
height: 1200
fit: true
quality: 90
```
---
## Chat Messages with Realtime
```yaml
collections:
- name: messages
fields:
- key: room
type: string
validation:
required: true
- key: sender
type: string
- key: senderName
type: string
- key: text
type: string
validation:
required: true
- key: type
type: string
validation:
in: [text, image, system]
permissions:
user:
methods:
get: true
post: true
hooks:
post.before: hooks/messages/before_send.js
post.after: hooks/messages/after_send.js
indexes:
- keys:
room: 1
insertTime: -1
- name: chat-stream
hooks:
get.before: hooks/messages/stream.js
permissions:
user:
methods:
get: true
```
**Hook: `hooks/messages/before_send.js`**
```js
var auth = user.auth();
context.data.sender = auth.id;
context.data.senderName = auth.username;
if (!context.data.type) {
context.data.type = "text";
}
```
**Hook: `hooks/messages/after_send.js`**
```js
context.channel.send("chat-" + context.data.room, {
id: context.data.id,
sender: context.data.sender,
senderName: context.data.senderName,
text: context.data.text,
type: context.data.type,
time: context.data.insertTime
});
```
**Hook: `hooks/messages/stream.js`**
```js
var req = request();
var room = req.query("room");
if (!room) {
throw { status: 400, json: { error: "room parameter required" } };
}
context.response.header("Content-Type", "text/event-stream");
context.response.header("Cache-Control", "no-cache");
context.response.writeHeader(200);
var sub = channel.subscribe("chat-" + room, {
bufferSize: 200,
lastN: 50
});
var msg;
while ((msg = sub.receive()) !== null) {
context.response.writeSSE("message", context.json.stringify(msg));
}
sub.close();
throw { status: 200 };
```
---
## File Storage with Metadata
```yaml
collections:
- name: documents
fields:
- key: title
type: string
validation:
required: true
- key: file
type: file
validation:
required: true
- key: mimeType
type: string
- key: size
type: int
- key: uploadedBy
type: string
- key: folder
type: string
- key: tags
type: string[]
permissions:
user:
methods:
get: true
post: true
put: true
delete: true
hooks:
post.before: hooks/documents/before_upload.js
indexes:
- keys:
folder: 1
- keys:
tags: 1
- keys:
uploadedBy: 1
```
**Hook: `hooks/documents/before_upload.js`**
```js
var auth = user.auth();
context.data.uploadedBy = auth.id;
// Extract mime type from data URI
if (context.data.file && context.data.file.indexOf("data:") === 0) {
var mime = context.data.file.split(";")[0].split(":")[1];
context.data.mimeType = mime;
}
```
---
## Cross-Referenced Collections
```yaml
collections:
- name: authors
fields:
- key: name
type: string
validation:
required: true
- key: email
type: string
- key: avatar
type: file
- name: articles
fields:
- key: title
type: string
validation:
required: true
- key: body
type: string
- key: authorId
type: string
validation:
required: true
- key: categoryIds
type: string[]
hooks:
get.after: hooks/articles/after_get.js
- name: categories
fields:
- key: name
type: string
- key: slug
type: string
```
**Hook: `hooks/articles/after_get.js`**
```js
// Resolve author references
var authorIds = [];
for (var i = 0; i < context.result.length; i++) {
if (context.result[i].authorId && authorIds.indexOf(context.result[i].authorId) === -1) {
authorIds.push(context.result[i].authorId);
}
}
if (authorIds.length > 0) {
var authors = db.find("authors", {
filter: { id: { "$in": authorIds } },
selector: { name: 1, avatar: 1 }
});
var authorMap = {};
for (var j = 0; j < authors.length; j++) {
authorMap[authors[j].id] = authors[j];
}
for (var k = 0; k < context.result.length; k++) {
context.result[k].author = authorMap[context.result[k].authorId] || null;
}
}
```
---
## Event Calendar
```yaml
collections:
- name: events
fields:
- key: title
type: string
validation:
required: true
- key: description
type: string
- key: location
type: string
- key: startDate
type: date
validation:
required: true
- key: endDate
type: date
- key: allDay
type: bool
- key: recurring
type: string
validation:
in: ["", daily, weekly, monthly, yearly]
- key: image
type: file
- key: organizer
type: string
- key: maxParticipants
type: int
- key: currentParticipants
type: int
permissions:
public:
methods:
get: true
filter:
startDate:
$gte:
$subst:
eval: "new Date().toISOString()"
hooks:
put.before: hooks/events/register.js
indexes:
- keys:
startDate: 1
- keys:
location: 1
```
---
## Settings / Key-Value Store
```yaml
collections:
- name: settings
fields:
- key: key
type: string
validation:
required: true
- key: value
type: string
- key: group
type: string
permissions:
"x-admin-token:${SETTINGS_TOKEN}":
methods:
get: true
post: true
put: true
delete: true
user:
methods:
get: true
indexes:
- keys:
key: 1
options:
unique: true
- keys:
group: 1
```
---
## Bulk Operations
### Bulk Update — Change Category for Multiple Documents
Update all products in category "old" to category "new":
```bash
curl -X PUT \
"https://example.com/api/v1/_/myproject/products?filter={\"category\":\"old\"}" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"category": "new"}'
```
### Bulk Update — Increase Prices by 10%
Use the `$mul` operator to increase all prices by 10%:
```bash
curl -X PUT \
"https://example.com/api/v1/_/myproject/products?filter={\"category\":\"premium\"}" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"price": {"$mul": 1.1}}'
```
### Bulk Update — Decrement Stock
Use `$inc` with a negative value to reduce stock for all matched items:
```bash
curl -X PUT \
"https://example.com/api/v1/_/myproject/products?filter={\"warehouse\":\"A\"}" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"stock": {"$inc": -5}}'
```
### Bulk Update — Remove a Field
Use `$unset` to remove a field from all matched documents:
```bash
curl -X PUT \
"https://example.com/api/v1/_/myproject/products?filter={\"deprecated\":true}" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"legacyField": {"$unset": true}}'
```
### Bulk Delete — Delete by Filter
Delete all archived posts:
```bash
curl -X DELETE \
"https://example.com/api/v1/_/myproject/posts?filter={\"status\":\"archived\"}" \
-H "Authorization: Bearer $TOKEN"
```
### Optimised Bulk Update with Hook
For large datasets, use a `bulkUpdate` hook step with `context.db.updateMany()` for a single database call instead of per-document processing:
```yaml
collections:
- name: products
hooks:
put.bulkUpdate: hooks/products/bulk_update.js
put.return: hooks/products/bulk_return.js
```
**hooks/products/bulk_update.js:**
```javascript
// Optimised: single DB call to update all matching documents
var filter = context.filter;
var body = context.body;
var result = context.db.updateMany("products", filter, { changes: body });
context.log("Updated " + result.modifiedCount + " documents");
```
### Optimised Bulk Update with Operators in Hook
Use the `update` parameter instead of `changes` to pass raw MongoDB update operators:
```javascript
// Increase price by 10% for all matching products
var filter = context.filter;
var result = context.db.updateMany("products", filter, {
update: { "$mul": { "price": 1.1 } }
});
context.log("Updated " + result.modifiedCount + " documents");
```
### Optimised Bulk Delete with Hook
```yaml
collections:
- name: products
hooks:
delete.bulkDelete: hooks/products/bulk_delete.js
delete.return: hooks/products/bulk_return.js
```
**hooks/products/bulk_delete.js:**
```javascript
var filter = context.filter;
var result = context.db.deleteMany("products", filter);
context.log("Deleted " + result.deletedCount + " documents");
```
src/content/docs/server/17-field-level-permissions.md
+550
View File
@@ -0,0 +1,550 @@
---
title: 'Field-Level Permissions'
---
# Field-Level Permissions
## Überblick
Field-Level Permissions erlauben den Zugriff auf **einzelne Felder** innerhalb einer Collection einzuschränken. Das System verwendet ein **3-Layer-Kaskadenmodell** mit **Fehler-Modus** (400 Bad Request) statt stillem Stripping.
Zusätzlich können `readonly` und `hidden` auf Field-Definition-Ebene als **Eval-Ausdrücke** (JavaScript) definiert werden, die **pro Dokument** dynamisch ausgewertet werden.
**Zwei Feld-Typen:**
| Typ | POST/PUT | GET | Config |
|-----|----------|-----|--------|
| `readonlyFields` | **400-Fehler** wenn im Body | Normal sichtbar | Felder bleiben in `fields[]`; Liste in `yourPermissions` |
| `hiddenFields` | **400-Fehler** wenn im Body | Aus Response **gestripped** | Felder aus `fields[]` **entfernt**; Liste nicht in `yourPermissions` |
**Admin-Bypass:** Admin (role=0) und Admin-Token → `GetEffectivePermissionSet()` gibt `nil` zurück → keine Einschränkungen.
---
## Konfiguration
### Drei Ebenen (Kaskade)
```yaml
name: orders
# Ebene 1: Collection-Level (Basis für alle PermissionSets)
readonlyFields: [createdBy]
hiddenFields: [costPrice]
permissions:
user:
methods:
get: true
post: true
put: true
# Ebene 2: PermissionSet-Level (addiert/subtrahiert von Collection-Level)
readonlyFields: [customerName]
hiddenFields: [internalMargin]
fields:
- name: alwaysReadonly
type: number
readonly: true # Ebene 3: Field-Definition (absolutes Override)
- name: alwaysHidden
type: string
hidden: true # Ebene 3: Field-Definition (absolutes Override)
- name: neverReadonly
type: string
readonly: false # Ebene 3: entfernt Feld aus JEDEM effektiven readonly-Set
- name: conditionalReadonly
type: string
readonly:
eval: "$.status === 'confirmed'" # Ebene 3: Eval — readonly nur wenn Bedingung zutrifft
- name: conditionalHidden
type: string
hidden:
eval: "$.status === 'confirmed'" # Ebene 3: Eval — hidden nur wenn Bedingung zutrifft
```
### Kaskadenmodell im Detail
Die effektive Feldliste wird in 4 Schritten berechnet (`ResolveEffectiveFields`):
1. **Collection-Level** → Basis-Set: `readonlyFields: [createdBy]`
2. **PermissionSet-Level** → Hinzufügen/Entfernen mit `-` Prefix-Negation:
- `readonlyFields: [customerName]` → fügt `customerName` hinzu
- `readonlyFields: [-createdBy]` → entfernt `createdBy` aus dem Set
3. **Parent-Expansion** → Elternfelder werden auf alle Sub-Felder expandiert (z.B. `payment` → `payment.method`, `payment.cardNumber`)
4. **Field-Definition Override** (letztes Wort):
- `readonly: true` → Feld wird **immer** hinzugefügt (egal ob in Liste oder nicht)
- `readonly: false` → Feld wird **immer** entfernt (überschreibt alles)
- `hidden: true` / `hidden: false` → analog für Hidden-Felder
- `readonly: {eval: "..."}` / `hidden: {eval: "..."}` → **Eval-Ausdruck**, wird pro Dokument ausgewertet (siehe [Eval-Ausdrücke](#eval-ausdrücke-dynamische-feld-permissions))
### Ergebnis-Beispiel
Mit der obigen Config ergibt sich:
| Feld | user effektiv readonly | user effektiv hidden |
|------|----------------------|---------------------|
| createdBy | ✅ (Collection-Level) | — |
| customerName | ✅ (PermissionSet) | — |
| alwaysReadonly | ✅ (Field-Def `readonly: true`) | — |
| neverReadonly | ❌ (Field-Def `readonly: false` überschreibt) | — |
| costPrice | — | ✅ (Collection-Level) |
| internalMargin | — | ✅ (PermissionSet) |
| alwaysHidden | — | ✅ (Field-Def `hidden: true`) |
### Verschachtelung (Dot-Notation)
Felder in verschachtelten Objekten werden per Dot-Notation adressiert:
```yaml
hiddenFields:
- payment.cardNumber
- payment.cvv
readonlyFields:
- address.zip
```
Wenn ein **Elternfeld** gelistet wird, werden automatisch **alle Sub-Felder** mit einbezogen (es sei denn, ein Sub-Feld hat ein Field-Def-Override `false`).
### Token-basierte Permissions
Tokens erhalten ein eigenes PermissionSet und unterliegen denselben Feld-Einschränkungen:
```yaml
permissions:
"token:MYTOKEN":
methods:
get: true
post: true
readonlyFields: [status]
# Erbt costPrice von Collection-Level hiddenFields
```
---
## Eval-Ausdrücke (Dynamische Feld-Permissions)
### Überblick
Neben statischem `readonly: true/false` und `hidden: true/false` kann auf Field-Definition-Ebene ein **JavaScript-Eval-Ausdruck** definiert werden. Dieser wird **pro Dokument** ausgewertet und ermöglicht konditionale Feld-Permissions basierend auf Dokumentinhalt, User-Kontext oder HTTP-Methode.
**Wichtig:** Eval-Ausdrücke nehmen **nicht** an der statischen Kaskade (Layer 1–3) teil. Sie werden als **separater Post-Pass** nach den statischen Checks ausgeführt.
### Syntax
```yaml
fields:
- name: confirmedAt
type: string
readonly:
eval: "$.status === 'confirmed'" # JS-Ausdruck → readonly wenn true
- name: secretNote
type: string
hidden:
eval: "$.status === 'confirmed'" # JS-Ausdruck → hidden wenn true
```
### Verfügbare Kontext-Variablen
| Variable | Typ | Beschreibung |
|----------|-----|-------------|
| `$` | `object` | Das vollständige Dokument (bei POST: der Body; bei PUT: das bestehende DB-Dokument + Body-Merge; bei GET: das DB-Dokument) |
| `$this` | `any` | Der Wert des aktuellen Felds im Dokument |
| `$auth` | `object` | Der authentifizierte User (mit `id`, `username`, `role`, `permissions`) |
| `$method` | `string` | Die HTTP-Methode (`POST`, `PUT`, `GET`) |
| `$project` | `string` | Der Projektname |
| `$namespace` | `string` | Der API-Namespace |
### Beispiele
```yaml
# Feld readonly nach Bestätigung
readonly:
eval: "$.status === 'confirmed'"
# Feld nur für bestimmte User hidden
hidden:
eval: "$auth.role > 1"
# Feld readonly bei PUT, aber nicht bei POST
readonly:
eval: "$method === 'PUT'"
# Kombination: Feld readonly wenn Betrag > 1000
readonly:
eval: "$.amount > 1000"
```
### Zwei-Phasen-Architektur
1. **Phase 1 (Statisch):** Die 3-Layer-Kaskade wird normal aufgelöst. Eval-Felder werden bei der Kaskade **ignoriert** (`.BoolValue()` gibt `nil` zurück).
2. **Phase 2 (Eval):** Nach den statischen Checks wird pro Dokument der JS-Ausdruck evaluiert. Ergibt er `true`, wird das Feld wie ein statisches readonly/hidden-Feld behandelt (400 bei POST/PUT, Strip bei GET).
### Vorkompilierung
Eval-Ausdrücke werden beim Laden der Config (in `AddCollection`) zu `goja.Program`-Objekten vorkompiliert. Syntaxfehler werden beim Config-Reload als Error geloggt. Zur Laufzeit wird nur noch `prog.Run()` aufgerufen.
### Admin-Bypass
Admin-User (role=0) und Admin-Tokens umgehen **alle** Eval-Checks. `GetEffectivePermissionSet()` gibt `nil` zurück → der gesamte Eval-Block wird übersprungen.
### Verhalten pro Endpoint
| Endpoint | Eval-Verhalten |
|----------|---------------|
| **POST** | Body wird als Dokument-Kontext (`$`) verwendet. Eval-readonly/hidden → 400 |
| **PUT** | Bestehendes DB-Dokument als Kontext. Eval-readonly/hidden → 400 (inkl. Operator-Bodies) |
| **GET** | Pro Ergebnis-Dokument: Eval-hidden → Feld wird aus Response gestripped |
| **Bulk-POST** | Pro Zeile individuelle Eval-Auswertung |
| **Bulk-Update** | Eval-Auswertung vor Batch-Operation |
### Config-Auslieferung
Eval-Definitionen (`readonly: {eval: "..."}`, `hidden: {eval: "..."}`) werden als Teil der Field-Definitionen in `fields[]` mit ausgeliefert. Der Client kann die Eval-Ausdrücke direkt aus der Config lesen und clientseitig vorvalidieren (da nur JS unterstützt wird, kann der gleiche Ausdruck im Browser ausgeführt werden).
Für **Non-Admins** werden statisch hidden Feld-Definitionen wie gewohnt aus `fields[]` entfernt. Eval-hidden-Felder bleiben jedoch sichtbar, da sie nur per Dokumentkontext ausgewertet werden können.
---
## Verhalten pro Endpoint
### POST / Bulk-POST
```
POST /api/v1/_/:project/:collection
```
Nach Body-Parsing, **vor** Validierung und Hooks:
1. `GetEffectivePermissionSet()` bestimmen (admin → `nil` → kein Check)
2. Effektive `readonlyFields` berechnen (3-Layer-Kaskade)
3. `CheckFieldsPresent(body, readonlyFields)` → bei Treffer: **400** `{"error": "readonly fields must not be sent", "fields": [...]}`
4. Effektive `hiddenFields` berechnen
5. `CheckFieldsPresent(body, hiddenFields)` → bei Treffer: **400** `{"error": "hidden fields must not be sent", "fields": [...]}`
6. **Eval-Phase:** `EvalFieldReadonly(body, ...)` + `EvalFieldHidden(body, ...)` → bei Treffer: **400** analog
Bei Bulk-POST: Check wird **pro Zeile** ausgeführt. Fehlgeschlagene Zeilen werden mit Index im Error-Array zurückgegeben, erfolgreiche Zeilen werden trotzdem verarbeitet.
### PUT / Bulk-Update
```
PUT /api/v1/_/:project/:collection/:id
```
Wie POST, zusätzlich werden **Operator-Felder** geprüft (`$set`, `$inc`, `$mul`, `$unset`, etc.):
```json
{"$set": {"customerName": "New"}, "$inc": {"price": 10}}
```
→ Prüfung ob `customerName` oder `price` in readonly/hidden ist. Felder in `$set`, `$inc`, `$mul`, `$unset`, `$min`, `$max`, `$rename`, `$push`, `$pull`, `$addToSet` werden alle geprüft.
Zusätzlich: **Eval-Phase** wie bei POST, das bestehende DB-Dokument dient als Eval-Kontext (`$`).
### GET
```
GET /api/v1/_/:project/:collection
GET /api/v1/_/:project/:collection/:id
```
Nach DB-Read, nach Lookups, **vor** Return-Hook:
- Hidden-Felder werden aus jedem Result-Dokument **gestripped** (`StripFields`)
- **Eval-Hidden:** Pro Dokument wird `EvalFieldHidden()` ausgewertet; bei `true` wird das Feld zusätzlich gestripped
- Readonly-Felder bleiben **sichtbar** (nur Schreibschutz, kein Leseschutz)
- Gilt für Einzel- und Listen-Abfragen
### Config-Auslieferung (GET /api/v1/project/:id)
Die `enrichData()`-Funktion in `project.go` liefert rollenabhängig unterschiedliche Configs:
**Nicht-Admin (User, Public, Token):**
- `yourPermissions[collection].readonlyFields` = effektive readonly-Liste
- `yourPermissions[collection].hiddenFields` wird **nicht** ausgeliefert
- Hidden-Feld-Definitionen werden aus `fields[]` **entfernt** (Frontend sieht sie nicht)
- Eval-Definitionen bleiben in `fields[]` sichtbar (auch eval-hidden, da nur per Dokument auswertbar)
- `hiddenFields`-Arrays werden aus Collection-Config und PermissionSets **entfernt**
**Admin:**
- `yourPermissions[collection].readonlyFields` wird **nicht** ausgeliefert (keine Einschränkungen)
- `yourPermissions[collection].hiddenFields` = nur als **Hint** (z.B. für UI-Markierung)
- Alle Feld-Definitionen bleiben vollständig in `fields[]`
**Lazy-Copy-Optimierung:** Die shared API-Struct wird nur kopiert wenn tatsächlich Hidden-Felder zu strippen sind (`apiCopied`-Flag). Admin-Requests erzeugen keine Kopie. So wird Mutation des geteilten API-Objekts verhindert.
---
## Vollständiges Config-Beispiel
```yaml
name: orders
defaultLanguage: german
# Collection-Level (Ebene 1): Basis für alle PermissionSets
readonlyFields: [createdBy]
hiddenFields: [costPrice]
permissions:
public:
methods:
get: true
post: false
put: false
delete: false
readonlyFields: [status, customerName]
hiddenFields: [internalMargin, costPrice, payment.cardNumber]
user:
methods:
get: true
post: true
put: true
delete: false
readonlyFields: [customerName, neverReadonly]
hiddenFields: [internalMargin]
fulfillment:
methods:
get: true
put: true
readonlyFields: [customerName, price]
# kein hiddenFields → erbt nur Collection-Level costPrice
"token:APITOKEN":
methods:
get: true
post: true
readonlyFields: [status]
fields:
- name: title
type: string
validator:
required: true
- name: status
type: string
- name: customerName
type: string
- name: price
type: number
- name: internalMargin
type: number
- name: costPrice
type: number
- name: createdBy
type: string
- name: alwaysReadonly
type: number
readonly: true # Field-Def: immer readonly für alle nicht-Admins
- name: alwaysHidden
type: string
hidden: true # Field-Def: immer hidden für alle nicht-Admins
- name: neverReadonly
type: string
readonly: false # Field-Def: NIEMALS readonly (überschreibt PermSet-Einträge)
- name: payment
type: object
subFields:
- name: method
type: string
- name: cardNumber
type: string
```
### Ergebnis-Matrix (nach Kaskade)
| Feld | admin | user | public | token:APITOKEN | fulfillment |
|------|-------|------|--------|----------------|-------------|
| title | read+write | read+write | read | read+write | read+write |
| status | read+write | read+write | **readonly** | **readonly** | read+write |
| customerName | read+write | **readonly** | **readonly** | read+write | **readonly** |
| price | read+write | read+write | read | read+write | **readonly** |
| createdBy | read+write | **readonly** | **readonly** | **readonly** | **readonly** |
| alwaysReadonly | read+write | **readonly** | **readonly** | **readonly** | **readonly** |
| neverReadonly | read+write | read+write | read+write | read+write | read+write |
| internalMargin | read+write | **hidden** | **hidden** | read+write | read+write |
| costPrice | read+write | **hidden** | **hidden** | **hidden** | **hidden** |
| alwaysHidden | read+write | **hidden** | **hidden** | **hidden** | **hidden** |
| payment.cardNumber | read+write | read+write | **hidden** | read+write | read+write |
---
## Implementierung
### Dateien
| Datei | Zweck |
|-------|-------|
| `internal/utils/field_filter.go` | Generische Kaskaden-Engine, Check/Strip-Funktionen |
| `internal/models/field_permissions.go` | Model-spezifische Wrapper, Eval-Funktionen (Eval/Resolve/Collect) |
| `internal/models/collection.go` | Struct-Definitionen (BoolOrEval, ReadonlyFields, HiddenFields) |
| `internal/models/api.go` | Config-Laden, `compileFieldEvals()` (JS-Vorkompilierung) |
| `internal/models/permission.go` | `GetEffectivePermissionSet()` — Admin-Bypass-Logik |
| `internal/models/project.go` | `enrichData()` — Config-Auslieferung mit Lazy-Copy + Eval-Felder |
| `internal/controller/collection_post.go` | POST-Enforcement (statisch + eval) |
| `internal/controller/collection_put.go` | PUT-Enforcement (statisch + eval, inkl. Operator-Check) |
| `internal/controller/collection_get.go` | GET Hidden-Stripping (statisch + eval pro Dokument) |
| `internal/controller/collection_bulk_post.go` | Bulk-POST per-row Enforcement (statisch + eval) |
| `internal/controller/collection_bulk_update.go` | Bulk-Update Enforcement (statisch + eval) |
### Structs
```go
// PermissionSet (collection.go)
type PermissionSet struct {
Methods map[string]MethodPermission
Filter dbcrud.M
ValidProjections []string
ReadonlyFields []string // Felder die für dieses PermSet readonly sind
HiddenFields []string // Felder die für dieses PermSet hidden sind
CompiledFilterSubs []*utils.CompiledSubst
}
// CollectionConfig (collection.go)
type CollectionConfig struct {
Name string
ReadonlyFields []string // Collection-Level readonly (Basis)
HiddenFields []string // Collection-Level hidden (Basis)
Fields []CollectionField
Permissions map[string]PermissionSet
// ...
}
// CollectionField (collection.go)
type CollectionField struct {
Name string
Type string
Readonly *BoolOrEval // Field-Def: true/false (statisch) oder {eval: "..."} (dynamisch)
Hidden *BoolOrEval // Field-Def: true/false (statisch) oder {eval: "..."} (dynamisch)
SubFields []CollectionField
// ...
}
// BoolOrEval (collection.go) — unterstützt bool oder eval-Ausdruck
type BoolOrEval struct {
Value *bool // statischer Wert (true/false)
Eval *string // JS-Ausdruck als String
JsProgram *JsProgram // vorkompiliertes goja-Programm (intern)
}
// FieldDef (utils/field_filter.go) — import-sichere Kopie für die Engine
type FieldDef struct {
Name string
SubFields []FieldDef
Readonly *bool
Hidden *bool
}
```
### Kernfunktionen
```go
// Kaskaden-Auflösung (utils/field_filter.go)
func ResolveEffectiveFields(
collectionLevel []string,
permSetLevel []string,
fields []FieldDef,
fieldDefFn func(f FieldDef) *bool,
) []string
// Prüfung ob geschützte Felder im Body vorhanden sind
func CheckFieldsPresent(doc map[string]interface{}, fields []string) []string
// Prüfung für Operator-Bodies ($set, $inc, etc.)
func CheckOperatorFieldsPresent(doc map[string]interface{}, fields []string) []string
// Felder aus Response-Dokument entfernen (für GET)
func StripFields(doc map[string]interface{}, fields []string)
// Feld-Definitionen aus Config entfernen (für enrichData)
func StripFieldDefsFromConfig(fields []FieldDef, hidden []string) []FieldDef
// Model-Wrapper (models/field_permissions.go)
func ResolveEffectiveReadonlyFields(collConfig, permSet, fields) []string
func ResolveEffectiveHiddenFields(collConfig, permSet, fields) []string
func CollectionFieldsToFieldDefs(fields []CollectionField) []FieldDef
func StripFieldsFromConfig(fields []CollectionField, hidden []string) []CollectionField
// Eval-Funktionen (models/field_permissions.go)
func EvalFieldReadonly(doc, collConfig, dbAPI, auth, method) ([]string, error)
func EvalFieldHidden(doc, collConfig, dbAPI, auth, method) ([]string, error)
func HasEvalFields(fields []CollectionField) bool
func HasEvalReadonlyFields(fields []CollectionField) bool
func HasEvalHiddenFields(fields []CollectionField) bool
// Vorkompilierung (models/api.go)
func compileFieldEvals(colName, fields, moduleCache, errs)
```
---
## Abwärtskompatibilität
- **Vollständig abwärtskompatibel**: Configs ohne `readonlyFields`/`hiddenFields` verhalten sich identisch
- `GetEffectivePermissionSet()` gibt für nicht vorkonfigurierte Felder leere Listen zurück → keine Checks
- `validProjections` arbeitet orthogonal und bleibt unberührt
- Admin-Bypass ist konsistent mit dem bestehenden Permission-System
---
## Tests
### Unit-Tests
| Datei | Tests | Abdeckung |
|-------|-------|-----------|
| `internal/utils/field_filter_test.go` | 22 | `ResolveEffectiveFields`, `CheckFieldsPresent`, `StripFields`, `StripFieldDefsFromConfig`, `CheckOperatorFieldsPresent` |
| `internal/models/field_permissions_test.go` | 22 | `CollectionFieldsToFieldDefs`, `ResolveEffectiveReadonlyFields`, `ResolveEffectiveHiddenFields`, `StripFieldsFromConfig` |
### Integration-Tests
| Datei | Tests | Abdeckung |
|-------|-------|-----------|
| `internal/controller/collection_field_permission_test.go` | 31 | Alle Endpoints, alle Rollen (statische Permissions) |
| `internal/controller/collection_field_eval_test.go` | 16 | Eval-basierte Permissions (readonly + hidden) |
**Test-Kategorien (statisch):**
| Kategorie | Tests | Was wird geprüft |
|-----------|-------|-----------------|
| POST readonly | 6 | User/Collection-Level/Field-Def/NeverReadonly/Admin-Bypass/Erlaubte Felder |
| POST hidden | 4 | User/Collection-Level/Field-Def/Admin-Bypass |
| PUT readonly | 3 | User-Rejection/Erlaubte Felder/Admin-Bypass |
| PUT Operator | 3 | $set readonly/$inc hidden/Erlaubte $inc |
| PUT hidden | 1 | User hidden field rejection |
| GET hidden | 5 | User-Strip/Admin-Vollzugriff/Public-Strip/SingleDoc/Readonly-sichtbar |
| Token | 4 | Readonly-Rejection/Erlaubte Felder/Hidden-Rejection/GET-Strip |
| yourPermissions | 2 | User (readonlyFields ja, hiddenFields nein) / Admin (hiddenFields-Hint) |
| Config-Delivery | 2 | User (hidden Feld-Defs entfernt) / Admin (alle sichtbar) |
| Mutation-Safety | 1 | Shared API nach User-Request nicht mutiert |
**Test-Config:** `testdata/api/testproject_fieldperm/collections/orders.yml`
**Test-Kategorien (Eval):**
| Kategorie | Tests | Was wird geprüft |
|-----------|-------|-----------------|
| POST eval readonly | 4 | Bedingung false=erlaubt, true=rejected, admin-bypass, ohne Feld=erlaubt |
| POST eval hidden | 2 | Bedingung false=erlaubt, true=rejected |
| POST statisch | 2 | Statisches readonly/hidden weiterhin korrekt |
| PUT eval readonly | 3 | Bedingung false=erlaubt, true=rejected, admin-bypass |
| GET eval hidden | 5 | Bedingung false=sichtbar, true=stripped, admin-sieht-alles, mixed-results, statisch-immer-stripped |
**Test-Config:** `testdata/api/testproject_fieldperm/collections/evalorders.yml`
src/content/docs/server/18-orgs-teams.md
+720
View File
@@ -0,0 +1,720 @@
---
title: 'Orgs & Teams'
---
# Orgs & Teams
## Overview
Tibi-Server uses a 3-level hierarchy for access control and data isolation: **Org → Team → User**. Orgs define the visibility boundary (which projects a user can see), while teams define working rights (what a user can do within a project's collections).
```text
┌─────────────────────────────────────────────────┐
│ Org │
│ (Visibility Boundary / Data Isolation) │
│ │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Team Alpha │ │ Team Beta │ │
│ │ (Working │ │ (Working │ │
│ │ Rights) │ │ Rights) │ │
│ │ │ │ │ │
│ │ 👤 User A │ │ 👤 User B │ │
│ │ 👤 User C │ │ 👤 User C │ ← multi │
│ └──────┬───────┘ └──────┬───────┘ team │
│ │ │ │
│ ▼ ▼ │
│ ┌──────────────────────────────┐ │
│ │ Project X │ │
│ │ teams: [Alpha, Beta] │ │
│ │ orgId: this-org │ │
│ └──────────────────────────────┘ │
└─────────────────────────────────────────────────┘
```
### Key Concepts
| Concept | Description |
|---------|-------------|
| **Org** | Isolation boundary. Org members can see all projects of their org. |
| **Team** | Working unit. Teams are assigned to projects and carry permission sets. |
| **OrgMembership** | Links a user to an org with a role (`admin` or `member`). |
| **TeamMembership** | Links a user to a team with a role (`admin` or `member`). |
| **Project.orgId** | The owning org of a project. |
| **Project.teams[]** | Which teams have working rights on this project. |
| **User.primaryOrgId** | The user's primary org (used for LLM budget billing). |
### Role Hierarchy
| Role | Scope | Description |
|------|-------|-------------|
| **SystemAdmin** (`role: 0`) | Global | Full access to everything. Bypasses all access control. |
| **OrgAdmin** (`orgs[].role: "admin"`) | Per Org | Can manage teams and users within their org(s). Can assign teams to projects. |
| **OrgMember** (`orgs[].role: "member"`) | Per Org | Can see projects of their org. Working rights come from team membership. |
| **TeamAdmin** (`teams[].role: "admin"`) | Per Team | Currently same as TeamMember. Reserved for future use. |
| **TeamMember** (`teams[].role: "member"`) | Per Team | Gets collection-level working rights when team is assigned to a project. |
## Data Model
### Org
```json
{
"id": "507f1f77bcf86cd799439011",
"name": "Acme Corp",
"llmBudgets": [
{
"provider": "openai",
"tokenBudget": 500000,
"tokensUsed": 123456,
"periodStart": "2026-01-01T00:00:00Z",
"periodEnd": "2026-04-01T00:00:00Z"
}
],
"meta": { "industry": "Tech" },
"insertTime": "2026-01-15T10:00:00Z",
"updateTime": "2026-01-15T10:00:00Z"
}
```
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | string | yes | Unique org name. Supports text search. |
| `llmBudgets` | LLMBudget[] | no | Org-level LLM token budgets (see [LLM Budget Cascade](#llm-budget-cascade)). |
| `meta` | object | no | Arbitrary metadata. |
**Collection name:** `org`
### Team
```json
{
"id": "507f1f77bcf86cd799439012",
"name": "Editors",
"orgId": "507f1f77bcf86cd799439011",
"permissions": ["editor", "reviewer"],
"meta": { "description": "Content editors" },
"insertTime": "2026-01-15T10:00:00Z",
"updateTime": "2026-01-15T10:00:00Z"
}
```
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | string | yes | Team name. Must be unique within the org. |
| `orgId` | string | yes | Reference to the parent org. Immutable after creation. |
| `permissions` | string[] | no | Named permission sets that this team grants (must match keys defined in collection configs). |
| `meta` | object | no | Arbitrary metadata. |
**Collection name:** `team`
**Composite identity:** `{name, orgId}` — team names are unique per org, not globally.
### User (extended fields)
The following fields have been added to the User model:
```json
{
"username": "alice",
"role": 1,
"orgs": [
{ "orgId": "507f1f77bcf86cd799439011", "role": "admin" },
{ "orgId": "507f1f77bcf86cd799439013", "role": "member" }
],
"teams": [
{ "teamId": "507f1f77bcf86cd799439012", "role": "member" },
{ "teamId": "507f1f77bcf86cd799439014", "role": "admin" }
],
"primaryOrgId": "507f1f77bcf86cd799439011"
}
```
| Field | Type | Description |
|-------|------|-------------|
| `orgs` | OrgMembership[] | Orgs the user belongs to, each with a role. |
| `teams` | TeamMembership[] | Teams the user belongs to, each with a role. |
| `primaryOrgId` | string | Primary org for LLM budget billing. Must reference an org in `orgs[]`. |
#### OrgMembership
| Field | Type | Values | Description |
|-------|------|--------|-------------|
| `orgId` | string | ObjectID | Reference to the org. |
| `role` | string | `"admin"`, `"member"` | User's role within this org. |
#### TeamMembership
| Field | Type | Values | Description |
|-------|------|--------|-------------|
| `teamId` | string | ObjectID | Reference to the team. |
| `role` | string | `"admin"`, `"member"` | User's role within this team. |
### Project (extended fields)
```json
{
"name": "website",
"namespace": "website_prod",
"orgId": "507f1f77bcf86cd799439011",
"teams": ["507f1f77bcf86cd799439012", "507f1f77bcf86cd799439014"]
}
```
| Field | Type | Description |
|-------|------|-------------|
| `orgId` | string | The owning org. Determines which users can see this project. |
| `teams` | string[] | Team IDs assigned to this project. Members of these teams get working rights. |
### JWT Token Claims (extended)
The JWT access token now includes org and team membership:
```json
{
"object": {
"id": "60a1b2c3d4...",
"username": "alice",
"role": 1,
"permissions": [],
"orgs": [
{ "orgId": "507f1f77bcf86cd799439011", "role": "admin" }
],
"teams": [
{ "teamId": "507f1f77bcf86cd799439012", "role": "member" }
],
"primaryOrgId": "507f1f77bcf86cd799439011"
},
"type": "User",
"exp": 1700000000
}
```
## API Endpoints
### Org CRUD
All org endpoints require authentication via `X-Auth-Token`.
#### List Orgs
```text
GET /api/v1/org
```
**Visibility:**
- SystemAdmin: sees all orgs
- Others: sees only orgs they are a member of
**Query parameters:** Standard collection query params (`_search`, `filter`, `sort`, `offset`, `limit`).
#### Create Org
```text
POST /api/v1/org
```
**Auth:** SystemAdmin only.
**Request:**
```json
{
"name": "Acme Corp",
"meta": { "industry": "Tech" }
}
```
#### Update Org
```text
PUT /api/v1/org/:id
```
**Auth:** SystemAdmin only.
#### Delete Org
```text
DELETE /api/v1/org/:id
```
**Auth:** SystemAdmin only.
**Cascade effects on delete:**
1. All teams within the org are deleted
2. Team references are removed from all projects (`teams[]`) and users (`teams[]`)
3. Projects are decoupled: `orgId` unset, `teams` cleared
4. Org membership is removed from all users (`orgs[]`)
---
### Team CRUD
#### List Teams
```text
GET /api/v1/team
```
**Visibility:**
- SystemAdmin: sees all teams
- OrgAdmin: sees all teams in orgs they admin, plus their own teams
- Others: sees only teams they are a member of
#### Create Team
```text
POST /api/v1/team
```
**Auth:** SystemAdmin or OrgAdmin of the team's org.
**Request:**
```json
{
"name": "Editors",
"orgId": "507f1f77bcf86cd799439011",
"permissions": ["editor"]
}
```
**Validation:**
- `orgId` is required and must reference an existing org
- `orgId` is **immutable** after creation
- Team name must be unique within the org
#### Update Team
```text
PUT /api/v1/team/:id
```
**Auth:** SystemAdmin or OrgAdmin of the team's org.
**Note:** `orgId` cannot be changed after creation.
#### Delete Team
```text
DELETE /api/v1/team/:id
```
**Auth:** SystemAdmin or OrgAdmin of the team's org.
**Cascade effects on delete:**
1. Team reference is removed from all projects (`teams[]`)
2. Team membership is removed from all users (`teams[]`)
---
### User Management (org/team-scoped)
#### Assign User to Org
```text
PUT /api/v1/user/:id
```
Add or modify org membership:
```json
{
"orgs": [
{ "orgId": "507f1f77bcf86cd799439011", "role": "admin" },
{ "orgId": "507f1f77bcf86cd799439013", "role": "member" }
]
}
```
**Auth by role:**
| Actor | Can Assign | Restrictions |
|-------|-----------|-------------|
| SystemAdmin | Any org, any role | None |
| OrgAdmin | Orgs they admin | Cannot modify entries for orgs outside their scope. Existing out-of-scope entries must remain unchanged. |
| Others | — | Cannot modify `orgs`. |
#### Assign User to Team
```text
PUT /api/v1/user/:id
```
```json
{
"teams": [
{ "teamId": "507f1f77bcf86cd799439012", "role": "member" }
]
}
```
**Auth:** Same rules as org assignment. OrgAdmin can only assign teams that belong to orgs they admin.
#### Set Primary Org
```text
PUT /api/v1/user/:id
```
```json
{
"primaryOrgId": "507f1f77bcf86cd799439011"
}
```
**Validation:** The referenced org must be in the user's `orgs[]` array. Can be set to `null` to clear.
#### User Visibility
- SystemAdmin: sees all users
- Org members: see all users that share at least one org (`orgs.orgId: {$in: [myOrgIDs]}`)
- No orgs: can only see own profile
---
### Project Team Assignment
```text
PUT /api/v1/project/:id
```
```json
{
"teams": ["507f1f77bcf86cd799439012", "507f1f77bcf86cd799439014"]
}
```
**Auth:**
- SystemAdmin: can set `orgId`, `teams`, and all other fields
- OrgAdmin: can only modify `teams[]`, and only with teams that belong to the project's org
**Validation:** All team IDs in `teams[]` must reference teams that belong to the same org as the project (`project.orgId`).
## Permission System
### How Team Permissions Map to Collections
Teams carry a `permissions[]` array of named permission set keys. These keys correspond to permission sets defined in collection configs:
**Collection config (YAML):**
```yaml
permissions:
public:
methods:
GET: { allow: true }
user:
methods:
GET: { allow: true }
POST: { allow: true }
editor:
methods:
GET: { allow: true }
POST: { allow: true }
PUT: { allow: true }
DELETE: { allow: true }
```
**Team definition:**
```json
{
"name": "Content Editors",
"orgId": "...",
"permissions": ["editor"]
}
```
When a user who is a member of the "Content Editors" team accesses a collection, they get the `editor` permission set — allowing GET, POST, PUT, and DELETE.
### Permission Resolution Priority
When determining access for a collection request, the following priority chain is evaluated top-to-bottom. The first match wins:
```text
1. Admin Token (namespace-level)
└─ Checked via X-Admin-Token header with namespace.*.collection.* permissions
2. Token Permission (query/header)
└─ Checked via ?token= or X-Collection-Token header
3. Public Permission Set
└─ The "public" key in the collection's permissions config
4. SystemAdmin (role=0)
└─ Always allowed — bypasses all checks
5. Team "user" Permission
└─ If user is a member of a team assigned to this project
AND the collection has a "user" permission set → apply it
6. Custom User Permissions
└─ User's auth.permissions[] matched against "namespace:permKey"
7. Custom Team Permissions (NEW)
└─ Team's permissions[] matched against collection permission keys
(e.g. team has ["editor"] and collection has an "editor" set)
8. Denied
```
### Effective Permission Set Selection
For field-level access control and projections, the system selects ONE effective permission set:
| Priority | Condition | Selected Set |
|----------|-----------|-------------|
| 1 | Admin token or SystemAdmin | `nil` (no restrictions) |
| 2 | Token permission matches | Token's permission set |
| 3 | User custom permission matches `namespace:key` | That custom permission set |
| 4 | Team permission key matches collection key | That team permission set |
| 5 | User is team member | `"user"` permission set |
| 6 | Fallback | `"public"` permission set |
### Permission Union from Multiple Teams
When a user is a member of multiple teams assigned to the same project, the permissions are **merged (union)**:
```text
Team Alpha: permissions=["editor"]
Team Beta: permissions=["reviewer"]
User in both → permissions checked: ["editor", "reviewer"]
```
If either team's permission set allows an operation, access is granted.
## Visibility Rules
### Project Visibility
```text
SystemAdmin (role=0) → sees ALL projects
User (role=1) → sees projects where project.orgId ∈ user's org IDs
No org membership → sees NO projects
```
Project visibility is **org-scoped**, not team-scoped. Being a member of an org grants visibility to all projects of that org, regardless of team assignment. Team assignment grants working rights, not visibility.
### Org Visibility
```text
SystemAdmin → sees ALL orgs
Others → sees only orgs they are a member of
```
### Team Visibility
```text
SystemAdmin → sees ALL teams
OrgAdmin → sees all teams in orgs they admin + their own teams
Others → sees only teams they are a member of
```
### User Visibility
```text
SystemAdmin → sees ALL users
Org members → sees users sharing at least one org
No orgs → sees only own profile
```
## Audit Visibility
Audit log access uses the team→project→collection chain (not org-based):
1. System collects all team IDs from the user's JWT
2. Finds all projects where `teams[]` overlaps with the user's teams
3. For each project, checks GET permission per collection
4. Builds a filter of allowed `{projectNamespace, collection}` pairs
5. Non-admin users **never** see `source.type: "system"` entries (login, reload, shutdown events)
```text
SystemAdmin → sees ALL audit entries (including system events)
Team member → sees audit entries for collections they have GET access to
No teams → sees NO audit entries
```
## LLM Budget Cascade
The LLM token budget system uses a two-level cascade: **User → Org**.
### Budget Check Flow
```text
1. Read user.llmBudgets[provider] ← REQUIRED (hard check)
└─ No budget → 429 Too Many Requests
└─ Period expired → 429
└─ Tokens exhausted → 429
2. Read org.llmBudgets[provider] ← OPTIONAL (soft check)
└─ via user.primaryOrgId
└─ No primaryOrgId → skip org check
└─ Org has no budget for provider → no restriction
└─ Org budget expired → 429
└─ Org tokens exhausted → 429
3. Execute LLM request
4. Record usage:
└─ Deduct from user.llmBudgets
└─ Deduct from org.llmBudgets (if applicable)
└─ Write LLMUsageLog with userId AND orgId
```
### Configuration Example
**Org with budget:**
```json
{
"name": "Acme Corp",
"llmBudgets": [{
"provider": "openai",
"tokenBudget": 1000000,
"tokensUsed": 0,
"periodStart": "2026-01-01T00:00:00Z",
"periodEnd": "2026-04-01T00:00:00Z"
}]
}
```
**User with personal budget and primary org:**
```json
{
"username": "alice",
"primaryOrgId": "acme-org-id",
"llmBudgets": [{
"provider": "openai",
"tokenBudget": 50000,
"tokensUsed": 0,
"periodStart": "2026-01-01T00:00:00Z",
"periodEnd": "2026-04-01T00:00:00Z"
}]
}
```
When Alice makes an LLM request:
- Her personal budget is checked first (hard check)
- Then Acme Corp's org budget is checked (soft check via `primaryOrgId`)
- Usage is deducted from both budgets
- The usage log entry includes both `userId` and `orgId`
### Shared Budget Scenario
Multiple users can share an org's budget:
```text
Org "Acme": tokenBudget=1000, tokensUsed=985
Alice (primaryOrgId=Acme): personal budget OK
Bob (primaryOrgId=Acme): personal budget OK
Alice sends request → 15 tokens used → Org tokensUsed becomes 1000
Bob sends request → 429 (org budget exhausted)
Alice sends request → 429 (org budget exhausted)
```
### Team Membership is Irrelevant for Billing
LLM billing goes through `user.primaryOrgId`, not through team membership. A user in multiple orgs/teams is always billed to their primary org:
```text
User in OrgA (primary) + OrgB → billing goes to OrgA only
```
## Cascade Consistency
### Org Deletion Cascade
When an org is deleted, the following cleanup occurs automatically:
1. **Teams**: All teams with `orgId == deletedOrg` are deleted
2. **Projects**: `orgId` is unset, `teams[]` is cleared for all affected projects
3. **Users**: Org membership removed from `orgs[]`, plus:
- Teams belonging to the deleted org are removed from `teams[]`
- `primaryOrgId` is cleared if it referenced the deleted org
### Team Deletion Cascade
When a team is deleted:
1. **Projects**: Team ID removed from `teams[]`
2. **Users**: Team membership removed from `teams[]`
### User Org Removal Consistency
When a user's `orgs[]` is modified and an org is removed:
1. `primaryOrgId` is cleared if it referenced the removed org
2. Teams belonging to the removed org are automatically removed from `teams[]`
## Access Control Summary
### Org Endpoints
| Action | SystemAdmin | OrgAdmin | Member | Unauthenticated |
|--------|-------------|----------|--------|-----------------|
| GET (list) | All | Own orgs | Own orgs | — |
| POST | ✅ | — | — | — |
| PUT | ✅ | — | — | — |
| DELETE | ✅ | — | — | — |
### Team Endpoints
| Action | SystemAdmin | OrgAdmin | Member | Unauthenticated |
|--------|-------------|----------|--------|-----------------|
| GET (list) | All | Org teams + own | Own teams | — |
| POST | ✅ | Own org teams | — | — |
| PUT | ✅ | Own org teams | — | — |
| DELETE | ✅ | Own org teams | — | — |
### User Endpoints (org/team-related)
| Action | SystemAdmin | OrgAdmin | TeamMember |
|--------|-------------|----------|------------|
| GET (list) | All | Shared-org users | Shared-org users |
| POST (create) | ✅ | ✅ (scoped) | — |
| PUT (modify orgs/teams) | ✅ | Own scope only | Own profile only |
| DELETE | ✅ | — | — |
### Project Endpoints
| Action | SystemAdmin | OrgAdmin | Member |
|--------|-------------|----------|--------|
| GET (list) | All | Org projects | Org projects |
| POST | ✅ | — | — |
| PUT (teams) | ✅ | ✅ (own org teams) | — |
| PUT (orgId, name, ...) | ✅ | — | — |
| DELETE | ✅ | — | — |
## Field Protection
When a non-SystemAdmin modifies a user, the following fields are protected:
| Actor | Protected Fields |
|-------|-----------------|
| **OrgAdmin** | `permissions`, `llmBudgets` — cannot modify. `orgs`, `teams`, `primaryOrgId` — scoped to own orgs. |
| **TeamAdmin/Member** | `permissions`, `llmBudgets`, `orgs`, `teams`, `role`, `primaryOrgId` — all protected. Can only modify own profile's basic fields. |
### OrgAdmin Scope Enforcement
When an OrgAdmin modifies a user's `orgs[]` or `teams[]`:
- **In-scope entries** (orgs the admin manages): can be added, removed, or role-changed
- **Out-of-scope entries** (other orgs): must remain unchanged — same number of entries, same roles
- Violation → `403 Forbidden`
Example: OrgAdmin of OrgA edits a user who is in OrgA and OrgB:
```json
// Before:
{ "orgs": [{"orgId": "A", "role": "member"}, {"orgId": "B", "role": "admin"}] }
// Allowed (only OrgA changed):
{ "orgs": [{"orgId": "A", "role": "admin"}, {"orgId": "B", "role": "admin"}] }
// Forbidden (OrgB modified):
{ "orgs": [{"orgId": "A", "role": "member"}, {"orgId": "B", "role": "member"}] }
```
src/content/docs/server/19-actions.md
+315
View File
@@ -0,0 +1,315 @@
---
title: 'Actions'
---
# Actions
## Übersicht
Actions sind Hook-basierte Endpunkte ohne eine zugrunde liegende Datenbank-Collection. Sie eignen sich für Logik, die nicht an ein CRUD-Datenmodell gebunden ist, z.B. Kontaktformulare, Newsletter-Anmeldungen, Webhook-Empfänger oder benutzerdefinierte API-Endpunkte.
Actions verwenden dasselbe Permission-, Hook-, CORS- und Audit-System wie Collections, haben aber **keine** Fields, Indexes, Uploads, Projections, Search oder Bulk-Operationen.
## Konfiguration
Actions werden im Projekt-Config unter dem `actions:`-Block definiert, analog zu `collections:`:
```yaml
# config.yml
namespace: myproject
collections:
- !include collections/events.yml
actions:
- !include actions/contact-form.yml
- !include actions/newsletter-signup.yml
```
### Action-Config
```yaml
# actions/contact-form.yml
name: contact-form
meta:
label: { de: "Kontaktformular", en: "Contact Form" }
description: { de: "Verarbeitet Kontaktanfragen", en: "Processes contact requests" }
permissions:
public:
methods:
post: true
user:
methods:
post: true
get: true
hooks:
post:
bind:
type: javascript
file: hooks/contact-form/post_bind.js
validate:
type: javascript
file: hooks/contact-form/post_validate.js
handle:
type: javascript
file: hooks/contact-form/post_handle.js
return:
type: javascript
file: hooks/contact-form/post_return.js
get:
handle:
type: javascript
file: hooks/contact-form/get_handle.js
cors:
allowOrigins: ["*"]
```
### Felder
| Feld | Typ | Beschreibung |
|------|-----|-------------|
| `name` | string | Eindeutiger Name der Action (URL-Pfad-Segment) |
| `meta` | object | Metadaten wie Label, Description für Admin-UI |
| `permissions` | object | Identisch zu Collection-Permissions (public/user/role-basiert) |
| `hooks` | object | Hook-Definitionen pro HTTP-Methode und Step |
| `cors` | object | Optionale CORS-Konfiguration (überschreibt Projekt-CORS) |
| `audit` | object | Optionale Audit-Konfiguration (siehe Audit-Abschnitt) |
## Admin-UI-Meta (tibi-admin-nova)
Der Server liefert `meta` unverändert als JSON an den Admin aus (Passthrough — keine Validierung, kein Stripping, keine serverseitige Auswertung). Alles unter `meta.methods` ist reine UI-Konvention für Nova; der Hook entscheidet allein über das Verhalten, Nova erkennt es an der Response. Guards: Nova zeigt Sektion und Features nur bei vorhandener Meta (defensiv gegen alte Server ohne Meta).
```yaml
# actions/import-snapshot.yml (Auszug)
meta:
label: { de: "Snapshot importieren", en: "Import Snapshot" }
muiIcon: file-upload-outline
description: { de: "Importiert eine CSV-Datei als Testlauf.", en: "Imports a CSV file as a test run." }
group: system
methods:
post:
data:
fields:
- name: file
type: file
validator:
required: true
accept: [".csv", "text/csv"]
maxFileSize: "2KB"
execution:
kind: sync # nur dokumentativ
output:
mapping:
summary:
fields:
- source: result.imported
type: number
```
Regeln:
- `query.fields` / `data.fields`: Collection-Feldsprache (`type`, `label`, `validator`, `dependsOn` …). Nova rendert über dieselben Widgets und validiert clientseitig (`required`, `accept`, `maxFileSize`, Datei-Hinweise aus `accept`). Serverseitig gilt nichts automatisch — wer serverseitig härten will, schreibt einen `validate`-Hook in JS.
- Datei-Inputs immer inline als Base64/Data-URI im JSON-Body (Nova-Primärpfad, kein Multipart).
- SSE/Progress: Der Hook streamt bei Bedarf auf derselben Route (`context.response.header("Content-Type", "text/event-stream")`, `writeSSE`, `flush`); Nova erkennt `text/event-stream` per Content-Type und zeigt Fortschritt (`output.mapping.progress` mit `statusField`/`messageField`/`indexField`/`countField`, Progress-Text nur aus explizit konfigurierten Status-/Message-Feldern). `execution.kind` (`sync`/`sse`) ist nur dokumentativ, Transport entscheidet der Hook.
- `output.mapping.summary`/`details`: kompakte Ergebnisdarstellung über `source`-Pfade (dot-Notation, `name` nur Legacy-Fallback); ohne Mapping zeigt Nova Raw-JSON.
- Audit-Verlauf: Einträge liegen unter Collection `_actions/<action-name>` und sind in Nova per Deep-Link (`/projects/:id/audit?collection=_actions/<name>`) erreichbar.
## Routing
Actions sind unter folgendem Pfad erreichbar:
```
POST /api/v1/_/:project/_actions/:action
GET /api/v1/_/:project/_actions/:action
```
Der `_actions`-Prefix verhindert Namenskonflikte mit Collections.
## Hook-Steps
### POST
| Step | Beschreibung |
|------|-------------|
| `bind` | Liest und transformiert den Request-Body. Kann Eingabedaten in `context` ablegen. |
| `validate` | Prüft die Eingabedaten. Wirft bei Fehlern eine Exception (HTTP 400). |
| `handle` | Führt die eigentliche Business-Logik aus. Kein DB-Roundtrip — alles passiert im Hook. |
| `return` | Transformiert die Ausgabe vor dem Senden der Response. |
Ausführungsreihenfolge: `bind` → `validate` → `handle` → `return`
### GET
| Step | Beschreibung |
|------|-------------|
| `handle` | Erzeugt die Antwortdaten. |
| `return` | Transformiert die Ausgabe vor dem Senden der Response. |
Ausführungsreihenfolge: `handle` → `return`
## Hook-Context
Action-Hooks haben Zugriff auf denselben Context wie Collection-Hooks, **ohne** die collection-/document-spezifischen Objekte:
**Verfügbar:**
- `context.request()` — Request-Daten (body, query, headers, params)
- `context.data` — geparster JSON-Body (bei POST), identisch zum Collection-Verhalten
- `api.*` — Zugriff auf andere Collections via `api.find()`, `api.create()`, etc.
- `http`, `smtp`, `jwt`, `bcrypt`, `crypto`, `cache`, `ratelimit` — alle Standard-Packages
**Nicht verfügbar:**
- `context.collection` — keine Collection vorhanden
- `context.document` — kein Dokument aus der DB
## Response-Verhalten
Die Response einer Action wird wie folgt bestimmt:
1. Wenn der letzte Hook (z.B. `return` oder `handle`) ein Objekt mit `data`-Key zurückgibt, wird `data` als Response verwendet.
2. Wenn ein Objekt ohne `data`-Key zurückgegeben wird, wird das gesamte Objekt als Response verwendet.
3. Wenn kein Hook eine Response liefert, wird `{"ok": true}` zurückgegeben.
### Beispiel: Response mit `data`-Key
```javascript
// hooks/contact-form/post_handle.js
;(function () {
var body = JSON.parse(context.request().body())
// E-Mail senden via smtp Package
context.smtp.sendMail({
to: "info@example.com",
subject: "Kontaktanfrage von " + body.name,
textBody: body.message
})
return { data: { sent: true, message: "Danke für Ihre Nachricht" } }
})()
```
Response: `{"sent": true, "message": "Danke für Ihre Nachricht"}`
## Permissions
Permissions funktionieren identisch zu Collections:
```yaml
permissions:
public:
methods:
post: true # Jeder kann POST aufrufen
get: false # GET nur für eingeloggte User
user:
methods:
post: true
get: true
admin:
methods:
post: true
get: true
```
Rollen, Filter und `$subst`-Expressions werden genauso unterstützt wie bei Collections.
## CORS
CORS-Konfiguration auf Action-Ebene überschreibt die Projekt-CORS-Konfiguration:
```yaml
cors:
allowOrigins: ["https://www.example.com"]
allowMethods: ["POST"]
allowHeaders: ["Content-Type", "X-Auth-Token"]
```
## Migration von Collection-Hacks
**Vorher** (Collection ohne Fields als Action-Workaround):
```yaml
collections:
- name: contact-form
strictFields: false
fields: []
permissions:
public:
methods:
post: true
hooks:
post:
create:
type: javascript
file: hooks/contact-form/handle.js
```
**Nachher** (Native Action):
```yaml
actions:
- name: contact-form
permissions:
public:
methods:
post: true
hooks:
post:
handle:
type: javascript
file: hooks/contact-form/handle.js
```
Vorteile:
- Keine unnötige DB-Collection
- Kein `fields: []`-Hack
- Eigener Hook-Step `handle` statt `create`
- Klarere Semantik und Trennung
## Audit
Actions unterstützen das gleiche Audit-System wie Collections. Audit-Logs werden unter der Collection `_actions/<action-name>` gespeichert.
### Konfiguration
```yaml
# actions/contact-form.yml
name: contact-form
audit:
enabled: true
# Optional: nur bestimmte Methoden auditieren (Standard: ["post", "get"])
actions:
- post
```
Die Konfiguration ist identisch zu `AuditCollectionConfig` bei Collections:
| Feld | Typ | Default | Beschreibung |
|------|-----|---------|-------------|
| `enabled` | bool | `false` | Audit für diese Action aktivieren |
| `actions` | []string | `["post", "get"]` | Welche HTTP-Methoden auditiert werden |
> **Hinweis:** Audit muss zusätzlich global aktiviert sein (`AUDIT_ENABLED=true` oder `audit.enabled: true` in der Server-Config).
### Audit-Einträge
Audit-Einträge für Actions enthalten:
| Feld | Beschreibung |
|------|-------------|
| `collection` | `_actions/<action-name>` (z.B. `_actions/contact-form`) |
| `action` | HTTP-Methode (`post` oder `get`) |
| `snapshot` | JSON-safe Kopie der Hook-Daten (bei POST) |
| `userId`, `username`, `userRole` | Auth-Informationen (falls vorhanden) |
| `timestamp` | Zeitpunkt der Aktion |
### Audit-Abfrage
Audit-Logs für Actions können über den Standard-Audit-Endpunkt abgefragt werden:
```
GET /api/v1/_/:project/audit?filter={"collection":"_actions/contact-form"}
```
src/content/docs/server/20-referenced-read-safety.md
+75
View File
@@ -0,0 +1,75 @@
---
title: 'Referenced Read Safety'
---
# Referenced Read Safety
## Problem
Features that read from a second collection during a request can silently bypass parts of the target collection's normal GET visibility model if they only check collection-level `get` permission.
Typical examples are:
- lookup-like relation expansion
- reverse aggregates / derived counts and sums
- future facet, join, or enrichment features over referenced collections
The dangerous gap is not only collection access itself, but also the field-level read model of the referenced collection.
That includes at least:
- permission-based pre-filters
- valid projections
- static hidden fields
- eval-based hidden fields
- read and return hooks
- user-supplied target filters that reference unreadable fields
- derived fields such as aggregate `valueField`
## Project Invariant
Any feature that reads or derives data from a referenced collection must go through the shared helper in [internal/controller/collection_reference_access.go](../internal/controller/collection_reference_access.go).
Today that shared path is used by:
- lookup target resolution
- aggregate target resolution
The helper centralizes:
- target collection existence and `get` permission checks
- effective pre-filter resolution
- projection validation / default projection selection for referenced reads
- hidden-field and projection-based field readability checks
- eval-hidden blocking for derived field access
- unreadable target field detection inside aggregate-style filters
- readable target join-field checks for lookup-style `foreignField` access
## Practical Rule
If a new feature touches a referenced collection, do not read target documents directly after `MethodIsAllowed` and `getCollectionPreFilter`.
Instead:
1. resolve referenced read access through the shared helper
2. validate any requested target field against that access model
3. validate any user-controlled target filter against readable fields
4. only then run target read hooks, DB reads, and return hooks
For lookup specifically:
- `foreignField` is a target read surface and must be readable for the caller
- static hidden fields may be pushed down into the referenced projection when that projection can express them safely
- eval-hidden stays a runtime sanitization concern and must not be treated as projection-only safe
## Why This Matters
Without this invariant, a feature can leak information even when the caller never receives the raw target document.
Examples:
- summing a hidden numeric field
- counting only documents where a hidden target field matches a secret condition
- using a target field blocked by projections as an aggregate input
Those are still data leaks, even if the endpoint only returns `_aggregate` values.
src/content/docs/server/21-backup-restore.md
+534
View File
@@ -0,0 +1,534 @@
---
title: 'Backup & Restore'
---
# Backup & Restore
## Overview
Die Backup & Restore-API ermöglicht das Exportieren eines kompletten Projekts (Collections + Media)
als ZIP-Datei sowie den Import auf dem gleichen oder einem anderen Server.
**Nur Admin-User** können diese Endpunkte nutzen (Server-Token, Projekt-Token mit `allowBackup`,
oder JWT mit AdminRole).
---
## Authentifizierung
Dreistufiges Auth-Modell (identisch mit anderen Admin-Endpunkten wie `reload`):
### 1. Server-Level Admin Token
```yaml
api:
adminTokens:
- token: "backup-agent-token"
label: "ci-backup"
permissions:
- project.backup # Nur Backup/Restore
# - project.* # Alle Projekt-Operationen inkl. Backup
```
Gesendet via `X-Admin-Token` Header oder `Authorization: Bearer <token>`.
### 2. Projekt-Level Admin Token
```yaml
# In der Projekt-Config (config.yml)
admin:
tokens:
- token: "ADMINTOKEN"
allowReload: true
allowRegenerateSearch: true
allowBackup: true # NEU: erlaubt Backup/Restore
```
Das Feld `allowBackup` steuert gezielt, ob dieser Token Backup/Restore verwenden darf.
### 3. JWT Login mit AdminRole
User mit `role: 0` (AdminRole) können Backup/Restore ausführen.
---
## Endpunkte
| Methode | Pfad | Beschreibung |
|--------|------|-------------|
| **GET** | `/_/:project/_/admin/backup` | Backup als ZIP herunterladen |
| **GET** | `/_/:project/_/admin/backup/download/:file` | Temp-ZIP aus SSE-Download abholen |
| **POST** | `/_/:project/_/admin/backup` | ZIP hochladen & analysieren → `restoreToken` (SSE via `?sse=1`) |
| **POST** | `/_/:project/_/admin/backup/:restoreToken` | Wiederherstellung ausführen |
| **DELETE** | `/_/:project/_/admin/backup/:restoreToken` | Temp-Daten manuell aufräumen |
---
## 1. Backup Download
```text
GET /_/:project/_/admin/backup?collections=contacts,pages&sse=1
```
| Query-Param | Default | Beschreibung |
|-------------|---------|-------------|
| `collections` | _(alle)_ | Komma-separierte Collection-Namen (selektiver Export) |
| `sse` | `0` | `1` = Fortschritt als SSE-Stream, `0` = direktes ZIP |
### Direkter Download (`sse=0`)
Response: `Content-Type: application/zip` mit folgender Struktur:
```
backup-<namespace>-<timestamp>.zip
├── backup-info.json
├── collections/
│ ├── <collection>.json # Array aller Dokumente
│ └── ...
└── media/ # Nur Dateien im konfigurierten uploadPath
└── <collection>/
└── <docId>/
└── <fieldName>/
└── <filename>
```
**Wichtig:** Es werden NUR Media-Dateien exportiert, die tatsächlich im konfigurierten
`uploadPath` der Collection existieren. Fehlende Dateien werden übersprungen und geloggt.
Die Datei `backup-info.json` enthält Metadaten, die beim Import zur Schema-Validierung
verwendet werden:
```json
{
"type": "tibi-backup",
"version": 1,
"sourceNamespace": "mein-projekt",
"createdAt": "2026-06-24T12:00:00Z",
"collections": {
"contacts": {
"count": 42,
"fields": ["email", "firstname", "lastname", "phone"]
}
}
}
```
### SSE-Modus (`sse=1`)
Response-Type: `text/event-stream` mit folgenden Events:
```
event: progress
data: {"phase":"init","totalCollections":3}
event: result
data: {"phase":"done","downloadUrl":"/_/:project/_/admin/backup/download/<token>","size":52428800,"expiresAt":"..."}
```
Der Client erhält eine temporäre Download-URL (TTL: 5 Minuten) und ruft die ZIP
anschließend separat ab:
```bash
# Schritt 1: SSE-Download initiieren
curl -H "X-Admin-Token: $TOKEN" \
"http://localhost:8080/api/v1/_/myproject/_/admin/backup?sse=1"
# Schritt 2: ZIP vom temp-Link abholen
curl -H "X-Admin-Token: $TOKEN" \
-o backup.zip \
"http://localhost:8080/api/v1/_/myproject/_/admin/backup/download/<token>"
```
Der Download-Link ist **One-Time-Use** — ein zweiter Request erhält 404.
---
## 2. Backup Upload & Analyse (Phase 1)
```text
POST /_/:project/_/admin/backup
Content-Type: multipart/form-data
```
**Body:** Feld `file` = backup.zip
| Query-Param | Default | Beschreibung |
|-------------|---------|-------------|
| `sse` | `0` | `1` = Fortschritt via SSE streamen |
### Direkter Modus (`sse=0`)
**Response (200):**
```json
{
"restoreToken": "a1b2c3d4e5f6...",
"sourceNamespace": "quelle-projekt",
"createdAt": "2026-06-24T12:00:00Z",
"expiresAt": "2026-06-24T13:00:00Z",
"collections": {
"contacts": {
"documentCount": 42,
"mediaFileCount": 5,
"backupFields": ["email","firstname","lastname","phone"],
"targetFields": ["email","firstname","lastname","phone","department"],
"warnings": [
"target has new fields not in backup: [department]"
],
"exists": true
},
"legacy_old": {
"documentCount": 7,
"mediaFileCount": 0,
"backupFields": ["data"],
"targetFields": null,
"warnings": ["collection \"legacy_old\" does not exist in target project"],
"exists": false
}
}
}
```
**In dieser Phase werden KEINE Daten geschrieben.** Das ZIP wird nur entpackt und analysiert.
Der `restoreToken` referenziert das Temp-Verzeichnis für die nächste Phase.
### SSE-Modus (`sse=1`)
Response-Type: `text/event-stream` mit folgenden Events:
```
event: progress
data: {"phase":"init","restoreToken":"a1b2c3d4e5f6..."}
event: progress
data: {"phase":"save"}
event: progress
data: {"phase":"extract"}
event: progress
data: {"phase":"validate"}
event: progress
data: {"phase":"analyze","totalCollections":2}
event: result
data: {"phase":"done","restoreToken":"a1b2c3d4e5f6...","sourceNamespace":"quelle-projekt","createdAt":"2026-06-24T12:00:00Z","expiresAt":"2026-06-24T13:00:00Z","collections":{...}}
```
### Schema-Kompatibilitäts-Prüfung
Die Prüfung ist **tolerant** — sie warnt, blockiert aber nicht:
| Situation | Verhalten |
|-----------|-----------|
| Backup & Ziel identisch | Kein Warning |
| Backup hat Felder, Ziel nicht | Warning (MongoDB speichert sie trotzdem) |
| Ziel hat Felder, Backup nicht | Warning (Dokumente bekommen null/leer) |
| Collection existiert nicht im Ziel | `exists: false`, wird beim Restore geskippt |
---
## 3. Restore ausführen (Phase 2)
```text
POST /_/:project/_/admin/backup/:restoreToken?collections=contacts&merge=true&sse=1
```
| Query-Param | Default | Beschreibung |
|-------------|---------|-------------|
| `collections` | _(alle)_ | Welche Collections aus dem Backup importieren |
| `merge` | `false` | `true` = Dokumente updaten, Orphans behalten; `false` = exakter 1:1-Abgleich (Orphans löschen) |
| `sse` | `0` | `1` = Fortschritt via SSE streamen |
### Direkter Restore (`sse=0`)
**Response (200):**
```json
{
"restoredCollections": ["contacts"],
"skippedCollections": ["legacy_old"],
"totalDocuments": 42,
"mediaFiles": 5
}
```
### Restore-Modi
**`merge=false` (Default) — Exakter 1:1-Abgleich:**
- Jedes Dokument aus dem Backup wird via atomarem Upsert (`ReplaceOne`) eingefügt oder ersetzt
- Nach dem Import werden alle Dokumente gelöscht, die vor dem Restore existierten, aber **nicht** im Backup enthalten sind (Orphans)
- Ergebnis: Die Collection ist eine exakte 1:1-Kopie des Backups — jedes Feld, jeder BSON-Typ
- Kein Zeitfenster, in dem die Collection leer ist (kein Pre-Delete mehr)
- Bricht der Import ab, sind bereits verarbeitete Docs korrekt, unverarbeitete unangetastet
**`merge=true` — Überschreiben, Orphans behalten:**
- Existierende Dokumente mit gleicher `_id` werden **ersetzt** (atomares Upsert)
- Neue `_id`s werden **angelegt**
- Dokumente im Ziel, die nicht im Backup sind, bleiben **erhalten**
- Geeignet für partielle Updates / Migration einzelner Collections
### SSE-Modus (`sse=1`)
```
event: progress
data: {"phase":"init"}
event: progress
data: {"phase":"collection","collection":"contacts","action":"importing"}
event: result
data: {"restoredCollections":["contacts"],"skippedCollections":[],"totalDocuments":42,"mediaFiles":5,"phase":"done"}
```
---
## 4. Temp-Daten aufräumen
```text
DELETE /_/:project/_/admin/backup/:restoreToken
```
Explizites Cleanup des Temp-Verzeichnisses. Antwort: `{"message":"ok"}`.
**Automatisches Cleanup:** Ein Background-Job (5-Minuten-Takt) löscht Temp-Verzeichnisse,
deren TTL abgelaufen ist:
- **Restore-Temp-Dirs**: 1 Stunde TTL
- **Download-ZIPs**: 5 Minuten TTL
---
## Typischer Workflow
### Szenario A: Projekt von Staging nach Production kopieren
```bash
# 1. Auf Staging: Backup herunterladen
curl -H "X-Admin-Token: $STAGING_TOKEN" \
-o staging-backup.zip \
"https://staging.example.com/api/v1/_/myproject/_/admin/backup"
# 2. Auf Production: Backup hochladen & analysieren
curl -H "X-Admin-Token: $PROD_TOKEN" \
-F "file=@staging-backup.zip" \
"https://prod.example.com/api/v1/_/myproject/_/admin/backup"
# → {"restoreToken":"abc...","collections":{...}}
# 3. Prüfung der Analyse-Warnings, dann Restore ausführen
curl -H "X-Admin-Token: $PROD_TOKEN" \
-X POST \
"https://prod.example.com/api/v1/_/myproject/_/admin/backup/abc..."
# → {"restoredCollections":["contacts","pages"],"totalDocuments":84,"mediaFiles":12}
```
### Szenario B: Nur eine Collection aktualisieren (Selective Restore)
```bash
# 1. Full Backup herunterladen (enthält alle Collections)
curl -H "X-Admin-Token: $TOKEN" -o full-backup.zip "..."
# 2. Upload & Analyse
RESTORE_TOKEN=$(curl -H "X-Admin-Token: $TOKEN" -F "file=@full-backup.zip" "..." | jq -r .restoreToken)
# 3. Nur "contacts" wiederherstellen
curl -H "X-Admin-Token: $TOKEN" -X POST \
"http://.../_/myproject/_/admin/backup/$RESTORE_TOKEN?collections=contacts"
```
### Szenario C: Mit Merge updaten (bestehende Daten erhalten)
```bash
# merge=true: Backup-Docs ersetzen bestehende, neue hinzufügen,
# aber Docs die nur im Ziel existieren bleiben erhalten
curl -H "X-Admin-Token: $TOKEN" -X POST \
"http://.../_/myproject/_/admin/backup/$TOKEN?merge=true"
```
### Szenario D: Mit SSE-Fortschritt (große Projekte)
```bash
# Download mit Fortschrittsanzeige
curl -H "X-Admin-Token: $TOKEN" -N \
"http://.../_/myproject/_/admin/backup?sse=1"
# → event:progress + event:result mit downloadUrl
# Restore mit Fortschrittsanzeige
curl -H "X-Admin-Token: $TOKEN" -N -X POST \
"http://.../_/myproject/_/admin/backup/$TOKEN?sse=1"
```
---
## ZIP-Struktur (Referenz)
```
backup-<namespace>-<timestamp>.zip
├── backup-info.json # Metadaten + Schema-Fingerprint
├── collections/
│ ├── contacts.json # Array aller Dokumente (mit _id)
│ └── pages.json
└── media/ # Nur Dateien im uploadPath
└── contacts/
└── <documentId>/
└── <fieldName>/
└── <filename>
```
Kein `project.json` oder `config.json` — das Projekt muss auf dem Zielsystem bereits existieren.
---
## Media-Erkennung
Beim Export werden **nur** Dateien berücksichtigt, die:
1. Über ein `file`- oder `file[]`-Feld im Dokument referenziert sind
2. **Tatsächlich** im konfigurierten `uploadPath` existieren (`os.Stat` → OK)
3. Innerhalb des `uploadPath` liegen (Path-Traversal-Schutz)
Algorithmus:
```
Für jedes Dokument einer Collection:
Für jedes Feld vom Typ "file" oder "file[]":
Relativer Pfad aus Feldwert extrahieren
Absoluter Pfad = <uploadPath> / <relativer Pfad>
os.Stat() → existiert → in ZIP aufnehmen
Existiert nicht → skip, Log-Warnung
```
---
## Sicherheit
| Maßnahme | Beschreibung |
|----------|-------------|
| **Auth** | Server-Token, Projekt-Token, oder JWT AdminRole erforderlich |
| **Path Traversal** | `filepath.Clean` + `strings.HasPrefix` bei ZIP-Extraktion und Media-Kopie |
| **Temp-Files** | Automatischer Cleanup nach TTL, keine Akkumulation |
| **One-Time Download** | SSE-Download-Links sind nach einmaligem Abruf ungültig |
| **Media außerhalb uploadPath** | Pfade außerhalb des konfigurierten `uploadPath` werden ignoriert |
---
## Fehlerbehandlung
| HTTP-Status | Ursache |
|-------------|---------|
| **400** | Keine Datei hochgeladen, ungültiges ZIP, fehlende `backup-info.json`, falscher Typ/Version |
| **403** | Fehlende Berechtigung (`project.backup` oder `allowBackup`) |
| **404** | `restoreToken` nicht gefunden oder abgelaufen, Projekt nicht gefunden |
| **410** | Download-Link abgelaufen (nur bei `/download/:file`) |
| **500** | Interner Fehler (DB-Lesefehler, ZIP-Erstellung fehlgeschlagen) |
---
## Temp-Datei-Management
```
Upload (Phase 1) → /tmp/tibi-backup/<restoreToken>/ (TTL: 1h)
Restore (Phase 2) → importiert, dann gelöscht
DELETE /:restoreToken → sofort gelöscht
TTL-Ablauf → Background-Cleanup (alle 5 Min)
SSE Download → /tmp/tibi-backup-download/<token>.zip (TTL: 5 Min)
GET /download/:file → ausgeliefert + sofort gelöscht
```
---
## Konfiguration
### Server-Level Token (config.yml)
```yaml
api:
adminTokens:
- token: "backup-token"
label: "backup-agent"
permissions:
- project.backup
```
`project.*` (Wildcard) inkludiert automatisch `project.backup`.
### Projekt-Level Token
```yaml
admin:
tokens:
- token: "PROJEKT-TOKEN"
allowBackup: true
```
### Keine Projekt-Config-Änderung nötig
Der Backup/Restore-Vorgang benötigt keine Änderungen an der Projekt-Config.
Collections, Felder, uploadPath etc. werden aus der bestehenden Konfiguration gelesen.
---
## Implementierung
### Dateien
| Datei | Inhalt |
|-------|--------|
| `internal/controller/admin_backup.go` | Alle Handler (Download, Analyze, Restore, Cleanup, SSE, Temp-Stores) |
| `internal/controller/admin_backup_auth_test.go` | 12 Auth-Tests |
| `internal/controller/admin_backup_analyze_test.go` | 8 Analyze- & Download-Tests |
| `internal/controller/admin_backup_restore_test.go` | 9 Restore-Tests |
| `internal/controller/admin_backup_integration_test.go` | 3 SSE/E2E-Tests |
| `internal/controller/setup.go` | Routen-Registrierung |
| `internal/models/project.go` | `AdminToken.AllowBackup` Feld |
| `cmd/server/main.go` | `StartBackupCleanupJob()` Aufruf |
### Test-Statistik
**33 Tests** (alle grün):
- 12 Auth-Tests (Server-Token, Projekt-Token, JWT, Berechtigungen)
- 6 Analyze-Tests (ZIP-Validierung, Schema-Diffs, unbekannte Collections)
- 2 Download-Tests (direkt, selektiv)
- 9 Restore-Tests (Import, Merge, Selective, Media-Copy, Cleanup)
- 3 SSE/E2E-Tests (SSE-Download, DownloadFile, SSE-Restore)
- 1 E2E-Test (skipped — Mock-Projekt-API-Limitierung)
Keine Regression auf bestehenden Admin-Tests (`TestAdminReload`, `TestAdminAuthReload` etc.).
---
## Betrieb: Speicherverhalten seit Streaming-Umbau (2026-10)
Erstellung und Wiederherstellung von Snapshots und Backups laufen in
**O(Batch)** statt O(Collection): Keyset-Pagination (`_id $gt`, Sort `_id`),
Datei-/ZIP-Streaming, sequenziell pro Collection. RAM ist keine Grenze mehr —
auch 10-GB-Collections (z. B. Outdoorworks: `productVariant` 91k × ~117 KB mit
`_search`-Bloat) laufen mit konstantem Heap.
### Batch-Konstanten (`internal/controller/snapshot_stream.go`)
| Konstante | Wert | Wirkung |
|---|---|---|
| `SnapshotBatchDocs` | 500 | Docs pro DB-Batch und Restore-Batch |
| `SnapshotBatchBytes` | 8 MiB | reserviert (Byte-Cap, greift pro großem Einzeldoc) |
| `SnapshotBufSize` | 64 KiB | `bufio`-Puffer für Snapshot-Dateien |
| `OrphanMapThreshold` | 200.000 | darunter In-RAM-Orphan-Set, darüber Temp-Collection in Ziel-DB |
| `orphanChunkSize` | 500 | `$in`-Chunks für Orphan-Deletes |
### Empfehlungen
- `GOMEMLIMIT` setzen (z. B. `GOMEMLIMIT=512MiB` im Container): begrenzt GC-Druck,
Streaming braucht es nicht — aber es schützt vor Nachbar-Last.
- Jeder Snapshot-Run loggt `snapshot <id>: done documents=… heapGrowthMB=…`
(`heapGrowthMB` = Heap-Wachstum über den Lauf, kein Peak; Peak ist per
Konstruktion batch-begrenzt).
- Retention `maxCount` behält die **neuesten** Einträge (Bugfix 2026-10-15:
zuvor wurden fälschlich die neuesten gelöscht).
### Nachweise (Tests)
- `TestExportStreaming_PeakScalesFlat` / `TestStreamRestore_PeakScalesFlat`:
10× Docs ⇒ kein 10× Peak (Kontrollmessung Alt-Pfad liegt ein Mehrfaches darüber).
- E2E Roundtrips (Mock + Real-Mongo): Snapshot NonMerge/Merge, Backup
Restore-Matrix, Download-Pipelines.
- OOM-Fixture: `testdata/outdoorworks-oom/` (echte `_search`-Form, Stats,
`dump-full.sh` für den 8-GB-Voll-Dump nach `/WM_Dev/.cortex/mcp-tmp/`).
src/content/docs/server/22-personal-access-tokens.md
+295
View File
@@ -0,0 +1,295 @@
---
title: '22. Personal Access Tokens'
---
# 22. Personal Access Tokens
Self-managed API tokens for users. Every user can create, list, update and delete
their own tokens. A token can be restricted to a subset of the owner's rights and
is **always** checked live against the current rights of its owner.
- [Token format & storage](#token-format--storage)
- [Quick start](#quick-start)
- [Scope model](#scope-model)
- [Scope reference](#scope-reference)
- [Which scopes a user may grant](#which-scopes-a-user-may-grant)
- [Endpoint → scope mapping](#endpoint--scope-mapping)
- [Management API](#management-api)
- [Security properties](#security-properties)
- [Status codes](#status-codes)
Personal Access Tokens are a core feature and always enabled.
## Token format & storage
```text
tibi_pat_<24-hex objectId>_<base64url secret>
```
- Only `sha256(secret)` is stored (`tokenHash`); the plaintext exists exactly once,
in the create response.
- `tokenPrefix` (`tibi_pat_` + 4 secret chars, 13 chars total) is stored for
display and audit.
- Lookup uses the embedded object id → one indexed `_id` read.
- The secret is compared in constant time.
- No expiry by default; `expiresAt` is optional (`expiresAt` must be in the
future when minting).
A PAT is sent via **`X-Auth-Token`** (like the JWT). `Authorization: Bearer` is
reserved for project-specific authentication and is *not* accepted for PATs.
## Quick start
```bash
# 1) create (requires a user login session, not a PAT)
curl -X POST https://host/api/v1/token \
-H "X-Auth-Token: <jwt>" \
-H "Content-Type: application/json" \
-d '{"name":"CI Deploy","scopes":["namespace.website.articles.read"]}'
# 2) use the returned plaintext token (shown exactly once)
curl https://host/api/v1/_/website/articles \
-H "X-Auth-Token: tibi_pat_..."
# 3) revoke (immediately effective)
curl -X DELETE https://host/api/v1/token/<id> -H "X-Auth-Token: <jwt>"
```
## Scope model
A scope is a dotted permission string. Three rules define the effective access of
a PAT request:
1. **Token ⊆ owner.** A scope can only be granted if the owner currently holds
that permission. The catalog is computed live from the database at
create/update time.
2. **Scope ∩ owner (AND).** At request time the action must be allowed by the
**token scope** *and* by the **owner's current rights**. Revoking a user right
(role, org, team, `permissions[]`) restricts the token immediately.
3. **Same syntax as admin tokens.** Dotted segments, `*` wildcard per segment,
and shorthands where fewer segments imply all descendants. Because of rule 2,
even a very broad scope (up to `*`) can never grant more than the owner
currently holds.
**Separate credentials authorize independently.** A valid **server/project admin
token** (`X-Admin-Token` or `Authorization: Bearer`) or a **collection config
token** (`name:value` header/query) authorizes on its own; passing an additional
narrow PAT does not restrict such a request. PAT scopes only restrict requests
that are authenticated *as a user* by the PAT.
Additional semantics:
- **Public baseline.** Data that is publicly readable stays readable with any PAT
(even with no scopes) — a token never adds or removes anonymous access.
- **Empty scope list** is allowed and yields a token that can only reach public
data. `*` yields the full (but still owner-capped) access.
- **Shorthands** are prefixes and never grant more than the owner has.
### Shorthand resolution
| Scope written | Matches (at most the owner's rights) |
|---------------|--------------------------------------|
| `*` | everything the owner may do |
| `namespace` | all collections in all namespaces |
| `namespace.<ns>` | every collection/action in `<ns>` the owner may use |
| `namespace.<ns>.<col>` | every action on `<col>` the owner may use |
| `namespace.<ns>.<col>.<action>` | exactly that action |
| `user` / `project` / `team` / `org` | all actions of that system domain |
| `user.read` / `project.reload` / `audit.read` / `llm.chat` | exactly that action |
| `namespace.*.<col>.read` / `user.*` | wildcard matches any single segment |
Because of rule 2, a shorthand or wildcard can never exceed the owner's rights —
it only avoids enumerating them. `*` is the convenient "everything this user may
do" scope; a request is still rejected as soon as the owner lacks the right.
## Scope reference
### Namespace (project collection) scopes
Namespace = the project's `namespace` (see project config).
| Scope | Grants |
|-------|--------|
| `namespace.<ns>.<col>.read` | `GET /_/<ns>/<col>` (list/one), `_count`, `_facet`, `_queue`, file reads, lookup/aggregate reads of `<col>` (as target), audit entries of `<col>` |
| `namespace.<ns>.<col>.create` | `POST /_/<ns>/<col>` (single and bulk) |
| `namespace.<ns>.<col>.update` | `PUT /_/<ns>/<col>/:id` (single and bulk) |
| `namespace.<ns>.<col>.delete` | `DELETE /_/<ns>/<col>/:id` (single and bulk; bulk additionally needs the owner's bulk right) |
| `namespace.<ns>.<col>` | all of the above the owner holds |
| `namespace.<ns>` | all of the above for every collection in `<ns>` |
| `namespace` | all of the above for every collection the owner may use |
| `*` | everything below and the system domains the owner may use |
### Action endpoint scopes
Actions (`/_/<ns>/_actions/<name>`) have their own scopes.
| Scope | Grants |
|-------|--------|
| `namespace.<ns>._actions.<name>.read` | `GET /_/<ns>/_actions/<name>` |
| `namespace.<ns>._actions.<name>.create` | `POST /_/<ns>/_actions/<name>` |
### System domain scopes
| Scope | Grants |
|-------|--------|
| `user` | all user actions below |
| `user.read` | `GET /api/v1/user`, `GET /api/v1/user/:id` (owner-visibility applied) |
| `user.create` | `POST /api/v1/user` |
| `user.update` | `PUT /api/v1/user/:id` |
| `user.delete` | `DELETE /api/v1/user/:id` |
| `project.read` | `GET /api/v1/project`, `GET /api/v1/project/:id` |
| `project.create` | `POST /api/v1/project` |
| `project.update` | `PUT /api/v1/project/:id` |
| `project.delete` | `DELETE /api/v1/project/:id` |
| `project.reload` | `POST /_/<ns>/_/admin/reload` |
| `project.backup` | `POST/GET/DELETE /_/<ns>/_/admin/backup…` |
| `project.regenerate-search` | `POST /_/<ns>/_/admin/regenerate-search` |
| `project.snapshot` | `/_/<ns>/_/admin/snapshots…` (list/detail/restore/download/prune) |
| `team` / `org` | all actions below |
| `team.read` / `team.create` / `team.update` / `team.delete` | `/api/v1/team…` |
| `org.read` / `org.create` / `org.update` / `org.delete` | `/api/v1/org…` |
| `audit.read` | `GET /api/v1/audit` (see below) |
| `llm.chat` | `POST /api/v1/llm/chat` |
**Audit is special:** a PAT needs `audit.read` **and** the read scope of a
collection (`namespace.<ns>.<col>.read`) to see that collection's audit entries.
`audit.read` alone does not expose collection snapshots. Non-admin users never see
system events (login/reload/shutdown) — this is unchanged by tokens.
**Not scope-relevant (public by design):** `GET /_/<ns>/_/assets/…`, `_meta`,
`_openapi/:project`, `/version`, `GET /api/v1/llm/providers`.
## Which scopes a user may grant
`GET /api/v1/token/_scopes` returns the exact catalog for the calling user. In
general:
| Owner role | System scopes available |
|------------|-------------------------|
| SystemAdmin (`role: 0`) | all of the above, including `project.reload/backup/regenerate-search/snapshot` |
| OrgAdmin (`orgs[].role: "admin"`) | `user.read/create/update`, `team.read/create/update/delete`, `project.read/update`, `org.read`, `audit.read`, `llm.chat` |
| User (`role: 1`) | `user.read/update`, `project.read`, `team.read`, `org.read`, `audit.read`, `llm.chat` |
Collection and action scopes are additionally limited to the projects/collections
the owner can reach (org visibility + team permissions). Public-only rights are
never part of the catalog.
## Endpoint → scope mapping
| Endpoint | Required scope |
|----------|----------------|
| `GET /_/<ns>/<col>` (+ `_count`, `_facet`, `_queue`) | `namespace.<ns>.<col>.read` |
| `POST /_/<ns>/<col>` | `namespace.<ns>.<col>.create` |
| `PUT /_/<ns>/<col>/:id` | `namespace.<ns>.<col>.update` |
| `DELETE /_/<ns>/<col>/:id` | `namespace.<ns>.<col>.delete` |
| `GET /_/<ns>/<col>/:id/:field/*path` (file) | `namespace.<ns>.<col>.read` (public collections: none) |
| `GET /_/<ns>/<col>?lookup=…:<target>` | `namespace.<ns>.<col>.read` **and** `namespace.<ns>.<target>.read` (public target: none) |
| `GET /_/<ns>/<col>?aggregate=…:<target>` | `namespace.<ns>.<col>.read` **and** `namespace.<ns>.<target>.read` (public target: none) |
| `GET /api/v1/<domain>…?lookup=…:<target>` (system lookup) | `<domain>.read` **and** `<target>.read`; missing target scope → target is silently not embedded |
| `GET/POST /_/<ns>/_actions/<name>` | `namespace.<ns>._actions.<name>.read` / `.create` |
| `/api/v1/user…` | `user.<action>` |
| `/api/v1/project…` | `project.<action>` |
| `/_/<ns>/_/admin/reload` etc. | `project.reload` / `.backup` / `.regenerate-search` / `.snapshot` |
| `/api/v1/team…`, `/api/v1/org…` | `team.<action>`, `org.<action>` |
| `GET /api/v1/audit` | `audit.read` (+ per-collection read scope) |
| `POST /api/v1/llm/chat` | `llm.chat` |
`GET /api/v1/token*` itself is **not** scope-controllable: token management
requires a user login session (a PAT gets `403`).
## Management API
All endpoints operate on the caller's own tokens; system admins can see and
manage all.
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/v1/token` | list own tokens (never contains secret/hash) |
| `POST` | `/api/v1/token` | create → returns the plaintext token **once** |
| `GET` | `/api/v1/token/_scopes` | available scopes for the current user |
| `PUT` | `/api/v1/token/:id` | update name/scopes/expiry (re-validated against the owner catalog) |
| `DELETE` | `/api/v1/token/:id` | delete (revoke); effective immediately |
Create response (the `token` field appears only here):
```json
{
"id": "60a1b2c3d4e5f6a7b8c9d0e1",
"name": "CI Deploy",
"tokenPrefix": "tibi_pat_ab12",
"scopes": ["namespace.website.articles.read"],
"expiresAt": null,
"token": "tibi_pat_60a1b2c3d4e5f6a7b8c9d0e1_..."
}
```
## Security properties
- **Never more than the owner:** scopes are validated against the owner's live
rights when minted; every request re-checks the owner's current rights
(scope ∩ owner).
- **Immediate revocation:** deleting a token invalidates it on the next request
(no validity cache); user permission/role/org/team changes take effect
immediately.
- **No secret leakage:** only the hash is persisted; `tokenHash` is never
serialized; audit entries record `authMethod="access-token"`, `tokenLabel`
(name) and `tokenPrefix`, never the secret or hash.
- **Ownership:** `PUT`/`DELETE` are restricted to the owner (or system admin).
- **Session-only management:** `/api/v1/token*` (including `/_scopes`) requires a
user login session; a PAT cannot mint/manage tokens or impersonate users.
- **Impersonation:** tokens cannot be created or managed while impersonating.
- **User deletion:** all tokens of the user are removed (cascade).
## UI integration notes (tibi-admin-nova)
- **Session required:** management calls use the normal login session
(`X-Auth-Token: <jwt>`). PATs are rejected on `/api/v1/token*` with `403`.
- **Scope picker:** `GET /api/v1/token/_scopes` returns `string[]` — the exact
scopes the current user may grant (concrete collection/action scopes, namespace
and collection shorthands, system actions). Offer them as checkboxes plus a
"select all" (= everything the user may do).
- **Power input:** arbitrary admin-token-style scopes are allowed (`*`, `user`,
`namespace.website`). The server validates them and answers `400` for invalid
syntax or scopes unrelated to the user's rights.
- **Create:** `POST /api/v1/token` with `{"name": "...", "scopes": [...],
"expiresAt": null}`. The response contains the plaintext `token` **exactly
once** — show a copy-once dialog and never persist it in the frontend state
longer than needed.
- **List:** `GET /api/v1/token` →
`[{id, name, tokenPrefix, scopes, expiresAt, lastUsedAt, insertTime, updateTime}]`.
Never contains `token` or `tokenHash`.
- **Update:** `PUT /api/v1/token/:id` with any of `name`, `scopes`, `expiresAt`
(re-validated; `expiresAt` must be in the future).
- **Revoke:** `DELETE /api/v1/token/:id` — effective immediately.
- **Errors:** `403` (no/insufficient session, foreign token, missing scope),
`400` (invalid syntax, scope not permitted, past `expiresAt`), `404` (unknown
token id). Show the `error` string.
- **Display list entries with** `name`, `tokenPrefix` and `lastUsedAt`; the
plaintext is never shown again.
## Known limitations
These are accepted, documented residual topics (deliberately not implemented):
- **No rate limit / cap on token creation.** Rate limiting (`ratelimit`) applies
to `POST /api/v1/login` only. An authenticated session can mint an arbitrary
number of tokens; every create/delete is audited, but there is no per-user cap.
If needed, add a limiter to `/api/v1/token` and/or a max-tokens-per-user config.
- **Invalid-token requests cost 1–2 indexed DB reads** (token + owner). Repeated
requests with arbitrary `tibi_pat_…` values are not throttled, which is a mild
amplification vector. Mitigation would be auth-failure rate limiting.
Neither is a privilege escalation: a token never exceeds its owner and never
exposes secrets.
## Status codes
| Situation | Status |
|-----------|--------|
| Missing/invalid/expired/deleted token | `403` |
| Token valid, scope missing | `403` |
| Scope present, owner right missing | `403` |
| `POST`/`PUT` with a syntactically invalid scope (bad segments, unknown domain) | `400` |
| `POST`/`PUT` with a scope not related to any owner right | `400` |
| `expiresAt` in the past | `400` |
| Foreign token `PUT`/`DELETE` | `403` (owner or system admin only) |
src/content/docs/server/23-mcp.md
+156
View File
@@ -0,0 +1,156 @@
---
title: '23. MCP-Server'
---
# 23. MCP-Server
Tibi-server ist zusätzlich ein MCP-Server (Streamable HTTP, JSON-RPC 2.0) für
Agenten, die Websites befüllen. Alle Tools laufen mit denselben Checks wie REST.
- [Scope-Modell](22-personal-access-tokens.md) · [Auth](05-authentication.md) ·
[Collections](04-collections.md) · [Actions](19-actions.md)
## Transport & Auth
```
POST /api/v1/mcp # JSON-RPC: initialize, tools/list, tools/call
GET /api/v1/mcp # Capability-Probe (PAT nötig)
```
- Auth **nur** per PAT: `X-Auth-Token: tibi_pat_...`
- JWT, `X-Admin-Token`, `Authorization: Bearer` (non-PAT) → `403`.
- Deaktiviert (`mcp.enabled: false`) → `404`.
```bash
# Tools listen
curl -X POST https://host/api/v1/mcp \
-H "X-Auth-Token: tibi_pat_..." -H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{}}'
# Tool aufrufen
curl -X POST https://host/api/v1/mcp \
-H "X-Auth-Token: tibi_pat_..." -H 'Content-Type: application/json' \
-d '{"jsonrpc":"2.0","id":2,"method":"tools/call",
"params":{"name":"collections_list","arguments":{"namespace":"website"}}}}'
```
JSON-RPC-Fehlercodes: `-32601` unbekanntes Tool, `-32004` Resource nicht gefunden,
`-32003` Scope/Permission (`403`), `-32602` Validierung (`400`), `-32603` intern (`500`),
`-32700` Parse-Error.
## Konfiguration (`config.yml`)
```yaml
mcp:
enabled: true
route: "/api/v1/mcp"
maxResultItems: 100
allowAdminActions: true # reload/backup/snapshot nur wenn Token-Scope es erlaubt
```
Env: `MCP_ENABLED`, `MCP_ROUTE`, `MCP_MAX_RESULT_ITEMS`, `MCP_ALLOW_ADMIN_ACTIONS`.
Kein `allowWrite`-Flag (nur PAT-Scopes), kein `llm.chat` via MCP.
## Tools (Hybrid, 36)
Discovery: `projects_list`, `project_get`, `collections_list` (braucht `project.read`),
`collection_describe` (braucht `project.read`; rechtegefilterte Wahrheit: Felder, Validatoren,
`yourPermissions`, Pre-Filter, Projektionen, `meta.agent` ohne Secrets, Perm-Keys maskiert),
`actions_list`, `action_describe`, `jobs_list` (lesend, kein Trigger),
`openapi_get`, `system_meta_get`, `agent_guide_get` (kuratiert aus `meta.agent`, inkl. `yourPermissions` bei Collection),
`assets_list`. `asset_read` ist REST-only (`GET /_/:project/_/assets/...` nutzen).
CRUD: `documents_list` (filter/sort/limit/offset/select/projection/lookup/aggregate/q
plus `extra`-Objekt für `meta.query`-Parameter, s. unten),
`document_get`, `documents_count`, `documents_facet`, `queue_status`,
`document_create` (single + `items[]`-Bulk, Dateien als base64 via `CollectionPost`),
`document_update` (single + `filter`-Bulk), `document_delete`,
`action_call` (nur get/post).
System: `users_list`, `user_get`, `orgs_list`, `org_get`, `teams_list`, `team_get`
(je mit `ValidateFilter`-Sichtbarkeit), `audit_query`
(`audit.read` + Collection-`read`; System-Events für Non-Admin unsichtbar).
REST-only in dieser Phase (klare `-32602`-Meldung, kein Silent-Fail):
`user_create/update/delete`, `project_reload`, `regenerate-search`,
`backup_*`, `snapshot_*`, `file_read`, `asset_read` → REST-Endpunkte nutzen.
Nie via MCP: `token`-Management, `login`/`impersonate`, `shutdown`, `llm.chat`.
## Endpoint → Scope (Auszug)
| MCP-Tool | Scope |
|---|---|
| `projects_list`/`project_get` | `project.read` |
| `collections_list` | `project.read` + je Collection `namespace.<ns>.<col>.read` |
| `collection_describe` | `project.read` + `namespace.<ns>.<col>.read` |
| `documents_*`, `queue_status` | `namespace.<ns>.<col>.read` |
| `document_create` | `namespace.<ns>.<col>.create` |
| `document_update`/`delete` (bulk) | `.update`/`.delete` (+ Owner-Bulk-Recht) |
| `action_call` | `namespace.<ns>._actions.<name>.read`/`.create` |
| `users_*`/`orgs_*`/`teams_*` | `user/org/team.read` (+ Write-Scopes für REST-only) |
| `audit_query` | `audit.read` (+ Collection-`read`) |
Dazu gelten Pre-Filter, Projektionen (`select` nur einengend), Bulk-Flags,
Lookup/Aggregate-Target-Scopes, Validatoren, Hooks (Hooks sind methoden-keyed:
jedes Tool nutzt seine REST-Methode, damit `get`/`put`/`delete`-Hooks laufen)
und Audit-Einträge mit `tokenLabel`/`tokenPrefix` (Client-IP vom Aufrufer) —
identisch zu REST, weil dieselben Controller laufen. MCP-Einträge tragen
zusätzlich `channel: "mcp"` (REST lässt das Feld leer), abfragbar per
`filter={"channel":"mcp"}`.
## Agent-Anleitung (`meta.agent`, YAML-Konvention)
`meta` ist free-form; der Unterschlüssel `agent` ist die kuratierte Anleitung.
Erlaubte Keys: `description`, `purpose`, `instructions`, `frontend`, `workflows`,
`hints`, `fieldHints`, `examples`. Secrets/Token/Pfade/Hooks werden nie
durchgereicht (Größe gecappt).
```yaml
# config.yml
meta:
agent:
description: { de: "Stadtportal", en: "City portal" }
frontend: { routes: "/:slug, /news/:id" }
workflows: ["Artikel anlegen → publizieren → prüfen"]
hints: ["slug im kebab-case"]
# collections/articles.yml
meta:
agent:
purpose: { de: "News-Artikel für Startseite" }
instructions: "Teaser max 160 Zeichen, publishDate immer setzen."
fieldHints: { teaser: "max 160 Zeichen" }
examples: [{ title: "...", published: true }]
```
Idempotenz: `filter`-vor-`create` + `unique`-Indexe; `409` = vorhanden;
stabile Caller-`_id` bei Retry.
## Zusätzliche Query-Parameter (`meta.query`)
Hooks werten oft Query-Parameter aus (`ctx.request().query(name)`). Via MCP sind
nur freigegebene Argumente erlaubt — Collections deklarieren weitere Parameter
in `meta.query` (Schema in `tibi-types/schemas/config/collection-meta.schema.json`):
```yaml
# collections/articles.yml
meta:
query:
hook: { type: string, enum: [fakedata, mail], description: { de: "Test-Dispatcher" } }
view: { type: string, maxLength: 32 }
```
- Typen: `string`, `number`, `boolean`; dazu `enum` und `maxLength`.
- MCP: als `extra`-Objekt an `documents_list`, `document_get`, `documents_count`,
`documents_facet` übergeben — validiert, dann als Query gesetzt. Unbekannte oder
falsch typisierte Keys → `-32602`. Sichtbar in `collection_describe` unter `query`.
- Niemals freigegeben (auch nicht konfigurierbar): Credential-Kanäle
(Permission-Token-Namen, `x-auth-token`, `x-admin-token`, `authorization`) und
Core-Keys (`filter`, `sort`, `limit`, `select`, `lookup`, …).
- REST bleibt unverändert offen; das Schema dient dort als Dokumentation
(`_meta`/OpenAPI) für Agenten. Details: [Collections](04-collections.md#metaquery).
## Beispiele
Siehe `examples/mcp/` (curl: tools/list, describe, Befüllungs-Workflow).
`test.http` enthält MCP-Snippets.
src/content/docs/server/24-ephemeral-rest-tokens.md
+75
View File
@@ -0,0 +1,75 @@
---
title: '24. Ephemere REST-Tokens via MCP (PLAN)'
---
# 24. Ephemere REST-Tokens via MCP (PLAN)
Status: Plan — noch nicht implementiert. Ziel: harness-neutrale Kurzzeit-Credentials,
damit Shell/`curl` alles tun kann, was MCP kann, ohne Base64 in den LLM-Kontext zu ziehen.
## Ausgangslage
- MCP-Uploads (`document_create`/`document_update`, `type:file` als
`{src: "data:<mime>;base64,...", path: "name.jpg"}`) funktionieren, landen aber bei
reinen Tool-Harness (kein JS-Runtime) als ~12k Tokens pro Bild im Prompt.
`documents_list` ohne `select` spült `file.data` ebenso zurück.
- Mit JS-Runtime (OpenCode `execute`: `fetch.bytes()` → `mcp_call`) bleibt Base64 im
Runtime-Speicher, zurück kommt nur `{id, file:{src,path,type}}`. Das hilft nur dort.
- Upload-Tickets (Presigned-PUT, S3, CORS, Quota) wären ein eigenes Projekt.
## Idee
Generische Kurzzeit-Token für alles, was MCP kann, via REST: MCP stellt aus,
Shell verbraucht. Binär/Base64 baut die Shell aus Dateien (`base64`, `jq`, `curl`),
im Prompt stehen nur Token (32 Zeichen) + Pfade.
```bash
# 1) via MCP ausstellen (Scope: ns.collection.method, TTL z. B. 300s)
# -> zurück: tibi_eph_... (einmalig im Klartext)
# 2) in der Shell verbrauchen, ohne LLM-Bloat
curl https://host/api/v1/_/website/articles \
-H "X-Auth-Token: tibi_eph_..." \
-H 'Content-Type: application/json' \
-d @/WM_Dev/.cortex/mcp-tmp/payload.json
curl https://host/api/v1/_/website/medialib/<id>/file/name.jpg \
-H "X-Auth-Token: tibi_eph_..."
```
## Design (angelehnt an 22-PAT)
- Format/Storage wie PAT wiederverwenden (`tokenHash` statt Klartext, `tokenPrefix`
für Anzeige/Audit, Lookup über eingebettete Objekt-ID, Constant-Time-Vergleich),
plus Pflichtfelder: `expiresAt` (max. 1h, Default 5 Min), `ephemeral:true`,
`parentTokenId` (Audit-Verkettung), enger Scope (z. B. genau eine Collection+Methode).
- Scope-Regeln wie PAT übernehmen: **Token ⊆ Owner** bei Ausstellung (live aus DB),
**Scope ∩ Owner** bei jedem Request (Owner-Revoke wirkt sofort). Kein neues
Rechtesystem, keine neuen Header: weiter `X-Auth-Token` wie PAT.
- Transport: neues MCP-Tool `token_issue_ephemeral` (braucht die zu delegierende
Scope-Menge selbst) + optional `token_revoke_ephemeral`. Kein `login`/`impersonate`
via MCP (bleibt verboten wie in 23-MCP). REST-Seite unverändert außer TTL-Check.
- Missbrauchsschutz: kurze TTL + vorhandener `expiresAt`-Pfad, Audit wie PAT,
Rate-Limit auf Ausstellung, kein Refresh (neu ausstellen statt verlängern),
keine Scope-Erweiterung über Owner hinaus.
## Nicht-Ziele
- Kein Presigned-S3, kein Multipart-Spezialfall, keine Änderung an
`handleFileInJSON` (`src`/`path`-Vertrag aus 08-File-Upload bleibt).
- Kein Ersatz für `clear_cache`-Hooks oder `filter_public`-Vertrag.
- Keine langlebigen CI-Token (dafür weiter normale PAT mit `expiresAt`).
## Umsetzungsskizze
1. `token_issue_ephemeral` im MCP (Scope-Check gegen Caller-PAT, TTL clamp 60s–3600s).
2. Token-Dokuments mit `expiresAt` + TTL-Index/Cleanup (prüfen, ob PAT-Cleanup das
schon abdeckt, sonst Job ergänzen).
3. Doku in 22-PAT (Ephemeral-Absatz) + 23-MCP (Tool-Zeile) + Beispiel-Shell-Snippet.
4. Tests: Ablauf nach TTL, Scope-Subset-Verweigerung, Owner-Revoke wirkt sofort,
Audit enthält `parentTokenId`.
## Offene Fragen
- Single-Use vs. Multi-Use innerhalb TTL (Shell-Skripte mögen Multi, Audit mag Single)?
- Soll `project_reload`/`backup`/`snapshot` (heute REST-only, `allowAdminActions`)
auch über Ephemeral erreichbar sein oder explizit ausgenommen bleiben?
- Braucht es IP-/User-Agent-Bindung, oder reicht kurze TTL + enger Scope?
src/content/docs/server/uebersicht.md
+21
View File
@@ -0,0 +1,21 @@
---
title: Tibi-Server Übersicht
description: Was der Tibi-Server kann und wo die Details stehen.
---
Der **Tibi-Server** ist ein dynamisches, konfigurationsgetriebenes Headless-CMS in Go: MongoDB plus YAML-definierte REST-APIs, JavaScript-Hooks, SSE-Realtime, LLM-Chat, Upload/Bilder, PDF, Audit, Cron, OpenAPI, Prometheus, Multi-Tenancy.
## Detailseiten
Die Detailseiten unter `server/` werden per `npm run sync` aus dem Server-Repo (`tibi-server/docs/`) übernommen und hier nur gerendert — Single Source bleibt das Server-Repo:
- Architektur, Konfiguration, API-Endpunkte, Collections
- Authentifizierung, Hooks, Realtime, Upload/Bilder
- LLM-Integration, Audit, Jobs, Deployment
- OpenAPI/Metriken, Security, Troubleshooting, Cookbook
- Feld-Permissions, Orgs/Teams, Aktionen, Backup/Restore, Tokens, MCP
## Konvention für Agenten
- Neue Server-Funktion → erst Code + `tibi-server/docs/*.md`, dann `npm run sync` hier, dann Build prüfen.
- Generierte Referenz (`reference/`) nie von Hand editieren.
src/styles/custom.css
+66
View File
@@ -0,0 +1,66 @@
/* Tibi Doku — an Tibi-Admin-Nova angelehntes Theme (Inter, Near-Black + Nova-Orange) */
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap');
:root {
--sl-font: 'Inter', system-ui, -apple-system, sans-serif;
/* Akzent = Nova Near-Black (Light Mode) */
--sl-color-accent-low: #e5e7eb;
--sl-color-accent: #111827;
--sl-color-accent-high: #000000;
--sl-color-white: #ffffff;
--sl-color-gray-1: #f8f9fa;
--sl-color-gray-2: #f1f3f5;
--sl-color-gray-3: #e5e7eb;
--sl-color-gray-4: #d1d5db;
--sl-color-gray-5: #9ca3af;
--sl-color-gray-6: #6b7280;
--sl-color-black: #111827;
/* Nova-Orange als Zweitakzent für Badges, Links-Hover, Hero */
--tibi-nova: #c2410c;
--tibi-nova-core: #4da8b8;
--tibi-nova-outer: #dc2626;
--tibi-sidebar: #f8f9fa;
--tibi-topbar: #ffffff;
}
:root[data-theme='dark'] {
--sl-color-accent-low: #2a2a2a;
--sl-color-accent: #e5e7eb;
--sl-color-accent-high: #ffffff;
--sl-color-white: #1e1e1e;
--sl-color-gray-1: #2a2a2a;
--sl-color-gray-2: #151515;
--sl-color-gray-3: #333333;
--sl-color-gray-4: #333333;
--sl-color-gray-5: #9ca3af;
--sl-color-gray-6: #6b7280;
--sl-color-black: #e5e7eb;
--tibi-sidebar: #1e1e1e;
--tibi-topbar: #1e1e1e;
}
/* Sidebar / Topbar leicht Nova-nah */
.sidebar-pane {
background-color: var(--tibi-sidebar);
}
.header {
background-color: var(--tibi-topbar);
}
/* Hero-Akzent: Nova-Flamme als dezenter Verlauf */
.hero {
background:
radial-gradient(ellipse 60% 50% at 50% 0%, rgba(194, 65, 12, 0.12), transparent),
radial-gradient(ellipse 40% 30% at 50% 0%, rgba(77, 168, 184, 0.08), transparent);
}
/* Externe Nova-Links / primäre Aktionen in Nova-Orange */
.sl-link-card:hover {
border-color: var(--tibi-nova);
}
a[href*='tibi-admin-nova'],
a[href*='tibicms'] {
text-decoration-color: var(--tibi-nova);
}