apairon
349fb9b2da
- Improved parsing of `lookup` and `aggregate` options to support JSON strings and arrays. - Added support for object format in `lookup` and `aggregate` to specify collections. - Simplified dependency tracking for SSR cache invalidation based on new formats.
308 lines
15 KiB
Markdown
308 lines
15 KiB
Markdown
---
|
||
name: tibi-ssr-caching
|
||
description: Implement and debug server-side rendering with goja (Go JS runtime) and dependency-based HTML cache invalidation for tibi-server. Use when working on SSR hooks, cache clearing, or the server-side Svelte rendering pipeline.
|
||
---
|
||
|
||
# tibi-ssr-caching
|
||
|
||
This skill should teach the **SSR architecture and implementation pattern** used in this repo family, not just describe one demo content setup. The important question is: **how is SSR built here, where is responsibility split, and which parts must be adapted per project?**
|
||
|
||
## SSR request flow
|
||
|
||
1. `ssr/get_read.js` receives a page request and calls `lib/ssr-server.js`.
|
||
2. `ssr/get_read.js` loads `lib/app.server.js` and calls `app.default.render({ url })`.
|
||
3. `frontend/src/ssr.ts` only initializes i18n and delegates rendering to `svelte/server`.
|
||
4. `frontend/src/App.svelte` owns the actual data loading for both browser and SSR.
|
||
5. During SSR, the app calls its normal page-loading path directly inside a `typeof window === "undefined"` guard.
|
||
6. During browser navigation, the same page-loading path is triggered from `$effect`.
|
||
7. API calls made during SSR are tracked as dependency strings (`col:id` or `col:*`) and cached in `window.__SSR_CACHE__`.
|
||
8. The rendered HTML + dependency list are stored in the `ssr` collection.
|
||
|
||
## Responsibility split
|
||
|
||
- `frontend/src/ssr.ts` should stay minimal.
|
||
- `frontend/src/ssr.ts` is responsible for SSR bootstrapping only: locale setup, SSR-safe render wrapper, and calling `render(App, { props: { url } })`.
|
||
- The app component should own data loading.
|
||
- Hooks under `api/hooks/ssr/` should own caching, cache lookup, and cache persistence.
|
||
- `api/hooks/lib/ssr.js` should own the shared API helper that works in both browser and SSR.
|
||
|
||
If these responsibilities get mixed together, SSR usually becomes harder to reason about and harder for an LLM to modify safely.
|
||
|
||
## Building the SSR bundle
|
||
|
||
```bash
|
||
yarn build:server
|
||
```
|
||
|
||
- Output: `api/hooks/lib/app.server.js`
|
||
- The project no longer uses Babel for SSR.
|
||
- The goja-compatible transform happens in `esbuild.config.server.js` via `supported`:
|
||
- `async-await: false`
|
||
- `async-generator: false`
|
||
- `dynamic-import: false`
|
||
- The SSR build writes directly to `api/hooks/lib/app.server.js`.
|
||
- Remove splitting-related frontend options (`outdir`, `splitting`, `entryNames`, `chunkNames`, `outExtension`) from the server build, otherwise esbuild will fail with `outfile`/`outdir` conflicts.
|
||
|
||
## Core design rule
|
||
|
||
- Prefer **one shared data-loading path** for browser and SSR.
|
||
- The browser should trigger it reactively.
|
||
- SSR should call that same path explicitly before rendering completes.
|
||
- Avoid maintaining a separate SSR-only content-loading implementation unless there is no viable alternative.
|
||
|
||
In this repo family, the practical pattern is:
|
||
|
||
- browser: `$effect(() => loadContent(...))`
|
||
- SSR: call the same `loadContent(...)` once inside a server guard
|
||
|
||
The main trap is assuming `$effect` alone is enough for SSR. It is not.
|
||
|
||
## Dependency-based cache invalidation
|
||
|
||
When content changes, `clear_cache.js` only invalidates SSR entries that depend on the changed collection/entry:
|
||
|
||
```js
|
||
// Each SSR cache entry stores dependency strings:
|
||
{
|
||
path: "/de/ueber-uns",
|
||
content: "...",
|
||
dependencies: ["content:abc123", "navigation:*", "medialib:*"]
|
||
}
|
||
```
|
||
|
||
- `col:id` means a detail dependency.
|
||
- `col:*` means a list dependency.
|
||
- `clear_cache.js` must handle `DELETE` robustly, because `context.data.id` and route params may be missing. Fallback to the last path segment if needed.
|
||
- `utils.clearSSRCache()` must clear:
|
||
- `col:*` on `POST`
|
||
- `col:id` OR `col:*` on `PUT`/`DELETE`
|
||
- everything on manual clear (`POST /ssr?clear=1` with no collection context)
|
||
|
||
## Limit: 1 ensures precise dependencies
|
||
|
||
By default, an API query for a collection (like `/api/v1/_/content?filter=...`) sets a list dependency `collection:*`. This means _any_ change to ANY entry in that collection will clear the SSR cache for this page.
|
||
|
||
If you are querying a single document (like a page or article based on its path or slug), you should ALWAYS append `limit: 1` to your API call (or pass `{ filter: {...}, limit: 1 }` to `getDBEntries`).
|
||
|
||
When `api/hooks/lib/ssr-server.js` intercepts a request with `limit === 1` and exactly one result is returned, it will register a precise `collection:id` dependency instead of a wildcard `collection:*`. This optimizes the cache drastically, because edits to _other_ pages won't invalidate this page.
|
||
|
||
### Automatic dependency tracking via `lookup` and `aggregate`
|
||
|
||
When options like `lookup` (e.g. `lookup: "image:medialib"`) or `aggregate` (e.g. `aggregate: "comments:contentId:count"`) are provided to an API call, `ssr-server.js` automatically parses these values and adds wildcard cache dependencies (`medialib:*` or `comments:*`) to the page. This guarantees that if a referenced image or child comment changes, the parent's SSR HTML is correctly flushed.
|
||
|
||
## How SSR data loading is supposed to work
|
||
|
||
- Keep `frontend/src/ssr.ts` thin. It should set up locale state and call `render(App, { props: { url } })`.
|
||
- Do not move application-specific prefetch logic into `ssr.ts` unless absolutely necessary.
|
||
- The app itself should own the page-loading behavior.
|
||
- In projects using this starter architecture, the correct pattern is:
|
||
- browser: `$effect(() => loadContent(...))`
|
||
- SSR: call the same `loadContent(...)` once inside `typeof window === "undefined"`
|
||
- This keeps SSR and client navigation on one shared code path.
|
||
- `loadContent(...)` must load **all data required for a fully rendered page**. In this repo that includes both navigation and page content. SSR is incomplete if only the main content entry is loaded.
|
||
- Because goja runs the transformed async path synchronously enough for this setup, the direct SSR call works. The problem was the reactive `$effect`, not the shared async loader itself.
|
||
|
||
## What is project-specific vs. architecture-specific
|
||
|
||
Architecture-specific rules:
|
||
|
||
- SSR entry goes through `api/hooks/ssr/get_read.js`
|
||
- HTML caching lives in the `ssr` collection
|
||
- SSR API calls are tracked through `context.ssrRequest`
|
||
- Client hydration reuses `window.__SSR_CACHE__`
|
||
- The app owns its own data-loading logic
|
||
|
||
Project-specific rules that an LLM must inspect before changing SSR:
|
||
|
||
- which collections contribute to rendered pages
|
||
- which routes should SSR vs. skip SSR
|
||
- whether URLs are language-prefixed
|
||
- whether DB paths are stored with or without language prefix
|
||
- which lookups are required to make a page fully render
|
||
- which collections need publication-aware invalidation
|
||
- whether there are canonical/alias paths
|
||
|
||
Do not hardcode demo assumptions into the skill. Instead, use the architecture rules above and inspect the current project's route model, collections, and page-loading code.
|
||
|
||
## SSR route validation
|
||
|
||
Route validation in `config.js` controls which paths get SSR treatment. Return:
|
||
|
||
- `1` to render the requested path as-is
|
||
- a string to rewrite to the canonical cache path
|
||
- `-1` for not found
|
||
|
||
For projects following this setup, route validation must understand the public URL shape used by the frontend router:
|
||
|
||
- `/` and `/{lang}` are valid SSR roots.
|
||
- Public content URLs are language-prefixed (`/de/...`, `/en/...`).
|
||
- Content entries in the DB are stored **without** the language prefix in `content.path`.
|
||
- `ssrValidatePath()` therefore needs to:
|
||
- extract the language prefix from the URL
|
||
- strip it before querying `content.path`
|
||
- include `{ lang }` in the content query
|
||
- support `alternativePaths.path`
|
||
- return a canonical language-prefixed URL when the request matched via an alternative path
|
||
|
||
If this mapping is wrong, SSR may appear to work for root pages while returning 404 or empty content for real CMS pages.
|
||
|
||
## Publication-aware SSR caching
|
||
|
||
- `config.js` exports `publishedFilter` and `ssrPublishCheckCollections`.
|
||
- `ssrPublishCheckCollections` should include every collection whose publication window can make cached HTML stale.
|
||
- In this starter, `content` is currently included.
|
||
- `ssr-server.js` uses `publication.from` / `publication.to` to compute `context.ssrCacheValidUntil`.
|
||
- `get_read.js` must reject expired cache entries and delete them before rendering anew.
|
||
|
||
## Hydration cache behavior
|
||
|
||
- `api/hooks/lib/ssr.js` uses the same API helper for browser and SSR.
|
||
- On the server, `apiRequest(...)` delegates to `context.ssrRequest(...)`.
|
||
- On the client, `window.__SSR_CACHE__` is checked first for GET requests.
|
||
- This means SSR is not just HTML prerendering; it also primes client-side data access.
|
||
- If HTML renders but `window.__SSR_CACHE__` is missing, the SSR pipeline is incomplete.
|
||
|
||
## SSR 404 signaling
|
||
|
||
When a page is not found during SSR, the framework returns the 404 page but with HTTP status **200** unless a 404 signal is set. The SSR hook (`get_read.js`) checks `context.is404` after rendering:
|
||
|
||
```js
|
||
// get_read.js, after app.default.render()
|
||
if (context.is404) {
|
||
status = 404
|
||
}
|
||
```
|
||
|
||
The signal is set from `NotFound.svelte` — when this component is rendered during SSR, it sets the flag directly. This keeps the 404 logic in the component that owns it:
|
||
|
||
```ts
|
||
// NotFound.svelte — top-level script, runs during render:
|
||
if (typeof window === "undefined") {
|
||
// @ts-ignore - context is the goja global in SSR runtime
|
||
context.is404 = true
|
||
}
|
||
```
|
||
|
||
**Why this works:**
|
||
|
||
- The `tibi-types` package declares `var context: HookContext` as a global (available because goja provides it during SSR).
|
||
- During SSR, `loadContent()` runs synchronously (goja transforms `async`/`await`).
|
||
- By the time `render(App)` returns in `ssr.ts`, `context.is404` is already `true`.
|
||
- `get_read.js` reads it, returns HTTP 404, and the rendered 404 page HTML is sent with the correct status.
|
||
- Caching is automatically skipped for 404 responses.
|
||
|
||
**Verification:** Test with a non-existent URL:
|
||
|
||
```bash
|
||
curl -w "\nHTTP Status: %{http_code}\n" "http://tibiserver:8080/api/v1/_/<namespace>/ssr?url=/de/nicht-existierend"
|
||
# Expected: HTTP 404, body contains the 404 page HTML
|
||
```
|
||
|
||
## What an LLM should inspect first when changing SSR
|
||
|
||
1. `api/hooks/ssr/get_read.js` to understand cache lookup, route validation, and template injection.
|
||
2. `api/hooks/lib/ssr-server.js` to understand dependency tracking and SSR-side API behavior.
|
||
3. `frontend/src/ssr.ts` to confirm how the SSR render wrapper is bootstrapped.
|
||
4. The top-level app/page-loading surface (for example `frontend/src/App.svelte`) to see where data is actually loaded.
|
||
5. `api/hooks/config.js` to understand route validation, canonicalization, and publication-aware collections.
|
||
6. `api/hooks/clear_cache.js` plus `api/hooks/lib/utils.js` to understand invalidation behavior.
|
||
|
||
This order helps an LLM separate infrastructure problems from app-loading problems.
|
||
|
||
## How to verify SSR correctly
|
||
|
||
- Do not rely only on the BrowserSync/frontend proxy when debugging SSR.
|
||
- Test the SSR API endpoint directly, for example:
|
||
|
||
```bash
|
||
curl "http://tibiserver:8080/api/v1/_/<namespace>/ssr?url=/de/ueber-uns"
|
||
```
|
||
|
||
- Verify all of the following:
|
||
- HTTP status is correct
|
||
- expected page content is present in the HTML
|
||
- all page-critical content is present in the HTML
|
||
- navigation labels are present in the HTML when navigation is part of the app shell
|
||
- `window.__SSR_CACHE__` exists
|
||
- no `error:` comment was injected into the template
|
||
- second request returns `X-SSR-Cache: true`
|
||
- `POST /ssr?clear=1` removes cache entries and the next request is a miss again
|
||
|
||
## Common pitfalls
|
||
|
||
- **Do not document Babel anymore**: the current SSR build is esbuild-only.
|
||
- **goja does not parse every modern syntax feature**: dynamic import must be downlevelled in the server build.
|
||
- **Do not leave frontend build options on the server build**: `splitting`/`outdir` inherited from the frontend config will break `build:server`.
|
||
- **No browser globals**: `window`, `document`, `localStorage` etc. don't exist in goja. Guard with `typeof window !== "undefined"`.
|
||
- **`$effect` does not solve SSR loading**: server-side content must be loaded outside browser-only reactive effects.
|
||
- **SSR can look healthy while content is missing**: a 200 response plus app shell is not enough; always verify actual DB content in the HTML.
|
||
- **Navigation is part of SSR**: if header/footer are missing, the SSR setup is still incomplete even when the page body renders.
|
||
- **SSR cache can go stale**: Always ensure `clear_cache.js` covers every collection that affects rendered output.
|
||
- **Do not overfit the skill to demo content**: the skill should explain the architecture and where to inspect project-specific route/content rules, not freeze one content model as universal.
|
||
|
||
## SSR data loading pattern
|
||
|
||
In Svelte 5, SSR data loading works via **top-level `loadData()` calls** (NOT inside `$effect`):
|
||
|
||
```typescript
|
||
// ✅ Richtig: Top-Level-Aufruf für SSR + Browser
|
||
loadData()
|
||
|
||
async function loadData() {
|
||
const data = await getCachedEntries(...)
|
||
state = data
|
||
}
|
||
|
||
// ❌ Falsch: $effect wird für SSR nicht rechtzeitig abgearbeitet
|
||
$effect(() => { loadData() })
|
||
```
|
||
|
||
**Warum das funktioniert:**
|
||
|
||
- `loadData()` läuft während der Component-Initialisierung (vor Template-Auswertung)
|
||
- `getCachedEntries` → `apiRequest` → SSR-Pfad → `context.ssrRequest()` → blockierender HTTP-Fetch in goja
|
||
- goja's `await` auf einem bereits aufgelösten Promise läuft synchron weiter (kein Microtask-Hickhack)
|
||
- State-Änderungen sind vor der Template-Auswertung sichtbar
|
||
|
||
**Browser-Reaktivität:** Wenn Props sich ändern (z.B. Navigation zu anderer Kategorie), wird die Component via `{#if}`/`{#key}` neu erstellt → `loadData()` läuft erneut.
|
||
|
||
## SSR-Cache in der Entwicklung
|
||
|
||
Der SSR-Cache ist das häufigste Debugging-Hindernis. Der Proxy in `esbuild.config.js` MUSS `&noCache=1` an den SSR-Request anhängen:
|
||
|
||
```javascript
|
||
// esbuild.config.js – SSR-Proxy
|
||
pathRewrite: function (path, req) {
|
||
return "/ssr?url=" + encodeURIComponent(path) + "&noCache=1"
|
||
}
|
||
```
|
||
|
||
Ohne `noCache` wird die erste SSR-Antwort gecached und bei Code-Änderungen nicht invalidiert. Der Entwickler sieht immer den alten Stand. **Immer zuerst den Cache-Bypass prüfen, bevor SSR-Fehler gesucht werden.**
|
||
|
||
**Erkennungsmerkmale für veralteten SSR-Cache:**
|
||
|
||
- `X-SSR-Cache: true` im Response-Header
|
||
- `<!--COMMENT--><!--SSR.ERROR-->` im HTML
|
||
- `__SSR_CACHE__` enthält nicht die erwarteten Daten
|
||
- Neustart von tibi-server nötig nach `app.server.js`-Änderungen (`docker restart <tibiserver>`)
|
||
|
||
## Build-Arbeitsschritte bei SSR-Änderungen
|
||
|
||
Nach jeder Änderung an Svelte-Komponenten oder `api.ts` ist folgendes nötig:
|
||
|
||
```bash
|
||
# 1. Frontend-Bundle bauen
|
||
yarn build
|
||
|
||
# 2. SSR-Bundle bauen (app.server.js)
|
||
yarn build:server
|
||
|
||
# 3. tibi-server neustarten (lädt neues app.server.js)
|
||
docker restart <tibiserver>
|
||
|
||
# 4. Frontend neustarten (für Entwicklungs-Proxy)
|
||
make docker-restart-frontend
|
||
```
|
||
|
||
**Wichtig:** `yarn build:server` allein reicht nicht – der tibi-server cached das Modul im Speicher und lädt es nur beim Start neu.
|