---
name: playwright-testing
description: Build, debug, and extend the current Playwright test setup for API, desktop E2E, mobile E2E, and visual checks. Use when changing tests, seeding deterministic content, or validating frontend/API behavior against the reverse-proxied CODING_URL.
---

# playwright-testing

## When to use this skill

Use this skill when:

- Adding or updating Playwright API, E2E, mobile, or visual tests
- Debugging failing tests in `tests/`
- Extending deterministic seed data for frontend or API coverage
- Verifying how `CODING_URL`, `ADMIN_TOKEN`, and collection permissions affect tests
- Deciding where a frontend assertion belongs: API spec, E2E spec, or visual regression spec

---

## Current test architecture

This starter uses Playwright across four slices:

- `tests/api/` for API-level checks
- `tests/e2e/` for desktop browser behavior
- `tests/e2e-admin/` for committed admin smoke coverage
- `tests/e2e-mobile/` for mobile behavior
- `tests/e2e-visual/` for screenshot-based regression tests

The current baseline is deterministic and seed-driven, not demo-content-driven.

### Core files

| File                                | Responsibility                                                                   |
| ----------------------------------- | -------------------------------------------------------------------------------- |
| `playwright.config.ts`              | Playwright projects, `baseURL`, global setup/teardown, BrowserSync-safe defaults |
| `tests/fixtures/test-constants.ts`  | `ADMIN_TOKEN`, `API_BASE`, `TEST_BASE_URL`, seeded route constants               |
| `tests/global-setup.ts`             | Verifies `TEST_BASE_URL`, probes `/api`, seeds deterministic content             |
| `tests/global-teardown.ts`          | Cleans seeded content and disposes shared API contexts                           |
| `tests/api/helpers/admin-api.ts`    | Shared admin CRUD helper using the static `Token:` header                        |
| `tests/api/helpers/seed-data.ts`    | Seed definitions and seed cleanup for deterministic content pages                |
| `tests/fixtures/console-monitor.ts` | Fails browser-based tests on unexpected page, console, or request errors         |
| `tests/e2e/fixtures.ts`             | Desktop browser fixtures and SPA helpers                                         |
| `tests/e2e-admin/fixtures.ts`       | Admin login helpers and admin smoke fixture setup                                |
| `tests/e2e-mobile/fixtures.ts`      | Mobile browser fixtures and hamburger-menu helpers                               |

---

## Environment prerequisites

### Always use the configured `CODING_URL`

Playwright uses `TEST_BASE_URL` from `tests/fixtures/test-constants.ts`.

Resolution order:

1. `process.env.CODING_URL`
2. `CODING_URL` from `.env`
3. fallback `http://localhost:3000`

For this project, prefer the reverse-proxied `CODING_URL` from `.env` whenever it serves both:

- `/`
- `/api/...`

If `/api/...` returns HTML instead of JSON, the seeded setup is not usable and `globalSetup` should fail fast.

### Admin host and default credentials

Admin browser tests use `TEST_ADMIN_BASE_URL` from `tests/fixtures/test-constants.ts`.

Resolution order:

1. `process.env.CODING_TIBIADMIN_URL`
2. `CODING_TIBIADMIN_URL` from `.env`
3. fallback `http://localhost:3000`

The current smoke setup assumes the default dev login unless overridden via env vars:

- `ADMIN_UI_USERNAME` default: `admin`
- `ADMIN_UI_PASSWORD` default: `admin`

Keep this only for local/dev smoke coverage. Do not turn production credentials into committed test defaults.

### Static project token vs JWT user auth

This distinction matters for tests:

- `Token:` is the static project/admin token from `api/config.yml.env`
- `X-Auth-Token` is a JWT user token from a login flow

Collection permissions under `user:` do **not** grant access for static `Token:` requests.
If tests seed or mutate a collection through `ADMIN_TOKEN`, that collection must define explicit token permissions like:

```yaml
permissions:
    "token:${ADMIN_TOKEN}":
        methods:
            get: true
            post: true
            put: true
            delete: true
```

This is required for collections the seed helper writes to, such as `content`.

---

## BrowserSync navigation rule

BrowserSync keeps a WebSocket open permanently. Because of that:

- do **not** wait for `networkidle`
- do **not** rely on `load`
- use `domcontentloaded`

The shared fixtures already patch navigation helpers accordingly.
When writing new tests, keep using the project fixtures rather than raw Playwright `test`.

## Console watcher

Browser-based fixtures attach `attachConsoleMonitor(page)` from `tests/fixtures/console-monitor.ts`.

This monitor records and fails tests on unexpected:

- `pageerror`
- `console.error`
- failed network requests except explicitly ignored infrastructure noise

The intent is to catch real frontend/runtime regressions even when visible assertions still pass.
Do not silence app bugs by broadening ignored patterns unless the noise is clearly external infrastructure.

---

## Deterministic seed strategy

The current setup seeds content through the public collection API plus the static `Token:` header.

Use a hidden per-collection marker field as the default seed identity strategy.
In this project the convention is `_testdata: true`.

### Seed lifecycle

1. `globalSetup` probes the configured base URL.
2. `globalSetup` verifies `/api/content` returns JSON.
3. `globalSetup` removes old seeded entries by their hidden test marker before recreating them.
4. `globalSetup` creates deterministic seed entries.
5. Tests run against those seeded routes.
6. `globalTeardown` removes seeded entries again.

Setup cleanup and teardown cleanup are both required.
The setup cleanup handles leftovers from aborted or previously failed runs.
The teardown cleanup keeps the environment clean after successful runs.

### Hidden seed marker pattern

Prefer this pattern for every collection that may receive test-created data:

1. Add a hidden boolean field named `_testdata` as the last field in the collection schema.
2. Set `_testdata: true` on every seeded entry.
3. Let cleanup match `_testdata === true` first.
4. Keep older identifiers such as fixed paths or translation keys only as migration fallbacks when existing seed data already used them.

This is more robust than relying on translation keys because not every collection has a natural grouping field.
It also makes leftovers from aborted runs discoverable across heterogeneous collection shapes.

### Parallel worker rule

Seed creation and seed cleanup must remain run-scoped, not worker-scoped.

- perform seed cleanup and creation in `globalSetup`
- perform final seed cleanup in `globalTeardown`
- do not create or delete shared seeded data in per-test hooks or worker fixtures
- keep seeded identifiers deterministic so many workers can read the same seeded dataset safely

This project runs with many workers.
Parallel safety depends on one shared deterministic seed pass before the suite and one shared cleanup pass after the suite, not on each worker mutating shared fixtures independently.

### Current seeded routes

Defined in `tests/fixtures/test-constants.ts`:

- `SEEDED_TEST_CONTENT.home.path`
- `SEEDED_TEST_CONTENT.contact.path`
- `SEEDED_TEST_CONTENT.inactive.path`

These are backed by DE/EN content entries in `tests/api/helpers/seed-data.ts`.

### What the seed currently covers

- localized page routing
- hero rendering
- features rendering
- richtext rendering
- accordion rendering
- contact-form rendering
- inactive route -> 404 behavior

When adding new deterministic coverage, extend the seed data instead of asserting against editorial demo content.

---

## Which test type to use

## Checklist-facing minimum contract for derived projects

When this starter is used to build a real website project, the testing layer should usually cover these contracts explicitly:

1. deterministic seed setup for the data the suite depends on
2. API smoke coverage for public reads and important write/action behavior
3. desktop E2E coverage for core public journeys such as homepage, navigation, language switching, and 404 behavior
4. admin smoke coverage for stable collection/admin contracts
5. pagebuilder registry plus real preview rendering when the project uses block-based authoring
6. SSR validation through direct endpoint checks, and committed tests where the SSR contract is central and stable enough

Do not treat the test suite as an optional polish step. It is one of the delivery contracts of the project.

### API tests

Use `tests/api/` when validating:

- collection filters
- public vs token-backed API behavior
- seeded content presence or absence
- mutation semantics independent of DOM rendering

Keep them narrow and data-oriented.

### Desktop E2E tests

Use `tests/e2e/` when validating:

- route changes
- language switching
- SPA navigation behavior
- block rendering in the real UI
- keyboard/a11y interactions such as skip links

### Admin smoke tests

Use `tests/e2e-admin/` when validating stable admin contracts such as:

- admin login still works in dev
- the project dashboard opens correctly
- core collections are still reachable
- critical collection views still render their configured labels/columns/actions
- collection lists render meaningful previews instead of broken placeholders
- important field widgets are configured and usable in entry forms
- pagebuilder block choosers, block forms, and live previews load correctly

These tests should stay intentionally narrow. They are regression guards for admin configuration, not full editor journey automation.

### Mobile E2E tests

Use `tests/e2e-mobile/` when validating:

- hamburger menu behavior
- responsive visibility
- mobile-specific navigation or controls

### Visual regression tests

Use `tests/e2e-visual/` only when layout/styling stability matters and a semantic DOM assertion is not enough.

## SSR validation placement

Do not try to prove every SSR property only through browser navigation.

Use direct SSR endpoint checks when:

- validating route acceptance and canonicalization
- validating SSR HTML content
- validating cache-hit / cache-miss behavior
- validating publication-window effects or cache invalidation after mutations

Use committed API/E2E tests when:

- the SSR-related behavior is stable enough to be a long-lived regression contract
- the project depends heavily on SSR for page-critical content
- a browser-level journey would otherwise hide SSR-specific regressions

Preferred rule:

- infrastructure-like SSR checks start as direct endpoint checks
- promote them into committed tests when the behavior is important and deterministic enough

## Admin config coverage strategy

Use a hybrid approach:

- committed Playwright smoke tests for stable, repeatable admin contracts
- one-shot MCP Playwright or VS Code browser checks for exploratory spot checks and ad-hoc audits

Committed tests should cover the admin paths that are expected to stay valid across everyday work, for example:

- login
- opening the Nova project dashboard
- visibility of the core collections
- opening important collection views like `content`
- checking that collection tables expose the intended columns, summaries, and preview thumbnails
- checking that key widgets like selects, foreign/media pickers, sidebars, and pagebuilder controls actually render
- checking that pagebuilder preview updates when block content changes

One-shot live browser checks are useful when:

- reviewing a newly added admin configuration once
- probing a flaky or hard-to-stabilize UI area before deciding what deserves a real test
- checking something highly visual or temporarily environment-specific

Do not rely on one-shot browser checks as the only safeguard for important admin paths. If a check matters repeatedly, promote it into `tests/e2e-admin/`.

---

## Current fixture conventions

### API

Use `tests/api/fixtures.ts`.

Current fixtures:

- `api`
- `adminApi`

`adminApi` is backed by the static `Token:` header from `ADMIN_TOKEN`.

### Desktop E2E

Use `tests/e2e/fixtures.ts`.

Helpers include:

- `waitForSpaReady(page)`
- `navigateToRoute(page, routePath)`
- `clickSpaLink(page, selector)`
- automatic console/page/request error monitoring via `attachConsoleMonitor(page)`

### Admin E2E

Use `tests/e2e-admin/fixtures.ts`.

Helpers include:

- `loginToAdmin(page)`
- `openNovaProjectDashboard(page)`
- automatic console/page/request error monitoring via `attachConsoleMonitor(page)`

### Mobile E2E

Use `tests/e2e-mobile/fixtures.ts`.

Helpers include:

- `waitForSpaReady(page)`
- `navigateToRoute(page, routePath)`
- `openHamburgerMenu(page)`
- `closeHamburgerMenuViaEscape(page)`
- automatic console/page/request error monitoring via `attachConsoleMonitor(page)`

### Visual E2E

Use `tests/e2e-visual/fixtures.ts`.

Helpers include:

- `waitForVisualReady(page)`
- `prepareForScreenshot(page)`
- `expectScreenshot(page, name, opts)`
- automatic console/page/request error monitoring via `attachConsoleMonitor(page)`

Do not reintroduce the old starter `authedPage` / `testUser` assumptions unless the project really needs JWT-user coverage again.

---

## Writing stable selectors

Prefer selectors that reflect current rendered structure and locale:

- scope language links to the relevant container (`header`, `main`, footer)
- avoid ambiguous `getByRole()` selectors when the same link text appears twice
- use the actual locale strings from `frontend/src/lib/i18n/locales/*.json`
- prefer stable block markers like `data-block="hero"`

Examples:

```ts
const header = page.locator("header")
await header.getByRole("link", { name: "en", exact: true }).click()

const homeLink = page.locator("main").getByRole("link", { name: "Zur Startseite" })
```

---

## Recommended commands

Run only the slice you changed.

```bash
/usr/bin/node ./node_modules/playwright/cli.js test tests/api/health.spec.ts --project=api
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e/home.spec.ts tests/e2e/demo.spec.ts --project=chromium
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e-admin/smoke.spec.ts --project=admin
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e-mobile/home.mobile.spec.ts --project=mobile-iphonese
```

If the shell environment is broken, calling the Playwright CLI through `/usr/bin/node` is acceptable in this workspace.

---

## Common failure patterns

### `/api/...` returns HTML

Cause:

- wrong `CODING_URL`
- fallback to BrowserSync without usable API proxy

Fix:

- verify the configured `CODING_URL` serves both `/` and `/api/...`

### `403 {"error":"empty token"}` on POST/PUT/DELETE

Cause:

- collection allows `user:` but not `"token:${ADMIN_TOKEN}":`

Fix:

- add explicit token permissions on that collection

### strict mode violations in role selectors

Cause:

- multiple matching links in header/footer/mobile menu

Fix:

- scope selectors to the intended container
- use `exact: true` where needed

### 404 assertions fail by link text

Cause:

- locale-specific text mismatch

Fix:

- use the real translated string from the locale JSON files

---

## Change workflow

When extending or fixing tests:

1. Start from the failing spec or the exact behavior to cover.
2. Check whether the needed content already exists in `seed-data.ts`.
3. Extend seed data only if the behavior is not already representable.
4. Run only the affected Playwright project/spec files.
5. Fix selectors or seed shape before widening scope.

Keep the test basis deterministic. Do not fall back to existing editorial demo content just because it is already present in the database.

## Admin E2E: Boot abwarten

Admin-SPA lädt Chunks asynchron. Vor Interaktionen auf sichtbares Login-Formular warten:

```ts
await page.goto("/login")
await expect(page.getByLabel(/Benutzername|Username/i)).toBeVisible({ timeout: 20000 })
```

Danach erst fill/click – das Formular erscheint erst wenn die App vollständig gebootet ist.

## MailDev E-Mail-Testing

MailDev läuft im Docker-Stack (SMTP Port 25, Web-API Port 1080). Die REST-API erlaubt E-Mails zu lesen und zu löschen:

```ts
const MAILDEV = "https://{project}-maildev.code.testversion.online"

// Alle E-Mails abrufen
const res = await request.get(`${MAILDEV}/email`)
const emails = await res.json()

// Alle löschen (vor Test)
await request.delete(`${MAILDEV}/email/all`)
```

### Polling-Pattern für asynchrone E-Mails

Formular → Action-Hook sendet E-Mail via `context.smtp.sendMail()`. MailDev braucht Zeit zum Verarbeiten:

```ts
// Nach Form-Submit auf E-Mails warten
for (let i = 0; i < 15; i++) {
    await new Promise((r) => setTimeout(r, 1000))
    const res = await request.get(`${MAILDEV}/email`)
    if (res.ok()) {
        const emails = await res.json()
        if (emails.length >= 2) break // Kunde + Betreiber
    }
}
emails.find((e) => e.to.some((t) => t.address === "kunde@test.de"))
```

Wichtig: Tests mit MailDev müssen sequentiell laufen (`--workers=1`), da parallele Tests sich gegenseitig die MailDev-Inbox überschreiben.

## Admin pagebuilder registry coverage

For starter-like projects, committed admin coverage should include both sides of the pagebuilder contract:

1. registry/chooser coverage on a new entry form
2. actual preview rendering on an existing seeded entry

The second check is important because it catches failures that the chooser alone does not see:

- broken `meta.pagebuilder.blockRegistry.file` wiring
- preview components that no longer mount through the shared block renderer
- missing `_lookup` hydration for foreign media fields
- image widgets that work on the public site but fail in admin preview because the API base or URL resolution is wrong

Preferred starter pattern:

- seed one deterministic medialib image through the collection API
- seed one deterministic content entry that references that image in at least one pagebuilder block
- open that entry in `tests/e2e-admin/pagebuilder.spec.ts`
- assert both block text and `img[data-entry-id]` preview rendering

Keep this test generic. Do not tie it to customer-specific block sets unless the project has already diverged from the starter pattern.

## Delivery-checklist alignment

When using this skill together with `.agents/BUILD_CHECKLIST.md`, the testing phase should leave behind explicit evidence for:

- which specs were run
- which seed data was extended or reused
- whether admin smoke coverage exists for the configured collections
- whether pagebuilder preview rendering is covered when pagebuilder is in scope
- whether SSR was verified by direct endpoint checks, committed tests, or both

If that evidence only exists in chat history and not in the repo or task notes, the testing work is too fragile for later agents.