tibi-svelte-starter/.agents/skills/playwright-testing/SKILL.md

name, description

name	description
playwright-testing	Build, debug, and extend the current Playwright test setup for API, desktop E2E, mobile E2E, and visual checks. Use when changing tests, seeding deterministic content, or validating frontend/API behavior against the reverse-proxied CODING_URL.

playwright-testing

When to use this skill

Use this skill when:

• Adding or updating Playwright API, E2E, mobile, or visual tests
• Debugging failing tests in tests/
• Extending deterministic seed data for frontend or API coverage
• Verifying how CODING_URL, ADMIN_TOKEN, and collection permissions affect tests
• Deciding where a frontend assertion belongs: API spec, E2E spec, or visual regression spec

---

Current test architecture

This starter uses Playwright across four slices:

• tests/api/ for API-level checks
• tests/e2e/ for desktop browser behavior
• tests/e2e-admin/ for committed admin smoke coverage
• tests/e2e-mobile/ for mobile behavior
• tests/e2e-visual/ for screenshot-based regression tests

The current baseline is deterministic and seed-driven, not demo-content-driven.

Core files

File	Responsibility
playwright.config.ts	Playwright projects, baseURL, global setup/teardown, BrowserSync-safe defaults
tests/fixtures/test-constants.ts	ADMIN_TOKEN, API_BASE, TEST_BASE_URL, seeded route constants
tests/global-setup.ts	Verifies TEST_BASE_URL, probes /api, seeds deterministic content
tests/global-teardown.ts	Cleans seeded content and disposes shared API contexts
tests/api/helpers/admin-api.ts	Shared admin CRUD helper using the static Token: header
tests/api/helpers/seed-data.ts	Seed definitions and seed cleanup for deterministic content pages
tests/fixtures/console-monitor.ts	Fails browser-based tests on unexpected page, console, or request errors
tests/e2e/fixtures.ts	Desktop browser fixtures and SPA helpers
tests/e2e-admin/fixtures.ts	Admin login helpers and admin smoke fixture setup
tests/e2e-mobile/fixtures.ts	Mobile browser fixtures and hamburger-menu helpers

---

Environment prerequisites

Always use the configured CODING_URL

Playwright uses TEST_BASE_URL from tests/fixtures/test-constants.ts.

Resolution order:

1. process.env.CODING_URL
2. CODING_URL from .env
3. fallback http://localhost:3000

For this project, prefer the reverse-proxied CODING_URL from .env whenever it serves both:

• /
• /api/...

If /api/... returns HTML instead of JSON, the seeded setup is not usable and globalSetup should fail fast.

Admin host and default credentials

Admin browser tests use TEST_ADMIN_BASE_URL from tests/fixtures/test-constants.ts.

Resolution order:

1. process.env.CODING_TIBIADMIN_URL
2. CODING_TIBIADMIN_URL from .env
3. fallback http://localhost:3000

The current smoke setup assumes the default dev login unless overridden via env vars:

• ADMIN_UI_USERNAME default: admin
• ADMIN_UI_PASSWORD default: admin

Keep this only for local/dev smoke coverage. Do not turn production credentials into committed test defaults.

Static project token vs JWT user auth

This distinction matters for tests:

• Token: is the static project/admin token from api/config.yml.env
• X-Auth-Token is a JWT user token from a login flow

Collection permissions under user: do not grant access for static Token: requests. If tests seed or mutate a collection through ADMIN_TOKEN, that collection must define explicit token permissions like:

permissions:
    "token:${ADMIN_TOKEN}":
        methods:
            get: true
            post: true
            put: true
            delete: true

This is required for collections the seed helper writes to, such as content.

---

BrowserSync navigation rule

BrowserSync keeps a WebSocket open permanently. Because of that:

• do not wait for networkidle
• do not rely on load
• use domcontentloaded

The shared fixtures already patch navigation helpers accordingly. When writing new tests, keep using the project fixtures rather than raw Playwright test.

Console watcher

Browser-based fixtures attach attachConsoleMonitor(page) from tests/fixtures/console-monitor.ts.

This monitor records and fails tests on unexpected:

• pageerror
• console.error
• failed network requests except explicitly ignored infrastructure noise

The intent is to catch real frontend/runtime regressions even when visible assertions still pass. Do not silence app bugs by broadening ignored patterns unless the noise is clearly external infrastructure.

---

Deterministic seed strategy

The current setup seeds content through the public collection API plus the static Token: header.

Use a hidden per-collection marker field as the default seed identity strategy. In this project the convention is _testdata: true.

Seed lifecycle

1. globalSetup probes the configured base URL.
2. globalSetup verifies /api/content returns JSON.
3. globalSetup removes old seeded entries by their hidden test marker before recreating them.
4. globalSetup creates deterministic seed entries.
5. Tests run against those seeded routes.
6. globalTeardown removes seeded entries again.

Setup cleanup and teardown cleanup are both required. The setup cleanup handles leftovers from aborted or previously failed runs. The teardown cleanup keeps the environment clean after successful runs.

Hidden seed marker pattern

Prefer this pattern for every collection that may receive test-created data:

1. Add a hidden boolean field named _testdata as the last field in the collection schema.
2. Set _testdata: true on every seeded entry.
3. Let cleanup match _testdata === true first.
4. Keep older identifiers such as fixed paths or translation keys only as migration fallbacks when existing seed data already used them.

This is more robust than relying on translation keys because not every collection has a natural grouping field. It also makes leftovers from aborted runs discoverable across heterogeneous collection shapes.

Parallel worker rule

Seed creation and seed cleanup must remain run-scoped, not worker-scoped.

• perform seed cleanup and creation in globalSetup
• perform final seed cleanup in globalTeardown
• do not create or delete shared seeded data in per-test hooks or worker fixtures
• keep seeded identifiers deterministic so many workers can read the same seeded dataset safely

This project runs with many workers. Parallel safety depends on one shared deterministic seed pass before the suite and one shared cleanup pass after the suite, not on each worker mutating shared fixtures independently.

Current seeded routes

Defined in tests/fixtures/test-constants.ts:

• SEEDED_TEST_CONTENT.home.path
• SEEDED_TEST_CONTENT.contact.path
• SEEDED_TEST_CONTENT.inactive.path

These are backed by DE/EN content entries in tests/api/helpers/seed-data.ts.

What the seed currently covers

• localized page routing
• hero rendering
• features rendering
• richtext rendering
• accordion rendering
• contact-form rendering
• inactive route -> 404 behavior

When adding new deterministic coverage, extend the seed data instead of asserting against editorial demo content.

---

Which test type to use

Checklist-facing minimum contract for derived projects

When this starter is used to build a real website project, the testing layer should usually cover these contracts explicitly:

1. deterministic seed setup for the data the suite depends on
2. API smoke coverage for public reads and important write/action behavior
3. desktop E2E coverage for core public journeys such as homepage, navigation, language switching, and 404 behavior
4. admin smoke coverage for stable collection/admin contracts
5. pagebuilder registry plus real preview rendering when the project uses block-based authoring
6. SSR validation through direct endpoint checks, and committed tests where the SSR contract is central and stable enough

Do not treat the test suite as an optional polish step. It is one of the delivery contracts of the project.

API tests

Use tests/api/ when validating:

• collection filters
• public vs token-backed API behavior
• seeded content presence or absence
• mutation semantics independent of DOM rendering

Keep them narrow and data-oriented.

Desktop E2E tests

Use tests/e2e/ when validating:

• route changes
• language switching
• SPA navigation behavior
• block rendering in the real UI
• keyboard/a11y interactions such as skip links

Admin smoke tests

Use tests/e2e-admin/ when validating stable admin contracts such as:

• admin login still works in dev
• the project dashboard opens correctly
• core collections are still reachable
• critical collection views still render their configured labels/columns/actions
• collection lists render meaningful previews instead of broken placeholders
• important field widgets are configured and usable in entry forms
• pagebuilder block choosers, block forms, and live previews load correctly

These tests should stay intentionally narrow. They are regression guards for admin configuration, not full editor journey automation.

Mobile E2E tests

Use tests/e2e-mobile/ when validating:

• hamburger menu behavior
• responsive visibility
• mobile-specific navigation or controls

Visual regression tests

Use tests/e2e-visual/ only when layout/styling stability matters and a semantic DOM assertion is not enough.

SSR validation placement

Do not try to prove every SSR property only through browser navigation.

Use direct SSR endpoint checks when:

• validating route acceptance and canonicalization
• validating SSR HTML content
• validating cache-hit / cache-miss behavior
• validating publication-window effects or cache invalidation after mutations

Use committed API/E2E tests when:

• the SSR-related behavior is stable enough to be a long-lived regression contract
• the project depends heavily on SSR for page-critical content
• a browser-level journey would otherwise hide SSR-specific regressions

Preferred rule:

• infrastructure-like SSR checks start as direct endpoint checks
• promote them into committed tests when the behavior is important and deterministic enough

Admin config coverage strategy

Use a hybrid approach:

• committed Playwright smoke tests for stable, repeatable admin contracts
• one-shot MCP Playwright or VS Code browser checks for exploratory spot checks and ad-hoc audits

Committed tests should cover the admin paths that are expected to stay valid across everyday work, for example:

• login
• opening the Nova project dashboard
• visibility of the core collections
• opening important collection views like content
• checking that collection tables expose the intended columns, summaries, and preview thumbnails
• checking that key widgets like selects, foreign/media pickers, sidebars, and pagebuilder controls actually render
• checking that pagebuilder preview updates when block content changes

One-shot live browser checks are useful when:

• reviewing a newly added admin configuration once
• probing a flaky or hard-to-stabilize UI area before deciding what deserves a real test
• checking something highly visual or temporarily environment-specific

Do not rely on one-shot browser checks as the only safeguard for important admin paths. If a check matters repeatedly, promote it into tests/e2e-admin/.

---

Current fixture conventions

API

Use tests/api/fixtures.ts.

Current fixtures:

• api
• adminApi

adminApi is backed by the static Token: header from ADMIN_TOKEN.

Desktop E2E

Use tests/e2e/fixtures.ts.

Helpers include:

• waitForSpaReady(page)
• navigateToRoute(page, routePath)
• clickSpaLink(page, selector)
• automatic console/page/request error monitoring via attachConsoleMonitor(page)

Admin E2E

Use tests/e2e-admin/fixtures.ts.

Helpers include:

• loginToAdmin(page)
• openNovaProjectDashboard(page)
• automatic console/page/request error monitoring via attachConsoleMonitor(page)

Mobile E2E

Use tests/e2e-mobile/fixtures.ts.

Helpers include:

• waitForSpaReady(page)
• navigateToRoute(page, routePath)
• openHamburgerMenu(page)
• closeHamburgerMenuViaEscape(page)
• automatic console/page/request error monitoring via attachConsoleMonitor(page)

Visual E2E

Use tests/e2e-visual/fixtures.ts.

Helpers include:

• waitForVisualReady(page)
• prepareForScreenshot(page)
• expectScreenshot(page, name, opts)
• automatic console/page/request error monitoring via attachConsoleMonitor(page)

Do not reintroduce the old starter authedPage / testUser assumptions unless the project really needs JWT-user coverage again.

---

Writing stable selectors

Prefer selectors that reflect current rendered structure and locale:

• scope language links to the relevant container (header, main, footer)
• avoid ambiguous getByRole() selectors when the same link text appears twice
• use the actual locale strings from frontend/src/lib/i18n/locales/*.json
• prefer stable block markers like data-block="hero"

Examples:

const header = page.locator("header")
await header.getByRole("link", { name: "en", exact: true }).click()

const homeLink = page.locator("main").getByRole("link", { name: "Zur Startseite" })

---

Recommended commands

Run only the slice you changed.

/usr/bin/node ./node_modules/playwright/cli.js test tests/api/health.spec.ts --project=api
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e/home.spec.ts tests/e2e/demo.spec.ts --project=chromium
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e-admin/smoke.spec.ts --project=admin
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e-mobile/home.mobile.spec.ts --project=mobile-iphonese

If the shell environment is broken, calling the Playwright CLI through /usr/bin/node is acceptable in this workspace.

---

Common failure patterns

/api/... returns HTML

Cause:

• wrong CODING_URL
• fallback to BrowserSync without usable API proxy

Fix:

• verify the configured CODING_URL serves both / and /api/...

403 {"error":"empty token"} on POST/PUT/DELETE

Cause:

• collection allows user: but not "token:${ADMIN_TOKEN}":

Fix:

• add explicit token permissions on that collection

strict mode violations in role selectors

Cause:

• multiple matching links in header/footer/mobile menu

Fix:

• scope selectors to the intended container
• use exact: true where needed

404 assertions fail by link text

Cause:

• locale-specific text mismatch

Fix:

• use the real translated string from the locale JSON files

---

Change workflow

When extending or fixing tests:

1. Start from the failing spec or the exact behavior to cover.
2. Check whether the needed content already exists in seed-data.ts.
3. Extend seed data only if the behavior is not already representable.
4. Run only the affected Playwright project/spec files.
5. Fix selectors or seed shape before widening scope.

Keep the test basis deterministic. Do not fall back to existing editorial demo content just because it is already present in the database.

Admin E2E: Boot abwarten

Admin-SPA lädt Chunks asynchron. Vor Interaktionen auf sichtbares Login-Formular warten:

await page.goto("/login")
await expect(page.getByLabel(/Benutzername|Username/i)).toBeVisible({ timeout: 20000 })

Danach erst fill/click – das Formular erscheint erst wenn die App vollständig gebootet ist.

MailDev E-Mail-Testing

MailDev läuft im Docker-Stack (SMTP Port 25, Web-API Port 1080). Die REST-API erlaubt E-Mails zu lesen und zu löschen:

const MAILDEV = "https://{project}-maildev.code.testversion.online"

// Alle E-Mails abrufen
const res = await request.get(`${MAILDEV}/email`)
const emails = await res.json()

// Alle löschen (vor Test)
await request.delete(`${MAILDEV}/email/all`)

Polling-Pattern für asynchrone E-Mails

Formular → Action-Hook sendet E-Mail via context.smtp.sendMail(). MailDev braucht Zeit zum Verarbeiten:

// Nach Form-Submit auf E-Mails warten
for (let i = 0; i < 15; i++) {
    await new Promise((r) => setTimeout(r, 1000))
    const res = await request.get(`${MAILDEV}/email`)
    if (res.ok()) {
        const emails = await res.json()
        if (emails.length >= 2) break // Kunde + Betreiber
    }
}
emails.find((e) => e.to.some((t) => t.address === "kunde@test.de"))

Wichtig: Tests mit MailDev müssen sequentiell laufen (--workers=1), da parallele Tests sich gegenseitig die MailDev-Inbox überschreiben.

Admin pagebuilder registry coverage

For starter-like projects, committed admin coverage should include both sides of the pagebuilder contract:

1. registry/chooser coverage on a new entry form
2. actual preview rendering on an existing seeded entry

The second check is important because it catches failures that the chooser alone does not see:

• broken meta.pagebuilder.blockRegistry.file wiring
• preview components that no longer mount through the shared block renderer
• missing _lookup hydration for foreign media fields
• image widgets that work on the public site but fail in admin preview because the API base or URL resolution is wrong

Preferred starter pattern:

• seed one deterministic medialib image through the collection API
• seed one deterministic content entry that references that image in at least one pagebuilder block
• open that entry in tests/e2e-admin/pagebuilder.spec.ts
• assert both block text and img[data-entry-id] preview rendering

Keep this test generic. Do not tie it to customer-specific block sets unless the project has already diverged from the starter pattern.

Delivery-checklist alignment

When using this skill together with .agents/BUILD_CHECKLIST.md, the testing phase should leave behind explicit evidence for:

• which specs were run
• which seed data was extended or reused
• whether admin smoke coverage exists for the configured collections
• whether pagebuilder preview rendering is covered when pagebuilder is in scope
• whether SSR was verified by direct endpoint checks, committed tests, or both

If that evidence only exists in chat history and not in the repo or task notes, the testing work is too fragile for later agents.