✨ feat: add playwright-testing skill documentation and enhance AGENTS.md with testing references
This commit is contained in:
@@ -0,0 +1,331 @@
|
||||
---
|
||||
name: playwright-testing
|
||||
description: Build, debug, and extend the current Playwright test setup for API, desktop E2E, mobile E2E, and visual checks. Use when changing tests, seeding deterministic content, or validating frontend/API behavior against the reverse-proxied CODING_URL.
|
||||
---
|
||||
|
||||
# playwright-testing
|
||||
|
||||
## When to use this skill
|
||||
|
||||
Use this skill when:
|
||||
|
||||
- Adding or updating Playwright API, E2E, mobile, or visual tests
|
||||
- Debugging failing tests in `tests/`
|
||||
- Extending deterministic seed data for frontend or API coverage
|
||||
- Verifying how `CODING_URL`, `ADMIN_TOKEN`, and collection permissions affect tests
|
||||
- Deciding where a frontend assertion belongs: API spec, E2E spec, or visual regression spec
|
||||
|
||||
---
|
||||
|
||||
## Current test architecture
|
||||
|
||||
This starter uses Playwright across four slices:
|
||||
|
||||
- `tests/api/` for API-level checks
|
||||
- `tests/e2e/` for desktop browser behavior
|
||||
- `tests/e2e-mobile/` for mobile behavior
|
||||
- `tests/e2e-visual/` for screenshot-based regression tests
|
||||
|
||||
The current baseline is deterministic and seed-driven, not demo-content-driven.
|
||||
|
||||
### Core files
|
||||
|
||||
| File | Responsibility |
|
||||
| ----------------------------------- | -------------------------------------------------------------------------------- |
|
||||
| `playwright.config.ts` | Playwright projects, `baseURL`, global setup/teardown, BrowserSync-safe defaults |
|
||||
| `tests/fixtures/test-constants.ts` | `ADMIN_TOKEN`, `API_BASE`, `TEST_BASE_URL`, seeded route constants |
|
||||
| `tests/global-setup.ts` | Verifies `TEST_BASE_URL`, probes `/api`, seeds deterministic content |
|
||||
| `tests/global-teardown.ts` | Cleans seeded content and disposes shared API contexts |
|
||||
| `tests/api/helpers/admin-api.ts` | Shared admin CRUD helper using the static `Token:` header |
|
||||
| `tests/api/helpers/seed-data.ts` | Seed definitions and seed cleanup for deterministic content pages |
|
||||
| `tests/fixtures/console-monitor.ts` | Fails browser-based tests on unexpected page, console, or request errors |
|
||||
| `tests/e2e/fixtures.ts` | Desktop browser fixtures and SPA helpers |
|
||||
| `tests/e2e-mobile/fixtures.ts` | Mobile browser fixtures and hamburger-menu helpers |
|
||||
|
||||
---
|
||||
|
||||
## Environment prerequisites
|
||||
|
||||
### Always use the configured `CODING_URL`
|
||||
|
||||
Playwright uses `TEST_BASE_URL` from `tests/fixtures/test-constants.ts`.
|
||||
|
||||
Resolution order:
|
||||
|
||||
1. `process.env.CODING_URL`
|
||||
2. `CODING_URL` from `.env`
|
||||
3. fallback `http://localhost:3000`
|
||||
|
||||
For this project, prefer the reverse-proxied `CODING_URL` from `.env` whenever it serves both:
|
||||
|
||||
- `/`
|
||||
- `/api/...`
|
||||
|
||||
If `/api/...` returns HTML instead of JSON, the seeded setup is not usable and `globalSetup` should fail fast.
|
||||
|
||||
### Static project token vs JWT user auth
|
||||
|
||||
This distinction matters for tests:
|
||||
|
||||
- `Token:` is the static project/admin token from `api/config.yml.env`
|
||||
- `X-Auth-Token` is a JWT user token from a login flow
|
||||
|
||||
Collection permissions under `user:` do **not** grant access for static `Token:` requests.
|
||||
If tests seed or mutate a collection through `ADMIN_TOKEN`, that collection must define explicit token permissions like:
|
||||
|
||||
```yaml
|
||||
permissions:
|
||||
"token:${ADMIN_TOKEN}":
|
||||
methods:
|
||||
get: true
|
||||
post: true
|
||||
put: true
|
||||
delete: true
|
||||
```
|
||||
|
||||
This is required for collections the seed helper writes to, such as `content`.
|
||||
|
||||
---
|
||||
|
||||
## BrowserSync navigation rule
|
||||
|
||||
BrowserSync keeps a WebSocket open permanently. Because of that:
|
||||
|
||||
- do **not** wait for `networkidle`
|
||||
- do **not** rely on `load`
|
||||
- use `domcontentloaded`
|
||||
|
||||
The shared fixtures already patch navigation helpers accordingly.
|
||||
When writing new tests, keep using the project fixtures rather than raw Playwright `test`.
|
||||
|
||||
## Console watcher
|
||||
|
||||
Browser-based fixtures attach `attachConsoleMonitor(page)` from `tests/fixtures/console-monitor.ts`.
|
||||
|
||||
This monitor records and fails tests on unexpected:
|
||||
|
||||
- `pageerror`
|
||||
- `console.error`
|
||||
- failed network requests except explicitly ignored infrastructure noise
|
||||
|
||||
The intent is to catch real frontend/runtime regressions even when visible assertions still pass.
|
||||
Do not silence app bugs by broadening ignored patterns unless the noise is clearly external infrastructure.
|
||||
|
||||
---
|
||||
|
||||
## Deterministic seed strategy
|
||||
|
||||
The current setup seeds content through the public collection API plus the static `Token:` header.
|
||||
|
||||
### Seed lifecycle
|
||||
|
||||
1. `globalSetup` probes the configured base URL.
|
||||
2. `globalSetup` verifies `/api/content` returns JSON.
|
||||
3. `globalSetup` removes old seeded entries by `translationKey`.
|
||||
4. `globalSetup` creates deterministic seed entries.
|
||||
5. Tests run against those seeded routes.
|
||||
6. `globalTeardown` removes seeded entries again.
|
||||
|
||||
### Current seeded routes
|
||||
|
||||
Defined in `tests/fixtures/test-constants.ts`:
|
||||
|
||||
- `SEEDED_TEST_CONTENT.home.path`
|
||||
- `SEEDED_TEST_CONTENT.contact.path`
|
||||
- `SEEDED_TEST_CONTENT.inactive.path`
|
||||
|
||||
These are backed by DE/EN content entries in `tests/api/helpers/seed-data.ts`.
|
||||
|
||||
### What the seed currently covers
|
||||
|
||||
- localized page routing
|
||||
- hero rendering
|
||||
- features rendering
|
||||
- richtext rendering
|
||||
- accordion rendering
|
||||
- contact-form rendering
|
||||
- inactive route -> 404 behavior
|
||||
|
||||
When adding new deterministic coverage, extend the seed data instead of asserting against editorial demo content.
|
||||
|
||||
---
|
||||
|
||||
## Which test type to use
|
||||
|
||||
### API tests
|
||||
|
||||
Use `tests/api/` when validating:
|
||||
|
||||
- collection filters
|
||||
- public vs token-backed API behavior
|
||||
- seeded content presence or absence
|
||||
- mutation semantics independent of DOM rendering
|
||||
|
||||
Keep them narrow and data-oriented.
|
||||
|
||||
### Desktop E2E tests
|
||||
|
||||
Use `tests/e2e/` when validating:
|
||||
|
||||
- route changes
|
||||
- language switching
|
||||
- SPA navigation behavior
|
||||
- block rendering in the real UI
|
||||
- keyboard/a11y interactions such as skip links
|
||||
|
||||
### Mobile E2E tests
|
||||
|
||||
Use `tests/e2e-mobile/` when validating:
|
||||
|
||||
- hamburger menu behavior
|
||||
- responsive visibility
|
||||
- mobile-specific navigation or controls
|
||||
|
||||
### Visual regression tests
|
||||
|
||||
Use `tests/e2e-visual/` only when layout/styling stability matters and a semantic DOM assertion is not enough.
|
||||
|
||||
---
|
||||
|
||||
## Current fixture conventions
|
||||
|
||||
### API
|
||||
|
||||
Use `tests/api/fixtures.ts`.
|
||||
|
||||
Current fixtures:
|
||||
|
||||
- `api`
|
||||
- `adminApi`
|
||||
|
||||
`adminApi` is backed by the static `Token:` header from `ADMIN_TOKEN`.
|
||||
|
||||
### Desktop E2E
|
||||
|
||||
Use `tests/e2e/fixtures.ts`.
|
||||
|
||||
Helpers include:
|
||||
|
||||
- `waitForSpaReady(page)`
|
||||
- `navigateToRoute(page, routePath)`
|
||||
- `clickSpaLink(page, selector)`
|
||||
- automatic console/page/request error monitoring via `attachConsoleMonitor(page)`
|
||||
|
||||
### Mobile E2E
|
||||
|
||||
Use `tests/e2e-mobile/fixtures.ts`.
|
||||
|
||||
Helpers include:
|
||||
|
||||
- `waitForSpaReady(page)`
|
||||
- `navigateToRoute(page, routePath)`
|
||||
- `openHamburgerMenu(page)`
|
||||
- `closeHamburgerMenuViaEscape(page)`
|
||||
- automatic console/page/request error monitoring via `attachConsoleMonitor(page)`
|
||||
|
||||
### Visual E2E
|
||||
|
||||
Use `tests/e2e-visual/fixtures.ts`.
|
||||
|
||||
Helpers include:
|
||||
|
||||
- `waitForVisualReady(page)`
|
||||
- `prepareForScreenshot(page)`
|
||||
- `expectScreenshot(page, name, opts)`
|
||||
- automatic console/page/request error monitoring via `attachConsoleMonitor(page)`
|
||||
|
||||
Do not reintroduce the old starter `authedPage` / `testUser` assumptions unless the project really needs JWT-user coverage again.
|
||||
|
||||
---
|
||||
|
||||
## Writing stable selectors
|
||||
|
||||
Prefer selectors that reflect current rendered structure and locale:
|
||||
|
||||
- scope language links to the relevant container (`header`, `main`, footer)
|
||||
- avoid ambiguous `getByRole()` selectors when the same link text appears twice
|
||||
- use the actual locale strings from `frontend/src/lib/i18n/locales/*.json`
|
||||
- prefer stable block markers like `data-block="hero"`
|
||||
|
||||
Examples:
|
||||
|
||||
```ts
|
||||
const header = page.locator("header")
|
||||
await header.getByRole("link", { name: "en", exact: true }).click()
|
||||
|
||||
const homeLink = page.locator("main").getByRole("link", { name: "Zur Startseite" })
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recommended commands
|
||||
|
||||
Run only the slice you changed.
|
||||
|
||||
```bash
|
||||
/usr/bin/node ./node_modules/playwright/cli.js test tests/api/health.spec.ts --project=api
|
||||
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e/home.spec.ts tests/e2e/demo.spec.ts --project=chromium
|
||||
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e-mobile/home.mobile.spec.ts --project=mobile-iphonese
|
||||
```
|
||||
|
||||
If the shell environment is broken, calling the Playwright CLI through `/usr/bin/node` is acceptable in this workspace.
|
||||
|
||||
---
|
||||
|
||||
## Common failure patterns
|
||||
|
||||
### `/api/...` returns HTML
|
||||
|
||||
Cause:
|
||||
|
||||
- wrong `CODING_URL`
|
||||
- fallback to BrowserSync without usable API proxy
|
||||
|
||||
Fix:
|
||||
|
||||
- verify the configured `CODING_URL` serves both `/` and `/api/...`
|
||||
|
||||
### `403 {"error":"empty token"}` on POST/PUT/DELETE
|
||||
|
||||
Cause:
|
||||
|
||||
- collection allows `user:` but not `"token:${ADMIN_TOKEN}":`
|
||||
|
||||
Fix:
|
||||
|
||||
- add explicit token permissions on that collection
|
||||
|
||||
### strict mode violations in role selectors
|
||||
|
||||
Cause:
|
||||
|
||||
- multiple matching links in header/footer/mobile menu
|
||||
|
||||
Fix:
|
||||
|
||||
- scope selectors to the intended container
|
||||
- use `exact: true` where needed
|
||||
|
||||
### 404 assertions fail by link text
|
||||
|
||||
Cause:
|
||||
|
||||
- locale-specific text mismatch
|
||||
|
||||
Fix:
|
||||
|
||||
- use the real translated string from the locale JSON files
|
||||
|
||||
---
|
||||
|
||||
## Change workflow
|
||||
|
||||
When extending or fixing tests:
|
||||
|
||||
1. Start from the failing spec or the exact behavior to cover.
|
||||
2. Check whether the needed content already exists in `seed-data.ts`.
|
||||
3. Extend seed data only if the behavior is not already representable.
|
||||
4. Run only the affected Playwright project/spec files.
|
||||
5. Fix selectors or seed shape before widening scope.
|
||||
|
||||
Keep the test basis deterministic. Do not fall back to existing editorial demo content just because it is already present in the database.
|
||||
@@ -178,6 +178,7 @@ These skills provide deep-dive documentation. Use them by phase instead of treat
|
||||
| --- | --- |
|
||||
| `frontend-architecture` | Routing, state management, Svelte 5 patterns, API layer, error handling |
|
||||
| `tibi-ssr-caching` | SSR rendering and cache invalidation |
|
||||
| `playwright-testing` | Extending or debugging seeded Playwright API, desktop, mobile, or visual tests |
|
||||
|
||||
### Quickstart roadmap for a new website
|
||||
|
||||
@@ -205,7 +206,7 @@ Use this order when building a project from scratch on this starter:
|
||||
Use `nova-ai-editor-features` only when there is a concrete editorial workflow for it.
|
||||
|
||||
8. Final verification.
|
||||
Confirm public site, admin authoring, pagebuilder rendering, navigation/media resolution, forms/actions, and SSR all work, then run `yarn build`, `yarn build:server`, and `yarn validate`.
|
||||
Use `playwright-testing` for deterministic API/E2E coverage, confirm public site, admin authoring, pagebuilder rendering, navigation/media resolution, forms/actions, and SSR all work, then run `yarn build`, `yarn build:server`, and `yarn validate`.
|
||||
|
||||
For one-off tasks, use the phase tables above as the lookup index instead of maintaining a second skill matrix here.
|
||||
|
||||
|
||||
+23
-4
@@ -2,6 +2,8 @@
|
||||
|
||||
Playwright tests for E2E, API, mobile, and visual regression. Config in `playwright.config.ts`.
|
||||
|
||||
For the full current workflow, prefer the `playwright-testing` skill.
|
||||
|
||||
## Running tests
|
||||
|
||||
- All tests: `yarn test`
|
||||
@@ -11,19 +13,36 @@ Playwright tests for E2E, API, mobile, and visual regression. Config in `playwri
|
||||
- Single file: `npx playwright test tests/e2e/filename.spec.ts`
|
||||
- Single test: `npx playwright test -g "test name"`
|
||||
- After code changes, run only affected spec files — not the full suite.
|
||||
- Prefer the configured `CODING_URL` from `.env` whenever it serves both `/` and `/api/...`.
|
||||
- The current test baseline is deterministic and seed-driven, not demo-content-driven.
|
||||
|
||||
## BrowserSync workaround
|
||||
|
||||
BrowserSync keeps a WebSocket open permanently, preventing `networkidle` and `load` from resolving. All fixture files override `page.goto()` and `page.reload()` to use `waitUntil: "domcontentloaded"` by default. Always use `domcontentloaded` for navigation waits.
|
||||
|
||||
## Console watcher
|
||||
|
||||
- Browser-based fixtures attach `tests/fixtures/console-monitor.ts` automatically.
|
||||
- Unexpected `pageerror`, `console.error`, and failed network requests fail the test.
|
||||
- Treat ignored patterns as infrastructure-only noise, not as a place to hide app bugs.
|
||||
|
||||
## Seeded setup
|
||||
|
||||
- `tests/global-setup.ts` verifies the configured base URL, probes `/api/content`, and seeds deterministic content pages.
|
||||
- `tests/global-teardown.ts` removes seeded content again and disposes shared API contexts.
|
||||
- Seed data lives in `tests/api/helpers/seed-data.ts`.
|
||||
- Seeded route constants live in `tests/fixtures/test-constants.ts`.
|
||||
- If tests write to a collection through `ADMIN_TOKEN`, that collection must define explicit permissions like `"token:${ADMIN_TOKEN}":`.
|
||||
|
||||
## Fixtures & helpers
|
||||
|
||||
- `e2e/fixtures.ts` — Shared fixtures (`authedPage`, `testUser`) and helpers (`waitForSpaReady`, `navigateToRoute`, `clickSpaLink`).
|
||||
- `e2e/fixtures.ts` — Shared desktop helpers (`waitForSpaReady`, `navigateToRoute`, `clickSpaLink`) with BrowserSync-safe navigation defaults.
|
||||
- `e2e-visual/fixtures.ts` — Visual test helpers (`waitForVisualReady`, `hideDynamicContent`, `prepareForScreenshot`, `expectScreenshot`, `getDynamicMasks`).
|
||||
- `e2e-mobile/fixtures.ts` — Mobile helpers (`openHamburgerMenu`, `isMobileViewport`, `isTabletViewport`, `isBelowLg`).
|
||||
- `api/fixtures.ts` — API fixtures (`api`, `authedApi`, `accessToken`).
|
||||
- `api/helpers/` — API test utilities (`test-user.ts`, `admin-api.ts`, `maildev.ts`).
|
||||
- `fixtures/test-constants.ts` — Central constants (`TEST_USER`, `ADMIN_TOKEN`, `API_BASE`).
|
||||
- `api/fixtures.ts` — API fixtures (`api`, `adminApi`).
|
||||
- `api/helpers/` — API test utilities (`admin-api.ts`, `seed-data.ts`, `maildev.ts`).
|
||||
- `fixtures/test-constants.ts` — Central constants (`ADMIN_TOKEN`, `API_BASE`, `TEST_BASE_URL`, `SEEDED_TEST_CONTENT`).
|
||||
- `api/helpers/test-user.ts` is legacy starter scaffolding and should only be reused if the project really needs JWT-user coverage again.
|
||||
|
||||
## Visual regression
|
||||
|
||||
|
||||
Vendored
+4
-4
@@ -4,10 +4,10 @@ import path from "node:path"
|
||||
/**
|
||||
* Zentrale Test-Konstanten für alle Tests.
|
||||
*
|
||||
* Passe diese Werte an dein Projekt an:
|
||||
* - TEST_USER: E-Mail/Passwort für den E2E-Test-User
|
||||
* - ADMIN_TOKEN: Token aus api/config.yml.env
|
||||
* - API_BASE: API-Pfad (Standard: /api)
|
||||
* Aktuell relevant fuer das deterministische Playwright-Setup:
|
||||
* - ADMIN_TOKEN: statischer Token aus api/config.yml.env fuer Seed/Cleanup
|
||||
* - TEST_BASE_URL: bevorzugt CODING_URL aus env/.env
|
||||
* - SEEDED_TEST_CONTENT: feste Routen fuer API- und E2E-Tests
|
||||
*/
|
||||
|
||||
export const TEST_USER = {
|
||||
|
||||
Reference in New Issue
Block a user