Testing

Playwright tests for E2E, API, mobile, and visual regression. Config in playwright.config.ts.

For the full current workflow, prefer the playwright-testing skill.

Running tests

All tests: yarn test
E2E: yarn test:e2e
API: yarn test:api
Admin smoke: npx playwright test tests/e2e-admin/smoke.spec.ts --project=admin
Visual regression: yarn test:visual
Single file: npx playwright test tests/e2e/filename.spec.ts
Single test: npx playwright test -g "test name"
After code changes, run only affected spec files — not the full suite.
Prefer the configured CODING_URL from .env whenever it serves both / and /api/....
Admin browser tests use CODING_TIBIADMIN_URL from .env and default to admin/admin unless overridden via env vars.
The current test baseline is deterministic and seed-driven, not demo-content-driven.

BrowserSync workaround

BrowserSync keeps a WebSocket open permanently, preventing networkidle and load from resolving. All fixture files override page.goto() and page.reload() to use waitUntil: "domcontentloaded" by default. Always use domcontentloaded for navigation waits.

Console watcher

Browser-based fixtures attach tests/fixtures/console-monitor.ts automatically.
Unexpected pageerror, console.error, and failed network requests fail the test.
Treat ignored patterns as infrastructure-only noise, not as a place to hide app bugs.

Seeded setup

tests/global-setup.ts verifies the configured base URL, probes /api/content, and seeds deterministic content pages.
tests/global-teardown.ts removes seeded content again and disposes shared API contexts.
Seed data lives in tests/api/helpers/seed-data.ts.
Seeded route constants live in tests/fixtures/test-constants.ts.
Prefer a hidden _testdata boolean field as the last field per collection as the primary marker for seeded test data.
Cleanup belongs both in globalSetup and globalTeardown.
Seed creation/cleanup must stay run-scoped so the suite works with many workers; do not create or delete shared seeded data in per-test hooks.
If tests write to a collection through ADMIN_TOKEN, that collection must define explicit permissions like "token:${ADMIN_TOKEN}":.

Fixtures & helpers

e2e/fixtures.ts — Shared desktop helpers (waitForSpaReady, navigateToRoute, clickSpaLink) with BrowserSync-safe navigation defaults.
e2e-admin/fixtures.ts — Shared admin helpers (loginToAdmin, openNovaProjectDashboard) for committed admin smoke coverage.
e2e-admin/content-config.spec.ts — Checks that collection config is actually reflected in Nova: sensible list columns/previews, usable widgets, and working pagebuilder preview.
e2e-visual/fixtures.ts — Visual test helpers (waitForVisualReady, hideDynamicContent, prepareForScreenshot, expectScreenshot, getDynamicMasks).
e2e-mobile/fixtures.ts — Mobile helpers (openHamburgerMenu, isMobileViewport, isTabletViewport, isBelowLg).
api/fixtures.ts — API fixtures (api, adminApi).
api/helpers/ — API test utilities (admin-api.ts, seed-data.ts, maildev.ts).
fixtures/test-constants.ts — Central constants (ADMIN_TOKEN, API_BASE, TEST_BASE_URL, SEEDED_TEST_CONTENT).
Use committed admin smoke tests for stable admin contracts. Use one-shot MCP/browser checks only as exploratory supplements, not as the sole regression guard for important admin paths.
Use admin tests specifically to catch broken collection configuration: empty/bad list previews, missing widgets, broken dependsOn behavior, or non-rendering pagebuilder previews.
api/helpers/test-user.ts is legacy starter scaffolding and should only be reused if the project really needs JWT-user coverage again.

Visual regression

Tests in e2e-visual/ with separate Playwright projects (visual-desktop, visual-iphonese, visual-ipad).
Run: yarn test:visual. Update baselines: yarn test:visual:update.
Screenshots in e2e-visual/__screenshots__/{projectName}/ — MUST be committed.
Tolerance: maxDiffPixelRatio: 0.02 (2%) for cross-OS/hardware rendering differences.
Always call prepareForScreenshot(page) before expectScreenshot().
Use waitForVisualReady(page) instead of waitForSpaReady() — it additionally waits for skeleton loaders and CSS settling.
Dynamic content: hideDynamicContent(page) disables BrowserSync overlay and animations; getDynamicMasks(page) returns locators for non-deterministic elements.
For AI review of screenshots, run ./scripts/export-visual-screenshots.sh.

Coordination

Always coordinate test adjustments with the user: do not fix broken tests to match buggy frontend or vice versa.
When tests fail, first clarify whether the test or the code is wrong.

4.7 KiB Raw Blame History