✨ feat: add admin smoke tests and enhance testing documentation with new strategies and configurations
This commit is contained in:
@@ -23,6 +23,7 @@ This starter uses Playwright across four slices:
|
||||
|
||||
- `tests/api/` for API-level checks
|
||||
- `tests/e2e/` for desktop browser behavior
|
||||
- `tests/e2e-admin/` for committed admin smoke coverage
|
||||
- `tests/e2e-mobile/` for mobile behavior
|
||||
- `tests/e2e-visual/` for screenshot-based regression tests
|
||||
|
||||
@@ -40,6 +41,7 @@ The current baseline is deterministic and seed-driven, not demo-content-driven.
|
||||
| `tests/api/helpers/seed-data.ts` | Seed definitions and seed cleanup for deterministic content pages |
|
||||
| `tests/fixtures/console-monitor.ts` | Fails browser-based tests on unexpected page, console, or request errors |
|
||||
| `tests/e2e/fixtures.ts` | Desktop browser fixtures and SPA helpers |
|
||||
| `tests/e2e-admin/fixtures.ts` | Admin login helpers and admin smoke fixture setup |
|
||||
| `tests/e2e-mobile/fixtures.ts` | Mobile browser fixtures and hamburger-menu helpers |
|
||||
|
||||
---
|
||||
@@ -63,6 +65,23 @@ For this project, prefer the reverse-proxied `CODING_URL` from `.env` whenever i
|
||||
|
||||
If `/api/...` returns HTML instead of JSON, the seeded setup is not usable and `globalSetup` should fail fast.
|
||||
|
||||
### Admin host and default credentials
|
||||
|
||||
Admin browser tests use `TEST_ADMIN_BASE_URL` from `tests/fixtures/test-constants.ts`.
|
||||
|
||||
Resolution order:
|
||||
|
||||
1. `process.env.CODING_TIBIADMIN_URL`
|
||||
2. `CODING_TIBIADMIN_URL` from `.env`
|
||||
3. fallback `http://localhost:3000`
|
||||
|
||||
The current smoke setup assumes the default dev login unless overridden via env vars:
|
||||
|
||||
- `ADMIN_UI_USERNAME` default: `admin`
|
||||
- `ADMIN_UI_PASSWORD` default: `admin`
|
||||
|
||||
Keep this only for local/dev smoke coverage. Do not turn production credentials into committed test defaults.
|
||||
|
||||
### Static project token vs JWT user auth
|
||||
|
||||
This distinction matters for tests:
|
||||
@@ -117,15 +136,46 @@ Do not silence app bugs by broadening ignored patterns unless the noise is clear
|
||||
|
||||
The current setup seeds content through the public collection API plus the static `Token:` header.
|
||||
|
||||
Use a hidden per-collection marker field as the default seed identity strategy.
|
||||
In this project the convention is `_testdata: true`.
|
||||
|
||||
### Seed lifecycle
|
||||
|
||||
1. `globalSetup` probes the configured base URL.
|
||||
2. `globalSetup` verifies `/api/content` returns JSON.
|
||||
3. `globalSetup` removes old seeded entries by `translationKey`.
|
||||
3. `globalSetup` removes old seeded entries by their hidden test marker before recreating them.
|
||||
4. `globalSetup` creates deterministic seed entries.
|
||||
5. Tests run against those seeded routes.
|
||||
6. `globalTeardown` removes seeded entries again.
|
||||
|
||||
Setup cleanup and teardown cleanup are both required.
|
||||
The setup cleanup handles leftovers from aborted or previously failed runs.
|
||||
The teardown cleanup keeps the environment clean after successful runs.
|
||||
|
||||
### Hidden seed marker pattern
|
||||
|
||||
Prefer this pattern for every collection that may receive test-created data:
|
||||
|
||||
1. Add a hidden boolean field named `_testdata` as the last field in the collection schema.
|
||||
2. Set `_testdata: true` on every seeded entry.
|
||||
3. Let cleanup match `_testdata === true` first.
|
||||
4. Keep older identifiers such as fixed paths or translation keys only as migration fallbacks when existing seed data already used them.
|
||||
|
||||
This is more robust than relying on translation keys because not every collection has a natural grouping field.
|
||||
It also makes leftovers from aborted runs discoverable across heterogeneous collection shapes.
|
||||
|
||||
### Parallel worker rule
|
||||
|
||||
Seed creation and seed cleanup must remain run-scoped, not worker-scoped.
|
||||
|
||||
- perform seed cleanup and creation in `globalSetup`
|
||||
- perform final seed cleanup in `globalTeardown`
|
||||
- do not create or delete shared seeded data in per-test hooks or worker fixtures
|
||||
- keep seeded identifiers deterministic so many workers can read the same seeded dataset safely
|
||||
|
||||
This project runs with many workers.
|
||||
Parallel safety depends on one shared deterministic seed pass before the suite and one shared cleanup pass after the suite, not on each worker mutating shared fixtures independently.
|
||||
|
||||
### Current seeded routes
|
||||
|
||||
Defined in `tests/fixtures/test-constants.ts`:
|
||||
@@ -173,6 +223,20 @@ Use `tests/e2e/` when validating:
|
||||
- block rendering in the real UI
|
||||
- keyboard/a11y interactions such as skip links
|
||||
|
||||
### Admin smoke tests
|
||||
|
||||
Use `tests/e2e-admin/` when validating stable admin contracts such as:
|
||||
|
||||
- admin login still works in dev
|
||||
- the project dashboard opens correctly
|
||||
- core collections are still reachable
|
||||
- critical collection views still render their configured labels/columns/actions
|
||||
- collection lists render meaningful previews instead of broken placeholders
|
||||
- important field widgets are configured and usable in entry forms
|
||||
- pagebuilder block choosers, block forms, and live previews load correctly
|
||||
|
||||
These tests should stay intentionally narrow. They are regression guards for admin configuration, not full editor journey automation.
|
||||
|
||||
### Mobile E2E tests
|
||||
|
||||
Use `tests/e2e-mobile/` when validating:
|
||||
@@ -185,6 +249,31 @@ Use `tests/e2e-mobile/` when validating:
|
||||
|
||||
Use `tests/e2e-visual/` only when layout/styling stability matters and a semantic DOM assertion is not enough.
|
||||
|
||||
## Admin config coverage strategy
|
||||
|
||||
Use a hybrid approach:
|
||||
|
||||
- committed Playwright smoke tests for stable, repeatable admin contracts
|
||||
- one-shot MCP Playwright or VS Code browser checks for exploratory spot checks and ad-hoc audits
|
||||
|
||||
Committed tests should cover the admin paths that are expected to stay valid across everyday work, for example:
|
||||
|
||||
- login
|
||||
- opening the Nova project dashboard
|
||||
- visibility of the core collections
|
||||
- opening important collection views like `content`
|
||||
- checking that collection tables expose the intended columns, summaries, and preview thumbnails
|
||||
- checking that key widgets like selects, foreign/media pickers, sidebars, and pagebuilder controls actually render
|
||||
- checking that pagebuilder preview updates when block content changes
|
||||
|
||||
One-shot live browser checks are useful when:
|
||||
|
||||
- reviewing a newly added admin configuration once
|
||||
- probing a flaky or hard-to-stabilize UI area before deciding what deserves a real test
|
||||
- checking something highly visual or temporarily environment-specific
|
||||
|
||||
Do not rely on one-shot browser checks as the only safeguard for important admin paths. If a check matters repeatedly, promote it into `tests/e2e-admin/`.
|
||||
|
||||
---
|
||||
|
||||
## Current fixture conventions
|
||||
@@ -211,6 +300,16 @@ Helpers include:
|
||||
- `clickSpaLink(page, selector)`
|
||||
- automatic console/page/request error monitoring via `attachConsoleMonitor(page)`
|
||||
|
||||
### Admin E2E
|
||||
|
||||
Use `tests/e2e-admin/fixtures.ts`.
|
||||
|
||||
Helpers include:
|
||||
|
||||
- `loginToAdmin(page)`
|
||||
- `openNovaProjectDashboard(page)`
|
||||
- automatic console/page/request error monitoring via `attachConsoleMonitor(page)`
|
||||
|
||||
### Mobile E2E
|
||||
|
||||
Use `tests/e2e-mobile/fixtures.ts`.
|
||||
@@ -265,6 +364,7 @@ Run only the slice you changed.
|
||||
```bash
|
||||
/usr/bin/node ./node_modules/playwright/cli.js test tests/api/health.spec.ts --project=api
|
||||
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e/home.spec.ts tests/e2e/demo.spec.ts --project=chromium
|
||||
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e-admin/smoke.spec.ts --project=admin
|
||||
/usr/bin/node ./node_modules/playwright/cli.js test tests/e2e-mobile/home.mobile.spec.ts --project=mobile-iphonese
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user