QA Guide -- Aggressive Product Testing¶

This guide covers every testable surface of AEO Bunny. It is designed for both human QA testers and AI agents performing automated QA. Work through each section methodically. Use the checkboxes to track progress. If a check fails, document the failure (screenshot, error message, expected vs. actual) before moving on.

Table of Contents¶

Test Environment Setup
Authentication & Authorization
Admin Portal Testing
Customer Portal Testing
Pipeline Gate Testing
Visibility Score Testing
Readiness Score Testing
Photo Collection Testing
Revision Workflow Testing
Notification System Testing
Mobile Responsive Testing
API Endpoint Testing
Security Testing
Edge Cases & Error States
Full Customer Journey (End-to-End)
Test Data Reference

1. Test Environment Setup¶

Seeding Test Data¶

The seed script creates 8 projects at different pipeline stages, 11 user accounts (3 admin, 8 customer), and all associated data (articles, batches, visibility scores, photos, notifications, etc.).

# From the aeo_bunny/ directory:

# Seed fresh (fails if data already exists):
DATABASE_URL=postgresql+asyncpg://... python3 -m scripts.seed_test_data

# Force re-seed (wipes existing seed data first):
DATABASE_URL=postgresql+asyncpg://... python3 -m scripts.seed_test_data --force

# Reset schema (drops and recreates all tables, then seeds):
DATABASE_URL=postgresql+asyncpg://... python3 -m scripts.seed_test_data --reset-schema

# Clean seed data only (no re-seed):
DATABASE_URL=postgresql+asyncpg://... python3 -m scripts.clean_test_data

Test User Credentials¶

All test users share the same password: TestPass123!

Email	Role	Project
`admin@test.aeobunny.com`	`super_admin`	None (sees all)
`ops1@test.aeobunny.com`	`admin`	None (sees all)
`ops2@test.aeobunny.com`	`admin`	None (sees all)
`dental@test.aeobunny.com`	`customer`	Bright Smile Dental (completed)
`roofing@test.aeobunny.com`	`customer`	Summit Roofing Co (batch 2 review)
`landscaping@test.aeobunny.com`	`customer`	Verde Landscaping (matrix review)
`plumbing@test.aeobunny.com`	`customer`	Apex Plumbing (photo gate)
`realty@test.aeobunny.com`	`customer`	Coastal Realty Group (running)
`mealprep@test.aeobunny.com`	`customer`	FreshFit Meal Prep (failed)
`vet@test.aeobunny.com`	`customer`	Mountain View Veterinary (HTML review)
`yoga@test.aeobunny.com`	`customer`	Harmony Yoga Studio (revision pending)

Portal Access¶

Portal	URL Pattern	Who
Admin	`https://<admin-domain>/admin`	`super_admin` and `admin` roles
Customer	`https://<customer-domain>/portal`	`customer` role
Login	`https://<domain>/login`	All roles
Onboarding	`https://<domain>/onboard`	New customers (PendingLead)

Required Environment Variables¶

At minimum, these must be set for the backend to serve requests:

DATABASE_URL -- Supabase connection string (Session Pooler)
SUPABASE_URL -- Supabase project URL (JWKS endpoint for JWT validation is derived automatically from this URL; no separate JWT secret or JWKS URL env var is needed)
SUPABASE_SERVICE_ROLE_KEY -- For seed script auth user creation

For full pipeline testing, also need: ANTHROPIC_API_KEY, R2_ACCOUNT_ID, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET_NAME, OPENAI_API_KEY, PERPLEXITY_API_KEY, GHL_WEBHOOK_URL, GHL_WEBHOOK_SECRET.

2. Authentication & Authorization¶

[ ] Login as admin@test.aeobunny.com / TestPass123! -- should redirect to admin dashboard
[ ] Login as ops1@test.aeobunny.com / TestPass123! -- should redirect to admin dashboard
[ ] Login as dental@test.aeobunny.com / TestPass123! -- should redirect to customer portal dashboard
[ ] Login as roofing@test.aeobunny.com / TestPass123! -- should redirect to customer portal dashboard
[ ] Verify GET /api/v1/me returns correct role and location_id for each user

Invalid Credentials¶

[ ] Login with valid email but wrong password -- should show error, no redirect
[ ] Login with nonexistent email -- should show error, no redirect
[ ] Login with empty email and password -- should show validation error
[ ] Login with SQL injection attempt in email field ('; DROP TABLE users; --) -- should fail safely

Token Expiry¶

[ ] Use an expired JWT token in Authorization header -- should return 401
[ ] Use a malformed JWT (truncated, wrong signature) -- should return 401
[ ] Make request with no Authorization header -- should return 401

Role-Based Access¶

Customer cannot access admin endpoints:

[ ] dental@test.aeobunny.com calls GET /api/v1/admin/projects -- should return 403
[ ] dental@test.aeobunny.com calls GET /api/v1/admin/stats?period=30d -- should return 403
[ ] dental@test.aeobunny.com calls POST /api/v1/admin/projects/{location_id}/gate -- should return 403
[ ] dental@test.aeobunny.com calls GET /api/v1/admin/settings -- should return 403
[ ] dental@test.aeobunny.com calls GET /api/v1/admin/revisions -- should return 403

Admin cannot access settings (only super_admin):

[ ] ops1@test.aeobunny.com calls GET /api/v1/admin/settings -- should return 403
[ ] ops1@test.aeobunny.com calls PUT /api/v1/admin/settings/pipeline -- should return 403
[ ] admin@test.aeobunny.com calls GET /api/v1/admin/settings -- should return 200 (super_admin)
[ ] admin@test.aeobunny.com calls PUT /api/v1/admin/settings/pipeline -- should return 200 (super_admin)

Customer can only see their own project:

[ ] dental@test.aeobunny.com calls GET /api/v1/portal/my-project -- should return Bright Smile Dental data
[ ] roofing@test.aeobunny.com calls GET /api/v1/portal/my-project -- should return Summit Roofing Co data
[ ] dental@test.aeobunny.com cannot see roofing's articles, photos, or visibility scores (verify IDOR in Section 13)

3. Admin Portal Testing¶

3.1 Dashboard / Overview Page¶

[ ] Login as admin@test.aeobunny.com or ops1@test.aeobunny.com
[ ] Verify 8 projects are listed
[ ] Verify correct status badges per project:
[ ] Bright Smile Dental -- "Completed" (green)
[ ] Summit Roofing Co -- "Paused" (yellow/amber)
[ ] Verde Landscaping -- "Paused" (yellow/amber)
[ ] Apex Plumbing -- "Paused" (yellow/amber)
[ ] Coastal Realty Group -- "Running" (blue)
[ ] FreshFit Meal Prep -- "Failed" (red)
[ ] Mountain View Veterinary -- "Paused" (yellow/amber)
[ ] Harmony Yoga Studio -- "Paused" (yellow/amber)
[ ] Verify metric cards display (total projects, active pipelines, pending approvals, failed)
[ ] Click a project row -- should navigate to project detail page

3.2 Project Detail Page¶

Test with Bright Smile Dental (completed project, most data):

[ ] Verify 3-column layout: article queue (left), content preview (center), pipeline journey (right)
[ ] Article queue shows all 50 articles
[ ] Click an article in the queue -- content preview shows article title, body (markdown), and meta description
[ ] Switch between Content and Preview tabs in the center panel
[ ] Pipeline journey shows all 5 batches as "Shipped"
[ ] Batch stepper shows batch 5 as the final completed batch

Test with Summit Roofing Co (mid-pipeline):

[ ] Batch stepper shows batch 1 as "Shipped", batch 2 at "Article Review", batches 3-5 as "Pending"
[ ] Article queue shows batch 2 articles with mixed review statuses (approved, pending, edited, flagged)
[ ] Content preview renders article body correctly for batch 2 articles
[ ] Approve button visible for this paused project

Test with FreshFit Meal Prep (failed):

[ ] Failed status banner visible with error message: "Anthropic API rate limit exceeded after 3 retries"
[ ] Retry button visible (or not, if max attempts = 3 and attempts = 3)

3.3 Approvals Queue¶

[ ] Navigate to Approvals page from sidebar
[ ] Verify Verde Landscaping appears (paused at gate_matrix_review)
[ ] Verify Harmony Yoga Studio appears (paused at gate_batch_article_review with revision pending)
[ ] Verify Summit Roofing Co appears (paused at gate_batch_article_review for customer review -- may or may not show in admin approvals depending on gate ownership)
[ ] Verify Mountain View Veterinary appears (paused at gate_batch_html_review for customer review -- dual gate customer side)
[ ] Verify Apex Plumbing appears (paused at gate_photo_upload)
[ ] Click Verde Landscaping approval -- should show matrix review content (50-article plan)
[ ] Click "Approve" -- verify pipeline state changes (if backend connected)
[ ] Click "Reject" -- verify feedback field appears, submit feedback

3.4 Stats Dashboard¶

[ ] Navigate to Stats page from sidebar
[ ] Verify 6 metric sections render:
[ ] Delivery Health (throughput, pipeline states)
[ ] Quality (revision rate, gate approval rate)
[ ] Visibility (engine averages, score trends)
[ ] Readiness (category bars)
[ ] Capacity (active pipelines, queue depth)
[ ] Cost (cost trend, agent breakdown)
[ ] Toggle period: 7d, 30d, 90d, YTD -- charts should update
[ ] Verify GET /api/v1/admin/stats?period=7d returns valid JSON
[ ] Verify GET /api/v1/admin/stats?period=30d returns valid JSON
[ ] Verify GET /api/v1/admin/stats?period=90d returns valid JSON

3.5 Settings (Super Admin Only)¶

[ ] Login as admin@test.aeobunny.com (super_admin) -- Settings link visible in sidebar
[ ] Login as ops1@test.aeobunny.com (admin) -- Settings link NOT visible in sidebar
[ ] Navigate to Settings page as super_admin
[ ] Verify 7 setting categories render:
[ ] Pipeline
[ ] Visibility
[ ] Readiness
[ ] Cost
[ ] Alerts
[ ] Notifications
[ ] General
[ ] Edit a setting value (e.g., pipeline_max_retries) and click Save
[ ] Verify the new value persists after page refresh
[ ] Test optimistic concurrency: open Settings in two tabs, save in one, then save in the other -- second save should warn about conflict
[ ] Verify System Broadcast card is visible (super_admin only)
[ ] Send a broadcast message -- verify it persists
[ ] Deactivate the broadcast -- verify it disappears

3.6 Profile¶

[ ] Navigate to Profile page from avatar dropdown
[ ] Verify current name displays (first_name, last_name)
[ ] Edit first_name and last_name -- click Save
[ ] Verify name change persists after refresh
[ ] Upload an avatar image (valid JPEG or PNG)
[ ] Verify avatar displays in the header and profile page
[ ] Delete avatar -- verify placeholder returns

3.7 Search¶

[ ] Click the search bar in the admin header
[ ] Type "Bright" -- should find Bright Smile Dental
[ ] Type "Denver" -- should find Summit Roofing Co (city search)
[ ] Type "dental" -- should find by industry or email
[ ] Type "realty@" -- should find Coastal Realty Group by email
[ ] Type "AZ" -- should find Verde Landscaping and Harmony Yoga Studio (state search)
[ ] Type gibberish -- should show "No results"
[ ] Verify ARIA combobox behavior: keyboard arrows navigate results, Enter selects, Escape closes
[ ] Verify search aborts previous in-flight request (AbortController) when typing new query

3.8 Notification Bell¶

[ ] Click the bell icon in admin header
[ ] Verify unread badge count matches actual unread notifications
[ ] Dropdown shows notification list with titles and timestamps
[ ] Click a notification -- verify it navigates to the correct project/page
[ ] Verify notification marked as read after clicking
[ ] Click "Mark all as read" -- verify all notifications become read and badge clears

4. Customer Portal Testing (Per Project State)¶

4.1 Bright Smile Dental -- `dental@test.aeobunny.com` (Completed)¶

[ ] Dashboard shows "Project Complete" state
[ ] Visibility Score card visible with composite score (~58.0 from latest scheduled scan)
[ ] Site Health card visible with readiness score (~78.0 from post-deployment check)
[ ] Content Delivered card shows "50 / 50 articles" across 5 batches
[ ] All 5 batches marked "Shipped" in batch display
[ ] Navigate to Content/Review tab -- all 50 articles visible, all marked "Approved"
[ ] Click an article -- article detail renders with body, Content/Preview tabs
[ ] Navigate to Visibility tab -- Overview, Trends, Competitors sub-tabs visible
[ ] Navigate to Download page -- ZIP download links for all 5 batches
[ ] Click a batch download link -- verify ZIP file downloads (or shows fake R2 URL for test data)
[ ] Navigate to Settings page -- profile info, password change

4.2 Summit Roofing Co -- `roofing@test.aeobunny.com` (Batch 2 Article Review)¶

[ ] Dashboard shows "Content Ready for Review" state for batch 2
[ ] Batch 1 shows as "Shipped", batch 2 shows as "In Review", batches 3-5 as "Pending"
[ ] Navigate to Content/Review tab
[ ] Verify batch 2 articles (10) visible with mixed review statuses:
[ ] 4 articles marked "Approved"
[ ] 3 articles marked "Pending"
[ ] 2 articles marked "Edited"
[ ] 1 article marked "Flagged"
[ ] Click a "Pending" article -- "Approve Article" button and "Request Changes" button visible
[ ] Click "Request Changes" on a pending article -- chat interface opens
[ ] Type a message in the chat -- should submit (or mock-submit if backend not connected)
[ ] Click "Approve Article" after providing feedback -- article status changes to "approved"
[ ] Click an "Edited" article -- verify chat messages are visible (2 messages: customer + agent)
[ ] Click "Approve" on a pending article -- status changes to "approved"
[ ] "Finish Review" button should NOT appear until all 10 articles are approved or edited (3 are still pending)

4.3 Verde Landscaping -- `landscaping@test.aeobunny.com` (Waiting for Operator)¶

[ ] Dashboard shows "waiting for review" or "in progress" state
[ ] No batches visible (pipeline is at matrix_review, before batch creation)
[ ] No articles visible on Content/Review tab
[ ] No visibility score yet (baseline not run since BI is technically complete but matrix not approved)
[ ] Photo upload should be accessible (customer can start uploading early)
[ ] Readiness Score may show intake score (45.0 composite)

4.4 Apex Plumbing -- `plumbing@test.aeobunny.com` (Photo Gate)¶

[ ] Dashboard shows photo upload needed state
[ ] Photo Progress card visible: "65 / 100 photos uploaded"
[ ] Progress bar shows 65% filled
[ ] Link to photo upload portal (/portal/photos) is prominent
[ ] Navigate to Photos tab:
[ ] 65 existing photos displayed in grid
[ ] Photos have quality badges (good, fair, poor)
[ ] Upload zone visible (drag-and-drop area)
[ ] Click a photo -- lightbox viewer opens with keyboard navigation (arrows, Escape)
[ ] Batch 1 articles (10) all approved, but batch is still at article_review status because photo gate not yet passed

4.5 Coastal Realty Group -- `realty@test.aeobunny.com` (Running)¶

[ ] Dashboard shows "Pipeline Running" state with progress indicator
[ ] 4 articles visible in Content/Review tab (partially written, generation_status="generated")
[ ] Articles show as "Pending" review status (not yet ready for review)
[ ] No batch completion or deployment visible

4.6 FreshFit Meal Prep -- `mealprep@test.aeobunny.com` (Failed)¶

[ ] Dashboard shows error/failed state
[ ] Error banner or message visible: "Anthropic API rate limit exceeded after 3 retries"
[ ] No content available for review
[ ] Customer should see some form of "We're working on it" or "Contact support" messaging
[ ] 3 partially written articles may be visible

4.7 Mountain View Veterinary -- `vet@test.aeobunny.com` (HTML Review)¶

[ ] Dashboard shows "Pages Ready for Review" state for batch 1
[ ] Navigate to Content/Review tab -- 10 articles visible, all "Approved"
[ ] HTML preview available via Preview tab or preview page
[ ] HTML preview renders in sandboxed iframe (CSP sandbox)
[ ] Approve/Reject buttons visible for HTML pages
[ ] Internal links visible in the preview
[ ] Schema markup section visible in Content tab

4.8 Harmony Yoga Studio -- `yoga@test.aeobunny.com` (Revision Pending)¶

[ ] Dashboard shows "Revision in Progress" or batch 1 under review
[ ] Batch 1 articles (10) visible with mixed review statuses:
[ ] 3 articles "Approved"
[ ] 4 articles "Edited" (with revision chat messages)
[ ] 2 articles "Pending"
[ ] 1 article "Flagged"
[ ] Click an "Edited" article -- chat messages visible (customer feedback + agent response)
[ ] Chat messages visible showing prior feedback exchange
[ ] Batch has revision_count=1 displayed
[ ] Dossier shows feedback from revision round (voice_and_tone, special_instructions populated)

5. Pipeline Gate Testing¶

Phase A Gates (Operator Only)¶

gate_bi_review:

[ ] Verde Landscaping is past BI review (research_status="complete"), verify BI data exists
[ ] For a project at gate_bi_review: approve via POST /api/v1/admin/projects/{location_id}/gate with action: "approve" -- pipeline should resume
[ ] Reject via same endpoint with action: "reject" and feedback -- pipeline stays paused, feedback stored

gate_matrix_review:

[ ] Verde Landscaping is at gate_matrix_review -- verify in admin approvals
[ ] Approve -- pipeline should advance to batch loop (create batches, start writing batch 1)
[ ] Reject with feedback -- pipeline stays paused

Batch Gates (Dual-Gate Pattern)¶

gate_batch_article_review (batch 1 -- operator first):

[ ] Harmony Yoga is at gate_batch_article_review, batch 1, operator_reviewed=False
[ ] Operator approves -- operator_reviewed flag set to True, pipeline re-pauses for customer
[ ] Customer then reviews and submits "Finish Review" -- pipeline resumes to media assembly

gate_batch_article_review (batch 2+ -- customer directly):

[ ] Summit Roofing is at gate_batch_article_review, batch 2, operator_reviewed=True (already done for batch 1)
[ ] Customer reviews directly (no operator step)
[ ] Customer clicks "Finish Review" after all articles approved/edited -- pipeline resumes

gate_batch_html_review (batch 1 -- operator first):

[ ] Mountain View Vet is at gate_batch_html_review, batch 1, operator_reviewed=True (operator already reviewed)
[ ] Customer reviews the HTML pages
[ ] Customer approves -- pipeline advances to deployment stage

Photo Gate¶

gate_photo_upload:

[ ] Apex Plumbing is at gate_photo_upload with 65/100 photos
[ ] Verify pipeline does NOT advance until 100 photos uploaded
[ ] Operator CANNOT approve this gate (it auto-resolves when photo count >= 100)
[ ] Upload 35 more photos via POST /api/v1/portal/my-project/photos -- gate should auto-clear
[ ] photo_gate_passed flag on PipelineRun set to True after gate clears
[ ] Batches 2-5 skip this gate (verify with Bright Smile Dental -- photo_gate_passed=True)

Deployment Confirmation Gate¶

gate_batch_deploy_confirm:

[ ] Always customer-facing (not operator)
[ ] Customer fills deployment checklist: pages_uploaded=True, google_search_console_submitted=True
[ ] Customer enters hub page URL
[ ] Confirm deployment -- batch status changes to "shipped"
[ ] Post-deployment visibility scan triggers in background

Gate Ownership Enforcement¶

[ ] Customer trying to approve an operator gate (gate_bi_review, gate_matrix_review) -- should fail
[ ] Operator trying to approve a customer gate (gate_batch_deploy_confirm) -- should fail
[ ] Verify error messages are clear about who can act on each gate

6. Visibility Score Testing¶

Overview Tab (dental@test.aeobunny.com)¶

[ ] Login as dental@test.aeobunny.com
[ ] Navigate to Visibility tab
[ ] Overview sub-tab shows:
[ ] Composite score (~58.0 from latest scheduled scan)
[ ] Engine breakdown bars: chatgpt, perplexity, google_aio, gemini
[ ] Each engine score varies around the composite (+/- 5)
[ ] Verify 4 visibility scores exist (baseline, 2x post_deployment, scheduled)
[ ] Score type labels display correctly

Trends Tab¶

[ ] Switch to Trends sub-tab
[ ] Area chart renders with 4+ data points
[ ] X-axis shows dates, Y-axis shows score 0-100
[ ] Engine toggles allow showing/hiding individual engine lines
[ ] Trend line shows: ~35 (baseline) -> ~48 -> ~62 -> ~58 (drop)

Competitors Tab¶

[ ] Switch to Competitors sub-tab
[ ] Competitor names from VisibilityCheck.competitors_cited display:
[ ] "SmileDirect Austin" should appear
[ ] "Capital City Dental" should appear
[ ] Competitor frequency/mention count displayed

Alerts¶

[ ] Score drop alert exists (score dropped from 62 to 58)
[ ] Alert visible in the UI (if surfaced to customer)
[ ] cooldown_until is in the past -- new alerts can fire

Consultation Request¶

[ ] Consultation request button visible
[ ] cooldown_until is in the past -- button should be clickable
[ ] Click button -- POST /api/v1/portal/my-project/request-consultation fires
[ ] After requesting, button shows cooldown state

API Verification¶

[ ] GET /api/v1/visibility/{location_id}/scores returns array of 4 scores
[ ] GET /api/v1/visibility/{location_id}/scores/latest returns the scheduled score
[ ] GET /api/v1/visibility/{location_id}/scores/{score_id}/checks returns 8 checks per score
[ ] GET /api/v1/visibility/{location_id}/trends returns trend data
[ ] GET /api/v1/visibility/{location_id}/competitors returns competitor aggregation
[ ] GET /api/v1/visibility/{location_id}/alerts (admin only) returns alert list

7. Readiness Score Testing¶

Customer Dashboard (dental@test.aeobunny.com)¶

[ ] Login as dental@test.aeobunny.com
[ ] Site Health card on dashboard shows readiness composite score (~78.0 post-deployment)
[ ] 4 category bars visible:
[ ] Crawlability: ~90.0 -- "Pass"
[ ] Schema: ~75.0 -- "Pass"
[ ] Speed: ~72.0 -- "Warn"
[ ] Structured Data: ~70.0 -- "Pass"
[ ] Earlier intake score was ~45.0 -- improvement is visible if history shown

Readiness API¶

[ ] GET /api/v1/readiness/{location_id}/scores/latest returns post-deployment score (composite=78.0)
[ ] GET /api/v1/readiness/{location_id}/scores returns array of 2 scores (intake + post_deployment)
[ ] GET /api/v1/readiness/{location_id}/scores/{score_id}/checks returns 4 checks for each score
[ ] Each check has: category, check_name, target_url, status (pass/warn/fail), value, message

Verde Landscaping (intake only)¶

[ ] Login as landscaping@test.aeobunny.com
[ ] Readiness score shows intake results (~45.0 composite, mixed warn/fail)
[ ] No post-deployment score yet (pipeline hasn't reached deployment)

8. Photo Collection Testing¶

Photo Upload Portal (plumbing@test.aeobunny.com)¶

[ ] Login as plumbing@test.aeobunny.com
[ ] Navigate to Photos tab (/portal/photos)
[ ] 65 existing photos displayed in a grid (2 columns on mobile, 4 on desktop)
[ ] Each photo shows a quality badge: Good (green), Fair (yellow), Poor (red)
[ ] Progress bar shows "65 / 100" with percentage fill

Upload Zone¶

[ ] Drag-and-drop area is visible and labeled
[ ] Drop a valid JPEG image -- upload initiates
[ ] Drop a valid PNG image -- upload initiates
[ ] Drop a non-image file (e.g., .txt) -- should be rejected with error message
[ ] Drop an oversized image (> max file size) -- should be rejected
[ ] Drop multiple files at once -- all valid files upload, invalid ones show errors
[ ] Per-file progress indicator visible during upload
[ ] Per-file retry button appears on failure

[ ] Click a photo in the grid -- lightbox modal opens
[ ] Photo displays at full/larger size in the modal
[ ] Quality badge visible in lightbox
[ ] Press right arrow key -- next photo displays
[ ] Press left arrow key -- previous photo displays
[ ] Press Escape key -- lightbox closes
[ ] Click outside the lightbox -- lightbox closes
[ ] Navigation arrows (on-screen) work for mouse users

Photo Gate Integration¶

[ ] Verify GET /api/v1/portal/my-project for plumbing shows photo gate status
[ ] Admin endpoint GET /api/v1/admin/projects/{location_id}/photo-status (as admin) returns:
[ ] Total count: 65
[ ] Quality breakdown: good count, fair count, poor count, unscored count
[ ] When photo count reaches 100, gate auto-clears (requires live backend upload)

Photo Limits¶

[ ] Verify 300-photo maximum per location is enforced
[ ] Attempt to upload photo #301 (on a project with 300) -- should be rejected
[ ] Duplicate filename upload -- should be rejected (UniqueConstraint on location_id + original_filename)

Pagination¶

[ ] If photo count exceeds page size, pagination controls appear
[ ] Navigate pages -- photos change, no duplicates

Bright Smile Dental Photos (120 photos, completed project)¶

[ ] Login as dental@test.aeobunny.com
[ ] Navigate to Photos tab -- 120 photos displayed
[ ] Quality distribution visible: ~78 good, ~23 fair, ~19 poor (seed bands 0.70-0.95/0.40-0.69/0.10-0.39 map to display thresholds: good >= 0.50, fair >= 0.30, poor < 0.30)
[ ] Upload zone may or may not be available (project is completed)

9. Revision Workflow Testing¶

Harmony Yoga Studio (yoga@test.aeobunny.com)¶

[ ] Login as yoga@test.aeobunny.com
[ ] Navigate to Content/Review tab
[ ] Batch 1 shows 10 articles with mixed statuses:
[ ] 3 "Approved" (green badge)
[ ] 4 "Edited" (orange/amber badge)
[ ] 2 "Pending" (gray badge)
[ ] 1 "Flagged" (red badge)
[ ] Click an "Edited" article:
[ ] Chat messages visible (customer feedback + agent response)
[ ] Chat messages visible showing prior feedback exchange
[ ] Article body displays current version
[ ] Click a "Flagged" article:
[ ] Customer feedback visible: "The tone doesn't match our brand voice"
[ ] Chat messages visible

Request Changes / Approve Article Flow¶

[ ] Click "Request Changes" on a "Pending" article -- chat interface opens
[ ] Submit a message describing requested changes
[ ] Click "Approve Article" after providing feedback -- article status changes to "approved"
[ ] Nav locking: while in chat, verify navigation warning if leaving without finishing

Finish Review Flow¶

[ ] "Finish Review" button should NOT appear if any articles are still "Pending"
[ ] Approve or edit all remaining pending articles
[ ] "Finish Review" button appears when all 10 articles are approved or edited
[ ] Click "Finish Review":
[ ] Progress bar shows during Haiku dossier extraction
[ ] Unprocessed chat messages are extracted into dossier
[ ] Pipeline resumes (or shows state change)

Summit Roofing Co (roofing@test.aeobunny.com) -- Active Review¶

[ ] Login as roofing@test.aeobunny.com
[ ] Batch 2 articles visible (10 articles, mixed statuses)
[ ] Edited articles (indices 17, 18) have 2 ReviewMessages each
[ ] ReviewMessages show:
[ ] Customer: "The tone feels too generic. Can you make it sound more like a local Denver expert..."
[ ] Agent: "Understood! I'll revise this article to use a more locally-grounded, Denver-specific tone..."
[ ] Some messages have processed_in_round=1 (already extracted), some processed_in_round=None (unprocessed)

Version History¶

[ ] On Bright Smile Dental articles (first 5) -- verify version history panel shows 2 versions each
[ ] On Summit Roofing edited articles -- verify version history panel shows 1 version
[ ] Version entries show: version number, body snapshot, timestamp

Admin Revision Queue¶

[ ] Login as ops1@test.aeobunny.com
[ ] Navigate to Approvals page
[ ] If Harmony Yoga has a BatchRevision with status=pending_approval, it appears here
[ ] Revision detail shows: affected articles, estimated cost, customer feedback
[ ] Approve revision -- revisions execute in background
[ ] Reject revision -- articles reset, customer notified

10. Notification System Testing¶

Notification Bell (dental@test.aeobunny.com)¶

[ ] Login as dental@test.aeobunny.com
[ ] Bell icon visible in header
[ ] Unread badge shows count: 3 (visibility_measured, batch_visibility_measured, project_complete)
[ ] Click bell -- dropdown opens with notification list
[ ] Notifications display:
[ ] "Visibility Score Ready" (unread)
[ ] "Batch Visibility Scored" (unread)
[ ] "Project Complete!" (unread)
[ ] "Batch 1 Deployed" (read)
[ ] "Content Approved" (read)
[ ] "Photos received!" (read)
[ ] "Upload your business photos" (read)
[ ] Each notification shows title, message, timestamp, business_name

Mark as Read¶

[ ] Click an unread notification -- it navigates to the relevant page
[ ] Return to bell -- notification now marked as read
[ ] Unread count decremented by 1
[ ] Click "Mark all as read" -- all notifications become read, badge clears

Cross-Tab Sync (BroadcastChannel)¶

[ ] Open the customer portal in Tab A
[ ] Open the customer portal in a second Tab B
[ ] In Tab A, mark a notification as read
[ ] In Tab B, verify the unread count updates without manual refresh
[ ] In Tab B, mark all as read
[ ] In Tab A, verify the badge clears

Admin Notifications¶

[ ] Login as ops1@test.aeobunny.com
[ ] Bell shows admin notifications (BI review needed, matrix review needed, etc.)
[ ] All 5 admin notifications for Bright Smile are pre-marked as read
[ ] Verify notifications from other projects (if seeded) appear

API Verification¶

[ ] GET /api/v1/notifications/unread-count returns correct count
[ ] GET /api/v1/notifications returns paginated notification list
[ ] PATCH /api/v1/notifications/{notification_id}/read marks one as read
[ ] POST /api/v1/notifications/mark-all-read marks all as read
[ ] POST /api/v1/admin/notifications/cleanup (admin only) cleans old notifications

11. Mobile Responsive Testing¶

Test at three breakpoints: 375px (phone), 768px (tablet), 1280px (desktop).

Use browser DevTools responsive mode or a real device.

[ ] Top navigation bar is hidden
[ ] Bottom tab bar appears with 5 items: Home, Review, Photos, Visibility, More
[ ] "More" overflow menu opens with: Settings, Download, Profile, and other secondary pages
[ ] Tab bar is fixed at the bottom of the screen
[ ] Active tab is highlighted
[ ] Tapping a tab navigates to the correct page

Dashboard (< 1024px)¶

[ ] Metric cards arranged in a horizontal carousel (snap scroll)
[ ] Swipe left/right to scroll between cards
[ ] Each card takes full width in the viewport
[ ] Snap points align correctly (card centers on swipe stop)

Pipeline Timeline (< 1024px)¶

[ ] Horizontal stepper is replaced by vertical timeline
[ ] Each stage is stacked vertically with status icon, label, and timestamp
[ ] Active stage is visually distinct

Photo Grid (< 1024px)¶

[ ] Photos display in 2-column grid (not 4-column)
[ ] Upload zone is full-width
[ ] Lightbox works on touch (tap to open, swipe to navigate)

Article Detail (< 1024px)¶

[ ] Breadcrumb navigation visible at top
[ ] Responsive padding (no content clipping at edges)
[ ] Safe-area adjustments for notched devices (viewport meta tag)
[ ] Chat interface readable and usable on small screens

Visibility Page (< 1024px)¶

[ ] Tabs (Overview, Trends, Competitors) accessible
[ ] Charts resize to fit viewport width
[ ] Engine breakdown bars readable

Settings Page (< 1024px)¶

[ ] Settings categories stack vertically
[ ] Form fields are full-width
[ ] Save button easily tappable

Review Page (< 1024px)¶

[ ] Article list scrollable
[ ] Content/Preview tab switching works
[ ] Approve/Edit buttons are tappable (large enough touch targets)

Desktop Verification (>= 1280px)¶

[ ] Top navigation visible (no bottom tab bar)
[ ] 3-column layout on project detail page
[ ] 4-column photo grid
[ ] Horizontal pipeline stepper
[ ] Metric cards in a grid (not carousel)

12. API Endpoint Testing¶

Direct API testing via curl, Postman, or automated test runner. All endpoints require a valid JWT in the Authorization: Bearer <token> header unless noted.

Health¶

[ ] GET /health -- returns 200 with {"status": "ok"} (no auth required)

Auth¶

[ ] GET /api/v1/me -- returns {user_id, email, role, location_id, first_name, last_name, avatar_url} for authenticated user
[ ] POST /api/v1/auth/request-reset with {"email": "dental@test.aeobunny.com"} -- returns 200
[ ] POST /api/v1/auth/reset-password with valid token and new password -- returns 200

Admin¶

[ ] GET /api/v1/admin/metrics -- returns dashboard metric counts
[ ] GET /api/v1/admin/projects -- returns list of 8 project summaries
[ ] GET /api/v1/admin/projects/{location_id}/summary -- returns single project detail
[ ] GET /api/v1/admin/approvals -- returns projects waiting for approval
[ ] POST /api/v1/admin/projects/{location_id}/gate with {"action": "approve"} -- returns 200
[ ] POST /api/v1/admin/projects/{location_id}/gate with {"action": "reject", "feedback": "..."} -- returns 200
[ ] GET /api/v1/admin/projects/{location_id}/batches -- returns batch list
[ ] GET /api/v1/admin/stats?period=30d -- returns stats response with 6 sections

Customer Portal¶

[ ] GET /api/v1/portal/my-project -- returns project for authenticated customer
[ ] GET /api/v1/portal/my-project/articles -- returns articles grouped by batch
[ ] POST /api/v1/portal/my-project/articles/{article_id}/approve -- marks article approved
[ ] POST /api/v1/portal/my-project/articles/{article_id}/finish-editing -- marks article edited
[ ] GET /api/v1/portal/my-project/articles/{article_id}/messages -- returns chat history
[ ] POST /api/v1/portal/my-project/review-chat with {"article_id": "...", "message": "..."} -- submits review message
[ ] POST /api/v1/portal/my-project/approve-all -- approve all (gated behind unprocessed message check)

Batches¶

[ ] GET /api/v1/portal/my-project/batches -- returns batch list with statuses
[ ] GET /api/v1/portal/my-project/batches/{batch_number}/download -- returns ZIP URL or file
[ ] POST /api/v1/portal/my-project/batches/{batch_number}/confirm-deployment -- confirms deployment
[ ] POST /api/v1/portal/my-project/batches/{batch_number}/finish-review -- triggers Haiku extraction + resume

Deployment¶

[ ] POST /api/v1/portal/my-project/confirm-deployment -- legacy endpoint (blocked for batch-mode projects)
[ ] GET /api/v1/portal/my-project/deployment -- returns latest deployment
[ ] GET /api/v1/portal/my-project/preview/{slug} -- returns HTML preview with CSP sandbox
[ ] POST /api/v1/portal/my-project/measure-visibility -- triggers on-demand visibility scan (rate-limited 1/hour)

Photos¶

[ ] POST /api/v1/portal/my-project/photos -- upload photo (multipart/form-data)
[ ] GET /api/v1/portal/my-project/photos -- returns paginated photo list
[ ] GET /api/v1/portal/my-project/photo-status -- returns photo count + quality breakdown

Visibility¶

[ ] GET /api/v1/visibility/{location_id}/scores -- returns all visibility scores
[ ] GET /api/v1/visibility/{location_id}/scores/latest -- returns most recent score
[ ] GET /api/v1/visibility/{location_id}/scores/{score_id}/checks -- returns checks for a score
[ ] POST /api/v1/visibility/{location_id}/scores/check -- triggers new visibility check (any authenticated user, rate-limited 3/hour)

Visibility Intelligence¶

[ ] GET /api/v1/visibility/{location_id}/trends -- returns trend data
[ ] GET /api/v1/visibility/{location_id}/competitors -- returns competitor aggregation
[ ] GET /api/v1/visibility/{location_id}/alerts -- returns alerts (admin only)

Readiness¶

[ ] POST /api/v1/readiness/{location_id}/check -- triggers readiness check
[ ] GET /api/v1/readiness/{location_id}/scores/latest -- returns latest readiness score
[ ] GET /api/v1/readiness/{location_id}/scores/{score_id}/checks -- returns checks for a score
[ ] GET /api/v1/readiness/{location_id}/scores -- returns all readiness scores

Notifications¶

[ ] GET /api/v1/notifications/unread-count -- returns {"count": N}
[ ] GET /api/v1/notifications -- returns paginated list
[ ] PATCH /api/v1/notifications/{notification_id}/read -- marks one read
[ ] POST /api/v1/notifications/mark-all-read -- marks all read
[ ] POST /api/v1/admin/notifications/cleanup -- admin-only cleanup

Profile¶

[ ] PATCH /api/v1/profile with {"first_name": "New", "last_name": "Name"} -- updates profile
[ ] POST /api/v1/profile/avatar -- uploads avatar image
[ ] DELETE /api/v1/profile/avatar -- removes avatar

Settings (super_admin only)¶

[ ] GET /api/v1/admin/settings -- returns all 7 categories
[ ] PUT /api/v1/admin/settings/{category} -- updates a category
[ ] DELETE /api/v1/admin/settings/{category} -- resets to defaults

Revisions (admin only)¶

[ ] GET /api/v1/admin/revisions -- returns pending revision list
[ ] GET /api/v1/admin/revisions/{revision_id} -- returns revision detail
[ ] POST /api/v1/admin/revisions/{revision_id}/approve -- approves revision
[ ] POST /api/v1/admin/revisions/{revision_id}/reject -- rejects revision
[ ] POST /api/v1/admin/revisions/{revision_id}/retry -- retries failed revision

Versions¶

[ ] GET /api/v1/portal/my-project/articles/{article_id}/versions -- returns version list
[ ] GET /api/v1/portal/my-project/articles/{article_id}/versions/{version_number} -- returns specific version

Broadcast¶

[ ] POST /api/v1/admin/broadcast (super_admin) -- creates broadcast
[ ] POST /api/v1/admin/broadcast/deactivate (super_admin) -- deactivates broadcast
[ ] GET /api/v1/admin/broadcast -- returns active broadcast
[ ] GET /api/v1/portal/broadcast -- returns active broadcast for customers

Onboarding¶

[ ] POST /api/v1/onboard with full intake form data -- creates business, location, user, starts pipeline

GHL Inbound Webhook¶

[ ] POST /api/v1/webhooks/ghl/purchase with X-GHL-Signature (Ed25519, preferred) or X-GHL-Secret (legacy fallback) header -- creates PendingLead

Consultation¶

[ ] GET /api/v1/portal/my-project/consultation-status -- returns consultation cooldown status
[ ] POST /api/v1/portal/my-project/request-consultation -- submits consultation request

13. Security Testing¶

IDOR (Insecure Direct Object Reference)¶

Customer A must never access Customer B's resources.

[ ] Login as dental@test.aeobunny.com, get Summit Roofing's location_id
[ ] Call GET /api/v1/visibility/{summit_location_id}/scores -- should return 403
[ ] Call GET /api/v1/readiness/{summit_location_id}/scores/latest -- should return 403
[ ] Call GET /api/v1/portal/my-project/articles/{summit_article_id}/messages -- should return 403 or 404
[ ] Call POST /api/v1/portal/my-project/articles/{summit_article_id}/approve -- should return 403 or 404
[ ] Call POST /api/v1/portal/my-project/photos and try to associate with another location -- should fail
[ ] Call GET /api/v1/portal/my-project/batches/{batch_number}/download for another project's batch -- should fail
[ ] Login as roofing@test.aeobunny.com, try accessing dental's visibility scores -- should fail

Authentication Enforcement¶

[ ] Every endpoint (except /health, /api/v1/webhooks/ghl/purchase, /api/v1/onboard, POST /api/v1/auth/request-reset, POST /api/v1/auth/reset-password) requires Authorization header
[ ] Calls without JWT return 401
[ ] Calls with expired JWT return 401
[ ] Calls with JWT for a deactivated user (is_active=false) return 401 or 403

SSRF Protection¶

[ ] Photo upload validates file content (magic bytes check), not just Content-Type header
[ ] Upload a file with .jpg extension but non-image magic bytes -- should be rejected
[ ] The shared SSRF validator (app/utils/ssrf.py) blocks internal/private IP addresses in URLs
[ ] Upload a profile avatar with a URL pointing to http://127.0.0.1 or http://169.254.169.254 -- should be blocked

Rate Limiting¶

[ ] POST /api/v1/portal/my-project/measure-visibility is rate-limited to 1/hour -- verify second request within an hour returns 429
[ ] Login endpoints have rate limiting -- rapid sequential login attempts eventually return 429
[ ] Photo upload has reasonable rate limiting

CSP (Content Security Policy)¶

[ ] GET /api/v1/portal/my-project/preview/{slug} returns HTML with CSP sandbox headers
[ ] The iframe preview does not allow JavaScript execution (sandbox restricts scripts)
[ ] Inline styles and external resource loading are appropriately restricted

CORS¶

[ ] API only allows requests from configured origins
[ ] Request from an unauthorized origin should be blocked or return no CORS headers
[ ] OPTIONS preflight requests return correct CORS headers for allowed origins

Webhook Security¶

[ ] POST /api/v1/webhooks/ghl/purchase accepts dual auth: X-GHL-Signature (Ed25519, preferred) with X-GHL-Secret (legacy fallback)
[ ] Call with valid X-GHL-Signature -- should succeed (creates PendingLead)
[ ] Call with invalid X-GHL-Signature -- should return 401 or 403 (does NOT fall back to X-GHL-Secret)
[ ] Call with only X-GHL-Secret (no X-GHL-Signature header) -- should succeed (legacy mode)
[ ] Call without either header -- should return 401 or 403
[ ] Call with wrong X-GHL-Secret value (no X-GHL-Signature) -- should return 401 or 403

Cross-Batch Article Mutation¶

[ ] A customer reviewing batch 2 cannot approve/edit articles belonging to batch 1 -- should fail with error
[ ] batch_number verification on all article mutation endpoints

14. Edge Cases & Error States¶

Empty States¶

[ ] Login as landscaping@test.aeobunny.com -- no articles, no batches: verify empty state UI (not a blank page)
[ ] Visibility page with 0 scores -- shows "No visibility data yet" or similar
[ ] Photo page with 0 photos -- shows upload zone prominently, no empty grid
[ ] Notification bell with 0 notifications -- shows "No notifications" message in dropdown
[ ] Admin overview with 0 projects (would require cleaning seed data) -- shows empty state

Pipeline Failure Recovery¶

[ ] FreshFit Meal Prep has attempts=3 (max) -- verify retry button is disabled or hidden
[ ] Batch with failed_at_status="writing" -- verify recovery flow starts from writing step
[ ] Concurrent gate approvals: two operators approve the same gate simultaneously -- only one succeeds (FOR UPDATE locking prevents double-processing)

Data Boundary Cases¶

[ ] Article with empty body_markdown -- Content tab shows empty or placeholder, Preview tab handles gracefully
[ ] Visibility score with all engines returning 0 -- composite shows 0
[ ] Readiness score where one category is skipped -- weights redistribute correctly
[ ] Photo with quality_score of exactly 0.30 (poor/fair boundary) -- verify correct badge (fair)
[ ] Photo with quality_score of exactly 0.50 (fair/good boundary) -- verify correct badge (good)
[ ] Photo with quality_score of 0.29 -- verify poor badge
[ ] Photo with quality_score of 0.49 -- verify fair badge

Pagination¶

[ ] Bright Smile Dental has 50 articles -- verify pagination if page size < 50
[ ] 120 photos for dental -- verify photo pagination works
[ ] Notification list pagination for a user with many notifications

System Broadcast¶

[ ] As super_admin, send a broadcast -- verify it appears on ALL customer dashboards
[ ] Customers cannot dismiss the banner
[ ] Send a new broadcast while one is active -- old one is deactivated
[ ] Deactivate broadcast -- verify it disappears from all customer dashboards
[ ] Regular admin cannot create broadcasts

Concurrent Operations¶

[ ] Two customers submitting "Finish Review" at the same time on the same batch -- only one should succeed
[ ] Rapid-fire photo uploads (10+ simultaneously) -- all should succeed or fail gracefully
[ ] Settings optimistic concurrency: two saves from different browser tabs -- second should warn about conflict

Input Validation¶

[ ] Onboarding form with missing required fields -- should show validation errors
[ ] Review chat with empty message -- should be rejected
[ ] Article slug with special characters -- should be sanitized or rejected
[ ] Photo with filename longer than 255 characters -- should be handled
[ ] Email with uppercase letters -- should be canonicalized to lowercase

15. Full Customer Journey (End-to-End)¶

This walkthrough tests the entire flow from purchase to completion. Requires a running backend with API keys configured.

Step 1: Simulate Purchase¶

[ ] Call POST /api/v1/webhooks/ghl/purchase with:

{
  "email": "newcustomer@test.aeobunny.com",
  "ghl_contact_id": "test-ghl-id-001",
  "product_tier": "$497"
}

Header: X-GHL-Secret: <configured secret>

[ ] Verify PendingLead created in database
[ ] Verify GHL webhook fires for purchase notification

Step 2: Complete Onboarding¶

[ ] Navigate to /onboard in browser
[ ] Fill intake form:
Business name: "Test QA Business"
Email: newcustomer@test.aeobunny.com
City: "Chicago"
State: "IL"
Services: "General contracting, home renovation"
Website URL: "https://example-qa-business.com"
[ ] Set password (meets strength requirements: uppercase, lowercase, number, special char, 8+ chars)
[ ] Submit form
[ ] Verify Business, Location, User, PipelineRun records created
[ ] Verify readiness intake check fires in background
[ ] Verify photos_upload_needed notification dispatched

[ ] Login with newcustomer@test.aeobunny.com and the password set above
[ ] GET /api/v1/me returns role: "customer", correct location_id
[ ] Dashboard shows early pipeline state ("Research in progress" or similar)

Step 4: Wait for BI Research + Operator Approval¶

[ ] Pipeline runs BI Agent (requires ANTHROPIC_API_KEY)
[ ] Pipeline pauses at gate_bi_review
[ ] Login as operator, review BI data, approve

Step 5: Wait for Content Strategy + Operator Approval¶

[ ] Pipeline runs Strategist Agent
[ ] Pipeline pauses at gate_matrix_review
[ ] Login as operator, review 50-article matrix, approve
[ ] 5 batches created after approval

Step 6: Upload Photos¶

[ ] Customer navigates to /portal/photos
[ ] Upload 100+ photos (JPEG/PNG images of the business)
[ ] Verify quality badges appear after Image Intel Agent scoring
[ ] Progress bar reaches 100/100
[ ] Photo gate auto-clears -- pipeline resumes

Step 7: Review Batch 1 Articles (Operator First)¶

[ ] Pipeline writes 10 articles for batch 1
[ ] Pipeline pauses at gate_batch_article_review
[ ] Operator reviews and approves batch 1 (dual gate)
[ ] Pipeline re-pauses for customer review
[ ] Customer reviews each article: approve some, edit others
[ ] Customer clicks "Finish Review"
[ ] Haiku extracts dossier from chat messages
[ ] Pipeline resumes

Step 8: Review HTML Pages (Operator First)¶

[ ] Pipeline assembles HTML pages with images and schema
[ ] Pipeline pauses at gate_batch_html_review
[ ] Operator reviews HTML preview, approves
[ ] Customer reviews HTML preview, approves
[ ] Pipeline advances to deployment

Step 9: Confirm Deployment¶

[ ] Batch 1 marked "ready_to_ship"
[ ] Customer downloads ZIP
[ ] Customer fills deployment checklist (pages_uploaded, google_search_console_submitted)
[ ] Customer enters hub page URL
[ ] Customer confirms deployment
[ ] Post-deployment visibility scan fires
[ ] Post-deployment readiness check fires

Step 10: Check Visibility Score¶

[ ] Navigate to Visibility tab
[ ] Post-deployment score appears
[ ] Compare to baseline (should be different)

Step 11: Batches 2-5 (Customer-Direct Review)¶

[ ] Batch 2 articles written (with dossier from batch 1 feedback)
[ ] Customer reviews directly (no operator gate)
[ ] Repeat approve/edit/finish review/deploy for batches 2-5
[ ] Each batch benefits from accumulated dossier

Step 12: Project Complete¶

[ ] After batch 5 deployment confirmed, pipeline status changes to "completed"
[ ] Customer dashboard shows "Project Complete"
[ ] All 50 articles deployed
[ ] Ongoing visibility monitoring active

16. Test Data Reference¶

Quick reference for what each seeded project tests.

#	Project	Email	Pipeline State	What It Tests
1	Bright Smile Dental	`dental@test.aeobunny.com`	`completed` (all 5 batches shipped)	Full data: 50 articles, 120 photos, 4 visibility scores (32 checks, 24 analyses), 2 readiness scores, 5 deployments, 12 notifications (3 unread), consultation request, version history, API usage, pipeline events
2	Summit Roofing Co	`roofing@test.aeobunny.com`	`paused` at `gate_batch_article_review` (batch 2, customer gate)	Batch 2 article review with mixed statuses (4 approved, 3 pending, 2 edited, 1 flagged), 6 ReviewMessages, ArticleVersions, batch 1 deployed, baseline visibility, intake readiness
3	Verde Landscaping	`landscaping@test.aeobunny.com`	`paused` at `gate_matrix_review`	Operator approval of content strategy (50-item matrix). No batches or articles yet. Intake readiness score. Minimal customer portal state
4	Apex Plumbing	`plumbing@test.aeobunny.com`	`paused` at `gate_photo_upload`	Photo gate testing: 65/100 photos uploaded, gate not passed. Batch 1 articles all approved. Upload portal, progress tracking, quality badges
5	Coastal Realty Group	`realty@test.aeobunny.com`	`running` at `article_writing` (batch 1)	Active pipeline: 4 articles partially written. In-progress state, no review available. Intake readiness score
6	FreshFit Meal Prep	`mealprep@test.aeobunny.com`	`failed` at `article_writing` (batch 1)	Pipeline failure: 3 partial articles, `attempts=3` (max retries exhausted), error message, failed batch with `failed_at_status="writing"`
7	Mountain View Veterinary	`vet@test.aeobunny.com`	`paused` at `gate_batch_html_review` (batch 1, customer gate)	HTML review: 10 articles approved, 10 HTML outputs, 10 schema markups, 105 photos. Operator already reviewed (dual gate customer side). HTML preview testing
8	Harmony Yoga Studio	`yoga@test.aeobunny.com`	`paused` at `gate_batch_article_review` (batch 1, operator gate)	Revision workflow: revision_round=1, 10 articles with mixed statuses (3 approved, 4 edited, 2 pending, 1 flagged), BatchRevision pending_approval, RevisionInstructions, dossier populated, 102 photos

Admin Users¶

Email	Role	Notes
`admin@test.aeobunny.com`	`super_admin`	Full access including Settings
`ops1@test.aeobunny.com`	`admin`	Standard operator (no Settings access)
`ops2@test.aeobunny.com`	`admin`	Second operator for concurrency testing

Key Location Slugs¶

All location slugs match the project prefix used in the seed script:

Project	Location Slug
Bright Smile Dental	`bright-smile`
Summit Roofing Co	`summit-roofing`
Verde Landscaping	`verde-landscaping`
Apex Plumbing	`apex-plumbing`
Coastal Realty Group	`coastal-realty`
FreshFit Meal Prep	`freshfit-mealprep`
Mountain View Veterinary	`mountain-view-vet`
Harmony Yoga Studio	`harmony-yoga`

Deterministic UUIDs¶

All seed data uses deterministic UUID-v5 derived from namespace a1b2c3d4-e5f6-7890-abcd-ef1234567890. To compute a specific entity's UUID:

import uuid
SEED_NS = uuid.UUID("a1b2c3d4-e5f6-7890-abcd-ef1234567890")
# Example: dental customer's location ID
location_id = uuid.uuid5(SEED_NS, "bright-smile-location")

This is useful for constructing API requests targeting specific seeded entities.

Checklist Summary¶

Section	Total Checks	Focus Area
1. Environment Setup	Setup steps	Seed, clean, credentials
2. Auth & Authorization	~25	Login, roles, token, access control
3. Admin Portal	~45	8 sub-sections: overview, detail, approvals, stats, settings, profile, search, bell
4. Customer Portal	~50	8 seeded projects, each a different state
5. Pipeline Gates	~25	Phase A gates, batch gates, photo gate, deploy confirm, ownership
6. Visibility Score	~20	Overview, trends, competitors, alerts, consultation, API
7. Readiness Score	~10	Dashboard card, categories, API
8. Photo Collection	~25	Upload, lightbox, gate integration, limits, pagination
9. Revision Workflow	~25	Request Changes/Approve Article, Finish Review, version history, admin queue
10. Notifications	~15	Bell, mark read, cross-tab sync, admin, API
11. Mobile Responsive	~25	Navigation, dashboard, timeline, photos, article, visibility, settings
12. API Endpoints	~60	All routers: health, auth, admin, customer, batch, deploy, photos, visibility, readiness, notifications, profile, settings, revisions, versions, broadcast, onboarding, GHL, consultation
13. Security	~25	IDOR, auth, SSRF, rate limiting, CSP, CORS, webhooks, cross-batch
14. Edge Cases	~25	Empty states, failure recovery, boundaries, pagination, broadcast, concurrency, validation
15. Full Journey	~20	End-to-end from purchase to completion
Total	~395

Known SQLAlchemy Quirks¶

Explicit flush() calls in seed scripts

The seed script uses explicit session.flush() calls to resolve foreign-key ordering before related records are inserted. This is intentional and correct. The idiomatic SQLAlchemy alternative is the session.no_autoflush context manager:

with session.no_autoflush:
    # insert parent + child without triggering premature FK checks
    session.add(parent)
    session.add(child)
session.flush()  # flush once when safe

Both patterns are valid. no_autoflush is preferred when you need to suppress flushes across a block rather than inserting an explicit flush at a specific point.

QA Guide -- Aggressive Product Testing¶

Table of Contents¶

1. Test Environment Setup¶

Seeding Test Data¶

Test User Credentials¶

Portal Access¶

Required Environment Variables¶

2. Authentication & Authorization¶

Login Flow¶

Invalid Credentials¶

Token Expiry¶

Role-Based Access¶

3. Admin Portal Testing¶

3.1 Dashboard / Overview Page¶

3.2 Project Detail Page¶

3.3 Approvals Queue¶

3.4 Stats Dashboard¶

3.5 Settings (Super Admin Only)¶

3.6 Profile¶

3.7 Search¶

3.8 Notification Bell¶

4. Customer Portal Testing (Per Project State)¶

4.1 Bright Smile Dental -- dental@test.aeobunny.com (Completed)¶

4.2 Summit Roofing Co -- roofing@test.aeobunny.com (Batch 2 Article Review)¶

4.3 Verde Landscaping -- landscaping@test.aeobunny.com (Waiting for Operator)¶

4.4 Apex Plumbing -- plumbing@test.aeobunny.com (Photo Gate)¶

4.5 Coastal Realty Group -- realty@test.aeobunny.com (Running)¶

4.6 FreshFit Meal Prep -- mealprep@test.aeobunny.com (Failed)¶

4.7 Mountain View Veterinary -- vet@test.aeobunny.com (HTML Review)¶

4.8 Harmony Yoga Studio -- yoga@test.aeobunny.com (Revision Pending)¶

5. Pipeline Gate Testing¶

Phase A Gates (Operator Only)¶

Batch Gates (Dual-Gate Pattern)¶

Photo Gate¶

Deployment Confirmation Gate¶

Gate Ownership Enforcement¶

6. Visibility Score Testing¶

Overview Tab (dental@test.aeobunny.com)¶

Trends Tab¶

Competitors Tab¶

Alerts¶

Consultation Request¶

API Verification¶

7. Readiness Score Testing¶

Customer Dashboard (dental@test.aeobunny.com)¶

Readiness API¶

Verde Landscaping (intake only)¶

8. Photo Collection Testing¶

Photo Upload Portal (plumbing@test.aeobunny.com)¶

Upload Zone¶

Photo Lightbox¶

Photo Gate Integration¶

Photo Limits¶

Pagination¶

Bright Smile Dental Photos (120 photos, completed project)¶

9. Revision Workflow Testing¶

Harmony Yoga Studio (yoga@test.aeobunny.com)¶

Request Changes / Approve Article Flow¶

Finish Review Flow¶

Summit Roofing Co (roofing@test.aeobunny.com) -- Active Review¶

Version History¶

Admin Revision Queue¶

10. Notification System Testing¶

Notification Bell (dental@test.aeobunny.com)¶

Mark as Read¶

Cross-Tab Sync (BroadcastChannel)¶

Admin Notifications¶

API Verification¶

11. Mobile Responsive Testing¶

Navigation (< 1024px)¶

Dashboard (< 1024px)¶

Pipeline Timeline (< 1024px)¶

Photo Grid (< 1024px)¶

Article Detail (< 1024px)¶

Visibility Page (< 1024px)¶

Settings Page (< 1024px)¶

Review Page (< 1024px)¶

Desktop Verification (>= 1280px)¶

12. API Endpoint Testing¶

Health¶

4.1 Bright Smile Dental -- `dental@test.aeobunny.com` (Completed)¶

4.2 Summit Roofing Co -- `roofing@test.aeobunny.com` (Batch 2 Article Review)¶

4.3 Verde Landscaping -- `landscaping@test.aeobunny.com` (Waiting for Operator)¶

4.4 Apex Plumbing -- `plumbing@test.aeobunny.com` (Photo Gate)¶

4.5 Coastal Realty Group -- `realty@test.aeobunny.com` (Running)¶

4.6 FreshFit Meal Prep -- `mealprep@test.aeobunny.com` (Failed)¶

4.7 Mountain View Veterinary -- `vet@test.aeobunny.com` (HTML Review)¶

4.8 Harmony Yoga Studio -- `yoga@test.aeobunny.com` (Revision Pending)¶