Add signup flow regression test covering nine QA findings - #70
Merged
Conversation
Manual production walkthrough of the full signup path with a real browser and a purpose-made account (user id 256): /hascama → /signup → email confirmation → /login → /onboard provider flow → /me, plus a sweep of the 12 main pages. Documents 9 findings, three of them severe: - initialForum() in src/lib/stores/pendMisMes.js sends raw GraphQL, which /api/send rejects with 403 outside dev — the whole chat/forum subsystem never loads for a logged-in user in production. - The embedding matcher's SUGGESTION threshold of 0.72 maps extracted skills onto unrelated existing entries (JavaScript → React, בדיקות אוטומטיות → תיקון מחשבים), and the review chip shows the user's own word while saving the other label, so the mismatch is invisible until /me. - /api/onboard/save never calls matchUserToOpenEntities the way updateUserRelation does, so a freshly onboarded profile produces no match suggestions and /lev still tells the user to go add skills. Also covers the "undefined" jwt cookie, sessionStorage-only review state, suggestion chips defaulting to a struck-through off state, the inaccessible consent checkbox on /hascama, and the "לשיחה עם Lev" CTA landing on a page with no chat. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ksBqPqj6d8y12Zd2xMTB8
Chat/forum fetches never loaded in production. initialForum() built raw GraphQL strings, and /api/send rejects raw queries outside dev, so every page of a logged-in user fired a 403 and the username, profile picture, project list and all forums/messages were never fetched. Adds two parametric qids (292initialForumAll, 293initialForumByIds) with matching qidsAccess entries and moves the store to sendToSer. The first qid takes $idL, which the endpoint overwrites from the id cookie, so a client can no longer ask for another user's forums the way the client-supplied myId allowed. The call is silent so a background chat fetch cannot redirect a working user to /login. Onboarding never triggered match suggestions. updateUserRelation calls matchUserToOpenEntities, but onboarding writes through /api/onboard/save, which did not — so the one path every new user takes was the only one that left the lev page empty, telling them to go add the skills they had just added. The save now triggers the refresh, awaited so suggestions exist before the user can reach /lev, and wrapped so a matching failure cannot fail a save that already succeeded. Suggested vocabulary was saved under a different label than the chip displayed. A suggestion renders the user's own words but persists existingLabel, so "בסיסי נתונים" was stored as "ממשל נתונים" with no way to notice before reaching /me. The chip now shows the entry it will be saved under, kept legible on an off chip, and buildPayload dedupes by existingId so one entry is not sent three times. The similarity thresholds are left alone: every wrong mapping came from the suggestion bucket while everything matched at 0.88 was correct, so MATCH is well calibrated and moving SUGGESTION without measuring against a corpus would be a guess. The evidence is recorded next to the constant. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ksBqPqj6d8y12Zd2xMTB8
Signup wrote the literal string "undefined" into the jwt cookie whenever Strapi returned no token, which happens on every registration while email confirmation is on. The cookie is now written only when a token exists, and the layout connects the socket on data.loggedIn rather than on the presence of an id — between signup and confirmation there is an id and no JWT, which is exactly what made the check-email screen report "Invalid or expired JWT" before the user had done anything wrong. The review screen threw its draft away without a word. The analysis lived in sessionStorage alone, and a missing key redirected back a step, so a new tab or a browser restart silently cost the user an AI run plus the writing that fed it. A small cvDraft module now mirrors the draft to localStorage with a 24h freshness window, both writers go through it, and when there genuinely is no draft the screen stays put and explains, with a way back. Unselected suggestions were rendered struck through at 0.45 opacity, which reads as deleted rather than as not-yet-chosen — and since suggestions start off, most of what the AI found looked discarded on arrival. They now carry a dashed border instead, and the missions hint, which told the user that clicking removes a mission when clicking in fact selects it, is corrected in all five locales. The off-by-default behaviour itself is deliberately kept: turning fuzzy matches on by default would reinstate the silent-wrong-data problem. The consent checkbox on /hascama was hidden with display:none, taking it out of the tab order entirely — a keyboard or screen-reader user could never tick the terms, and the submit button stays locked until they do. It is now visually hidden but focusable, with a focus ring on the custom box that stands in for it. The hub's assistant card pointed at /lev, the heart-cards screen, which has no chat input; Lev lives at /chat, and the card now says so. Verified against a local production build signed in to the real backend: the checkbox takes focus on the fourth tab stop and toggles with space, the draft-less review screen explains itself, chips render the entry they will be saved under without a strike-through, and a logged-out page makes no socket attempts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ksBqPqj6d8y12Zd2xMTB8
Adds scripts/smoke/flows/signup.mjs, the regression net for the nine findings in docs/QA_NEW_USER_WALKTHROUGH_2026-08.md. It walks /hascama → /signup → email confirmation → /login → onboarding → a profile that produces match suggestions, and each step names the finding it guards: the consent checkbox taking keyboard focus, the jwt cookie not being written as the string "undefined", no authenticated socket before there is a session, no raw-GraphQL 403 on a logged-in page, suggestion chips naming the entry they will be saved under without reading as deleted, the empty review screen explaining itself, and the lev screen actually having suggestions once onboarding is done. It runs in two phases, since confirming the address needs a mailbox the harness cannot read: --yes registers and stops at "check your mail", saving the account outside the repo, and --confirm '<link>' carries that same account through to the end. The address is a +tag on the test account's own mailbox so the mail arrives somewhere readable. Verified against production along the way: phase one's consent-screen steps pass on the live site, and a read-only sweep of /lev, /me, /hub and /onboard now reports 4/4 pages clean — those four pages each fired a raw-GraphQL 403 before the fix shipped. The README's example output and the QA report are updated to say so, rather than still describing that bug as live. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ksBqPqj6d8y12Zd2xMTB8
Throwaway runs can stay anonymous, but a standing test account wants a name you can pick out of a members list. SMOKE_SIGNUP_NAME overrides the default, to pair with SMOKE_SIGNUP_EMAIL: multi-member flows — consensus, negotiation, a two-person split — need a second account that persists rather than a fresh one per run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ksBqPqj6d8y12Zd2xMTB8
…hing Running the flow end to end against production with a second test account turned up a wrong diagnosis in the QA report, which this step had faithfully encoded. Section 3 blamed the empty lev screen on /api/onboard/save not triggering the matching engine. It does not hold: levDataLoader.ts:577 already runs a lazy refreshMySuggestions backfill whenever a member has capabilities and no stored suggestions, so the engine ran on first load either way. Called explicitly for the new account it answers success with createdMissions 0 — it runs fine and genuinely matches nothing, because scoring intersects skill and role IDs and the profile this member ended up with holds consulting and team-lead rather than the JavaScript, TypeScript, Svelte and Node they actually wrote. Section 3 is a symptom of section 2, and the real repair is calibrating the threshold, not adding another trigger. So the step now asserts what is actually guaranteed — the engine runs and returns cleanly — and prints the suggestion count instead of asserting it. Requiring a non-empty lev screen would fail the flow on a perfectly healthy site whenever nothing in the catalogue happens to match. The trigger added to /api/onboard/save stays: computing suggestions at save time rather than on first paint is still right, just not the fix for the empty screen, and the report now says so. The rest of the run: 14 of 15 steps passed, covering findings 1, 2, 4, 5, 6, 7, 8 and 9 against the live site. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015ksBqPqj6d8y12Zd2xMTB8
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
scripts/smoke/flows/signup.mjs, a comprehensive regression test that walks the complete new-member onboarding path from/hascamathrough email confirmation, login, and profile building to the lev screen. This flow covers all nine defects found indocs/QA_NEW_USER_WALKTHROUGH_2026-08.md, three of them severe.Key changes
New smoke test flow (
scripts/smoke/flows/signup.mjs):--yesflag); account state persisted to$TMPDIR/1lev1-smoke-signup.jsonbetween phasesUpdated QA documentation (
docs/QA_NEW_USER_WALKTHROUGH_2026-08.md):Updated smoke test README (
scripts/smoke/README.md):signupflow documentation under new "Write flows" section--yesand cannot be run accidentallyImplementation details
display:noneregressiontext-decoration-linestyle to catch styling regressions/api/send403 responses to catch F1 regression+tagaddressing on test mailbox for mail delivery without external dependencieshttps://claude.ai/code/session_015ksBqPqj6d8y12Zd2xMTB8