padhle - post migration
This commit is contained in:
+331
-2
@@ -307,11 +307,12 @@ Supabase CLI (local dev via `npx supabase start`).
|
||||
|
||||
| Key | Value |
|
||||
|---|---|
|
||||
| **Publishable Key** (client-side) | `sb_publishable_ACJWlzQHlZjBrEguHvfOxg_3BJgxAaH` |
|
||||
| **Secret Key** (server-side) | `sb_secret_N7UND0UgjKTVK-Uodkm0Hg_xSvEMPvz` |
|
||||
| **Publishable Key** (client-side, public by design) | `sb_publishable_ACJWlzQHlZjBrEguHvfOxg_3BJgxAaH` |
|
||||
| **Secret Key** (server-side) | `<rotated-local-dev-service-role-key>` — **rotated 2026-08-19; never commit the real value** |
|
||||
|
||||
> **Never expose the Secret Key in client code.** Use the publishable key in frontend, secret key in backend.
|
||||
> Local dev uses shared default keys — **do not use in production**.
|
||||
> **Security note:** the previous `sb_secret_...` value was committed to git in this doc, `IMPLEMENTATION_PLAN.md`, and `backend/scripts/setup-db.js`. All three were scrubbed; rotate the local keys (`npx supabase stop && npx supabase start` or Studio) and update only `backend/.env`.
|
||||
|
||||
### Database
|
||||
|
||||
@@ -704,3 +705,331 @@ Before context work has value, the **frontend session-continuity bug** must be f
|
||||
- Ensure `.env` secrets are never committed or exposed.
|
||||
- Rotate any secret key that has been shared outside the local environment.
|
||||
- Add monitoring for rate-limit hits and database errors.
|
||||
|
||||
---
|
||||
|
||||
## Textbook Grounding — Future Plan (touchpoint)
|
||||
|
||||
> Direction agreed 2026-08-23. Endgoal: selected grade → subject → chapter resolves to actual textbook markdown content, and the model answers **only from that supplied content**. Conversation memory stays separate from textbook evidence (the model-authored rolling summary is never a source of factual truth). No code shipped yet — this is the agreed design and the V-order touchpoint for ANALYSIS.md.
|
||||
|
||||
### The seam (core invariant)
|
||||
|
||||
```
|
||||
SYSTEM — base tutor identity (grade/subject/chapter already validated server-side)
|
||||
MEMORY — rolling conversation summary; continuity ONLY, never a fact source
|
||||
TEXTBOOK EVIDENCE — retrieved markdown sections; the ONLY factual authority
|
||||
RECENT CONVERSATION — last 10 messages
|
||||
CURRENT QUESTION — the student's message
|
||||
```
|
||||
|
||||
Model rule: answer exclusively from the supplied textbook sections; if they don't cover the question, say so. The summary guides continuity but can never override textbook retrieval.
|
||||
|
||||
### Content model
|
||||
|
||||
- One markdown file per chapter, plain text read from disk by the backend — **the model never receives a filesystem path**; the backend resolves the selection, reads the file, and injects the parsed content into the prompt (this is where `ai.js` already sits).
|
||||
- Filesystem layout (grade = directory layer, subject = namespace, chapter = file granularity):
|
||||
```
|
||||
content/
|
||||
└── grade-10/
|
||||
└── science/
|
||||
├── 01-chemical-reactions.md
|
||||
└── 02-acids-bases-salts.md
|
||||
```
|
||||
- **No build-time content indexing needed for V1** — files are authored markdown, not derived.
|
||||
|
||||
### Registry (decision: UI identity ≠ filename ≠ display label)
|
||||
|
||||
Do **not** couple application IDs to filesystem paths. A small hand-maintained registry maps trusted `{gradeId, subjectId, chapterId}` → `{ file, title }`:
|
||||
|
||||
```js
|
||||
// backend/src/services/content/textbookRegistry.js
|
||||
export const TEXTBOOK_REGISTRY = {
|
||||
"10": {
|
||||
"science": {
|
||||
"chemical-reactions": {
|
||||
title: "Chemical Reactions & Equations",
|
||||
file: "content/grade-10/science/01-chemical-reactions.md",
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
export const getChapter = (grade, subject, chapterId) =>
|
||||
TEXTBOOK_REGISTRY[grade]?.[subject]?.[chapterId] ?? null;
|
||||
```
|
||||
|
||||
- Lookup is registry-only; the resolved `file` path is what touches disk — never raw user values (path traversal becomes a non-issue by construction, not by relying on input validation).
|
||||
- Renames (`chemical-reactions.md` → `chapter-01-…`) don't break sessions or frontend ids.
|
||||
- Unknown `{grade, subject, chapter}` → hard 500 (leak nothing), **never** fall back to generic Qwen.
|
||||
|
||||
### Retrieval mode — RECONCILED 2026-08-24 (section-level retrieval is the default)
|
||||
|
||||
> **Supersedes** the earlier staged framing (V1 whole-chapter / V2 section-retrieval-"only if needed"). Real scanned textbooks (30k-line chapters, e.g. Rajeev Bansal Class XII Economics at an estimated 60–90k tokens/chapter — estimate pending measurement) proved chapters are too large to inject whole per request. Section-level retrieval is therefore the **default** path. Whole-chapter injection is kept only as a corner case for genuinely small chapters (< a few k tokens).
|
||||
|
||||
```
|
||||
Grade 10 → Science → Chemical Reactions
|
||||
→ registry resolves chapter (never raw user values → no traversal)
|
||||
→ read PRE-NORMALIZED section file (built offline; runtime never parses raw OCR)
|
||||
→ retriever keyword-scores question vs section text → top-N sections
|
||||
→ top-N injected as TEXTBOOK EVIDENCE
|
||||
→ model answers from them only; refuses if uncovered (strict)
|
||||
```
|
||||
|
||||
- Parser + retriever share one artifact: `markdownParser` (`##`-split) runs at **offline ingestion** only, never per-request; `retriever` scores the pre-split sections at request time.
|
||||
- Keep boring: keyword/regex section scoring against the raw query — no embeddings / pgvector / vector DB / relevance-threshold tuning.
|
||||
- Strict grounding confirmed: answer exclusively from supplied sources; say so when the content doesn't cover the question.
|
||||
|
||||
### Explicitly deferred / not in scope now
|
||||
|
||||
- `textbooks` / `chapters` / `textbook_sections` DB tables (filesystem + registry is enough for the first books; revisit only at content scale).
|
||||
- Fine-tuning a small model to replace Qwen (answer from grounded retrieval first; fine-tune only if quality demands it later).
|
||||
- Separating summary into a distinct persisted memory store — keep the current `sessions.summary` seam honed, just never let it inject textbook-fact authority.
|
||||
|
||||
---
|
||||
|
||||
## Next Conversation — Starting Point (2026-08-24)
|
||||
|
||||
> This is the handoff anchor. If the conversation context is lost, resume here. Everything below is **decided and agreed** — not speculative. No code shipped yet; plan mode.
|
||||
|
||||
### Problem statement
|
||||
|
||||
Padhle's grade → subject → chapter selector currently drives **prompt context only** — the model has no access to actual textbook content. A student selecting "Grade 10 → Science → Chemical Reactions" gets a generic Qwen answer, not an answer grounded in the selected textbook. The endgoal: make that selection resolve to the exact textbook sections the model may answer from, and **nothing else**. The model must answer exclusively from the supplied content and say so when it can't. Conversation memory (rolling summary) must never become a source of factual truth — textbook evidence and conversation memory are separate trust domains.
|
||||
|
||||
### Detailed solution
|
||||
|
||||
```
|
||||
STUDENT selects Grade 12 → Economics → Price Elasticity of Demand
|
||||
↓
|
||||
BACKEND resolves {gradeId, subjectId, chapterId} via textbookRegistry
|
||||
↓ (never raw user values → path traversal impossible by construction)
|
||||
BACKEND reads the chapter's PRE-NORMALIZED section file (built offline)
|
||||
↓
|
||||
RETRIEVER keyword-scores the question against section text, returns top-N
|
||||
↓
|
||||
AI receives: SYSTEM(base) + MEMORY(summary) + TEXTBOOK EVIDENCE(sections) + RECENT + QUESTION
|
||||
↓
|
||||
Qwen answers ONLY from the supplied sections; refuses if uncovered
|
||||
```
|
||||
|
||||
Offline ingestion / online application are **separate systems**: messy OCR/MinerU cleanup happens once per book (script proposes boundaries → you review), never inside a student request.
|
||||
|
||||
### Decisions locked (in order)
|
||||
|
||||
1. **Source format: markdown (md), not the scanner's JSON.** md is ~⅓ the tokens (JSON carries `bbox`/`page_idx`/per-page header+footer noise), renders native `##` / `$…$` / `$$…$$`, and matches the existing KaTeX rendering. JSON's only edge is `text_level`/`type` for offline section disambiguation.
|
||||
2. **Registry, not filesystem-as-model.** Stable keys `{grade,subject,chapter}` → `{file,title}`; UI identity ≠ filename ≠ display label; renames don't break sessions. Unknown selection → hard 500, never silent generic fallback.
|
||||
3. **Section-level retrieval is the default** (not whole-chapter). Chapters are too large to inject whole. Whole-chapter injection only for genuinely small chapters.
|
||||
4. **Conversation memory ≠ textbook evidence.** Structurally separated prompt blocks. Summary may never inject textbook-fact authority.
|
||||
5. **Model never reads a file path.** Backend resolves + reads + injects. That's where `ai.js` already sits.
|
||||
6. **Strict grounding:** answer exclusively from supplied textbook sources; refuse if uncovered.
|
||||
|
||||
### Backend shape (proposed)
|
||||
|
||||
```
|
||||
backend/src/services/content/
|
||||
├── textbookRegistry.js # authored map {grade,subject,chapter} → {file,title}
|
||||
├── retriever.js # keyword-scans pre-normalized sections; top-N
|
||||
├── markdownParser.js # ##-split (OFFLINE ingestion only, not per-request)
|
||||
└── contentLoader.js # compose: registry → read → (parse, if raw) → sections
|
||||
```
|
||||
|
||||
`ai.js`: `buildSystemPrompt(grade,subject,chapter,sources)` gains strict-grounding instructions; `buildMessages` appends sources as a separate block (never merged into summary). Route (`chat.js`): resolve → sources → inject; unknown → hard 500.
|
||||
|
||||
### Content file format — author-facing spec (YOU provide this) — LOCKED 2026-08-24
|
||||
|
||||
> Decision: you hand me **clean, `##`-sectioned markdown, one file per chapter** (human-readable form, same shape as the confirmed-clean NCERT `file.md`). My ingester pre-splits it into retrieval-ready sections **offline** — you never maintain machine JSON by hand.
|
||||
|
||||
**File naming & location** — one `.md` per chapter under the raw store:
|
||||
```
|
||||
backend/content/raw/
|
||||
└── grade-12/
|
||||
└── economics/
|
||||
├── ch01-demand.md
|
||||
├── ch02-supply.md
|
||||
└── ...
|
||||
```
|
||||
- `grade-<N>/` = grade directory (numeric), `<subject>/` = subject slug, `ch<NN>-<slug>.md` = chapter.
|
||||
- File basename (`ch01-demand`) is the stable **chapter id** the registry keys on; it won't collide across subjects because the subject dir namespaces it.
|
||||
|
||||
**Heading rules**
|
||||
- `#` (H1) = chapter title, once at top.
|
||||
- `##` = **one logical section per heading** — numbered (`## 1.1 Introduction`, `## 1.2 …`) OR standalone names (`## EXERCISE 1.1`, `## Miscellaneous Exercise on Chapter 1`).
|
||||
- Do **not** use `##` for boxes/figures/callouts (that overloaded it in Rajeev Bansal — reject that source's style). Keep those as plain paragraphs inside a section.
|
||||
- `###` + deeper = fine inside a section; the ingester splits only on `##`.
|
||||
|
||||
**What to include / strip**
|
||||
- Keep: body text, `$…$` / `$$…$$` LaTeX math (renders via existing KaTeX), tables, `## EXERCISE` blocks (they're legit sections).
|
||||
- Strip before handing me: raw `<details>…</details>` image blocks, `` placeholders, running headers/footers repeating the book title / page numbers. (The scanner's JSON carries this noise; your clean md should not.)
|
||||
- No `bbox`/`page_idx`/JSON — this is plain markdown.
|
||||
|
||||
**Ingestion target** (what my ingester emits, NOT yours to write):
|
||||
```
|
||||
{ grade, subject, chapter, chapter_title, section, section_id, text }
|
||||
// e.g. { grade:12, subject:"economics", chapter:5,
|
||||
// section:"5.2 Factors Affecting Elasticity",
|
||||
// section_id:"eco12-ch5-s5.2", text:"…" }
|
||||
```
|
||||
|
||||
### Storage model — LOCKED 2026-08-24 (two stores, never merged)
|
||||
|
||||
```
|
||||
backend/content/
|
||||
├── raw/ ← Store 1: YOUR clean per-chapter md. Durable, human-authored,
|
||||
│ └── grade-12/ editable, committed, never read by the running server.
|
||||
│ └── economics/
|
||||
│ └── ch01-demand.md
|
||||
└── generated/ ← Store 2: built by the ingester from raw/ at deploy/boot.
|
||||
└── grade-12/ Pre-split sections; the ONLY thing runtime reads.
|
||||
└── economics/
|
||||
└── ch01-demand.sections.json # [{id,heading,text}, …]
|
||||
```
|
||||
- Registry (`textbookRegistry.js`) points at **Store 2** (`generated/grade-12/economics/ch01-demand.sections.json`). Runtime never touches Store 1.
|
||||
- Regenerate Store 2 from Store 1 whenever a chapter changes (a small script or a deploy step) — never hand-edit Store 2.
|
||||
- Rationale: raw = "the book, editable"; generated = "retrieval-ready, no parse at request time". This is the offline/online split the plan already calls for.
|
||||
|
||||
### Locked decisions (2026-08-24 — all five answered)
|
||||
|
||||
1. **Content source:** YOU provide clean per-chapter `##`-sectioned md (Store 1 above). I build the ingester that pre-splits it.
|
||||
2. **Scope of first vertical slice:** exactly ONE grade + ONE subject (Grade 12 → Economics) end-to-end (registry, retrieval, grounding) before scaling. Prove it, then grow.
|
||||
3. **Grounding strictness — LOCKED strict-only:** answer exclusively from supplied textbook sections. The model may be creative with **examples**, but only derived from the supplied text — no outside/general knowledge. Refuses if the content doesn't cover the question.
|
||||
4. **Grade normalization seam:** frontend/DB keep sending `"Grade 10"` (label); the content loader boundary normalizes to `"10"` (registry key) before lookup. Frontend/DB untouched.
|
||||
5. **Tokenizer measurement:** deferred to the future **admin-dashboard** build — not Phase A. Section-retrieval decision holds regardless.
|
||||
|
||||
### Blocking inputs before code
|
||||
|
||||
1. The **first cleaned chapter md** (Store 1) for Grade 12 → Economics, in the format above. Registry + parser contract anchor against it; I confirm the `##`-split against real input, then build the ingester + registry around that one file set.
|
||||
2. ~~Strictness~~ — **locked** (strict-only, decision #3).
|
||||
3. ~~Tokenizer count~~ — deferred to admin dashboard (decision #5).
|
||||
|
||||
### Where we are vs endgoal (2026-08-24 reconciliation)
|
||||
|
||||
- **MVP chat infrastructure: ~75–80% done** — auth, sessions, SSE, rate limiting, ownership isolation, anonymous trials, routing, markdown+KaTeX rendering, rolling summaries: all implemented, tested, green.
|
||||
- **The endgoal (grounded textbook answering): ~5%** — the entire content pipeline (registry, parser, retriever, loader, source injection) is **0% shipped**. It's additive into a deliberately left-open seam (`ai.js` prompt builders), not a rewrite.
|
||||
- The endgoal reduces to: 4 content files + extend 2 `ai.js` functions + route resolution + chapter ids in selector data. Everything existing stays.
|
||||
|
||||
### Execution checklist — next session (do in order)
|
||||
|
||||
**Phase A — Offline ingestion** (not in `/api/chat` path)
|
||||
1. Ingest raw md → split on `## <n.n>` → strip `<details>`/image placeholders/LaTeX spills → emit sections `{ section_id, heading, text }`.
|
||||
2. Attach Grade/Subject/Chapter metadata; produce one normalized section file per chapter. (Optional: measure chapter tokens here with the target tokenizer.)
|
||||
|
||||
**Phase B — Content backend**
|
||||
3. `backend/src/services/content/textbookRegistry.js` — authored `{grade,subject,chapter}` → `{file,title}`.
|
||||
4. `backend/src/services/content/retriever.js` — keyword-score question vs sections → top-N.
|
||||
5. `markdownParser.js` — `##`-split (offline only).
|
||||
|
||||
**Phase C — Wire into `ai.js`** (the existing seam)
|
||||
6. `buildSystemPrompt(grade,subject,chapter,sources)` gains strict-grounding instructions; `buildMessages` appends sources as a **separate block**, never merged into summary.
|
||||
7. `chat.js` route: resolve `{grade,subject,chapter}` → sources → inject; unknown → hard 500 (no silent generic fallback).
|
||||
|
||||
**Phase D — Frontend + tests**
|
||||
8. Give chapters stable ids (mirror subject id→label pattern; `activeChapter` currently stores labels) — compareField becomes `"id"` for chapters.
|
||||
9. Tests: parser split, registry miss/traversal, retriever scoring, sources-block injection (absent sources = byte-identical old prompt), full-suite green.
|
||||
|
||||
**Phase E — only if needed**: measured cost/latency tuning of the retrieval threshold (already section-retrieving; no new infra).
|
||||
|
||||
**Resolved micro-decisions (2026-08-24):**
|
||||
- Grade normalization: done at `contentLoader`/registry boundary (`"Grade 10"` → `"10"`); frontend/DB keep the label. Locked, decision #4.
|
||||
- Chapter id naming: the file basename (`ch01-demand`) IS the registry key / frontend chapter id — stable, namespaced by subject dir. The raw file spec (§ Content file format) defines it.
|
||||
|
||||
---
|
||||
|
||||
## Security Hardening — 2026-08-19 (implemented + tested)
|
||||
|
||||
> Full plan: `docs/security-hardening-plan.md`. All changes verified against the running local Supabase stack.
|
||||
|
||||
### Secret scrub 🔴
|
||||
- Hardcoded `sb_secret_...` (Supabase service-role, bypasses RLS) removed from `ANALYSIS.md`, `IMPLEMENTATION_PLAN.md`, and `backend/scripts/setup-db.js`.
|
||||
- `setup-db.js` now refuses to run without `SUPABASE_SERVICE_ROLE_KEY` (no fallback).
|
||||
- Added `scripts/check-secrets.mjs` — pre-commit scan; passes on the clean tree. **Rotate local keys** (`npx supabase stop && start`) since the old value is in git history.
|
||||
|
||||
### Auth
|
||||
- `GET /api/auth/me` now derives identity from the **verified JWT** (`verifyToken`), never the unsigned `padhle.user` cookie — forged cookies return 401 (tested).
|
||||
- Backend enforces password ≥ 6 chars on signup (matches frontend).
|
||||
|
||||
### Input validation
|
||||
- New `validateChapter()` / `validateSessionMeta()` in `chatValidation.js`: chapter capped at 200 chars, rejects `< > \n \r` (prompt-injection vector into the AI system prompt — closed, tested).
|
||||
- Applied to `POST /api/chat` and `POST /api/sessions`.
|
||||
|
||||
### AI provider
|
||||
- CVE-2026-004 fully closed: the non-streaming Google path in `getChatResponse` now uses the safe `URL` constructor (key in query string, never in path).
|
||||
- Every outbound AI call (OpenAI/Anthropic/Google + summarizer) has a 120 s `AbortSignal.timeout`.
|
||||
|
||||
### Middleware / robustness
|
||||
- Token cache in `supabaseAuth.js` capped at 2000 entries (FIFO eviction) — bounded memory.
|
||||
- New `securityHeaders` middleware (nosniff, frame-deny, referrer-policy, COOP; removes `X-Powered-By`) — tested.
|
||||
- `TRUST_PROXY` env to fix rate-limit client IP behind a reverse proxy.
|
||||
- Dev-mode rate limiter skips all loopback IPs (tests no longer trip the auth 5/15min limit).
|
||||
|
||||
### Session-leak verification (the core ask)
|
||||
- `backend/test/leak.test.js` — end-to-end against real local Supabase, in-process app on an ephemeral port, real cookies:
|
||||
- B cannot read / list / delete / clear / continue A's session (404s, listings exclude it).
|
||||
- `/api/auth/me` rejects forged cookies; unauthenticated protected routes → 401.
|
||||
- Chapter injection → 400; security headers present.
|
||||
- **Result:** `node --test` → 14/14 pass (9 unit + 5 integration). Frontend Vite build passes.
|
||||
|
||||
### Out of scope (documented)
|
||||
Refresh-token flow, server-side session revocation list, CSRF tokens, Redis rate-limit store, `run-migration.js` local Postgres password, legacy `sessionStore.js` XSS note.
|
||||
|
||||
---
|
||||
|
||||
## Anonymous 5-Message Trial — 2026-08-19 (implemented + tested)
|
||||
|
||||
> Agreed design: unregistered users can use the full app and send **5 free messages**; the backend hard-stops the 6th with `403 { code: "SIGNIN_REQUIRED" }`. Refresh or sign-out starts a fresh trial (pure UX funnel, not a security boundary — by design). No cookies, no DB writes for anonymous traffic.
|
||||
|
||||
### Backend
|
||||
- **`backend/src/services/anonTrial.js`** — in-memory trial store (factory `createTrialStore()` + default singleton with reaper):
|
||||
- Crypto-UUID trial ids (unguessable), 500-trial LRU cap, 30-min idle TTL reaper (unref'd interval), history trimmed (last 10 for AI context, 20 stored).
|
||||
- Only `user` messages consume the 5-message budget.
|
||||
- **`backend/src/services/sse.js`** — shared SSE writer: JSON-escapes `\n`/`U+2028`/`U+2029` (CVE-2026-003) and no-ops after `res.end()` (prevents write-after-end crashes).
|
||||
- **`backend/src/routes/chat.js`** — now `createChatRouter({ streamFn })` (injectable for tests). `POST /api/chat`:
|
||||
- **Anonymous** (`req.user` unset): resolve trial (`chatId` continues in-memory trial; unknown/forged id → fresh trial) → **if budget spent → 403 SIGNIN_REQUIRED before any AI call** → else consume 1 message, stream via SSE emitting `session` (trial id) + `limit` (remaining) events. **No DB writes at all.**
|
||||
- **Signed-in**: unchanged (DB session, rolling summary, unlimited).
|
||||
- `GET /api/chat/sessions` and `GET /api/chat/:id` now require auth (401 for anonymous) — anonymous trials are invisible to the sessions API.
|
||||
- **`backend/src/app.js`** — `/api/chat` mounts `optionalAuth` (not `supabaseAuth`); `/api/sessions` stays strictly authenticated.
|
||||
|
||||
### Frontend
|
||||
- **`App.jsx`** — hard auth gate removed; app renders for everyone. New state: `showSignIn` (on-demand modal), `anonRemaining` (5). `handleSend` blocks client-side at 0 remaining; handles `403 SIGNIN_REQUIRED`; applies `limit` SSE events. Auth-transition effect resets the budget (fresh trial per refresh/sign-out).
|
||||
- **`ChatInput.jsx`** — `exhausted` prop → replaces the input with a "You've used your 5 free messages — **Sign in to continue**" card (opens the modal).
|
||||
- **`TopNav.jsx` / `Sidebar.jsx`** — anonymous users get a "Sign in" button; signed-in users keep the avatar/sign-out.
|
||||
- **`SignIn.jsx`** — modal is now on-demand: optional `onClose` ("Maybe later" + × + overlay click).
|
||||
|
||||
### Tests (all passing — `cd backend && node --test` → 34/34; `cd frontend && npm run build` → passes)
|
||||
- `backend/test/anonTrial.test.js` (10) — budget semantics, UUID uniqueness, LRU cap, TTL reaper, context trimming, **worst-case concurrent interleaving (100 parallel → ≤5 pass)**.
|
||||
- `backend/test/anonFlow.test.js` (8) — integration with injected fake AI streamer (no real AI / no Supabase):
|
||||
- 5 messages succeed with `session`+`limit` events; 6th → 403 SIGNIN_REQUIRED with **no AI call**.
|
||||
- Continuity (chatId echoes same trial, AI sees prior turns); refresh/unknown id → fresh budget; forged id → fresh trial, never touches sessions; AI failure consumes message + clean error event; chapter injection → 400; unauth GET chat routes → 401.
|
||||
- Existing suites unchanged and green (chatValidation, aiMessages, leak against live Supabase).
|
||||
|
||||
### Security posture of the anonymous path
|
||||
- No DB writes → no data to leak; forged/session-id-shaped `chatId`s resolve to fresh trials, never to other users' data.
|
||||
- Bounded memory (LRU + TTL) → no DoS growth from anonymous traffic.
|
||||
- `chatLimiter` (30/min/IP) still applies on top.
|
||||
- Rate limiting + validation run before the trial branch (same rules as signed-in).
|
||||
|
||||
---
|
||||
|
||||
## Client-Side Routing — 2026-08-19 (implemented)
|
||||
|
||||
> The app previously had **no router** — it was a pure state-driven SPA where every "view" (selector flow, chat, modal) was a conditional render off React state, so the URL never changed from `/`. Added `react-router-dom` so URLs are meaningful and chats are deep-linkable.
|
||||
|
||||
### Changes
|
||||
- **`frontend/package.json`** — added `react-router-dom@^7`.
|
||||
- **`frontend/src/main.jsx`** — wrapped the app in `<BrowserRouter>` with explicit routes: `/`, `/chat/:chatId`, and a `*` catch-all (all render the same `<App/>` shell; `useParams()` exposes `chatId`).
|
||||
- **`frontend/src/App.jsx`**:
|
||||
- `useNavigate` + `useParams` — `handleNewChat` now goes to `/`; opening/creating a chat navigates to `/chat/:id` (`replace`). The SSE `session` event drives the URL for both signed-in sessions and anonymous trials.
|
||||
- A **deep-link effect** loads a conversation on cold-load of `/chat/:id` (signed-in users only; anonymous trial URLs just start fresh, matching the "refresh loses the conversation" rule).
|
||||
- A `chatLoadHandledRef` guards against duplicate loads (StrictMode double-invoke, sidebar-then-URL, auth restore) and drops stale in-app trial/session URLs on auth transitions.
|
||||
|
||||
### Result
|
||||
- Sharing/pasting `/chat/<id>` now lands on that conversation (**signed-in only**).
|
||||
- URL reflects the active chat; back/forward and refresh behave sensibly.
|
||||
- Anonymous users keep the URL at `/` — their trial conversation works, but the trial id is **never** written into the address bar (see security note below).
|
||||
- Verified: `vite preview` serves both `/` and `/chat/<id>` as the SPA shell (200, app `#root` present) — production deep links resolve.
|
||||
|
||||
### Security note (important)
|
||||
Anonymous trial ids are **capabilities** — there is no server-side ownership check on them; `POST /api/chat` with `chatId: <trialId>` rejoins the trial. Earlier, `navigate(\`/chat/${data.id}\`)` ran for *both* signed-in and anonymous, which baked an anonymous trial id into the URL — leaking the capability via browser history, server logs, referer headers, shared links, etc. **Fixed:** only signed-in sessions are reflected in the URL; anonymous trial ids stay in in-memory React state and the POST body only. Signed-in `/chat/:id` URLs are safe because the server ownership-checks them (404 unless the current user owns the session).
|
||||
|
||||
### Log redaction (defense-in-depth)
|
||||
- **`backend/src/services/logRedact.js`** — `redactSecrets()` scrubs UUIDs (both anonymous trial ids *and* signed-in session ids, since UUID format overlaps) and obvious API keys (`sb_secret_…`, `sk-…`) from any string; `logError(tag, err)` wraps `console.error` and returns non-strings unchanged.
|
||||
- **`backend/src/routes/chat.js`** — all five `console.error(err.message)` sites now route through `logError`, so a trial/session id embedded in an error message (e.g. an AI provider echoing an id back) is redacted before it reaches stderr. Covers the risk of a future request-logger (morgan/pino) or error tracker re-introducing bearer token / capability ids into logs.
|
||||
- **`backend/test/logRedact.test.js`** (2, included in the 34) — unit tests for the scrubber + an integration test that drives a real anonymous chat where the AI throws a message *containing* the actual trial id and an `sk-…` key, spies on `console.error`, and asserts neither the id nor the key reach the log output (with `[REDACTED_UUID]`/`[REDACTED_KEY]` markers instead). Proves the redaction is real, not vacuous (verifies the raw id was present pre-redaction).
|
||||
- `Referrer-Policy: no-referrer` was already set globally in `securityHeaders.js`, so the referer vector is closed too.
|
||||
|
||||
Reference in New Issue
Block a user