1036 lines
58 KiB
Markdown
1036 lines
58 KiB
Markdown
# padhle — Codebase Analysis & Feature Tracker
|
||
|
||
## File Map
|
||
|
||
| File | Purpose |
|
||
|---|---|
|
||
| `frontend/src/App.jsx` | Main orchestrator — owns all state, SSE chat logic, session CRUD, auth gating |
|
||
| `frontend/src/styles/global.css` | Global CSS variables (colors, fonts, spacing, radius, shadows) |
|
||
| `frontend/src/main.jsx` | React entry point — wraps App in `AuthProvider` |
|
||
| `frontend/src/lib/auth/SupabaseAuth.jsx` | Auth context — `signIn`, `signUp`, `signOut`, `getToken` (all via backend) |
|
||
| `frontend/src/lib/auth/backendAuth.js` | Backend auth client — `signup`, `signin`, `signout`, `me`, `check` |
|
||
| `frontend/src/lib/auth/SignIn.jsx` | Sign-in/Sign-up modal form |
|
||
| `frontend/src/lib/auth/SignIn.css` | Modal styles |
|
||
| `frontend/src/components/selector-flow/SelectorFlow.jsx` | Generic step-based selector — grade, subject, chapter (grid layout, conditional icons) |
|
||
| `frontend/src/components/selector-flow/SelectorFlow.css` | Grid layout for grade options |
|
||
| `frontend/src/components/top-nav/TopNav.jsx` | Header with grade/subject/chapter dropdown pills + search/notifications/user avatar |
|
||
| `frontend/src/components/top-nav/TopNav.css` | Pill buttons, dropdown menus, avatar, sign-out styles |
|
||
| `frontend/src/components/sidebar/Sidebar.jsx` | Brand, New Chat button, subjects list, chat history list, user info, sign-out, upgrade card |
|
||
| `frontend/src/components/sidebar/Sidebar.css` | Sidebar layout, hover indicators, chat item delete, upgrade card, user info |
|
||
| `frontend/src/components/chat-history/ChatHistory.jsx` | Welcome screen with bento suggestions OR message list OR selector flow |
|
||
| `frontend/src/components/chat-history/ChatHistory.css` | Welcome header, bento grid, info panel, message list |
|
||
| `frontend/src/components/message/Message.jsx` | Single message bubble (user or assistant), markdown+math rendering for assistant, action buttons, typing dots |
|
||
| `frontend/src/components/message/Message.css` | Bubble styling, avatar, actions, typing animation |
|
||
| `frontend/src/components/chat-input/ChatInput.jsx` | Textarea with auto-expand, mic, send button, attachment |
|
||
| `frontend/src/components/chat-input/ChatInput.css` | Input bar styling, send button states, disclaimer |
|
||
| `backend/src/index.js` | Express app setup, middleware, route registration |
|
||
| `backend/src/middleware/cookies.js` | Cookie parser middleware (cookie-parser) |
|
||
| `backend/src/middleware/supabaseAuth.js` | JWT verification middleware — reads from `padhle.token` httpOnly cookie |
|
||
| `backend/src/routes/auth.js` | POST /api/auth/signup, /api/auth/signin, /api/auth/signout; GET /api/auth/me |
|
||
| `backend/src/routes/chat.js` | POST /api/chat (stream), GET /api/chat/sessions, GET /api/chat/:id (load session+messages); validates metadata and enforces ownership |
|
||
| `backend/src/routes/sessions.js` | GET/POST/DELETE/PATCH /api/sessions; explicit ownership checks |
|
||
| `backend/src/services/db.js` | Supabase database service for profiles, sessions, and messages |
|
||
| `backend/src/routes/chatValidation.js` | Pure chat input validation and session ownership helpers |
|
||
| `backend/src/stores/sessionStore.js` | Legacy in-memory Map-based session + message store; no longer used by chat/session routes |
|
||
| `frontend/src/appState.js` | Central default/reset state for auth transitions |
|
||
| `backend/src/services/ai.js` | AI provider abstraction (OpenAI/Anthropic/Google), streaming, system prompt builder |
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
```
|
||
Frontend (React + Vite) ──SSE──▶ Backend (Express) ──API──▶ AI Provider
|
||
│ │
|
||
├─ State: activeGrade, └─ Supabase Postgres service
|
||
activeSubject, └─ Explicit req.user.uid ownership checks
|
||
activeChapter,
|
||
messages,
|
||
selectorStep (grade → subject → chapter → chat)
|
||
│
|
||
└─ Auth: httpOnly cookies (padhle.token, padhle.user)
|
||
└─ Browser API requests use credentials: 'include'
|
||
```
|
||
|
||
---
|
||
|
||
## Authentication — How It Works
|
||
|
||
All authentication is handled server-side via httpOnly cookies. The frontend never sees raw tokens.
|
||
|
||
### Flow
|
||
|
||
```
|
||
1. User enters email + password → Frontend POST /api/auth/signin
|
||
2. Backend calls Supabase Auth API → receives JWT
|
||
3. Backend sets httpOnly cookies:
|
||
- padhle.token = JWT (for API verification)
|
||
- padhle.user = JSON({ uid, email }) (safe user metadata)
|
||
4. Frontend never sees the JWT — browser stores cookies automatically
|
||
5. All API calls include cookies via credentials: 'include'
|
||
6. Backend middleware reads padhle.token, verifies via /auth/v1/user
|
||
7. If valid, req.user = { uid, email } is attached to request
|
||
```
|
||
|
||
### Auth Endpoints
|
||
|
||
| Route | Method | Purpose | Auth Required |
|
||
|---|---|---|---|
|
||
| `/api/auth/signup` | POST | Create account via Supabase Auth | No |
|
||
| `/api/auth/signin` | POST | Sign in via Supabase Auth | No |
|
||
| `/api/auth/signout` | POST | Revoke session + clear cookies | Yes |
|
||
| `/api/auth/me` | GET | Get current user from cookie | Yes |
|
||
|
||
### Protected Routes
|
||
|
||
| Route | Middleware | Purpose |
|
||
|---|---|---|
|
||
| `/api/chat` | supabaseAuth | Send message, load conversation |
|
||
| `/api/sessions` | supabaseAuth | CRUD chat sessions |
|
||
|
||
### Security
|
||
|
||
- **httpOnly cookies**: JWT is never accessible to JavaScript
|
||
- **sameSite: lax**: CSRF protection
|
||
- **secure flag**: Enabled in production (HTTPS)
|
||
- **JWT verification**: Each request calls Supabase Auth server to validate
|
||
- **No localStorage tokens**: Sensitive data never stored in browser
|
||
- **Session revocation**: Signout calls Supabase `/auth/v1/logout` server-side
|
||
|
||
---
|
||
|
||
## Grade Selector — Current State
|
||
|
||
### Where it lives
|
||
|
||
| Component | File | Role |
|
||
|---|---|---|
|
||
| `SelectorFlow` | `frontend/src/components/selector-flow/SelectorFlow.jsx` | Generic step-based selector — grade, subject, chapter (grid layout) |
|
||
| `TopNav` grade pill | `frontend/src/components/top-nav/TopNav.jsx` (lines ~46-75) | Always-visible header dropdown |
|
||
| State owner | `frontend/src/App.jsx` | `selectorStep` + `activeGrade/Subject/Chapter` state |
|
||
| `ChatHistory` | `frontend/src/components/chat-history/ChatHistory.jsx` | Renders SelectorFlow when `selectorStep !== "chat"` |
|
||
|
||
### Sequential selector flow
|
||
|
||
```
|
||
New Chat → selectorStep = "grade"
|
||
→ Pick Grade → selectorStep = "subject" → fade-in
|
||
→ Pick Subject → selectorStep = "chapter" → fade-in (subject-aware chapters)
|
||
→ Pick Chapter → selectorStep = "chat" → welcome screen / messages
|
||
```
|
||
|
||
### State values
|
||
|
||
| State | Type | Default | Stored in DB |
|
||
|---|---|---|---|
|
||
| `activeGrade` | label | `"Choose standard"` | `"Grade 10"` |
|
||
| `activeSubject` | ID | `"choose-subject"` | `"math"` |
|
||
| `activeChapter` | label | `"choose-chapter"` | `"Chapter 1: Rational Numbers"` |
|
||
| `selectorStep` | enum | `"grade"` | — |
|
||
|
||
| Value | Used in |
|
||
|---|---|
|
||
| `"Choose standard"` | Default for `activeGrade` |
|
||
| `Grade 6` .. `Grade 12` | SelectorFlow + TopNav grade options |
|
||
| `"math"` .. `"geography"` | `activeSubject` IDs (subjectItems) |
|
||
| `"choose-subject"` | Default/placeholder for `activeSubject` |
|
||
| `"choose-chapter"` | Default/placeholder for `activeChapter` |
|
||
|
||
### Data flow
|
||
|
||
```
|
||
User clicks grade in SelectorFlow
|
||
→ handleGradeSelect(item) → setActiveGrade(item.label) → pill shows label
|
||
|
||
User clicks subject in SelectorFlow
|
||
→ handleSubjectSelect(item) → setActiveSubject(item.id) → "math"
|
||
→ getSubjectLabel("math") → "Mathematics" (for pill/placeholder display)
|
||
→ handleChatSelect restores: data.session.subject = "math" → matches subject.id
|
||
|
||
User clicks subject in Sidebar
|
||
→ onSubjectChange(id) → setActiveSubject(id) → resets selectorStep to "subject"
|
||
|
||
User clicks subject in TopNav dropdown
|
||
→ selectOption("subject", subject.id) → onSubjectChange(id)
|
||
→ setActiveSubject(id) → subject-aware chapter dropdown updates
|
||
|
||
User clicks grade/chapter in TopNav
|
||
→ selectOption("grade", grade.label) / selectOption("chapter", chapter.label)
|
||
→ directly updates state
|
||
```
|
||
|
||
### Subject label resolution (central pattern)
|
||
|
||
```
|
||
activeSubject (stores ID) = "math"
|
||
→ getSubjectLabel("math") → "Mathematics"
|
||
→ TopNav pill: resolvedSubjectLabel
|
||
→ ChatInput placeholder: `${getSubjectLabel(activeSubject)}`
|
||
→ SelectorFlow subtitle: `Subject – ${getSubjectLabel(activeSubject)}`
|
||
→ TopNav dropdown: activeSubject === subject.id → highlights Mathematics
|
||
→ Sidebar: activeSubject === subject.id → highlights Mathematics
|
||
```
|
||
|
||
### Chapter label resolution
|
||
|
||
```
|
||
activeChapter (stores ID/label) = "choose-chapter" → getChapterLabel("choose-chapter") → "Choose Chapter"
|
||
activeChapter (stored label) = "Chapter 1: Rational Numbers" → getChapterLabel(...) → "Chapter 1: Rational Numbers"
|
||
→ TopNav pill: resolvedChapterLabel
|
||
→ ChatInput placeholder: `${getChapterLabel(activeChapter)}`
|
||
```
|
||
|
||
### TopNav chapter dropdown (subject-aware)
|
||
|
||
```
|
||
chapterData[activeSubject] → chapterOptions array
|
||
→ ["Choose Chapter", ...chaptersFromSubject]
|
||
→ Changes dynamically when activeSubject changes
|
||
→ Previously: static list mixing chapters from all subjects
|
||
```
|
||
|
||
### How grade is used
|
||
|
||
- Sent in `POST /api/chat` body → `grade` field (only when not default)
|
||
- Used in AI system prompt via `buildSystemPrompt()` → `"You are teaching students in grade X"`
|
||
- Displayed in chat input placeholder: `"Ask anything about {grade} {subject} – {chapter}..."`
|
||
- Restored when loading a session from `data.session.grade`
|
||
|
||
### SelectorFlow component
|
||
- Generic: accepts `title`, `subtitle`, `items`, `selectedValue`, `onSelect`, `step`, `compareField` props
|
||
- `compareField` determines which property to compare against `selectedValue` for active highlighting:
|
||
- `"label"` → `selectedValue === item.label` (used for grades and chapters)
|
||
- `"id"` → `selectedValue === item.id` (used for subjects)
|
||
- Renders any item type via 3-column grid with fade-in animation
|
||
- Conditional icon rendering: `item.icon && (<span>icon</span>)`
|
||
- Fade-in animation on step change via `key={step}` — CSS `@keyframes selectorFadeIn` with `opacity` + `translateY(16px→0)` over 0.35s
|
||
|
||
### Selector data
|
||
- **Grades**: 7 options (Grade 6–12), each with icon (Material Symbols)
|
||
- **Subjects**: 6 options (Math, Science, History, Language, English, Geography) — label only, no icons
|
||
- **Chapters**: 6 per subject, subject-aware via `chapterData` object in App.jsx — label only, no icons
|
||
- **Chapter mapping**: `chapterData[subjectId]` → array of chapters (e.g. math → Rational Numbers, Linear Equations, etc.)
|
||
|
||
### Current features ✅
|
||
|
||
- Sequential selector flow: grade → subject → chapter → chat (fade-in transitions)
|
||
- Grade selection via SelectorFlow (new-chat grid)
|
||
- Grade selection via TopNav (header pill dropdown)
|
||
- Subject selection in SelectorFlow (label-only, no icons)
|
||
- Chapter selection in SelectorFlow (subject-aware, label-only, no icons)
|
||
- Grade persists across session restore
|
||
- Grade sent to AI backend for context-aware responses
|
||
- 7 grades available (Grade 6–12)
|
||
- Upgrade-to-Pro card hidden via `showUpgrade={false}` prop
|
||
|
||
---
|
||
|
||
## Feature Tracker
|
||
|
||
### Grade Selector
|
||
- **Status**: Working ✅
|
||
- **Verified**: Yes — hover effect confirmed
|
||
- **Last updated**: 2026-01-18
|
||
|
||
### Hover effect (✅ verified)
|
||
- Subtle lift: `translateY(-2px)`
|
||
- Box-shadow pop via `--shadow-card-hover`
|
||
- Smooth transition on transform and shadow only
|
||
- Press-down feel on `:active` (`translateY(0)`)
|
||
- File: `frontend/src/components/selector-flow/SelectorFlow.css`
|
||
|
||
### Sequential selector flow (✅ verified)
|
||
- Steps: grade → subject → chapter → chat (via `selectorStep` state)
|
||
- Fade-in animation on each step change: `opacity 0→1` + `translateY(16px→0)` over 0.35s
|
||
- Triggered by `key={step}` on `.selector-flow` — forces React re-mount
|
||
- Subject-aware chapters: `chapterData[subjectId]` returns relevant chapters
|
||
- File: `frontend/src/App.jsx` (selector data + step logic), `frontend/src/components/chat-history/ChatHistory.jsx` (conditional rendering)
|
||
|
||
### Subject ID/label resolution pattern (✅ verified in browser)
|
||
- `activeSubject` now stores **IDs** (e.g. `"math"`, `"science"`) instead of labels
|
||
- Default is `"choose-subject"` (a reserved placeholder ID)
|
||
- `getSubjectLabel(id)` resolves IDs to display names: `"math"` → `"Mathematics"`
|
||
- Used in: TopNav pill text, ChatInput placeholder, SelectorFlow subtitle, `handleSend` body
|
||
- `handleSubjectSelect` stores `item.id` (not `item.label`)
|
||
- `handleChatSelect` restoration: backend returns `subject: "math"` which matches `subject.id`
|
||
- **TopNav**: uses `subjectItems` prop; `activeSubject === subject.id` comparison works; pill uses `resolvedSubjectLabel`
|
||
- **TopNav chapter dropdown**: uses `chapterData` prop; generates `chapterOptions[activeSubject]` dynamically (subject-aware)
|
||
- **Sidebar**: `activeSubject === subject.id` comparison now works correctly; sidebar subject change resets selector to `subject` step
|
||
- **SelectorFlow**: `compareField="id"` for subjects, `compareField="label"` for grades/chapters
|
||
- **ChatInput placeholder**: uses `getSubjectLabel(activeSubject)` and `getChapterLabel(activeChapter)` for full label resolution
|
||
- Files: `App.jsx`, `TopNav.jsx`, `Sidebar.jsx`, `SelectorFlow.jsx`, `ChatInput.jsx`
|
||
|
||
### Chapter pill resolution (✅ verified in browser)
|
||
- `activeChapter` stores chapter labels (e.g. `"Chapter 1: Rational Numbers"`) after selection
|
||
- Default is `"choose-chapter"` (placeholder ID)
|
||
- `getChapterLabel(chapterId)` resolves placeholder: `"choose-chapter"` → `"Choose Chapter"`
|
||
- Used in: TopNav pill, ChatInput placeholder
|
||
- Files: `App.jsx`, `TopNav.jsx`
|
||
|
||
### TopNav subject-aware chapter dropdown (✅ verified in browser)
|
||
- Replaced static 6-item chapters array in TopNav with dynamic `chapterOptions` generated from `chapterData[activeSubject]`
|
||
- Shows subject-specific chapters based on current selection:
|
||
- Math → Rational Numbers, Linear Equations, Coordinate Geometry, Algebra, Geometry, Trigonometry
|
||
- Science → Nutrition in Plants, Photosynthesis, Human Physiology, etc.
|
||
- (all 6 subjects × 6 chapters)
|
||
- Dropdown placeholder highlights correctly when `activeChapter` is `"choose-chapter"`
|
||
- Files: `TopNav.jsx` (chapterOptions generation, chapterData prop), `App.jsx` (chapterData pass-through)
|
||
|
||
### Upgrade-to-Pro card hidden (✅ verified)
|
||
- `Sidebar` component accepts `showUpgrade` prop (default `true`)
|
||
- `App.jsx` passes `showUpgrade={false}`
|
||
- Card is wrapped in `{showUpgrade && (...)}` conditional
|
||
- Can be re-enabled by setting `showUpgrade={true}`
|
||
- File: `frontend/src/components/sidebar/Sidebar.jsx`, `frontend/src/App.jsx`
|
||
|
||
---
|
||
|
||
## Supabase
|
||
|
||
### Installation
|
||
|
||
Supabase CLI (local dev via `npx supabase start`).
|
||
- **Project path**: `supabase/` (CLI-managed)
|
||
- **Start**: `npx supabase start`
|
||
- **Stop**: `npx supabase stop`
|
||
- **Status**: `npx supabase status`
|
||
- **Services**: Postgres, GoTrue (Auth), PostgREST (REST), Storage, Supavisor (pooler), Realtime, Edge Runtime, Studio (dashboard), Kong (API gateway), Mailpit (email testing), Analytics (Logflare), Vector (logging)
|
||
|
||
### Dashboard (Studio)
|
||
|
||
| Setting | Value |
|
||
|---|---|
|
||
| **URL** | `http://localhost:54323` |
|
||
| **Mailpit** | `http://localhost:54324` |
|
||
|
||
### API Keys (local dev)
|
||
|
||
| Key | Value |
|
||
|---|---|
|
||
| **Publishable Key** (client-side, public by design) | `sb_publishable_ACJWlzQHlZjBrEguHvfOxg_3BJgxAaH` |
|
||
| **Secret Key** (server-side) | `<rotated-local-dev-service-role-key>` — **rotated 2026-08-19; never commit the real value** |
|
||
|
||
> **Never expose the Secret Key in client code.** Use the publishable key in frontend, secret key in backend.
|
||
> Local dev uses shared default keys — **do not use in production**.
|
||
> **Security note:** the previous `sb_secret_...` value was committed to git in this doc, `IMPLEMENTATION_PLAN.md`, and `backend/scripts/setup-db.js`. All three were scrubbed; rotate the local keys (`npx supabase stop && npx supabase start` or Studio) and update only `backend/.env`.
|
||
|
||
### Database
|
||
|
||
| Setting | Value |
|
||
|---|---|
|
||
| **Host** | `localhost` |
|
||
| **Port** | `54322` |
|
||
| **Database** | `postgres` |
|
||
| **User** | `postgres` |
|
||
| **Password** | `postgres` |
|
||
|
||
**Connection string:**
|
||
```bash
|
||
psql 'postgres://postgres:postgres@localhost:54322/postgres'
|
||
```
|
||
|
||
### API Endpoints (via Kong gateway)
|
||
|
||
| Service | Endpoint |
|
||
|---|---|
|
||
| **REST** | `http://localhost:54321/rest/v1/` |
|
||
| **Auth** | `http://localhost:54321/auth/v1/` |
|
||
| **Storage** | `http://localhost:54321/storage/v1/` |
|
||
| **Realtime** | `http://localhost:54321/realtime/v1/` |
|
||
| **GraphQL** | `http://localhost:54321/graphql/v1` |
|
||
| **Edge Functions** | `http://localhost:54321/functions/v1/` |
|
||
| **MCP** | `http://localhost:54321/mcp` |
|
||
|
||
### Supabase MCP
|
||
|
||
- **Endpoint**: `http://localhost:54321/mcp`
|
||
- **Mode**: Read-only (local dev)
|
||
- **Auth**: Local (no cloud OAuth)
|
||
- **Tools**: 10 (database, debugging, development, docs)
|
||
- **Config**: `~/.pi/agent/supabase.json` (callback port 54326 — default 54324 conflicts with Mailpit)
|
||
- **CLI**: `pi install npm:pi-supabase` → then `supabase_mcp_connect()`
|
||
|
||
### Database State
|
||
|
||
- **Custom tables**: `profiles`, `sessions`, and `messages` exist in the connected local Supabase database
|
||
- **Auth tables**: Supabase-managed (`auth.users`, `auth.sessions`, `auth.refresh_tokens`, etc.)
|
||
- **RLS**: Enabled on custom tables and auth tables
|
||
- **Permissions (FIXED)**: API roles lacked table grants, causing PostgreSQL `42501 permission denied`. Fixed via grant to `anon`/`authenticated`/`service_role`; tracked in `supabase/migrations/20260819120000_grant_api_role_table_access.sql`, which also sets `ALTER DEFAULT PRIVILEGES` so future tables inherit the grants.
|
||
- **Profiles (control)**: `profiles` has `role` (default `user`) and `is_banned` (default `false`) columns. A trigger on `auth.users` auto-creates a profile row on any user creation (signup or admin). `authenticated` may update only `display_name`/`avatar_url` — it cannot change `role`/`is_banned`; `service_role` has full control. Migration: `supabase/migrations/20260819130000_profiles_control_and_trigger.sql`.
|
||
- **Signups**: enabled (email verification not yet wired to the app).
|
||
- **User data**: all previous test accounts were deleted (2026-08-19). `auth.users`/`profiles` are empty; no admin user yet. Admin user is added manually in Studio (Auth → Add user, email_confirm on) and then promoted to `role='admin'`.
|
||
- **Verified**: self-signup and admin-created users both auto-create a `profiles` row with `role:'user'`; the role is_banned self-edit is blocked at the column-privilege level.
|
||
|
||
### Environment Files
|
||
|
||
- `supabase/config.toml` — CLI project config
|
||
- `supabase/.temp/` — CLI-generated files (do not commit)
|
||
|
||
---
|
||
|
||
## Global Design Tokens (global.css)
|
||
|
||
| Token | Value |
|
||
|---|---|
|
||
| `--color-primary` | `#000000` |
|
||
| `--color-on-primary` | `#ffffff` |
|
||
| `--color-background` | `#fcf8fa` |
|
||
| `--color-surface-lowest` | `#ffffff` |
|
||
| `--color-on-surface` | `#111827` |
|
||
| `--color-on-surface-variant` | `#6b7280` |
|
||
| `--font-heading` | `"Montserrat", sans-serif` |
|
||
| `--font-body` | `"Inter", system-ui, sans-serif` |
|
||
| `--sidebar-width` | `280px` |
|
||
| `--container-max` | `800px` |
|
||
|
||
---
|
||
|
||
## Recent Implementation
|
||
|
||
### Backend Authentication (completed 2026-08-18)
|
||
|
||
The sign-in/sign-out flow was moved from the frontend (direct Supabase Auth calls) to the backend (cookie-based proxy).
|
||
|
||
#### What was changed
|
||
|
||
| File | Change |
|
||
|---|---|
|
||
| `backend/src/routes/auth.js` | New — `POST /api/auth/signup`, `/signin`, `/signout`; `GET /api/auth/me` |
|
||
| `backend/src/middleware/supabaseAuth.js` | Updated — reads JWT from `padhle.token` httpOnly cookie instead of `Authorization` header |
|
||
| `backend/src/index.js` | Updated — added `cookie-parser` middleware; applied `supabaseAuth` to `/api/chat` and `/api/sessions` |
|
||
| `backend/src/stores/sessionStore.js` | Updated — stores `userId` per session; `listSessions(userId)` filters by user |
|
||
| `frontend/src/lib/auth/backendAuth.js` | New — frontend auth client (`signup`, `signin`, `signout`, `me`, `check`) |
|
||
| `frontend/src/lib/auth/SupabaseAuth.jsx` | Updated — removed all `supabase.auth.*` calls; now uses backend client |
|
||
|
||
#### How it works
|
||
|
||
```
|
||
1. User enters email + password → Frontend POST /api/auth/signin
|
||
2. Backend calls Supabase Auth API → receives JWT
|
||
3. Backend sets httpOnly cookies:
|
||
- padhle.token = JWT (for API verification)
|
||
- padhle.user = JSON({ uid, email }) (safe user metadata)
|
||
4. Frontend never sees the JWT — browser stores cookies automatically
|
||
5. All API calls include cookies via `credentials: 'include'`
|
||
6. Backend middleware reads `padhle.token`, verifies via `/auth/v1/user`
|
||
7. If valid, `req.user = { uid, email }` is attached to request
|
||
```
|
||
|
||
#### Verification
|
||
- ✅ Build passes (Vite 6.4.3)
|
||
- ✅ Sign-in → user authenticated, sees main app
|
||
- ✅ Sign-out → cookies cleared, back to sign-in modal
|
||
- ✅ Protected routes (`/api/sessions`) properly require auth (401 on missing cookie)
|
||
- ✅ `padhle.token` cookie confirmed as `HttpOnly` in browser DevTools
|
||
- ✅ No `localStorage` tokens used anywhere in frontend
|
||
|
||
---
|
||
|
||
## Security Audit — Vulnerabilities & Fixes
|
||
|
||
An audit was performed on all backend files on 2026-08-18.
|
||
|
||
### 🔴 Critical — Fixed
|
||
|
||
#### CVE-2026-011: Cross-user identity leak via truncated token cache key
|
||
**File:** `backend/src/middleware/supabaseAuth.js`
|
||
|
||
**Problem:** `verifyToken` cached verified users keyed by `token.substring(0, 50)`. Every JWT from the same GoTrue instance shares the header (including `kid`) and the opening claims, so users A and B produced **identical 50-char fingerprints**. B's request then hit A's cached entry and `req.user.uid` resolved to A — so B's `/api/sessions` returned A's sessions/history. Reproduced end-to-end: two fresh users, A created a session, B listed A's session.
|
||
|
||
**Fix:** Key the cache by the **full token**, which is unique per user:
|
||
```js
|
||
const cached = tokenCache.get(token); // full token
|
||
...
|
||
tokenCache.set(token, { ...result, expires }); // full token
|
||
```
|
||
Removed the `FINGERPRINT_LEN` constant. `listSessions`/RLS filtering was already correct; the leak was purely identity resolution.
|
||
|
||
**Verified:** On fixed code, A has 1 session, B has 0; no cross-user leak. (Note: the running backend on :3001 must be restarted to load the fix — it runs `node src/index.js` without `--watch`.)
|
||
|
||
|
||
#### CVE-2026-001: IDOR on `GET /api/chat/:chatId`
|
||
**File:** `backend/src/routes/chat.js`
|
||
|
||
**Problem:** The route checks `supabaseAuth` middleware (so user is authenticated), but never verifies the session belongs to `req.user.uid`. Any authenticated user can read another user's conversation history by guessing a `chatId`.
|
||
|
||
**Fix:** Add ownership check before returning session data:
|
||
```js
|
||
router.get("/:chatId", (req, res) => {
|
||
const session = getSession(req.params.chatId);
|
||
if (!session || session.userId !== req.user.uid) {
|
||
return res.status(404).json({ error: "Session not found" });
|
||
}
|
||
// ... rest of route
|
||
});
|
||
```
|
||
Also apply to `GET /api/chat/sessions` to filter by `req.user.uid`.
|
||
|
||
---
|
||
|
||
#### CVE-2026-002: No rate limiting on auth endpoints
|
||
**File:** `backend/src/routes/auth.js`
|
||
|
||
**Problem:** Zero throttling on signup/signin/signout. Allows unlimited password brute-force, account enumeration via signup attempts, and session flooding.
|
||
|
||
**Fix:** Apply `express-rate-limit` to auth routes:
|
||
```js
|
||
import rateLimit from 'express-rate-limit';
|
||
const authLimiter = rateLimit({ windowMs: 15 * 60 * 1000, max: 20 });
|
||
router.post('/signup', authLimiter, ...);
|
||
router.post('/signin', authLimiter, ...);
|
||
router.post('/signout', authLimiter, ...);
|
||
```
|
||
|
||
---
|
||
|
||
#### CVE-2026-003: SSE stream injection — untrusted data in `data: ` prefix
|
||
**File:** `backend/src/routes/chat.js`
|
||
|
||
**Problem:** User messages (AI responses built from user input) are written raw into SSE events:
|
||
```js
|
||
res.write(`data: ${JSON.stringify({ type: "chunk", content: chunk })}\n\n`);
|
||
```
|
||
If `chunk` contains `}\n\n` or other control characters, it breaks the SSE stream and can inject fake events.
|
||
|
||
**Fix:** Sanitize the JSON output before writing:
|
||
```js
|
||
const safe = JSON.stringify({ type: "chunk", content: chunk })
|
||
.replace(/\n/g, '\\n')
|
||
.replace(/\u2028/g, '\\u2028')
|
||
.replace(/\u2029/g, '\\u2029');
|
||
res.write(`data: ${safe}\n\n`);
|
||
```
|
||
|
||
---
|
||
|
||
#### CVE-2026-004: Google Gemini API URL path break
|
||
**File:** `backend/src/services/ai.js`
|
||
|
||
**Problem:** API key concatenated into URL path — if the key contains `/` or `..`, the URL breaks:
|
||
```js
|
||
const url = `${GOOGLE_BASE_URL}${apiKey}/chat/models/...`;
|
||
// = https://generativelanguage.googleapis.com/v1beta/openai/{key}/chat/... ← wrong
|
||
```
|
||
|
||
**Fix:** Use `URL` constructor to build safely:
|
||
```js
|
||
const url = new URL(`chat/models/${model}:streamGenerateContent`, GOOGLE_BASE_URL);
|
||
url.searchParams.set('key', apiKey);
|
||
```
|
||
|
||
---
|
||
|
||
### 🟡 High — Fixes Applied
|
||
|
||
#### CVE-2026-005: Supabase error details leaked to client
|
||
**File:** `backend/src/routes/auth.js`
|
||
|
||
**Problem:** `data.error_description` from Supabase returns detailed error text (e.g., "Email already registered") that reveals system behavior.
|
||
|
||
**Fix:** Return generic messages from the backend; log raw errors server-side only:
|
||
```js
|
||
// Log the real error
|
||
console.error(`Auth error: ${data.error_description}`);
|
||
// Return generic message
|
||
return res.status(400).json({ error: "Sign up failed" });
|
||
```
|
||
|
||
---
|
||
|
||
#### CVE-2026-006: No timeout on Supabase JWT verification HTTP calls
|
||
**File:** `backend/src/middleware/supabaseAuth.js`
|
||
|
||
**Problem:** Every request to a protected route makes an HTTP call to `Supabase /auth/v1/user` with no timeout. If Supabase is slow or unreachable, the Express server hangs indefinitely.
|
||
|
||
**Fix:** Add `AbortSignal` timeout:
|
||
```js
|
||
const controller = new AbortController();
|
||
const timeout = setTimeout(() => controller.abort(), 3000);
|
||
const res = await fetch(url, { headers, signal: controller.signal });
|
||
clearTimeout(timeout);
|
||
```
|
||
|
||
---
|
||
|
||
#### CVE-2026-007: No token caching — every request hits Supabase
|
||
**File:** `backend/src/middleware/supabaseAuth.js`
|
||
|
||
**Problem:** Every single request makes an HTTP call to verify the token. A token is valid for 3600s; it should be cached.
|
||
|
||
**Fix:** Cache verified tokens in a `Map` with TTL:
|
||
```js
|
||
const tokenCache = new Map(); // tokenFingerprint -> { uid, email, expires }
|
||
const FINGERPRINT_TTL = 5 * 60 * 1000;
|
||
const TOKEN_EXPIRY_TTL = 3600 * 1000;
|
||
|
||
export async function verifyToken(token) {
|
||
const fp = token.substring(0, 50);
|
||
const cached = tokenCache.get(fp);
|
||
if (cached && Date.now() < cached.expires) {
|
||
return cached;
|
||
}
|
||
// ... verify via Supabase, then:
|
||
tokenCache.set(fp, { uid, email, expires: Date.now() + FINGERPRINT_TTL });
|
||
return { uid, email };
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
### 🟢 Medium — Fixes Applied
|
||
|
||
#### CVE-2026-008: No input length limits on chat messages
|
||
**File:** `backend/src/routes/chat.js`
|
||
|
||
**Problem:** No max on `text` length. Can fill the in-memory store or blow up the AI context window.
|
||
|
||
**Fix:** Add length check:
|
||
```js
|
||
if (!text || !text.trim()) return 400;
|
||
if (text.length > 10000) return 400;
|
||
```
|
||
|
||
---
|
||
|
||
#### CVE-2026-009: No XSS sanitization on stored messages
|
||
**File:** `backend/src/stores/sessionStore.js`
|
||
|
||
**Problem:** User messages stored and returned raw. If the frontend renders them as HTML, user input containing `<script>` tags is a stored XSS vector.
|
||
|
||
**Fix:** Backend should at least flag or truncate dangerous content:
|
||
```js
|
||
const XSS_PATTERN = /[<>"'&]/;
|
||
if (XSS_PATTERN.test(message.text)) {
|
||
message.text = message.text.replace(/[<>]/g, (c) => c === '<' ? '<' : '>');
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
#### CVE-2026-0010: AI keys exposed in process environment
|
||
**File:** `backend/src/services/ai.js`
|
||
|
||
**Problem:** API keys read from `process.env` and used in `fetch()` calls. On some systems they appear in `ps aux`.
|
||
|
||
**Fix:** Not a code fix — ensure `.env` is in `.gitignore`, use secret managers in production, and avoid `console.log` of config values.
|
||
|
||
---
|
||
|
||
## Current Implementation Status — 2026-08-19
|
||
|
||
### Completed and verified
|
||
|
||
- Backend cookie-based authentication remains implemented; frontend never reads JWTs.
|
||
- Auth, chat, and session rate limiters are installed and mounted.
|
||
- Auth rate-limit boundary verified: requests 1–5 returned validation responses; request 6 returned HTTP 429.
|
||
- Chat validation rejects empty text, text over 10,000 characters, invalid grades, and invalid subjects.
|
||
- Chat/session ownership checks are enforced against `req.user.uid`.
|
||
- `GET /api/chat/sessions` is registered before `GET /api/chat/:chatId`, preventing `sessions` from being interpreted as a chat ID.
|
||
- Frontend auth transitions clear user-scoped conversation state and abort in-flight requests.
|
||
- Frontend API requests include `credentials: "include"` for httpOnly cookies.
|
||
- Added regression tests: `backend/test/chatValidation.test.js` and `frontend/test/appState.test.js`.
|
||
- Backend tests, backend syntax checks, frontend regression test, and Vite production build pass.
|
||
- Chrome MCP verified the sign-in screen renders after fixing a missing `useAuth` import exposed by the browser console.
|
||
- Cross-user session leak fixed (token cache keyed by full token — see CVE-2026-011).
|
||
- AI connected via OpenRouter; streaming verified in the browser with both user+assistant messages persisted.
|
||
- Assistant replies render as markdown + KaTeX math; verified in the headed browser.
|
||
- Current user message is included in the AI context (persisted before streaming).
|
||
- Session continuity fixed: backend emits `session` id in SSE; follow-ups reuse the same session.
|
||
- Rolling conversation summary implemented and persisted (bounded context + long-range memory).
|
||
|
||
### Database status
|
||
|
||
- Supabase API is reachable at `http://127.0.0.1:54321`.
|
||
- The `profiles`, `sessions`, and `messages` tables exist.
|
||
- Table grants were missing, causing `42501 permission denied`. **Fixed and verified** (see Database State above); migration tracked in `supabase/migrations/20260819120000_grant_api_role_table_access.sql`.
|
||
|
||
---
|
||
|
||
## Data Storage & Conversation Context Model (2026-08-19)
|
||
|
||
### Q: Each message is a new row — is that sustainable?
|
||
|
||
**A (yes):** Each message is one row in `public.messages` (`id`, `session_id`, `role`, `text`, `created_at`), written append-only. This is the standard chat design (Slack/OpenAI-style). It is sustainable because:
|
||
- Storage is tiny (average turn is ~KB scale; a million messages ≈ a few GB).
|
||
- `idx_messages_session_id` keeps per-session history reads fast even with many rows.
|
||
- Append-only is desirable: simple pagination/partial loads, immutable history, easy to add edit/regenerate later.
|
||
- One-row-per-session-with-big-text-blob would be worse (can't load partial history, bad indexing).
|
||
- DB only needs attention at scale: partitioning by date/user past ~10s of millions of rows, or archiving old sessions.
|
||
|
||
### The real scaling concern: AI context window (the "memory" issue)
|
||
|
||
- The DB is fine, but on **every** `/api/chat` call the backend loads the **entire** message history and sends it **all** to the model.
|
||
- Token count (and therefore **cost and latency**) grows with conversation length; eventually it exceeds the model's context limit and the call fails.
|
||
- **This is the lever worth engineering** — not the database.
|
||
|
||
### Blocked-on note
|
||
|
||
Before context work has value, the **frontend session-continuity bug** must be fixed (see What's Next #1): follow-ups in a fresh chat currently create a new session with no history, so long-running context never even accumulates.
|
||
|
||
---
|
||
|
||
## What's Next
|
||
|
||
### 1. Frontend session continuity — DONE
|
||
- Backend now emits a `{ type: "session", id }` SSE event when a conversation is created/loaded; the frontend sets `activeChat` on it, so follow-ups reuse the same session and history accumulates.
|
||
- Remaining minor: sidebar refresh (`loadSessions()`) only runs on the successful SSE path — an AI error still leaves the sidebar stale even though the session row was created.
|
||
|
||
### 1b. Rolling conversation summary (B+D) — implemented 2026-08-19
|
||
- `sessions.summary` column (migration `20260819150000_add_sessions_summary.sql`).
|
||
- On each `/api/chat`, when a session exceeds `RECENT_LIMIT` (10) messages, older messages are folded into a persisted summary via a non-streaming model call (`summarizeConversation`), stored with `setSummary`.
|
||
- The model receives `system(+summary) + last 10 messages` (`buildMessages`), so context stays bounded while long-range context persists.
|
||
- Verified end-to-end: fresh chat emits session id; a 12-message session folded its overflow into a stored summary; follow-up answered with bounded context.
|
||
- Tests: `backend/test/aiMessages.test.js` (capping + summary injection).
|
||
|
||
### 2. Frontend rendering — fixed
|
||
- Assistant replies now render via `react-markdown` (headings, bold, italics, lists, blockquotes, code, tables, `---`) and math (`$H_2O$`, `$CO_2$`) via KaTeX. User messages stayed plain text. `ReactMarkdown` escapes raw HTML by default, so stored-XSS (CVE-2026-009) stays closed while rendering rich content.
|
||
- Protected-route 401s still return to the sign-in modal.
|
||
- Selector state persists per user.
|
||
|
||
### AI Provider — OpenRouter (connected 2026-08-19)
|
||
|
||
- Backend uses the OpenAI-compatible path (`AI_PROVIDER=openai`) pointed at OpenRouter.
|
||
- `OPENAI_BASE_URL=https://openrouter.ai/api/v1` (added to `ai.js` as an optional override).
|
||
- Model: `qwen/qwen3.7-flash` (final choice; earlier tried `~deepseek/deepseek-v4-flash-latest` and `upstage/solar-pro4`).
|
||
- OpenRouter "latest" aliases use a required `~` prefix (e.g. `~deepseek/...`); the non-tilde `-latest` is an invalid ID.
|
||
- Reasoning disabled for OpenRouter (`reasoning: { enabled: false }` in `streamOpenai`/`getChatResponse`) so `delta.content` streams immediately instead of sitting empty during the thinking phase (which made the chat look blank).
|
||
- Verified end-to-end: `/api/chat` streams a real reply and persists both `user` and `assistant` messages.
|
||
- The user message is now persisted **before** streaming, so the model sees the current question (fixes canned-greeting responses) and the user message survives AI failures.
|
||
|
||
**Env (non-secret):** wire via `backend/.env`, and the placeholders are in `backend/.env.example`.
|
||
|
||
**Secrets:** `backend/.env` contains the real `OPENAI_API_KEY` (OpenRouter) plus `SUPABASE_SERVICE_ROLE_KEY`. Both are git-ignored; never commit. Rotate if exposed.
|
||
|
||
### Operational hardening
|
||
- Ensure `.env` secrets are never committed or exposed.
|
||
- Rotate any secret key that has been shared outside the local environment.
|
||
- Add monitoring for rate-limit hits and database errors.
|
||
|
||
---
|
||
|
||
## Textbook Grounding — Future Plan (touchpoint)
|
||
|
||
> Direction agreed 2026-08-23. Endgoal: selected grade → subject → chapter resolves to actual textbook markdown content, and the model answers **only from that supplied content**. Conversation memory stays separate from textbook evidence (the model-authored rolling summary is never a source of factual truth). No code shipped yet — this is the agreed design and the V-order touchpoint for ANALYSIS.md.
|
||
|
||
### The seam (core invariant)
|
||
|
||
```
|
||
SYSTEM — base tutor identity (grade/subject/chapter already validated server-side)
|
||
MEMORY — rolling conversation summary; continuity ONLY, never a fact source
|
||
TEXTBOOK EVIDENCE — retrieved markdown sections; the ONLY factual authority
|
||
RECENT CONVERSATION — last 10 messages
|
||
CURRENT QUESTION — the student's message
|
||
```
|
||
|
||
Model rule: answer exclusively from the supplied textbook sections; if they don't cover the question, say so. The summary guides continuity but can never override textbook retrieval.
|
||
|
||
### Content model
|
||
|
||
- One markdown file per chapter, plain text read from disk by the backend — **the model never receives a filesystem path**; the backend resolves the selection, reads the file, and injects the parsed content into the prompt (this is where `ai.js` already sits).
|
||
- Filesystem layout (grade = directory layer, subject = namespace, chapter = file granularity):
|
||
```
|
||
content/
|
||
└── grade-10/
|
||
└── science/
|
||
├── 01-chemical-reactions.md
|
||
└── 02-acids-bases-salts.md
|
||
```
|
||
- **No build-time content indexing needed for V1** — files are authored markdown, not derived.
|
||
|
||
### Registry (decision: UI identity ≠ filename ≠ display label)
|
||
|
||
Do **not** couple application IDs to filesystem paths. A small hand-maintained registry maps trusted `{gradeId, subjectId, chapterId}` → `{ file, title }`:
|
||
|
||
```js
|
||
// backend/src/services/content/textbookRegistry.js
|
||
export const TEXTBOOK_REGISTRY = {
|
||
"10": {
|
||
"science": {
|
||
"chemical-reactions": {
|
||
title: "Chemical Reactions & Equations",
|
||
file: "content/grade-10/science/01-chemical-reactions.md",
|
||
},
|
||
},
|
||
},
|
||
};
|
||
export const getChapter = (grade, subject, chapterId) =>
|
||
TEXTBOOK_REGISTRY[grade]?.[subject]?.[chapterId] ?? null;
|
||
```
|
||
|
||
- Lookup is registry-only; the resolved `file` path is what touches disk — never raw user values (path traversal becomes a non-issue by construction, not by relying on input validation).
|
||
- Renames (`chemical-reactions.md` → `chapter-01-…`) don't break sessions or frontend ids.
|
||
- Unknown `{grade, subject, chapter}` → hard 500 (leak nothing), **never** fall back to generic Qwen.
|
||
|
||
### Retrieval mode — RECONCILED 2026-08-24 (section-level retrieval is the default)
|
||
|
||
> **Supersedes** the earlier staged framing (V1 whole-chapter / V2 section-retrieval-"only if needed"). Real scanned textbooks (30k-line chapters, e.g. Rajeev Bansal Class XII Economics at an estimated 60–90k tokens/chapter — estimate pending measurement) proved chapters are too large to inject whole per request. Section-level retrieval is therefore the **default** path. Whole-chapter injection is kept only as a corner case for genuinely small chapters (< a few k tokens).
|
||
|
||
```
|
||
Grade 10 → Science → Chemical Reactions
|
||
→ registry resolves chapter (never raw user values → no traversal)
|
||
→ read PRE-NORMALIZED section file (built offline; runtime never parses raw OCR)
|
||
→ retriever keyword-scores question vs section text → top-N sections
|
||
→ top-N injected as TEXTBOOK EVIDENCE
|
||
→ model answers from them only; refuses if uncovered (strict)
|
||
```
|
||
|
||
- Parser + retriever share one artifact: `markdownParser` (`##`-split) runs at **offline ingestion** only, never per-request; `retriever` scores the pre-split sections at request time.
|
||
- Keep boring: keyword/regex section scoring against the raw query — no embeddings / pgvector / vector DB / relevance-threshold tuning.
|
||
- Strict grounding confirmed: answer exclusively from supplied sources; say so when the content doesn't cover the question.
|
||
|
||
### Explicitly deferred / not in scope now
|
||
|
||
- `textbooks` / `chapters` / `textbook_sections` DB tables (filesystem + registry is enough for the first books; revisit only at content scale).
|
||
- Fine-tuning a small model to replace Qwen (answer from grounded retrieval first; fine-tune only if quality demands it later).
|
||
- Separating summary into a distinct persisted memory store — keep the current `sessions.summary` seam honed, just never let it inject textbook-fact authority.
|
||
|
||
---
|
||
|
||
## Next Conversation — Starting Point (2026-08-24)
|
||
|
||
> This is the handoff anchor. If the conversation context is lost, resume here. Everything below is **decided and agreed** — not speculative. No code shipped yet; plan mode.
|
||
|
||
### Problem statement
|
||
|
||
Padhle's grade → subject → chapter selector currently drives **prompt context only** — the model has no access to actual textbook content. A student selecting "Grade 10 → Science → Chemical Reactions" gets a generic Qwen answer, not an answer grounded in the selected textbook. The endgoal: make that selection resolve to the exact textbook sections the model may answer from, and **nothing else**. The model must answer exclusively from the supplied content and say so when it can't. Conversation memory (rolling summary) must never become a source of factual truth — textbook evidence and conversation memory are separate trust domains.
|
||
|
||
### Detailed solution
|
||
|
||
```
|
||
STUDENT selects Grade 12 → Economics → Price Elasticity of Demand
|
||
↓
|
||
BACKEND resolves {gradeId, subjectId, chapterId} via textbookRegistry
|
||
↓ (never raw user values → path traversal impossible by construction)
|
||
BACKEND reads the chapter's PRE-NORMALIZED section file (built offline)
|
||
↓
|
||
RETRIEVER keyword-scores the question against section text, returns top-N
|
||
↓
|
||
AI receives: SYSTEM(base) + MEMORY(summary) + TEXTBOOK EVIDENCE(sections) + RECENT + QUESTION
|
||
↓
|
||
Qwen answers ONLY from the supplied sections; refuses if uncovered
|
||
```
|
||
|
||
Offline ingestion / online application are **separate systems**: messy OCR/MinerU cleanup happens once per book (script proposes boundaries → you review), never inside a student request.
|
||
|
||
### Decisions locked (in order)
|
||
|
||
1. **Source format: markdown (md), not the scanner's JSON.** md is ~⅓ the tokens (JSON carries `bbox`/`page_idx`/per-page header+footer noise), renders native `##` / `$…$` / `$$…$$`, and matches the existing KaTeX rendering. JSON's only edge is `text_level`/`type` for offline section disambiguation.
|
||
2. **Registry, not filesystem-as-model.** Stable keys `{grade,subject,chapter}` → `{file,title}`; UI identity ≠ filename ≠ display label; renames don't break sessions. Unknown selection → hard 500, never silent generic fallback.
|
||
3. **Section-level retrieval is the default** (not whole-chapter). Chapters are too large to inject whole. Whole-chapter injection only for genuinely small chapters.
|
||
4. **Conversation memory ≠ textbook evidence.** Structurally separated prompt blocks. Summary may never inject textbook-fact authority.
|
||
5. **Model never reads a file path.** Backend resolves + reads + injects. That's where `ai.js` already sits.
|
||
6. **Strict grounding:** answer exclusively from supplied textbook sources; refuse if uncovered.
|
||
|
||
### Backend shape (proposed)
|
||
|
||
```
|
||
backend/src/services/content/
|
||
├── textbookRegistry.js # authored map {grade,subject,chapter} → {file,title}
|
||
├── retriever.js # keyword-scans pre-normalized sections; top-N
|
||
├── markdownParser.js # ##-split (OFFLINE ingestion only, not per-request)
|
||
└── contentLoader.js # compose: registry → read → (parse, if raw) → sections
|
||
```
|
||
|
||
`ai.js`: `buildSystemPrompt(grade,subject,chapter,sources)` gains strict-grounding instructions; `buildMessages` appends sources as a separate block (never merged into summary). Route (`chat.js`): resolve → sources → inject; unknown → hard 500.
|
||
|
||
### Content file format — author-facing spec (YOU provide this) — LOCKED 2026-08-24
|
||
|
||
> Decision: you hand me **clean, `##`-sectioned markdown, one file per chapter** (human-readable form, same shape as the confirmed-clean NCERT `file.md`). My ingester pre-splits it into retrieval-ready sections **offline** — you never maintain machine JSON by hand.
|
||
|
||
**File naming & location** — one `.md` per chapter under the raw store:
|
||
```
|
||
backend/content/raw/
|
||
└── grade-12/
|
||
└── economics/
|
||
├── ch01-demand.md
|
||
├── ch02-supply.md
|
||
└── ...
|
||
```
|
||
- `grade-<N>/` = grade directory (numeric), `<subject>/` = subject slug, `ch<NN>-<slug>.md` = chapter.
|
||
- File basename (`ch01-demand`) is the stable **chapter id** the registry keys on; it won't collide across subjects because the subject dir namespaces it.
|
||
|
||
**Heading rules**
|
||
- `#` (H1) = chapter title, once at top.
|
||
- `##` = **one logical section per heading** — numbered (`## 1.1 Introduction`, `## 1.2 …`) OR standalone names (`## EXERCISE 1.1`, `## Miscellaneous Exercise on Chapter 1`).
|
||
- Do **not** use `##` for boxes/figures/callouts (that overloaded it in Rajeev Bansal — reject that source's style). Keep those as plain paragraphs inside a section.
|
||
- `###` + deeper = fine inside a section; the ingester splits only on `##`.
|
||
|
||
**What to include / strip**
|
||
- Keep: body text, `$…$` / `$$…$$` LaTeX math (renders via existing KaTeX), tables, `## EXERCISE` blocks (they're legit sections).
|
||
- Strip before handing me: raw `<details>…</details>` image blocks, `` placeholders, running headers/footers repeating the book title / page numbers. (The scanner's JSON carries this noise; your clean md should not.)
|
||
- No `bbox`/`page_idx`/JSON — this is plain markdown.
|
||
|
||
**Ingestion target** (what my ingester emits, NOT yours to write):
|
||
```
|
||
{ grade, subject, chapter, chapter_title, section, section_id, text }
|
||
// e.g. { grade:12, subject:"economics", chapter:5,
|
||
// section:"5.2 Factors Affecting Elasticity",
|
||
// section_id:"eco12-ch5-s5.2", text:"…" }
|
||
```
|
||
|
||
### Storage model — LOCKED 2026-08-24 (two stores, never merged)
|
||
|
||
```
|
||
backend/content/
|
||
├── raw/ ← Store 1: YOUR clean per-chapter md. Durable, human-authored,
|
||
│ └── grade-12/ editable, committed, never read by the running server.
|
||
│ └── economics/
|
||
│ └── ch01-demand.md
|
||
└── generated/ ← Store 2: built by the ingester from raw/ at deploy/boot.
|
||
└── grade-12/ Pre-split sections; the ONLY thing runtime reads.
|
||
└── economics/
|
||
└── ch01-demand.sections.json # [{id,heading,text}, …]
|
||
```
|
||
- Registry (`textbookRegistry.js`) points at **Store 2** (`generated/grade-12/economics/ch01-demand.sections.json`). Runtime never touches Store 1.
|
||
- Regenerate Store 2 from Store 1 whenever a chapter changes (a small script or a deploy step) — never hand-edit Store 2.
|
||
- Rationale: raw = "the book, editable"; generated = "retrieval-ready, no parse at request time". This is the offline/online split the plan already calls for.
|
||
|
||
### Locked decisions (2026-08-24 — all five answered)
|
||
|
||
1. **Content source:** YOU provide clean per-chapter `##`-sectioned md (Store 1 above). I build the ingester that pre-splits it.
|
||
2. **Scope of first vertical slice:** exactly ONE grade + ONE subject (Grade 12 → Economics) end-to-end (registry, retrieval, grounding) before scaling. Prove it, then grow.
|
||
3. **Grounding strictness — LOCKED strict-only:** answer exclusively from supplied textbook sections. The model may be creative with **examples**, but only derived from the supplied text — no outside/general knowledge. Refuses if the content doesn't cover the question.
|
||
4. **Grade normalization seam:** frontend/DB keep sending `"Grade 10"` (label); the content loader boundary normalizes to `"10"` (registry key) before lookup. Frontend/DB untouched.
|
||
5. **Tokenizer measurement:** deferred to the future **admin-dashboard** build — not Phase A. Section-retrieval decision holds regardless.
|
||
|
||
### Blocking inputs before code
|
||
|
||
1. The **first cleaned chapter md** (Store 1) for Grade 12 → Economics, in the format above. Registry + parser contract anchor against it; I confirm the `##`-split against real input, then build the ingester + registry around that one file set.
|
||
2. ~~Strictness~~ — **locked** (strict-only, decision #3).
|
||
3. ~~Tokenizer count~~ — deferred to admin dashboard (decision #5).
|
||
|
||
### Where we are vs endgoal (2026-08-24 reconciliation)
|
||
|
||
- **MVP chat infrastructure: ~75–80% done** — auth, sessions, SSE, rate limiting, ownership isolation, anonymous trials, routing, markdown+KaTeX rendering, rolling summaries: all implemented, tested, green.
|
||
- **The endgoal (grounded textbook answering): ~5%** — the entire content pipeline (registry, parser, retriever, loader, source injection) is **0% shipped**. It's additive into a deliberately left-open seam (`ai.js` prompt builders), not a rewrite.
|
||
- The endgoal reduces to: 4 content files + extend 2 `ai.js` functions + route resolution + chapter ids in selector data. Everything existing stays.
|
||
|
||
### Execution checklist — next session (do in order)
|
||
|
||
**Phase A — Offline ingestion** (not in `/api/chat` path)
|
||
1. Ingest raw md → split on `## <n.n>` → strip `<details>`/image placeholders/LaTeX spills → emit sections `{ section_id, heading, text }`.
|
||
2. Attach Grade/Subject/Chapter metadata; produce one normalized section file per chapter. (Optional: measure chapter tokens here with the target tokenizer.)
|
||
|
||
**Phase B — Content backend**
|
||
3. `backend/src/services/content/textbookRegistry.js` — authored `{grade,subject,chapter}` → `{file,title}`.
|
||
4. `backend/src/services/content/retriever.js` — keyword-score question vs sections → top-N.
|
||
5. `markdownParser.js` — `##`-split (offline only).
|
||
|
||
**Phase C — Wire into `ai.js`** (the existing seam)
|
||
6. `buildSystemPrompt(grade,subject,chapter,sources)` gains strict-grounding instructions; `buildMessages` appends sources as a **separate block**, never merged into summary.
|
||
7. `chat.js` route: resolve `{grade,subject,chapter}` → sources → inject; unknown → hard 500 (no silent generic fallback).
|
||
|
||
**Phase D — Frontend + tests**
|
||
8. Give chapters stable ids (mirror subject id→label pattern; `activeChapter` currently stores labels) — compareField becomes `"id"` for chapters.
|
||
9. Tests: parser split, registry miss/traversal, retriever scoring, sources-block injection (absent sources = byte-identical old prompt), full-suite green.
|
||
|
||
**Phase E — only if needed**: measured cost/latency tuning of the retrieval threshold (already section-retrieving; no new infra).
|
||
|
||
**Resolved micro-decisions (2026-08-24):**
|
||
- Grade normalization: done at `contentLoader`/registry boundary (`"Grade 10"` → `"10"`); frontend/DB keep the label. Locked, decision #4.
|
||
- Chapter id naming: the file basename (`ch01-demand`) IS the registry key / frontend chapter id — stable, namespaced by subject dir. The raw file spec (§ Content file format) defines it.
|
||
|
||
---
|
||
|
||
## Security Hardening — 2026-08-19 (implemented + tested)
|
||
|
||
> Full plan: `docs/security-hardening-plan.md`. All changes verified against the running local Supabase stack.
|
||
|
||
### Secret scrub 🔴
|
||
- Hardcoded `sb_secret_...` (Supabase service-role, bypasses RLS) removed from `ANALYSIS.md`, `IMPLEMENTATION_PLAN.md`, and `backend/scripts/setup-db.js`.
|
||
- `setup-db.js` now refuses to run without `SUPABASE_SERVICE_ROLE_KEY` (no fallback).
|
||
- Added `scripts/check-secrets.mjs` — pre-commit scan; passes on the clean tree. **Rotate local keys** (`npx supabase stop && start`) since the old value is in git history.
|
||
|
||
### Auth
|
||
- `GET /api/auth/me` now derives identity from the **verified JWT** (`verifyToken`), never the unsigned `padhle.user` cookie — forged cookies return 401 (tested).
|
||
- Backend enforces password ≥ 6 chars on signup (matches frontend).
|
||
|
||
### Input validation
|
||
- New `validateChapter()` / `validateSessionMeta()` in `chatValidation.js`: chapter capped at 200 chars, rejects `< > \n \r` (prompt-injection vector into the AI system prompt — closed, tested).
|
||
- Applied to `POST /api/chat` and `POST /api/sessions`.
|
||
|
||
### AI provider
|
||
- CVE-2026-004 fully closed: the non-streaming Google path in `getChatResponse` now uses the safe `URL` constructor (key in query string, never in path).
|
||
- Every outbound AI call (OpenAI/Anthropic/Google + summarizer) has a 120 s `AbortSignal.timeout`.
|
||
|
||
### Middleware / robustness
|
||
- Token cache in `supabaseAuth.js` capped at 2000 entries (FIFO eviction) — bounded memory.
|
||
- New `securityHeaders` middleware (nosniff, frame-deny, referrer-policy, COOP; removes `X-Powered-By`) — tested.
|
||
- `TRUST_PROXY` env to fix rate-limit client IP behind a reverse proxy.
|
||
- Dev-mode rate limiter skips all loopback IPs (tests no longer trip the auth 5/15min limit).
|
||
|
||
### Session-leak verification (the core ask)
|
||
- `backend/test/leak.test.js` — end-to-end against real local Supabase, in-process app on an ephemeral port, real cookies:
|
||
- B cannot read / list / delete / clear / continue A's session (404s, listings exclude it).
|
||
- `/api/auth/me` rejects forged cookies; unauthenticated protected routes → 401.
|
||
- Chapter injection → 400; security headers present.
|
||
- **Result:** `node --test` → 14/14 pass (9 unit + 5 integration). Frontend Vite build passes.
|
||
|
||
### Out of scope (documented)
|
||
Refresh-token flow, server-side session revocation list, CSRF tokens, Redis rate-limit store, `run-migration.js` local Postgres password, legacy `sessionStore.js` XSS note.
|
||
|
||
---
|
||
|
||
## Anonymous 5-Message Trial — 2026-08-19 (implemented + tested)
|
||
|
||
> Agreed design: unregistered users can use the full app and send **5 free messages**; the backend hard-stops the 6th with `403 { code: "SIGNIN_REQUIRED" }`. Refresh or sign-out starts a fresh trial (pure UX funnel, not a security boundary — by design). No cookies, no DB writes for anonymous traffic.
|
||
|
||
### Backend
|
||
- **`backend/src/services/anonTrial.js`** — in-memory trial store (factory `createTrialStore()` + default singleton with reaper):
|
||
- Crypto-UUID trial ids (unguessable), 500-trial LRU cap, 30-min idle TTL reaper (unref'd interval), history trimmed (last 10 for AI context, 20 stored).
|
||
- Only `user` messages consume the 5-message budget.
|
||
- **`backend/src/services/sse.js`** — shared SSE writer: JSON-escapes `\n`/`U+2028`/`U+2029` (CVE-2026-003) and no-ops after `res.end()` (prevents write-after-end crashes).
|
||
- **`backend/src/routes/chat.js`** — now `createChatRouter({ streamFn })` (injectable for tests). `POST /api/chat`:
|
||
- **Anonymous** (`req.user` unset): resolve trial (`chatId` continues in-memory trial; unknown/forged id → fresh trial) → **if budget spent → 403 SIGNIN_REQUIRED before any AI call** → else consume 1 message, stream via SSE emitting `session` (trial id) + `limit` (remaining) events. **No DB writes at all.**
|
||
- **Signed-in**: unchanged (DB session, rolling summary, unlimited).
|
||
- `GET /api/chat/sessions` and `GET /api/chat/:id` now require auth (401 for anonymous) — anonymous trials are invisible to the sessions API.
|
||
- **`backend/src/app.js`** — `/api/chat` mounts `optionalAuth` (not `supabaseAuth`); `/api/sessions` stays strictly authenticated.
|
||
|
||
### Frontend
|
||
- **`App.jsx`** — hard auth gate removed; app renders for everyone. New state: `showSignIn` (on-demand modal), `anonRemaining` (5). `handleSend` blocks client-side at 0 remaining; handles `403 SIGNIN_REQUIRED`; applies `limit` SSE events. Auth-transition effect resets the budget (fresh trial per refresh/sign-out).
|
||
- **`ChatInput.jsx`** — `exhausted` prop → replaces the input with a "You've used your 5 free messages — **Sign in to continue**" card (opens the modal).
|
||
- **`TopNav.jsx` / `Sidebar.jsx`** — anonymous users get a "Sign in" button; signed-in users keep the avatar/sign-out.
|
||
- **`SignIn.jsx`** — modal is now on-demand: optional `onClose` ("Maybe later" + × + overlay click).
|
||
|
||
### Tests (all passing — `cd backend && node --test` → 34/34; `cd frontend && npm run build` → passes)
|
||
- `backend/test/anonTrial.test.js` (10) — budget semantics, UUID uniqueness, LRU cap, TTL reaper, context trimming, **worst-case concurrent interleaving (100 parallel → ≤5 pass)**.
|
||
- `backend/test/anonFlow.test.js` (8) — integration with injected fake AI streamer (no real AI / no Supabase):
|
||
- 5 messages succeed with `session`+`limit` events; 6th → 403 SIGNIN_REQUIRED with **no AI call**.
|
||
- Continuity (chatId echoes same trial, AI sees prior turns); refresh/unknown id → fresh budget; forged id → fresh trial, never touches sessions; AI failure consumes message + clean error event; chapter injection → 400; unauth GET chat routes → 401.
|
||
- Existing suites unchanged and green (chatValidation, aiMessages, leak against live Supabase).
|
||
|
||
### Security posture of the anonymous path
|
||
- No DB writes → no data to leak; forged/session-id-shaped `chatId`s resolve to fresh trials, never to other users' data.
|
||
- Bounded memory (LRU + TTL) → no DoS growth from anonymous traffic.
|
||
- `chatLimiter` (30/min/IP) still applies on top.
|
||
- Rate limiting + validation run before the trial branch (same rules as signed-in).
|
||
|
||
---
|
||
|
||
## Client-Side Routing — 2026-08-19 (implemented)
|
||
|
||
> The app previously had **no router** — it was a pure state-driven SPA where every "view" (selector flow, chat, modal) was a conditional render off React state, so the URL never changed from `/`. Added `react-router-dom` so URLs are meaningful and chats are deep-linkable.
|
||
|
||
### Changes
|
||
- **`frontend/package.json`** — added `react-router-dom@^7`.
|
||
- **`frontend/src/main.jsx`** — wrapped the app in `<BrowserRouter>` with explicit routes: `/`, `/chat/:chatId`, and a `*` catch-all (all render the same `<App/>` shell; `useParams()` exposes `chatId`).
|
||
- **`frontend/src/App.jsx`**:
|
||
- `useNavigate` + `useParams` — `handleNewChat` now goes to `/`; opening/creating a chat navigates to `/chat/:id` (`replace`). The SSE `session` event drives the URL for both signed-in sessions and anonymous trials.
|
||
- A **deep-link effect** loads a conversation on cold-load of `/chat/:id` (signed-in users only; anonymous trial URLs just start fresh, matching the "refresh loses the conversation" rule).
|
||
- A `chatLoadHandledRef` guards against duplicate loads (StrictMode double-invoke, sidebar-then-URL, auth restore) and drops stale in-app trial/session URLs on auth transitions.
|
||
|
||
### Result
|
||
- Sharing/pasting `/chat/<id>` now lands on that conversation (**signed-in only**).
|
||
- URL reflects the active chat; back/forward and refresh behave sensibly.
|
||
- Anonymous users keep the URL at `/` — their trial conversation works, but the trial id is **never** written into the address bar (see security note below).
|
||
- Verified: `vite preview` serves both `/` and `/chat/<id>` as the SPA shell (200, app `#root` present) — production deep links resolve.
|
||
|
||
### Security note (important)
|
||
Anonymous trial ids are **capabilities** — there is no server-side ownership check on them; `POST /api/chat` with `chatId: <trialId>` rejoins the trial. Earlier, `navigate(\`/chat/${data.id}\`)` ran for *both* signed-in and anonymous, which baked an anonymous trial id into the URL — leaking the capability via browser history, server logs, referer headers, shared links, etc. **Fixed:** only signed-in sessions are reflected in the URL; anonymous trial ids stay in in-memory React state and the POST body only. Signed-in `/chat/:id` URLs are safe because the server ownership-checks them (404 unless the current user owns the session).
|
||
|
||
### Log redaction (defense-in-depth)
|
||
- **`backend/src/services/logRedact.js`** — `redactSecrets()` scrubs UUIDs (both anonymous trial ids *and* signed-in session ids, since UUID format overlaps) and obvious API keys (`sb_secret_…`, `sk-…`) from any string; `logError(tag, err)` wraps `console.error` and returns non-strings unchanged.
|
||
- **`backend/src/routes/chat.js`** — all five `console.error(err.message)` sites now route through `logError`, so a trial/session id embedded in an error message (e.g. an AI provider echoing an id back) is redacted before it reaches stderr. Covers the risk of a future request-logger (morgan/pino) or error tracker re-introducing bearer token / capability ids into logs.
|
||
- **`backend/test/logRedact.test.js`** (2, included in the 34) — unit tests for the scrubber + an integration test that drives a real anonymous chat where the AI throws a message *containing* the actual trial id and an `sk-…` key, spies on `console.error`, and asserts neither the id nor the key reach the log output (with `[REDACTED_UUID]`/`[REDACTED_KEY]` markers instead). Proves the redaction is real, not vacuous (verifies the raw id was present pre-redaction).
|
||
- `Referrer-Policy: no-referrer` was already set globally in `securityHeaders.js`, so the referer vector is closed too.
|