feat: rolling conversation summary + session continuity

- Backend emits session id in SSE; frontend sets activeChat so follow-ups
  reuse the same session (fixes fragmented history).
- sessions.summary column; messages past RECENT_LIMIT (10) are folded into
  a persisted summary via a non-streaming model call.
- buildMessages sends system(+summary) + last 10 messages, bounding context
  while preserving long-range memory.
- Tests: aiMessages (capping + summary injection).
This commit is contained in:
2026-08-19 10:23:56 -04:00
parent 56d6f355d1
commit 534b51b41b
7 changed files with 175 additions and 26 deletions
+49 -11
View File
@@ -19,7 +19,7 @@
| `frontend/src/components/sidebar/Sidebar.css` | Sidebar layout, hover indicators, chat item delete, upgrade card, user info |
| `frontend/src/components/chat-history/ChatHistory.jsx` | Welcome screen with bento suggestions OR message list OR selector flow |
| `frontend/src/components/chat-history/ChatHistory.css` | Welcome header, bento grid, info panel, message list |
| `frontend/src/components/message/Message.jsx` | Single message bubble (user or assistant), action buttons, typing dots |
| `frontend/src/components/message/Message.jsx` | Single message bubble (user or assistant), markdown+math rendering for assistant, action buttons, typing dots |
| `frontend/src/components/message/Message.css` | Bubble styling, avatar, actions, typing animation |
| `frontend/src/components/chat-input/ChatInput.jsx` | Textarea with auto-expand, mic, send button, attachment |
| `frontend/src/components/chat-input/ChatInput.css` | Input bar styling, send button states, disclaimer |
@@ -630,6 +630,12 @@ if (XSS_PATTERN.test(message.text)) {
- Added regression tests: `backend/test/chatValidation.test.js` and `frontend/test/appState.test.js`.
- Backend tests, backend syntax checks, frontend regression test, and Vite production build pass.
- Chrome MCP verified the sign-in screen renders after fixing a missing `useAuth` import exposed by the browser console.
- Cross-user session leak fixed (token cache keyed by full token — see CVE-2026-011).
- AI connected via OpenRouter; streaming verified in the browser with both user+assistant messages persisted.
- Assistant replies render as markdown + KaTeX math; verified in the headed browser.
- Current user message is included in the AI context (persisted before streaming).
- Session continuity fixed: backend emits `session` id in SSE; follow-ups reuse the same session.
- Rolling conversation summary implemented and persisted (bounded context + long-range memory).
### Database status
@@ -639,24 +645,56 @@ if (XSS_PATTERN.test(message.text)) {
---
## Data Storage & Conversation Context Model (2026-08-19)
### Q: Each message is a new row — is that sustainable?
**A (yes):** Each message is one row in `public.messages` (`id`, `session_id`, `role`, `text`, `created_at`), written append-only. This is the standard chat design (Slack/OpenAI-style). It is sustainable because:
- Storage is tiny (average turn is ~KB scale; a million messages ≈ a few GB).
- `idx_messages_session_id` keeps per-session history reads fast even with many rows.
- Append-only is desirable: simple pagination/partial loads, immutable history, easy to add edit/regenerate later.
- One-row-per-session-with-big-text-blob would be worse (can't load partial history, bad indexing).
- DB only needs attention at scale: partitioning by date/user past ~10s of millions of rows, or archiving old sessions.
### The real scaling concern: AI context window (the "memory" issue)
- The DB is fine, but on **every** `/api/chat` call the backend loads the **entire** message history and sends it **all** to the model.
- Token count (and therefore **cost and latency**) grows with conversation length; eventually it exceeds the model's context limit and the call fails.
- **This is the lever worth engineering** — not the database.
### Blocked-on note
Before context work has value, the **frontend session-continuity bug** must be fixed (see What's Next #1): follow-ups in a fresh chat currently create a new session with no history, so long-running context never even accumulates.
---
## What's Next
### 1. Frontend chat-flow design gap
- Backend does not return the new `session.id` to the client (SSE only sends `chunk`/`done`/`error`).
- Sidebar refresh (`loadSessions()`) only runs on the successful SSE path, so an AI error leaves the sidebar stale even though the session row is created.
- Decide: return `session.id` in SSE + refresh sidebar on error path (Option 1), or only persist sessions after a successful AI response (Option 2).
### 1. Frontend session continuity — DONE
- Backend now emits a `{ type: "session", id }` SSE event when a conversation is created/loaded; the frontend sets `activeChat` on it, so follow-ups reuse the same session and history accumulates.
- Remaining minor: sidebar refresh (`loadSessions()`) only runs on the successful SSE path an AI error still leaves the sidebar stale even though the session row was created.
### 2. Frontend authentication handling
- Handle protected-route HTTP 401 responses by returning to the sign-in modal.
- Persist selector state per user.
### 1b. Rolling conversation summary (B+D) — implemented 2026-08-19
- `sessions.summary` column (migration `20260819150000_add_sessions_summary.sql`).
- On each `/api/chat`, when a session exceeds `RECENT_LIMIT` (10) messages, older messages are folded into a persisted summary via a non-streaming model call (`summarizeConversation`), stored with `setSummary`.
- The model receives `system(+summary) + last 10 messages` (`buildMessages`), so context stays bounded while long-range context persists.
- Verified end-to-end: fresh chat emits session id; a 12-message session folded its overflow into a stored summary; follow-up answered with bounded context.
- Tests: `backend/test/aiMessages.test.js` (capping + summary injection).
### 2. Frontend rendering — fixed
- Assistant replies now render via `react-markdown` (headings, bold, italics, lists, blockquotes, code, tables, `---`) and math (`$H_2O$`, `$CO_2$`) via KaTeX. User messages stayed plain text. `ReactMarkdown` escapes raw HTML by default, so stored-XSS (CVE-2026-009) stays closed while rendering rich content.
- Protected-route 401s still return to the sign-in modal.
- Selector state persists per user.
### AI Provider — OpenRouter (connected 2026-08-19)
- Backend uses the OpenAI-compatible path (`AI_PROVIDER=openai`) pointed at OpenRouter.
- `OPENAI_BASE_URL=https://openrouter.ai/api/v1` (added to `ai.js` as an optional override).
- Model: `~deepseek/deepseek-v4-flash-latest` (the `~` prefix is REQUIRED — it is OpenRouter's "redirect to latest" alias; the non-tilde `deepseek/deepseek-v4-flash-latest` is an invalid model ID).
- Verified end-to-end: `/api/chat` streams a real reply and persists both `user` and `assistant` messages (lost messages bug from the no-AI path is gone).
- Note: this is a reasoning model — it streams thinking in `delta.reasoning` before `delta.content`; the app reads `delta.content`, so there is a brief (sub-second) gap while reasoning, then the answer streams.
- Model: `qwen/qwen3.7-flash` (final choice; earlier tried `~deepseek/deepseek-v4-flash-latest` and `upstage/solar-pro4`).
- OpenRouter "latest" aliases use a required `~` prefix (e.g. `~deepseek/...`); the non-tilde `-latest` is an invalid ID.
- Reasoning disabled for OpenRouter (`reasoning: { enabled: false }` in `streamOpenai`/`getChatResponse`) so `delta.content` streams immediately instead of sitting empty during the thinking phase (which made the chat look blank).
- Verified end-to-end: `/api/chat` streams a real reply and persists both `user` and `assistant` messages.
- The user message is now persisted **before** streaming, so the model sees the current question (fixes canned-greeting responses) and the user message survives AI failures.
**Env (non-secret):** wire via `backend/.env`, and the placeholders are in `backend/.env.example`.