feat: rolling conversation summary + session continuity
- Backend emits session id in SSE; frontend sets activeChat so follow-ups reuse the same session (fixes fragmented history). - sessions.summary column; messages past RECENT_LIMIT (10) are folded into a persisted summary via a non-streaming model call. - buildMessages sends system(+summary) + last 10 messages, bounding context while preserving long-range memory. - Tests: aiMessages (capping + summary injection).
This commit is contained in:
+49
-11
@@ -19,7 +19,7 @@
|
||||
| `frontend/src/components/sidebar/Sidebar.css` | Sidebar layout, hover indicators, chat item delete, upgrade card, user info |
|
||||
| `frontend/src/components/chat-history/ChatHistory.jsx` | Welcome screen with bento suggestions OR message list OR selector flow |
|
||||
| `frontend/src/components/chat-history/ChatHistory.css` | Welcome header, bento grid, info panel, message list |
|
||||
| `frontend/src/components/message/Message.jsx` | Single message bubble (user or assistant), action buttons, typing dots |
|
||||
| `frontend/src/components/message/Message.jsx` | Single message bubble (user or assistant), markdown+math rendering for assistant, action buttons, typing dots |
|
||||
| `frontend/src/components/message/Message.css` | Bubble styling, avatar, actions, typing animation |
|
||||
| `frontend/src/components/chat-input/ChatInput.jsx` | Textarea with auto-expand, mic, send button, attachment |
|
||||
| `frontend/src/components/chat-input/ChatInput.css` | Input bar styling, send button states, disclaimer |
|
||||
@@ -630,6 +630,12 @@ if (XSS_PATTERN.test(message.text)) {
|
||||
- Added regression tests: `backend/test/chatValidation.test.js` and `frontend/test/appState.test.js`.
|
||||
- Backend tests, backend syntax checks, frontend regression test, and Vite production build pass.
|
||||
- Chrome MCP verified the sign-in screen renders after fixing a missing `useAuth` import exposed by the browser console.
|
||||
- Cross-user session leak fixed (token cache keyed by full token — see CVE-2026-011).
|
||||
- AI connected via OpenRouter; streaming verified in the browser with both user+assistant messages persisted.
|
||||
- Assistant replies render as markdown + KaTeX math; verified in the headed browser.
|
||||
- Current user message is included in the AI context (persisted before streaming).
|
||||
- Session continuity fixed: backend emits `session` id in SSE; follow-ups reuse the same session.
|
||||
- Rolling conversation summary implemented and persisted (bounded context + long-range memory).
|
||||
|
||||
### Database status
|
||||
|
||||
@@ -639,24 +645,56 @@ if (XSS_PATTERN.test(message.text)) {
|
||||
|
||||
---
|
||||
|
||||
## Data Storage & Conversation Context Model (2026-08-19)
|
||||
|
||||
### Q: Each message is a new row — is that sustainable?
|
||||
|
||||
**A (yes):** Each message is one row in `public.messages` (`id`, `session_id`, `role`, `text`, `created_at`), written append-only. This is the standard chat design (Slack/OpenAI-style). It is sustainable because:
|
||||
- Storage is tiny (average turn is ~KB scale; a million messages ≈ a few GB).
|
||||
- `idx_messages_session_id` keeps per-session history reads fast even with many rows.
|
||||
- Append-only is desirable: simple pagination/partial loads, immutable history, easy to add edit/regenerate later.
|
||||
- One-row-per-session-with-big-text-blob would be worse (can't load partial history, bad indexing).
|
||||
- DB only needs attention at scale: partitioning by date/user past ~10s of millions of rows, or archiving old sessions.
|
||||
|
||||
### The real scaling concern: AI context window (the "memory" issue)
|
||||
|
||||
- The DB is fine, but on **every** `/api/chat` call the backend loads the **entire** message history and sends it **all** to the model.
|
||||
- Token count (and therefore **cost and latency**) grows with conversation length; eventually it exceeds the model's context limit and the call fails.
|
||||
- **This is the lever worth engineering** — not the database.
|
||||
|
||||
### Blocked-on note
|
||||
|
||||
Before context work has value, the **frontend session-continuity bug** must be fixed (see What's Next #1): follow-ups in a fresh chat currently create a new session with no history, so long-running context never even accumulates.
|
||||
|
||||
---
|
||||
|
||||
## What's Next
|
||||
|
||||
### 1. Frontend chat-flow design gap
|
||||
- Backend does not return the new `session.id` to the client (SSE only sends `chunk`/`done`/`error`).
|
||||
- Sidebar refresh (`loadSessions()`) only runs on the successful SSE path, so an AI error leaves the sidebar stale even though the session row is created.
|
||||
- Decide: return `session.id` in SSE + refresh sidebar on error path (Option 1), or only persist sessions after a successful AI response (Option 2).
|
||||
### 1. Frontend session continuity — DONE
|
||||
- Backend now emits a `{ type: "session", id }` SSE event when a conversation is created/loaded; the frontend sets `activeChat` on it, so follow-ups reuse the same session and history accumulates.
|
||||
- Remaining minor: sidebar refresh (`loadSessions()`) only runs on the successful SSE path — an AI error still leaves the sidebar stale even though the session row was created.
|
||||
|
||||
### 2. Frontend authentication handling
|
||||
- Handle protected-route HTTP 401 responses by returning to the sign-in modal.
|
||||
- Persist selector state per user.
|
||||
### 1b. Rolling conversation summary (B+D) — implemented 2026-08-19
|
||||
- `sessions.summary` column (migration `20260819150000_add_sessions_summary.sql`).
|
||||
- On each `/api/chat`, when a session exceeds `RECENT_LIMIT` (10) messages, older messages are folded into a persisted summary via a non-streaming model call (`summarizeConversation`), stored with `setSummary`.
|
||||
- The model receives `system(+summary) + last 10 messages` (`buildMessages`), so context stays bounded while long-range context persists.
|
||||
- Verified end-to-end: fresh chat emits session id; a 12-message session folded its overflow into a stored summary; follow-up answered with bounded context.
|
||||
- Tests: `backend/test/aiMessages.test.js` (capping + summary injection).
|
||||
|
||||
### 2. Frontend rendering — fixed
|
||||
- Assistant replies now render via `react-markdown` (headings, bold, italics, lists, blockquotes, code, tables, `---`) and math (`$H_2O$`, `$CO_2$`) via KaTeX. User messages stayed plain text. `ReactMarkdown` escapes raw HTML by default, so stored-XSS (CVE-2026-009) stays closed while rendering rich content.
|
||||
- Protected-route 401s still return to the sign-in modal.
|
||||
- Selector state persists per user.
|
||||
|
||||
### AI Provider — OpenRouter (connected 2026-08-19)
|
||||
|
||||
- Backend uses the OpenAI-compatible path (`AI_PROVIDER=openai`) pointed at OpenRouter.
|
||||
- `OPENAI_BASE_URL=https://openrouter.ai/api/v1` (added to `ai.js` as an optional override).
|
||||
- Model: `~deepseek/deepseek-v4-flash-latest` (the `~` prefix is REQUIRED — it is OpenRouter's "redirect to latest" alias; the non-tilde `deepseek/deepseek-v4-flash-latest` is an invalid model ID).
|
||||
- Verified end-to-end: `/api/chat` streams a real reply and persists both `user` and `assistant` messages (lost messages bug from the no-AI path is gone).
|
||||
- Note: this is a reasoning model — it streams thinking in `delta.reasoning` before `delta.content`; the app reads `delta.content`, so there is a brief (sub-second) gap while reasoning, then the answer streams.
|
||||
- Model: `qwen/qwen3.7-flash` (final choice; earlier tried `~deepseek/deepseek-v4-flash-latest` and `upstage/solar-pro4`).
|
||||
- OpenRouter "latest" aliases use a required `~` prefix (e.g. `~deepseek/...`); the non-tilde `-latest` is an invalid ID.
|
||||
- Reasoning disabled for OpenRouter (`reasoning: { enabled: false }` in `streamOpenai`/`getChatResponse`) so `delta.content` streams immediately instead of sitting empty during the thinking phase (which made the chat look blank).
|
||||
- Verified end-to-end: `/api/chat` streams a real reply and persists both `user` and `assistant` messages.
|
||||
- The user message is now persisted **before** streaming, so the model sees the current question (fixes canned-greeting responses) and the user message survives AI failures.
|
||||
|
||||
**Env (non-secret):** wire via `backend/.env`, and the placeholders are in `backend/.env.example`.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user