Document Intelligence Platform
DocuMind
Chat interface for PDFs that answers only from what's actually in the document, no invented citations. An async ingestion pipeline chunks and embeds files in the background so uploads never block the API, and answers stream back over SSE.
targetProblem Statement
Chatting with a PDF usually goes wrong one of two ways: the model answers from its training data instead of the document, or a heavy synchronous ingestion step freezes the UI while a large file gets parsed and embedded.
boltReal-world Impact
Ingestion runs entirely off the request thread. BullMQ workers handle chunking and embedding in the background while the API stays responsive, and the answer streams back over SSE as soon as the model starts generating.
account_treeSystem Architecture
Asynchronous BullMQ worker queues decouple the Next.js API thread from the Redis & pgvector ingestion pipeline. Data flows into Supabase (Postgres) where hybrid lexical and vector semantics isolate context efficiently.
┌───────────────────────────────────────────┐
│ DOCUMIND RAG PIPELINE │
└───────────────────────────────────────────┘
[Client Upload] ──► [API Server (NestJS)]
│
(Job Enqueued)
▼
[Redis/BullMQ]
│
[Background Worker]
(Extract text → Chunking → OpenAI Embedding)
│
▼
[PostgreSQL (pgvector) in Supabase]
│
[Client Query] ──► (Vector Search Top-K)
│
[LLM Prompt Assembly]
│
[SSE Streaming Response to NextJS]memoryAI / System Intelligence
OpenAI models are grounded strictly on retrieval chunks. Strict system prompts make the LLM decline to answer when the retrieved context doesn't contain it, instead of guessing.
psychologyKey Architectural Decisions
pgvector over Dedicated SaaS DBs
Chose PostgreSQL with pgvector natively within Supabase to keep relational (users, per-document chats, access roles) and semantic (embeddings) data logically tied, severely reducing transaction complexity.
Async Message Queues (BullMQ)
Parsing heavy 100-page PDFs blocks the JavaScript event loop. Pushing ingestion to a dedicated Redis-backed worker guarantees concurrent user processing without dropping API requests.