PDF RAG Chatbot

Secure async document intelligence with hybrid retrieval, OCR, abstention, and grounded citations.

FastAPICeleryRedisQdrantPostgreSQLPyMuPDFTesseract OCRDocker

Case study

What this project demonstrates

Built as an infrastructure-complete RAG backend rather than a chat demo. The system treats ingestion, retrieval quality, confidence calibration, document security, citation validity, and regression testing as first-class engineering problems.

Architecture

PDF validation → async queue → extraction/OCR → structure-aware chunks → embeddings → Qdrant + lexical retrieval → fusion/reranking → confidence gate → grounded generation → validated citations

Engineering highlights

Asynchronous PDF ingestion with worker-based processing and persistent metadata.
Dense, hybrid, and hybrid-rerank retrieval with evidence-based confidence gating and abstention.
Native, scanned, and mixed PDF handling with local OCR plus layout/table-aware extraction.
Validated page citations, retrieval traces, reindex fingerprints, regression evaluation, and security hardening.

Boundaries / limitations

Evaluation is repository-defined and domain-dependent; OCR and layout heuristics are bounded by configured limits and do not imply universal document understanding.