Notes on RAG & Vector Retrieval

Most retrieval failures in RAG systems stem from bad chunking rather than model limitations. When indexing university circulars and schedules, fixed-size token splitting often bisects critical tables or rule lists.

Preprocessing documents to preserve table structure (such as converting bounding boxes to markdown tables) significantly improved embedding similarity matches.

For small to medium datasets (~1,000 documents), an in-memory FAISS flat index provides near-instant search (<10ms) without the overhead or cost of running an external vector database.