What I learned building a RAG assistant for scanned Hindi and English university circulars

Engineering · October 18, 2026 · 4 min read

Scanned paper, two scripts, tables everywhere, and staff who need an answer they can trust. Notes from building an assistant that answers from university circulars with citations, on the web and in a Flutter app.

Most RAG demos start with clean PDFs in English. The SVSU Knowledge Assistant started with a cupboard of paper circulars in Hindi and English, photocopied, scanned at an angle, full of ruled tables, that staff across a university needed answers from. This is what I learned building it, in the order I learned it.

OCR quality decides everything downstream

I spent the first weeks on retrieval and models. Wrong order. If the text coming out of OCR is garbage, nothing after it can be good. Three things made the biggest difference:

• Detect the script per page before running Tesseract, and run the right language model. Mixed pages get both and the better confidence wins.
• Rebuild tables cell by cell. Circulars are mostly tables: fee schedules, date sheets, code lists. Plain OCR output turns a table into word soup. Detecting the ruled lines and reading each cell keeps "Course code 302" next to "Fee ₹1,200" instead of three rows away.
• Flag low-confidence words and keep them visible to reviewers, instead of silently guessing.

The pipeline runs in the background, so a staff member uploads a scan and gets on with their day.

Store the location of every passage, not just the text

Each paragraph and each table row is embedded and stored in PostgreSQL with pgvector, along with the page number and the bounding box it came from. That sounds like bookkeeping until you see the payoff: an answer can cite "Circular 42, page 3, row 7", and clicking the citation opens the original scan scrolled to that row, highlighted. For people who have to act on the answer, the citation is the product.

Vectors alone miss the things that matter most

Semantic search is great at "what is the policy on late fees" and terrible at "what does code EX-2024-17 say". Circulars are full of codes, dates and exact amounts, and those are what people search for. The fix was hybrid retrieval:

1. Rewrite the question into an English query and a Hindi query, because the user may ask in Hinglish and the document may be in either.
2. Run vector search for meaning.
3. Run exact matching for codes, dates and rare tokens.
4. Merge, then re-rank the top candidates with a small cross-encoder model.

The re-ranker was the cheapest big improvement in the whole project.

Answer only from the passages, then check the answer

The language model gets the retrieved passages and is told to answer only from them, with numbered citations. Then a second, cheaper model reads the answer and the sources and flags any claim it cannot find support for. The flagged answers are rare, but they are the ones that would have cost someone's trust. Answers stream in as they are written, which makes the whole thing feel fast even when retrieval takes a second.

The boring parts are the product

• A draft-to-active approval workflow, so a document is reviewed before the assistant can answer from it.
• Roles and per-document visibility: some circulars are for staff only.
• Bulk import and automatic expiry for circulars with an end date.
• Two-step verification and lockouts, because this is university data.
• An automatic answer-quality test: a set of questions with known answers runs in CI on every push. If retrieval regresses, the deploy fails.

CI/CD runs from GitHub Actions to an AWS server with Docker and Caddy. Nothing exotic, which is the point.

One API, two clients

The web app is Next.js. The Android app is Flutter, on the same API and the same accounts, with camera upload for photographing paper letters and in-app update notices. Widget tests open every screen on a phone-sized viewport in CI. Building the app second, on an API that was already solid, took a fraction of the time the web app did.

What I would do differently

• Start with the OCR evaluation set, not the chat UI.
• Keep the chunking tied to document structure (rows, sections) from day one instead of fixed token windows.
• Budget time for the approval workflow early; it shaped the data model more than the AI did.

If you are sitting on a pile of scanned documents and want something your team can actually trust, this is the kind of work I take on. The AI assistants service page has the short version, and the project page has screenshots.

Tags: RAG, OCR, pgvector, FastAPI, Hindi, AI