Building a PDF Question-Answering Chatbot with Spring AI: From PDF Upload to RAG-Powered Answers

2026-06-22 · Source: Machine Learning on Medium · Field: Technology & Digital — Artificial Intelligence & Machine Learning, Software Development & Engineering · Depth: Intermediate, long

Summary

A practical guide details building a PDF question-answering chatbot leveraging a Retrieval-Augmented Generation (RAG) pipeline with Spring AI. The system integrates Gemini 2.5 Flash for language model generation and Ollama's nomic-embed-text for local, cost-free embeddings, which are 768-dimensional. PostgreSQL with the PGVector extension serves as the vector store. The architecture involves two flows: PDF ingestion, where documents are parsed, chunked into ~500-token segments with 100-token overlap, embedded, and stored; and question answering, where user queries are embedded, relevant chunks retrieved via cosine similarity search, and then used as context for Gemini to generate grounded answers. The project, built with Java 21 and Spring Boot 3.5.x, highlights the importance of RAG, vector databases, and local embedding solutions for enterprise AI applications.

Key takeaway

For AI Engineers building enterprise-grade document Q&A systems, prioritize a robust RAG architecture over solely relying on large LLMs. Your focus should be on effective chunking, local embedding solutions like Ollama for cost and privacy, and integrating a vector store like PGVector. This approach ensures answers are accurate and verifiable, transforming general LLMs into domain experts for your specific data. Consider Spring AI for its abstraction layer, simplifying provider swaps.

Key insights

RAG systems enhance LLM accuracy by grounding answers in retrieved, relevant private data, outperforming larger models without context.

Principles

• Retrieval quality is key for RAG performance.
• Vector databases are essential for AI applications.
• Local embeddings offer privacy and cost savings.

Method

The RAG pipeline involves PDF ingestion (parse, chunk, embed, store) and query answering (embed question, retrieve similar chunks, inject context into LLM, generate answer).

In practice

• Use Ollama for local, private, and cost-effective embeddings.
• Implement "TokenTextSplitter" for optimal chunking with overlap.
• Store metadata (source, page) with chunks for citations.

Topics

• Retrieval-Augmented Generation
• Spring AI
• Ollama Embeddings
• PGVector
• Document Q&A
• Semantic Search

Best for: AI Engineer, Machine Learning Engineer, Software Engineer

Related on AIssential

• Vector Search Using Ollama for Retrieval-Augmented Generation (RAG) — RAG combines vector search with LLMs to provide context-aware, fact-grounded responses from external data.
• HybridRAG: A Practical LLM-based ChatBot Framework based on Pre-Generated Q&A over Raw Unstructured Documents — HybridRAG pre-generates QA pairs from unstructured documents to enhance RAG chatbot speed and accuracy.
• Perplexity Is Just a Pipeline. Here’s How to Build Your Own, Private and Local — A trustworthy AI answer engine is a local, private pipeline of open-source components, enforced by strict citation prompting.
• AI tips & tricks (Ep. 307) — RAG systems enable secure, cost-effective, and performant AI for institutional knowledge by localizing data and leveraging smaller models.
• Enterprise Document Intelligence: A Series on Building RAG Brick by Brick, from Minimal to Corpus scale — Enterprise RAG success hinges on deep document and domain expertise, not just advanced AI models.
• A RAG Chatbot with Incremental Context Retrieval based on Local LLMs for Hospital Documents — Adaptive context control is crucial for reliable and efficient RAG systems using local LLMs in healthcare.

Open in AIssential →

Editorial summary, takeaway, and curation by AIssential. Original article published by Machine Learning on Medium.