Choosing an LLM for Enterprise RAG: Retrieval Fit Beats Model Hype

This article argues that selecting an LLM for enterprise RAG should prioritize retrieval fit over model size or novelty. It outlines a method that starts with understanding source quality, access control, retrieval behavior, and test questions before shortlisting models. Key criteria include grounded answer quality, citation discipline, context handling, and failure behavior, rather than general intelligence. The article provides a selection flow and evaluation scorecard design, emphasizing that a model must work with the retrieval design, latency, cost, data controls, and evaluation requirements. It avoids naming specific models due to rapid changes, focusing instead on a practical framework for enterprise teams.