Retrieval-Augmented Generation At Enterprise Scale: Chunking Strategies, Vector Index Optimization, And Confidence-Calibrated Retrieval For Mission-Critical LLM Applications

Abstract

Retrieval-Augmented Generation has emerged as the primary architectural pattern for grounding large language model outputs in enterprise knowledge assets that cannot be encoded in model weights alone. Despite broad adoption, production deployments routinely leave three core design decisions at default configurations — fixed-size chunking, flat vector indices, and globally applied confidence thresholds — accepting retrieval quality penalties that accumulate with document volume and query diversity. This article presents a sequenced optimisation framework addressing each decision layer in turn. Chunking strategy selection is examined across five approaches — fixed-size, recursive, semantic, structure-aware, and hierarchical — with empirical evidence demonstrating that structure-aware parsing achieves measurably higher top-K retrieval effectiveness than token-boundary segmentation on technical and regulatory enterprise corpora. Vector index architecture is analysed across the scale spectrum from development-level flat search to multi-tenant approximate nearest-neighbour configurations, with hybrid dense-sparse retrieval via Reciprocal Rank Fusion presented as the configuration that consistently recovers retrieval signal lost by either approach in isolation. Confidence calibration is examined through per-field threshold adaptation, with production evidence demonstrating up to 38% reduction in false-positive review traffic through routing thresholds calibrated to observed accuracy by field type rather than applied uniformly. The integrated framework sequences corpus analysis, chunking selection, index architecture, hybrid search configuration, and per-field calibration into a repeatable deployment process validated across enterprise knowledge retrieval applications operating at million-document scale with sub-200ms latency requirements.