Closed-Loop RAG Optimization System Based on User Feedback

College of Information Science and Technology, Beijing University of Chemical Technology, Beijing, China

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

Retrieval-Augmented Generation (RAG) effectively mitigates large language model (LLM) hallucinations, yet traditional systems suffer from high cold-start costs, fragmented retrieval-generation optimization, and low feedback utilization in low-resource scenarios. To tackle these pain points, the paper designs a closed-loop RAG optimisation framework that introduces three complementary modules, a Causal Feedback Labeling (CFL) subsystem that builds and maintains a transparent a “feedback-type– root-cause–optimization-strategy” lookup table, a Few-Shot Cold-Start (FCS) component that bootstraps performance by manufacturing synthetic pseudo-feedback, filtering the most informative samples through active learning, and then blending them with the trickle of real user ratings, and a Retrieval-Generation Collaborative Adapter (RGA) that lets gradient signals hop back and forth between retriever and generator via lightweight cross-attention layers so both modules update in lockstep. Experiments on FeedbackQA and HotpotQA-small, comparing our system with six strong baselines, reveal gains of 5.2 percentage points in F1, a 4.5-point drop in hallucination rate, and annotation expenses that shrink to only 17.5 % of the standard supervised budget. And it’s cold-start performance curve climbs more than 60 % faster than the best rival, confirming that the framework can adapt quickly in feedback-starved settings and offering engineers a practical route to deploying truly closed-loop RAG services.