Hybrid User-LLM: Bridging Sequential Embeddings and Text Summarization for Universal Sequential Recommendation

Department of Computer Science Central University of Rajasthan Ajmer, Rajasthan, India

^* PhD Scholar, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.
^** Assistant Professor, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it. (corresponding author)
^*** Assistant Professor, e-mail: This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

Large Language Models (LLMs) have become powerful tools in personalized recommendation, but the methodologies developed up to now have a salient drawback of not being capable of sufficiently responding to cold-start users with limited interaction histories. Based on the User-LLM architecture [2], the current paper proposes one hybrid architecture that covers 100% of users, but also achieves a Hit@10 of 10.1%, which is a 13.5% improvement compared to the baseline.The suggested Hybrid User-LLM system is an integrated system that dynamically combines latent sequential embeddings of warm users with interpretable textual summarization of cold-start users.Paper use a pretraining phase first, during which we train an autoregressive transformer encoder to predict the next item, and then a joint fine-tuning, where the encoder is also trained together with an LLM using gated cross-attention. Experiment on MovieLens 20M (50k users, 7.2M ratings) shows that the model provides universal coverage to all cohorts of users. The statistical analysis indicates that there are significant performance differences in the user groups (p<0.001), with the hybrid approach having a graceful degradation in cold users (11.8% Hit@10) compared to warm users (11.2% Hit@10).The architecture provides a token cutoff of 93.6 per cent compared to raw text prompting with interpretability through structured summarization. In line with this, this work fills a critical gap in the literature of LLM-based recommendation systems and makes them be deployed in the real world, within the heterogeneous user population.