LLM derived item and user comprehension, bridged with collaborative filtering
Three augmentation patterns distinct from the generative retrieval and ranking direction.
A language model's view of items and users captures different signal from what a collaborative filtering recommender has learned from interactions. That difference is both why combining the two is useful and why it is not trivial.
The previous post identified three places where a language model contributes signal the recommender doesn't have on its own: items whose similarity is determined by content the catalog has no schema for, items the interaction log says little or nothing about, and behavioral patterns the recommender records but cannot interpret.
This post covers three approaches to getting an LLM derived view of items and users, and some ideas on how to fuse that view with the embeddings the recommender already has. For items, a language model reads each item's knowledge graph subgraph and produces a semantic representation. The user side approach takes a similar shape: a language model reads each user's interaction history and produces a text profile, with training that optimizes for recommendation accuracy. The fusion question has multiple active positions, including alignment of the two representation spaces, parallel branch architectures, and fusion that preserves each view's distinctive signal. These techniques are particularly interesting because they layer on top of traditional recsys models instead of purely relying on generative retrieval and ranking techniques in production (TIGER, OneRec etc).
Item comprehension
Generally, items have semantic relationships that exist in reviews, in product descriptions, in world knowledge about how things relate to each other. However those relationships often cannot be reached through co-occurrence or any structured attribute an internal or external catalog/knowledge graph as they are often incomplete.
This is where the CoLaKG paper was particularly interesting. The authors propose to build such a representation entirely offline with the aid of LLMs. For each item, the system extracts a 1-hop and 2-hop subgraph from the knowledge graph centered on that item, serializes the triples into natural language, and passes them to a language model. The model produces a comprehension of the item that draws on its parametric world knowledge to describe relationships the graph never encoded, and a text encoder turns that comprehension into an embedding. Cosine similarity across the catalog then produces a top-k semantic similarity graph, which is what the serving system queries at request time. Both the language model and the encoder run only at build time.
The interesting evidence for this approach comes from deliberately damaging the underlying knowledge graph. When CoLaKG removes thirty percent of the KG’s facts, the LLM comprehension approach degrades less than any other method tested, because the language model fills in what the graph fails to represent. The graph and the language model end up doing complementary work, with the LLM handling the parts of item similarity that no schema ever captured and the graph handling the structured parts the catalog has explicitly modeled.
User comprehension
User profiles, in the LLM augmented sense, are explicit textual descriptions of a user's preferences and behavioral patterns, derived from their interaction history. The motivation, as the LettinGo paper articulates it, is that embedding based user representations "lack interpretability and adaptability." Embeddings are opaque (you can't read what they encode), rigid (fixed dimensionality and format), and locked to whatever model produced them. Text based profiles are "semantically richer and more transparent", interpretable (auditable for bias, eventually editable by the user themselves), domain adaptive (the format can vary by recommendation task), and compact enough to feed any downstream component (LLM prompts, retrieval queries, rankers, agents). The catch is that the profile has to actually be predictive of user behavior, not just a readable summary.
LettinGo treats profile generation as an optimization problem with downstream recommendation accuracy as the reward signal, where “accuracy” here is the rate at which an 8B LLaMA scorer, given the user’s history, the candidate profile, and a held-out item, correctly predicts the user’s actual rating class for that item (dislike, neutral, or like). The training loop runs four language models at high temperature to generate ten candidate profiles per user, scores each against the held-out interactions, and uses winning and losing profiles for the same user as DPO preference pairs. The generator learns to produce profiles that work, without anyone ever specifying what a “good” profile looks like.
The first finding from LettinGo is that an 8B model trained with DPO outperforms GPT-4o profiles produced without alignment, which means the alignment process matters more than the base model capacity. The second is that the profile format emerges from the optimization itself: MovieLens profiles end up looking narrative, Yelp profiles end up structured, and neither was specified in advance. The third is that longer history does not monotonically improve profile quality; performance peaks at a dataset-specific breakpoint and then declines, which suggests the compression and selection are doing real work and not just summarization.
LettinGo validates its profiles only against an LLM evaluator. The complementary path, LLM-generated profile consumed by a traditional ranker with no LLM at inference, has older work behind it. KAR prompts an LLM for user and item knowledge text, encodes both with a frozen text encoder, and feeds the resulting vectors as side features into nine CTR backbones (DCNv2, DCNv1, DeepFM, FiBiNET, AutoInt, FiGNN, xDeepFM, DIEN, DIN) and four reranking models (DLCM, PRM, SetRank, MIR), with consistent lifts across all of them. ONCE does the analogous thing for news recommendation, plugging LLM-generated user representations into NAML, NRMS, LSTUR, Fastformer, and MINER. These two papers are the cleanest existing demonstrations that an LLM-derived user profile can be consumed by a conventional ranker.
However, while textual representation of user profiles have been explored, the more common approach is where LLMs are used as an offline text encoder for items or interaction behaviors. LEARN uses a frozen LLM as a content encoder for items, and the user representation is built by aggregating the resulting item embeddings through a transformer module. BAHE does something similar at the behavior level, precomputing per-interaction embeddings via shallow LLM layers and aggregating with deeper layers into a user embedding. Neither produces a written user profile, but both validate the offline LLM into traditional ranker pattern at production scale. There is a structural reason this lane is more populated than the user profile lane: text profiles are easier for other LLMs to consume than for traditional rankers to consume.
Cross view fusion techniques
Cross view fusion of LLM and Collaborative Filtering based representations has two main approaches in the published literature: alignment between the two embedding spaces, and fusion that preserves what is distinct to each representation.
The alignment approach treats the semantic and collaborative views as the same underlying signal expressed in different coordinates, and learns a transformation that brings them into register. RLMRec implements this directly, with an MLP that maps collaborative embeddings into the language model’s semantic space and an in-batch contrastive loss that pulls matched user-item pairs together across the two spaces, producing a fused representation that outperforms either view alone. LLM-ESR runs semantic and collaborative branches in parallel, ablates each, and shows that the semantic branch is what holds up tail performance while the collaborative branch is what holds up head performance. The asymmetry is real and reproducible, and the alignment based architectures that dominate production deployments today were built around it.
The complementarity approach treats the two views as fundamentally different representations that should not be forced into a common space. “Rethinking Semantic-Collaborative Integration: Why Alignment Is Not Enough” evaluates this They present that across the Amazon Movies, Books, and Games datasets, only 6-14% percent of correct “hits” overlap between the two views, while 55-63% are unique to one view or the other. Furthermore, the semantic unique hits target items with up to 18% lower log-popularity, which formalizes the head vs. tail asymmetry. The practical consequence is that simple normalized concatenation of the two embeddings beats RLMRec, CARec, and AlphaRec by 13-42% across three datasets based on the results published in the paper.
What ties these techniques together
The three mechanisms address different gaps, but they share three structural properties worth naming:
The first is that the expensive language model work happens offline and is stored, not repeated per request. CoLaKG runs subgraph comprehension at build time and serves cached embeddings. LettinGo generates the profile offline; the cached profile is what downstream components consume at inference, whether that is a CTR ranker, a precomputed text-encoder embedding, or an LLM ranker prompt. Cross view fusion uses precomputed semantic and collaborative embeddings.
The language model is being used as a representation engine, with the actual recommendation task handled by a separate, much lighter model. Each mechanism uses the language model to produce an artifact that something else consumes: a similarity graph, a user profile, a fused embedding. The separation is what makes serving costs tractable, and it is also what allows teams to swap language models in and out without having to re-architect the recommender itself.
All three mechanisms populate different parts of the same underlying structure. The semantic similarity graph encodes entity level relationships, the user profile encodes individual behavioral patterns, and the cross view fusion encodes how an individual relates to those entities through the lens of cross-user signal.
In my opinion, none of them produces the full picture alone, and the same composition patterns accommodate additional artifacts discussed in literature: semantic IDs, domain specialized encoders, and parameter space adaptation of LLMs/SLMs. I will cover these in upcoming posts.
References
Entity comprehension and behavioral profiles
Cui et al., Comprehending Knowledge Graphs with LLMs for Recommendation (CoLaKG), 2025
Wang et al., User Profile Generation for Recommendation (LettinGo), 2025
Xi et al., Towards Open-World Recommendation with Knowledge Augmentation from LLMs (KAR), 2023
Liu et al., ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source LLMs, WSDM 2024
Jia et al., LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application, Kuaishou, AAAI 2025
Geng et al., Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors (BAHE), Alibaba, SIGIR 2024
Cross-view fusion
Ren et al., Representation Learning with LLMs for Recommendation (RLMRec), 2024
Liu et al., LLM Enhancement for Long-Tailed Sequential Recommendation (LLM-ESR), 2024
Wang et al., Rethinking Semantic-Collaborative Integration: Why Alignment Is Not Enough, 2025