Sweta Karlekar

Transformer-based large language models (LLMs) encode many high-level concepts as linear directions in activation space. Such concept directions support measurement, quantifying a concept’s presence, and intervention, steering the model’s behavior. In practice, a direction learned in one context often fails in another, such as a new prompt style, source dataset, or target group. We treat these contexts as environments and cast concept-direction extraction as a multi-environment out-of-distribution (OOD) generalization problem. This framing exposes two reasons a direction can fail to transfer: it may pick up features that merely correlate with the concept in the training environments, or the concept may genuinely be encoded differently across environments. Invariance methods from the OOD literature address the first problem but not the second, since forcing a single invariant direction discards context-specific structure that can carry the concept. We introduce Context-Aided Representation Extraction (CARE), which addresses both. CARE decomposes each concept direction into a shared direction, common to all environments and optionally regularized for invariance, plus an orthogonal environment-specific residual. We evaluate CARE on subject–verb agreement, refusal of harmful prompts, and toxicity across four LLMs. Under shifts in the environment–label relationship, CARE measures concepts more accurately than standard and OOD baselines while remaining effective for intervention. The shared direction transfers to environments and datasets never seen in training, and the residuals can improve intervention when the environment is known. In a semi-synthetic study with known concept structure, CARE’s decomposition recovers the shared and context-specific components of the concept.

Many applications seek to optimize LLM outputs at test time by iteratively proposing, scoring, and refining candidates over a discrete output space. Existing methods use a calibrated scalar evaluator for the target objective to guide search, but for many tasks such scores are unavailable, too sparse, or unreliable. Pairwise comparisons, by contrast, are often easier to elicit, still provide useful signal on improvement directions, and can be obtained from the LLM itself without external supervision. Building on this observation, we introduce Duel-Evolve, an evolutionary optimization algorithm that replaces external scalar rewards with pairwise preferences elicited from the same LLM used to generate candidates. Duel-Evolve aggregates these noisy candidate comparisons via a Bayesian Bradley–Terry model, yielding uncertainty-aware estimates of candidate quality. These quality estimates guide allocation of the comparison budget toward plausible optima using Double Thompson Sampling, as well as selection of high-quality parents to generate improved candidates. We evaluate Duel-Evolve on MathBench, where it achieves 20 percentage points higher accuracy over existing methods and baselines, and on LiveCodeBench, where it improves over comparable iterative methods by over 12 percentage points. Notably, the method requires no reward model, no ground-truth labels during search, and no hand-crafted scoring function. Results show that pairwise self-preferences provide strong optimization signal for test-time improvement over large, discrete output spaces.

Traditional topic models are effective at uncovering latent themes in large text collections. However, due to their reliance on bag-of-words representations, they struggle to capture semantically abstract features. While some neural variants use richer representations, they are similarly constrained by expressing topics as word lists, which limits their ability to articulate complex topics. We introduce Mechanistic Topic Models (MTMs), a class of topic models that operate on interpretable features learned by sparse autoencoders (SAEs). By defining topics over this semantically rich space, MTMs can reveal deeper conceptual themes with expressive feature descriptions. Moreover, uniquely among topic models, MTMs enable controllable text generation using topic steering vectors. To properly evaluate MTM topics against word list approaches, we propose topic judge, an LLM-based pairwise comparison evaluation framework. Across eight datasets, MTMs match or exceed traditional and neural baselines on coherence metrics, are consistently preferred by topic judge, and enable effective LLM steering.11 1 github.com/blei-lab/mechanistic-topic-models

This paper presents a method for estimating the hallucination rate for in-context learning (ICL) with generative AI. In ICL, a conditional generative model (CGM) is prompted with a dataset and a prediction question and asked to generate a response. One interpretation of ICL assumes that the CGM computes the posterior predictive of an unknown Bayesian model, which implicitly defines a joint distribution over observable datasets and latent mechanisms. This joint distribution factorizes into two components: the model prior over mechanisms and the model likelihood of datasets given a mechanism. With this perspective, we define a hallucination as a generated response to the prediction question with low model likelihood given the mechanism. We develop a new method that takes an ICL problem and estimates the probability that a CGM will generate a hallucination. Our method only requires generating prediction questions and responses from the CGM and evaluating its response log probability. We empirically evaluate our method using large language models for synthetic regression and natural language ICL tasks.