Publications

Transformer-based large language models (LLMs) encode many high-level concepts as linear directions in the latent activation space. Once identified, these directions support both measurement, the quantification of a concept's presence, and intervention, the steering of the model's behavior. In practice, however, a direction learned from one context often fails when applied to prompts from a new context. This poor transfer can have two distinct sources: spurious correlation between the concept label and dataset-specific features, and genuine heterogeneity in how the concept is encoded across contexts. Methods from out-of-distribution (OOD) literature can address the first, but worsen the second by discarding context-specific structure that may carry concept signal. We introduce Context-Aided Representation Extraction (CARE), which bridges OOD generalization and mechanistic interpretability. Given activations labeled with a concept and an environment, CARE jointly learns a shared direction, optimized to be invariant across environments, and orthogonal environment-specific residuals that capture how the concept varies by context. We evaluate CARE on subject--verb agreement, refusal of harmful prompts, and toxicity. CARE produces directions that measure concepts more reliably under distribution shift and in unseen environments and datasets than existing methods while remaining effective for intervention. Its decomposition also supports a swap-cost diagnostic that identifies when context-specific structure carries concept-relevant signal.

@inproceedings{care,
  title = {{Do You CARE to Generalize? Extracting Robust Concept Directions from LLMs}},
  author = {Sweta Karlekar and Claudia Shi and Aahlad Manas Puli and Carolina Zheng and Maggie Makar and John Bowlan and Michal Kucer and David Blei},
  year = {2026},
  booktitle = {NeurIPS 2026},
  note = {Accepted; forthcoming}
}

Many applications seek to optimize LLM outputs at test time by iteratively proposing, scoring, and refining candidates over a discrete output space. Existing methods use a calibrated scalar evaluator for the target objective to guide search, but for many tasks such scores are unavailable, too sparse, or unreliable. Pairwise comparisons, by contrast, are often easier to elicit, still provide useful signal on improvement directions, and can be obtained from the LLM itself without external supervision. Building on this observation, we introduce Duel-Evolve, an evolutionary optimization algorithm that replaces external scalar rewards with pairwise preferences elicited from the same LLM used to generate candidates. Duel-Evolve aggregates these noisy candidate comparisons via a Bayesian Bradley–Terry model, yielding uncertainty-aware estimates of candidate quality. These quality estimates guide allocation of the comparison budget toward plausible optima using Double Thompson Sampling, as well as selection of high-quality parents to generate improved candidates. We evaluate Duel-Evolve on MathBench, where it achieves 20 percentage points higher accuracy over existing methods and baselines, and on LiveCodeBench, where it improves over comparable iterative methods by over 12 percentage points. Notably, the method requires no reward model, no ground-truth labels during search, and no hand-crafted scoring function. Results show that pairwise self-preferences provide strong optimization signal for test-time improvement over large, discrete output spaces.

@inproceedings{duel-evolve,
  title = {{Duel-Evolve: Reward-Free Test-Time Scaling via LLM Self-Preferences}},
  author = {Sweta Karlekar and Carolina Zheng and Magnus Saebo and Nicolas Beltran-Velez and Shuyang Yu and John Bowlan and Michal Kucer and David Blei},
  year = {2026},
  booktitle = {ICLR 2026 Workshop on Recursive Self Improvement (RSI)},
  eprint = {2602.21585},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.21585}
}

Transformer-based large language models (LLMs) encode many high-level concepts as linear directions in the latent activation space. Isolating such concept directions enables both the measurement of said concepts and interventions to modify them. Directions extracted from one dataset, however, often fail to generalize to new contexts or environments. We introduce Context-Aided Representation Extraction (CARE), a method that decomposes concept representations into two complementary components: (1) an invariant direction capturing shared aspects of the concept across environments, and (2) environment-specific directions capturing contextual variations. We evaluate CARE on subject-verb agreement and harmful prompt detection across datasets with varying degrees of induced correlation between concept labels and environmental attributes. Our results demonstrate that CARE achieves robust performance on both measurement and intervention under distribution shift. A fitted CARE model reveals whether concepts admit unified representations or require multiple directions to fully capture their structure. With this, CARE provides insight into the complexity of how concepts are encoded in language models and provides measurement and intervention strategies that are robust to distribution shift.

@inproceedings{robust-representations,
  title = {{Extracting Representations in LLMs Robust to Distribution Shifts}},
  author = {Sweta Karlekar and Claudia Shi and Aahlad Manas Puli and Carolina Zheng and Maggie Makar and John Bowlan and Michal Kucer and David M. Blei},
  year = {2026},
  booktitle = {ICLR 2026 Workshop on Unifying Concept Representation Learning (UCRL)},
  url = {https://openreview.net/forum?id=KzH1cLVU1R}
}

Traditional topic models are effective at uncovering latent themes in large text collections. However, due to their reliance on bag-of-words representations, they struggle to capture semantically abstract features. While some neural variants use richer representations, they are similarly constrained by expressing topics as word lists, which limits their ability to articulate complex topics. We introduce Mechanistic Topic Models (MTMs), a class of topic models that operate on interpretable features learned by sparse autoencoders (SAEs). By defining topics over this semantically rich space, MTMs can reveal deeper conceptual themes with expressive feature descriptions. Moreover, uniquely among topic models, MTMs enable controllable text generation using topic steering vectors. To properly evaluate MTM topics against word list approaches, we propose topic judge, an LLM-based pairwise comparison evaluation framework. Across eight datasets, MTMs match or exceed traditional and neural baselines on coherence metrics, are consistently preferred by topic judge, and enable effective LLM steering.11 1 github.com/blei-lab/mechanistic-topic-models

@article{mechanistic-topic-models,
  title = {{Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders}},
  author = {Carolina Zheng and Nicolas Beltran-Velez and Sweta Karlekar and Claudia Shi and Achille Nazaret and Asif Mallik and Amir Feder and David M. Blei},
  year = {2026},
  journal = {Transactions of the Association for Computational Linguistics},
  eprint = {2507.23220},
  archivePrefix = {arXiv},
  note = {Presented at EMNLP 2026},
  url = {https://arxiv.org/abs/2507.23220}
}

This paper presents a method for estimating the hallucination rate for in-context learning (ICL) with generative AI. In ICL, a conditional generative model (CGM) is prompted with a dataset and a prediction question and asked to generate a response. One interpretation of ICL assumes that the CGM computes the posterior predictive of an unknown Bayesian model, which implicitly defines a joint distribution over observable datasets and latent mechanisms. This joint distribution factorizes into two components: the model prior over mechanisms and the model likelihood of datasets given a mechanism. With this perspective, we define a hallucination as a generated response to the prediction question with low model likelihood given the mechanism. We develop a new method that takes an ICL problem and estimates the probability that a CGM will generate a hallucination. Our method only requires generating prediction questions and responses from the CGM and evaluating its response log probability. We empirically evaluate our method using large language models for synthetic regression and natural language ICL tasks.

@inproceedings{hallucination-rate,
  title = {{Estimating the Hallucination Rate of Generative AI}},
  author = {Andrew Jesson and Nicolas Beltran-Velez and Quentin Chu and Sweta Karlekar and Jannik Kossen and Yarin Gal and John P. Cunningham and David Blei},
  year = {2024},
  booktitle = {NeurIPS 2024},
  url = {https://proceedings.neurips.cc/paper_files/paper/2024/hash/3791f5fc0e8e43730466afd2bcdb7493-Abstract-Conference.html}
}

With the recent rise of #MeToo, an increasing number of personal stories about sexual harassment and sexual abuse have been shared online. In order to push forward the fight against such harassment and abuse, we present the task of automatically categorizing and analyzing various forms of sexual harassment, based on stories shared on the online forum SafeCity. For the labels of groping, ogling, and commenting, our single-label CNN-RNN model achieves an accuracy of 86.5%, and our multi-label model achieves a Hamming score of 82.5%. Furthermore, we present analysis using LIME, first-derivative saliency heatmaps, activation clustering, and embedding visualization to interpret neural model predictions and demonstrate how this helps extract features that can help automatically fill out incident reports, identify unsafe areas, avoid unsafe practices, and ‘pin the creeps’.

@inproceedings{safecity,
  title = {{SafeCity: Understanding Diverse Forms of Sexual Harassment Personal Stories}},
  author = {Sweta Karlekar and Mohit Bansal},
  year = {2018},
  booktitle = {EMNLP 2018},
  url = {https://arxiv.org/pdf/1809.04739.pdf}
}

Alzheimer’s disease (AD) is an irreversible and progressive brain disease that can be stopped or slowed down with medical treatment. Language changes serve as a sign that a patient’s cognitive functions have been impacted, potentially leading to early diagnosis. In this work, we use NLP techniques to classify and analyze the linguistic characteristics of AD patients using the DementiaBank dataset. We apply three neural models based on CNNs, LSTM-RNNs, and their combination, to distinguish between language samples from AD and control patients. We achieve a new independent benchmark accuracy for the AD classification task. More importantly, we next interpret what these neural models have learned about the linguistic characteristics of AD patients, via analysis based on activation clustering and first-derivative saliency techniques. We then perform novel automatic pattern discovery inside activation clusters, and consolidate AD patients’ distinctive grammar patterns. Additionally, we show that first derivative saliency can not only rediscover previous language patterns of AD patients, but also shed light on the limitations of neural models. Lastly, we also include analysis of gender-separated AD data.

@inproceedings{alzheimers-language,
  title = {{Detecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models}},
  author = {Sweta Karlekar and Tong Niu and Mohit Bansal},
  year = {2018},
  booktitle = {NAACL 2018},
  url = {https://arxiv.org/pdf/1804.06440.pdf}
}

The detection and classification of domestic abuse stories shared online has ever-increasing importance in today’s social activism sphere. With massive numbers of stories shared, automatic detection can aggregate stories from around the internet and help push forward the fight against domestic abuse from a social campaign to social change. We develop CNN, LSTM-RNN, and CNN-LSTM neural models to detect domestic abuse stories in the Reddit Domestic Abuse dataset. We achieved 95.8% accuracy in classifying posts as containing abuse stories versus not containing abuse stories, outperforming the current state-of-the-art. More importantly, we next present sentiment-only classification feasibility as well as interpretable and explainable analysis of the neural model’s predictions using activation clustering techniques to automatically discover linguistic features.

@inproceedings{metoo-abuse-stories,
  title = {{\#MeToo: Neural Detection and Explanation of Language in Personal Abuse Stories}},
  author = {Sweta Karlekar and Mohit Bansal},
  year = {2018},
  booktitle = {NAACL WiNLP 2018},
  url = {https://www.cs.unc.edu/~mbansal/papers/winlp2018-karlekar-bansal.pdf}
}

The US Army has an increasing focus on using automated remote sensing techniques with commercial multispectral imagery (MSI) to map urban and peri-urban agricultural and vegetative features; however, similar spectral profiles between trees (i.e., forest canopy) and other vegetation result in confusion between these cover classes. Established vegetation indices, like the Normalized Difference Vegetation Index (NDVI), are typically not effective in reliably differentiating between trees and other vegetation. Previous research in tree mapping has included integration of hyperspectral imagery (HSI) and LiDAR for tree detection and species identification, as well as the use of MSI to distinguish tree crowns from non-vegetated features. This project developed a straightforward method to model and also mask out trees from eight-band WorldView-2 (1.85 meter x 1.85 meter resolution at nadir) satellite imagery at the Beltsville Agricultural Research Center in Beltsville, MD spanning 2012 - 2015. The study site included tree cover, a range of agricultural and vegetative cover types, and urban features. The modeling method exploits the product of the red and red edge bands and defines accurate thresholds between trees and other land covers. Results show this method outperforms established vegetation indices including the NDVI, Soil Adjusted Vegetation Index, Normalized Difference Water Index, Simple Ratio, and Normalized Difference Red Edge Index in correctly masking trees while preserving the other information in the imagery. This method is useful when HSI and LiDAR collection are not possible or when using archived MSI.

@inproceedings{multispectral-tree-masking,
  title = {{Developing a Method to Mask Trees in Commercial Multispectral Imagery}},
  author = {S. J. Becker and C. S. T. Daughtry and D. Jain and S. S. Karlekar},
  year = {2015},
  booktitle = {American Geophysical Union Fall Meeting 2015},
  url = {http://adsabs.harvard.edu/abs/2015AGUFM.B43C0569B}
}