About The Meeting
The New England Mechanistic Interpretability (NEMI) workshop aims to bring together academic and industry researchers from the New England and surrounding regions who are advancing the field of mechanistic interpretability in machine learning systems. The workshop will serve as a forum to share recent progress, challenges, and ideas in reverse-engineering, circuit analysis, and other techniques that seek to understand how models compute internally. NEMI seeks to foster a participatory and collaborative environment where researchers at all levels—including graduate students, early-career scientists, and established experts—can engage in discussion and feedback. We particularly encourage submissions from rising researchers currently enrolled in graduate programs at New England-based universities. Topics of interest include, but are not limited to, interpretability of neural circuits, activation patching, probe-based analysis, feature attribution methods, model simplification, scaling laws and applications as applied to interpretability. The workshop will feature a dynamic program including invited keynote speakers, selected oral presentations, interactive poster sessions, and opportunities for open discussion.
Livestream
Schedule
Time |
Event |
Venue |
| 08:30 AM - 9:00 AM | Setup time for Poster Session 1 | CSC Second Floor Suites |
| 09:00 AM - 09:30 AM | Breakfast & Registration | CSC Ballroom |
| 09:30 AM - 09:40 AM | Opening Remarks | CSC Ballroom |
| 09:40 AM - 10:00 AM | Keynote 1: Lee Sharkey: "Mech Interp: Where should we go from here?" | CSC Ballroom |
| 10:00 AM - 10:10 AM | Student Talk 1: Amil Dravid"Vision Transformers Don't Need Trained Registers" | CSC Ballroom |
| 10:10 AM - 10:20 AM | Student Talk 2: Andrew Lee"Shared Global and Local Geometry of Language Model Embeddings" | CSC Ballroom |
| 10:20 AM - 10:40 AM | Keynote 2: Tamar Rott Shaham: "Can Language Models Interpret Humans?" | CSC Ballroom |
| 10:40 AM - 11:45 AM | Round 1 of LLM Roundtables | CSC Ballroom |
| 11:45 AM - 1:00 PM | Poster Session 1 + Coffee Break | Poster Session at CSC Second Floor Suites, Coffee Break at CSC Ballroom |
| 01:00 PM - 02:00 PM | Lunch + Group Photo + Continued LLM Roundtables | CSC Ballroom |
| 01:45 PM - 02:00 PM | Setup time for Poster Session 2 | CSC Second Floor Suites |
| 02:00 PM - 03:15 PM | Poster Session 2 | CSC Second Floor Suites |
| 03:15 PM - 04:15 PM | NDIF/NNsight | CSC Ballroom |
| 04:15 PM - 04:25 PM | Coffee Break | CSC Ballroom |
| 04:25 PM - 04:45 PM | Keynote 3: Aaron Mueller: "Beyond Human Concepts: Evaluating and Applying Unsupervised Interpretability" | CSC Ballroom |
| 04:45 PM - 04:55 PM | Student Talk 3: Helena Casademunt"Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning" | CSC Ballroom |
| 04:55 PM - 05:05 PM | Student Talk 4: Michael Hanna"Circuit-tracer: A New Library for Feature Circuits" | CSC Ballroom |
| 05:05 PM - 05:25 PM | Keynote 4: Ekdeep Singh Lubana: "Looking Inwards: Implicit Assumptions Formally Constrain Mechanistic Interpretability" | CSC Ballroom |
| 05:25 PM - 05:55 PM | Panel Discussion: "Bridging the Gap: From Lowest-Level Mechanisms to High-Level Behaviors" | CSC Ballroom |
| 05:55 PM - 06:00 PM | Closing Remarks | CSC Ballroom |
| 06:00 PM+ | Optional Social | CSC Ballroom |
Registration
Register by August 4, 2025.
Submission Guidelines
We invite submissions for the NEMI 2025 workshop, a one-day event dedicated to exploring the latest developments in mechanistic interpretability research. We welcome submissions on all aspects of interpretability. Some of them will be selected for oral presentations and the remaining will be presented as posters. We encourage submissions from rising researchers who are enrolled in graduate programs at universities located in the New England region.
Dates
- Registration deadline: August 4, 2025, (AOE)
- Submission deadline: August 9, 2025, (AOE)
- Notification: August 12, 2025, (AOE)
- Event Date: August 22, 2025
Keynote Speakers
Aaron
Mueller
Boston University
Ekdeep Singh Lubana
Harvard UniversityStudent Organizers
Koyena Pal
Northeastern University
Alex Loftus
Northeastern University
Emma Bortz
Northeastern University
Aruna Sankaranarayanan
MITSenior Program Committee
David Bau
Northeastern University
Jacob Andreas
MIT
Himabindu Lakkaraju
Harvard University
Najoung Kim
Boston UniversityLogistics Support
Heather Sciacca
Northeastern UniversityVenue
Curry Student Center, 360 Huntington Ave, Boston, MA 02115
Sponsors
We thank our sponsors for their generous support!
Amil Dravid
UC Berkeley
Vision Transformers Don't Need Trained Registers
Abstract
We investigate the mechanism underlying a previously identified phenomenon in Vision Transformers – the emergence of high-norm tokens that lead to noisy attention maps (Darcet et al., 2024). We observe that in multiple models (e.g., CLIP, DINOv2), a sparse set of neurons is responsible for concentrating high-norm activations on outlier tokens, leading to irregular attention patterns and degrading downstream visual processing. While the existing solution for removing these outliers involves retraining models from scratch with additional learned register tokens, we use our findings to create a training-free approach to mitigate these artifacts. By shifting the high-norm activations from our discovered register neurons into an additional untrained token, we can mimic the effect of register tokens on a model already trained without registers. We demonstrate that our method produces cleaner attention and feature maps, enhances performance over base models across multiple downstream visual tasks, and achieves results comparable to models explicitly trained with register tokens. We then extend test-time registers to off-the-shelf vision-language models to improve their interpretability. Our results suggest that test-time registers effectively take on the role of register tokens at test-time, offering a training-free solution for any pre-trained model released without them.
Michael Hanna
University of Amsterdam
Circuit-tracer: A New Library for Feature Circuits
Abstract
Feature circuits aim to shed light on LLM behavior by identifying the features that are causally responsible for a given LLM output, and connecting them into a directed graph, or circuit, that explains how both each feature and each output arose. However, performing circuit analysis is challenging: the tools for finding, visualizing, and verifying feature circuits are complex and spread across multiple libraries. To solve this, we introduce circuit-tracer, an open-source library for identifying feature circuits. circuit-tracer provides an integrated pipeline for finding, visualizing, annotating, and performing interventions on such feature circuits, tested on models with up to 14 billion parameters. We make circuit-tracer available to both developers and end users, via integration with tools such as Neuronpedia, which provides a user-friendly interface.
Helena Casademunt
Harvard University
Steering Out-of-Distribution Generalization with Concept Ablation Fine-Tuning
Abstract
Fine-tuning large language models (LLMs) can lead to unintended out-of-distribution generalization. Standard approaches to this problem rely on modifying training data, for example by adding data that better specify the intended generalization. However, this is not always practical. We introduce Concept Ablation Fine-Tuning (CAFT), a technique that leverages interpretability tools to control how LLMs generalize from fine-tuning, without needing to modify the training data or otherwise use data from the target distribution. Given a set of directions in an LLM's latent space corresponding to undesired concepts, CAFT works by ablating these concepts with linear projections during fine-tuning, steering the model away from unintended generalizations. We successfully apply CAFT to three fine-tuning tasks, including emergent misalignment, a phenomenon where LLMs fine-tuned on a narrow task generalize to give egregiously misaligned responses to general questions. Without any changes to the fine-tuning data, CAFT reduces misaligned responses by 10x without degrading performance on the training distribution. Overall, CAFT represents a novel approach for steering LLM generalization without modifying training data.
Andrew Lee
Harvard University
Shared Global and Local Geometry of Language Model Embeddings
Abstract
Researchers have recently suggested that models share common representations. In our work, we find numerous geometric similarities across the token embeddings of large language models. First, we find ``global'' similarities: token embeddings often share similar relative orientations. Next, we characterize local geometry in two ways: (1) by using Locally Linear Embeddings, and (2) by defining a simple measure for the intrinsic dimension of each embedding. Both characterizations allow us to find local similarities across token embeddings. Additionally, our intrinsic dimension demonstrates that embeddings lie on a lower dimensional manifold, and that tokens with lower intrinsic dimensions often have semantically coherent clusters, while those with higher intrinsic dimensions do not. Based on our findings, we introduce EMB2EMB, a simple application to linearly transform steering vectors from one language model to another, despite the two models having different dimensions.
Tamar Rott Shaham
MIT
Can Language Models Interpret Humans?
Bio
Tamar Rott Shaham is an incoming assistant professor at the Faculty of Mathematics and Computer Science at Weizmann. She is currently a postdoctoral researcher at MIT CSAIL in Antonio Torralba’s lab. She received her PhD from the ECE Technion, supervised by Prof. Tomer Michaeli. Tamar has received several awards, including the ICCV 2019 Best Paper Award (Marr Prize), the Google WTM Scholarship, the Adobe Research Fellowship, the Rothchild Postdoctoral Fellowship, the Vatat-Zuckerman Postdoctoral Scholarship and the Schmidt Postdoctoral Award.
Ekdeep Singh Lubana
Harvard
Looking Inwards: Implicit Assumptions Formally Constrain Mechanistic Interpretability
Bio
Ekdeep Singh Lubana is a research fellow at the CBS-NTT Program in Physics of Intelligence at Harvard University. His research focuses on developing predictive theories for phenomena observed during evaluation and deployment of neural networks—e.g., emergent learning curves, in-context learning novel concept structures, and effects of post-training. His recent work has focused on making explicit assumptions underlying tools popularly used for neural network interpretability. Ekdeep completed a PhD co-affiliated at University of Michigan and Center for Brain Science, Harvard.
Lee Sharkey
Goodfire
Mech Interp: Where should we go from here?
Bio
Lee is a Principal Investigator at Goodfire (London) where his team work on developing new interpretability methods. His previous work includes Attribution-based Parameter Decomposition and sparse autoencoders as a solution to superposition. Prior to Goodfire, Lee co-founded Apollo Research and led the interpretability team there.
Aaron Mueller
Boston University
Beyond Human Concepts: Evaluating and Applying Unsupervised Interpretability
Bio
Aaron Mueller is an assistant professor in the Department of Computer Science at Boston University. His research centers on developing methods and evaluations for mechanistic interpretability inspired by causal and linguistic principles, and applying these for precise model control and improved efficiency. He completed a Ph.D. at Johns Hopkins University and was a Zuckerman postdoctoral fellow at Northeastern and the Technion. His work has been published in ML and NLP venues (such as ICLR, ACL, and EMNLP) and has won awards at TMLR and ACL. He has recently been featured in the New York Times (2023) and IEEE Spectrum (2024) as an organizer of the BabyLM Challenge.