Dong-Kyum Kim

dong-kyum.kim [at] mpi-sp.org

I am a postdoc in the Data Science for Humanity Group at the Max Planck Institute for Security and Privacy (MPI-SP) in Germany, working with Meeyoung Cha. I study how large language models (LLMs) encode knowledge in their parameters & activations and how they decode it during inference. Topics I work on:

Memory, unlearning, and editing: localizing the parameters that hold a specific memory (an engram), and testing whether it can be truly erased or changed without damaging unrelated abilities, and whether the change generalizes beyond the targeted memory (AI Engram, ICML 2026 Oral; Erase or Hide?, ICLR 2026; Moir, NeurIPS 2026)
Out-of-context reasoning: how LLMs reason without verbalizing their thoughts to infer facts their training data implies but never states. For example, why models trained on “A is B” often fail to infer “B is A” (the reversal curse), and which internal representations fix this (Bilinear representation mitigates reversal curse, ICLR 2026)

Previously, I did my PhD in physics at the Korea Advanced Institute of Science and Technology (KAIST), where I worked with Hawoong Jeong on using deep learning to study nonequilibrium statistical physics.

I am currently on the job market and seeking research positions in academia and industry.

news

Sep 30, 2026	Moir, on preserving reasoning under knowledge editing by estimating the preservation covariance from the model’s own samples, was accepted at NeurIPS 2026.
Jul 24, 2026	Presented AI research using agents and notebooks as one of five speakers at Spark(l)ing Science, MPI-SP, in the session AI Agents, Show and Tell: Real Workflows, Real Speedups.
May 24, 2026	AI Engram, on identifying memory traces in neural networks, was selected for an Oral at ICML 2026 (168 orals, top 0.7% of submissions).
May 13, 2026	Recognized as a Gold Reviewer for ICML 2026, placing among the conference’s top reviewers.

selected papers

AI Engram: In Search of Memory Traces in Artificial Intelligence

Jea Kwon, Dong-Kyum Kim, Jiwon Kim, Yonghyun Kim, Woong Kook, and Meeyoung Cha

ICML 2026Oral26.6% acceptance, Oral top 0.7%

Abstract arXiv

Memory formation is fundamental to intelligence, yet whether deep neural networks preserve identifiable memory traces—analogous to biological memory units—remains an open question. This work introduces a geometric framework to identify such "AI engrams," by formalizing the neuroscientific criteria of specificity, reactivation, sufficiency, and necessity into a constrained inverse problem. We derive a closed-form estimator that isolates individual memory traces from globally entangled parameters. Theoretical analysis reveals that this biologically-derived solution corresponds to a natural gradient update on the parameter manifold. AI engrams enable surgical manipulation of learned knowledge: any subset of memories can be composed or erased through linear arithmetic, without iterative optimization. Experiments ranging from simple MLPs to LLMs demonstrate the causal validity and substantial scalability of AI engrams. Together, these results bridge theories of biological memory and artificial representation learning, offering geometric insight into how deep networks simultaneously support functional specificity within distributed storage.

Bilinear representation mitigates reversal curse and enables consistent model editing

Dong-Kyum Kim, Minsung Kim, Jea Kwon, Nakyeong Yang, and Meeyoung Cha

ICLR 202627.0% acceptance

Abstract arXiv Paper

The reversal curse—a language model’s inability to infer an unseen fact "B is A" from a learned fact "A is B"—is widely considered a fundamental limitation. We show that this is not an inherent failure but an artifact of how models encode knowledge. Our results demonstrate that training from scratch on synthetic relational knowledge graphs leads to the emergence of a bilinear relational structure within the models’ hidden representations. This structure alleviates the reversal curse and facilitates inference of unseen reverse facts. Crucially, this bilinear geometry is foundational for consistent model editing: updates to a single fact propagate correctly to its reverse and logically dependent relations. In contrast, models lacking this representation suffer from the reversal curse and fail to generalize model edits, leading to logical inconsistencies. Our results establish that training on a relational knowledge dataset induces the emergence of bilinear internal representations, which in turn support language models in behaving in a logically consistent manner after editing. This suggests that the efficacy of language model editing depends not only on the choice of algorithm but on the underlying representational geometry of the knowledge itself.

Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearning

Nakyeong Yang, Dong-Kyum Kim, Jea Kwon, Minsung Kim, Kyomin Jung, and Meeyoung Cha

ICLR 202627.0% acceptance

Abstract arXiv Paper

Large language models trained on web-scale data can memorize private or sensitive knowledge, raising significant privacy risks. Although some unlearning methods mitigate these risks, they remain vulnerable to "relearning" during subsequent training, allowing a substantial portion of forgotten knowledge to resurface. In this paper, we show that widely used unlearning methods cause shallow alignment: instead of faithfully erasing target knowledge, they generate spurious unlearning neurons that amplify negative influence to hide it. To overcome this limitation, we introduce Ssiuu, a new class of unlearning methods that employs attribution-guided regularization to prevent spurious negative influence and faithfully remove target knowledge. Experimental results confirm that our method reliably erases target knowledge and outperforms strong baselines across two practical retraining scenarios: (1) adversarial injection of private data, and (2) benign attack using an instruction-following benchmark. Our findings highlight the necessity of robust and faithful unlearning methods for safe deployment of language models.

Transformer as a hippocampal memory consolidation model based on NMDAR-inspired nonlinearity

Dong-Kyum Kim, Jea Kwon, Meeyoung Cha, and C. Justin Lee

NeurIPS 202326.1% acceptance

Abstract Paper

The hippocampus plays a critical role in learning, memory, and spatial representation, processes that depend on the NMDA receptor (NMDAR). Here we build on recent findings comparing deep learning models to the hippocampus and develop a new nonlinear activation function based on NMDAR dynamics. We find that NMDAR-like nonlinearity is essential for shifting short-term working memory into long-term reference memory in transformers, thus enhancing a process that resembles memory consolidation in the mammalian brain. We design a navigation task assessing these two memory functions and show that manipulating the activation function (i.e., mimicking the Mg^2+-gating of NMDAR) disrupts long-term memory processes. Our experiments suggest that place cell-like functions and reference memory reside in the feed-forward network layer of transformers and that nonlinearity drives these processes. We discuss the role of NMDAR-like nonlinearity in establishing this striking resemblance between transformer architecture and hippocampal spatial representation.

Learning Entropy Production via Neural Networks

Dong-Kyum Kim, Youngkyoung Bae, Sangyun Lee, and Hawoong Jeong

PRL 2020IF 9.0

Abstract arXiv Paper

This Letter presents a neural estimator for entropy production (NEEP), that estimates entropy production (EP) from trajectories of relevant variables without detailed information on the system dynamics. For steady state, we rigorously prove that the estimator, which can be built up from different choices of deep neural networks, provides stochastic EP by optimizing the objective function proposed here. We verify the NEEP with the stochastic processes of the bead spring and discrete flashing ratchet models and also demonstrate that our method is applicable to high-dimensional data and can provide coarse-grained EP for Markov systems with unobservable states.