This research introduces a framework for mapping hidden-state attractors in TinyLlama, aiming to build a runtime map of LLM dynamics during inference. The focus is on tracking the model's internal state transitions rather than explaining semantics directly. The approach explores linear subspaces and dynamics within the model's hidden layers to understand state evolution. This work contributes to interpretability research by offering a novel perspective on LLM behavior through runtime trajectory analysis.

Read original