Researchers investigated the training timelines and dataset compositions of frontier large language models through "probing" techniques, including Wikipedia-based historical quizzes and self-identification queries. By analyzing error rate timelines across 8-way multiple-choice quizzes on daily facts, the study estimated pre-training checkpoint dates. Findings suggest Anthropic’s Opus 4.7 and subsequent models share a training run with an effective knowledge cutoff around late December 2025. In contrast, OpenAI’s GPT-5.6 family appears to stem from a separate checkpoint completed near late February 2026. Interestingly, while Opus 5 claims a May 2026 cutoff, probing reveals its knowledge density aligns with January 2026 models. The study also utilized "Self-reported Dates" and "Self-reported Identity" to observe post-training patterns. Vertical shifts in self-reported dates suggest active post-training on recency-biased datasets or distillation from older models. Furthermore, identity probing revealed that models often reflect the era of their training data; for example, Anthropic’s Sonnet 5 occasionally self-identifies as GPT-4, suggesting that historical user chat sessions from ChatGPT.com may be present in Anthropic's training mixtures. When prompted to emulate specific models, Claude reproduced OpenAI’s quirks at a 68% rate, while OpenAI models only matched Claude’s quirks at 8%.
Read original
hackernews