The 0xSero/ai-data-extraction toolkit is a comprehensive Python-based utility designed to harvest high-fidelity datasets from local AI coding assistants for machine learning training. The tool automates the discovery and extraction of conversation histories from several major platforms, including Claude (Code and Desktop), Cursor, Codex, Trae, Windsurf, Continue AI, Google Gemini CLI, and OpenCode. It targets diverse storage formats, including SQLite databases (such as Cursor's state.vscdb and cursorDiskKV), JSONL session files, and Tauri .dat files across macOS, Linux, and Windows. Beyond simple chat logs, the extraction process captures critical technical context: user and assistant messages, multi-file code snippets, line number ranges, tool execution results, and code diffs/suggested edits. For the Cursor CLI, the tool implements a DFS-walking algorithm to reconstruct ordered message sequences from content-addressed SQLite blob stores using SHA256 hex IDs. Extracted data is organized into timestamped JSONL files containing rich metadata like model names, timestamps, and project paths. Additionally, the toolkit includes a privacy filtering module utilizing the openai/privacy-filter model to redact sensitive information and a utility to synthesize "Agent Skills" from the extracted corpus using OpenAI-compatible APIs.

Read original