Evaluation datasets maintained by EleutherAI
AI & ML interests
Large language models, scaling laws, AI Alignment, democratization of DL
Recent Activity
View all activity
Papers
Agent Memory Is a Surface for Endogenous Authorization Laundering
Soohak: A Mathematician-Curated Benchmark for Evaluating Research-level Math Capabilities of LLMs
Pythia is the first LLM suite designed specifically to enable scientific research on LLMs. To learn more see https://github.com/EleutherAI/pythia
-
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
Paper • 2304.01373 • Published • 9 -
EleutherAI/pythia-14m-deduped
Text Generation • 39.2M • Updated • 32.6k • 31 -
EleutherAI/pythia-70m
95.6M • Updated • 541k • 86 -
EleutherAI/pythia-160m
Text Generation • 0.2B • Updated • 3.46M • 45
Leave-1%-out retrained models, per-query loss changes, and datasets for computing LDS.
T5 trained on the Pile with Llama Tokenizer
T5x Checkpoints for Pile-T5
This collection contains the model and data artifacts from O'Brien et al. (2025). https://deepignorance.ai
-
Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
Paper • 2508.06601 • Published • 7 -
EleutherAI/deep-ignorance-unfiltered
Text Generation • 7B • Updated • 1.27k • 6 -
EleutherAI/deep-ignorance-e2e-strong-filter
Text Generation • 7B • Updated • 5.97k • 1 -
EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal
Text Generation • 7B • Updated • 41 • 1
A collection of datasets and finetuned models that can be used for Eliciting Latent Knowledge (ELK) research.
Models and datasets for benchmarking ELK methods. NOTE: These models correspond to our first arxiv release. See our newer Quirky Models collection.
SAEs are tools for understanding the internal representations of neural networks. These can be loaded using https://github.com/EleutherAI/sae
Evaluation datasets maintained by EleutherAI
This collection contains the model and data artifacts from O'Brien et al. (2025). https://deepignorance.ai
-
Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs
Paper • 2508.06601 • Published • 7 -
EleutherAI/deep-ignorance-unfiltered
Text Generation • 7B • Updated • 1.27k • 6 -
EleutherAI/deep-ignorance-e2e-strong-filter
Text Generation • 7B • Updated • 5.97k • 1 -
EleutherAI/deep-ignorance-strong-filter-pt-weak-filter-anneal
Text Generation • 7B • Updated • 41 • 1
Pythia is the first LLM suite designed specifically to enable scientific research on LLMs. To learn more see https://github.com/EleutherAI/pythia
-
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
Paper • 2304.01373 • Published • 9 -
EleutherAI/pythia-14m-deduped
Text Generation • 39.2M • Updated • 32.6k • 31 -
EleutherAI/pythia-70m
95.6M • Updated • 541k • 86 -
EleutherAI/pythia-160m
Text Generation • 0.2B • Updated • 3.46M • 45
Leave-1%-out retrained models, per-query loss changes, and datasets for computing LDS.
A collection of datasets and finetuned models that can be used for Eliciting Latent Knowledge (ELK) research.
T5 trained on the Pile with Llama Tokenizer
Models and datasets for benchmarking ELK methods. NOTE: These models correspond to our first arxiv release. See our newer Quirky Models collection.
T5x Checkpoints for Pile-T5
SAEs are tools for understanding the internal representations of neural networks. These can be loaded using https://github.com/EleutherAI/sae