SlayerLab/Slayer1B-1.08

Slayer1B 1.08

Arkadiusz Słota, Anna Sieczka, Piotr Styła / Fabryka AI (SlayerLab)

Experimental checkpoint — acceptance guard FAIL, 9 November 2026. Three preregistered endpoint gains are confirmed, but the ARC-Easy retention guard fails. This is not an accepted successor to 1.07. The new 1.08 is distinct from the earlier 1.08b experiment.

Public training trace.

Run objective

Train on as many approved token positions as the fixed compute budget permits, recording actual progress and expense at the spot prices observed during the run. Synthetic speed, real-data training and downstream evaluation are separate measurements. The expense report covers the billed session, including preparation and evaluation; steady-state cost per billion tokens is an extrapolation, not that bill.

Pre-registered evaluation and outcome

FINAL and 1.07 were evaluated on the same B200 hardware. Reported 95% CIs use paired bootstrap with 10,000 draws; one-sided p-values use Holm correction over three confirmatory endpoints. One run means sampling uncertainty, not training-seed variance. Mathematics uses base mode, three-shot, strict pass@1; its baseline exceeds the prespecified 5% validity threshold. Domain bpb uses matched fixed samples of at most two million native tokens, scored per UTF-8 byte.

Confirmatory endpoint	n	1.07	1.08	Difference [95% CI]	Holm p
Polish mathematics pass@1	300 items	6.33%	12.00%	+5.67 pp [+1.67, +9.67]	0.0035
Polish logic bpb	314 documents	0.78727	0.70895	−0.07832 [−0.09000, −0.06865]	0.0003
Within-source English technical VALID bpb	487 documents	0.66661	0.38549	−0.28112 [−0.34537, −0.23652]	0.0003

All three endpoints pass their confirmatory tests. Technical VALID shares source repositories with TRAIN; only exact text duplicates were removed. Its gain is within training-source projects, not evidence of generalization to unseen technical repositories.

Preregistered guard	n	1.07	1.08	Difference	Allowed difference	Outcome
ARC-Easy	2,376	58.67%	56.19%	−2.48 pp	≥ −1.00 pp	FAIL
BLiMP	67,000	77.90%	76.91%	−0.99 pp	≥ −1.00 pp	PASS, near boundary
ARC-Easy-PL	2,376	46.93%	47.14%	+0.21 pp	≥ −1.00 pp	PASS
MultiBLiMP-PL	3,272	98.84%	98.47%	−0.37 pp	≥ −1.00 pp	PASS
Polish H bpb	1,056 documents	0.75459	0.74835	−0.00624	≤ +0.005	PASS

The frozen rule makes any guard failure prevent acceptance; endpoint gains do not override it.

Descriptive diagnostic, not confirmatory	n	1.07	1.08
Held-out reasoning v2 pass@1	360	13.33%	9.44%
Arithmetic grid pass@1	500	12.20%	21.20%
WikiText-2 byte perplexity	1,292,008 bytes	2.1837	2.1939
Polish H hit-free subset bpb	711 documents	0.78425	0.77805

Model and starting point

The run initializes from SlayerLab/Slayer1B-1.07, revision a090b54e658fb57dc5166c1fcc90cc6bf5c80106. Its published card describes a Polish-dominant decoder-only model with 1,073,912,851 parameters, a 32,768-token v6 vocabulary, and final checkpoint 22,327. These are base-model properties.

The continuation preserves that architecture and tokenizer. Its task is causal text continuation, not instruction tuning. The objective is greater exposure to Polish analytical and English technical text while retaining general-language coverage.

Frozen launch recipe

Setting	Selected configuration
Initialization	Public 1.07 weights; fresh optimizer, rather than optimizer-state resume
Trainer and optimizer	Existing r24.2; Muon with AdamW
Batch and context	256 sequences × 4,096 token positions
Parallel layout	Eight GPU ranks; microbatch 8 × accumulation 4; activation checkpointing disabled
Schedule	300 warmup updates, then WSD with decay over the final 25% to a nonzero floor of 0.00006
Checkpoints	Every 250 updates and at the end; FINAL fixed in advance for evaluation

Training execution and cost

Observation	Value and scope
Hardware	Eight NVIDIA B200 GPUs, Nebius spot; USD 0.99 per GPU-hour
Completed updates	1,709
Processed positions	1,792,016,384 at the stated global batch; approximately 0.64 passes through the prepared input
Training window	58.6 minutes, including compilation and checkpoints; not the full billed session
Steady interval throughput	Approximately 572,000 positions/s; not positions divided by the whole training window
Training loss	Approximately 2.20 at update 10 to 1.99 at completion; not an evaluation score
Training GPU cost	Approximately USD 7.73, or USD 4.32 per billion processed positions, calculated from the observed window and spot rate
Attributed session cost	Approximately USD 21.5 through evaluation completion; preparation ≈11.2, training 7.73, evaluation/export ≈2.6. Provider-counter attribution, not a separate settled invoice; the VM continues under 1.09

Input capacity, processed positions, steady interval speed and the complete billed session remain separate quantities. The failed acceptance verdict is unchanged by permission to publish this experiment.

Training inputs

Input	Pinned revision
Slayer PL Logika 2B	e24c8b71526ca1dc56a280100f88c2f09a79ecdd
English Technical 1B	3b833d36bd3ba9aeecf46f39605c9ac849bb96a3
Slayer 1.07 replay	6a337bbfba518d88f14eb54f7d0a9b9f297df0d1

Replay uses pl_web, en_fineweb_edu and en_mix: previously seen material, not fresh web. Reference v2 contains 2,800,806,882 native token positions including EOS, after source selection and filtering; this is input capacity, not observed training exposure. The prepared TRAIN digest is b73a69eaf399bb594300e7d1914a967a38b80b5bc35505fbc6459a6a27be496f. GPL/GFDL and mixed GPL/LGPL sources are excluded. English validation/test remain outside TRAIN, and Polish held-out sections are grouped by source book.

Comparison limits

A replay-only continued-pretraining control is not budgeted for the priority run. A difference from 1.07 therefore combines data, fresh optimizer initialization and learning-rate decay: 1.07 was released from a constant-LR checkpoint without decay. It does not isolate the new datasets' effect. Inherited exposure in the base model is an additional limitation.

Evaluation records identify checkpoints, evaluator, split, sample count, prompting, normalization and test-read history.

Licensing and limitations

Weights are released under Apache-2.0, as 1.07; source corpora keep their own obligations, with attributions in the data card. FinePDFs compilation terms and its TDM rationale do not settle rights to every PDF. Book, translation and technical-source notices must be preserved where required.

A base language model can generate inaccurate, biased, offensive or personal information. Translated reasoning and technical text can contain errors; their presence does not demonstrate reasoning, technical reliability or instruction following. Privacy filtering is heuristic, and technical or tax material is not professional advice.