- Rewrite README into a structured Hugging Face model card with YAML
metadata (license, tags, pipeline_tag), badges, Model Summary/Details,
Intended Use, Limitations, and an FAQ
- Fix usage instructions and directory tree (use model_updated.pt +
tokenizer/; historical split model is under experiment_data/historical_version/)
- Align all figures with the PLOS ONE paper (16.09M sequences / 17.4B nt,
350M params, model dimension 1280, 1024-token context, BPE vocab 1024)
- Add BibTeX / plain-text citation and paper + preprint DOI links
- Add CITATION.cff and .gitignore
Fix finetuning RuntimeError and add pyproject.toml
- Fix block_size mismatch causing "params, grads, exp_avgs, and
exp_avg_sqs must have same dtype" RuntimeError when finetuning with
block_size different from checkpoint. Set finetuning config
block_size=1024 to match pretrained model_updated.pt.
- Add warning when config model parameters differ from checkpoint to
help users diagnose shape mismatch issues.
- Fix GradScaler logic: enabled=(dtype=='float32') was inverted,
corrected to enabled=(dtype=='float16').
- Add pyproject.toml with complete dependency list (transformers, tqdm,
tiktoken were missing) and project metadata.
- Add description about newer model in README.md (991fc3f484aaad404966f65fb619de239434c901)
Co-authored-by: YichongEricZhao <Eric2333@users.noreply.huggingface.co>
