tadad/midf-sanskrit-ocr

MIDF Sanskrit OCR v1.0

The default pipeline combines a Kraken BLLA v1.0 topline segmenter, Kraken polygon tracing, and a PP-OCRv6 v1.0 Sanskrit line recognizer. Both models train on 1,916 reviewed pages and 18,990 lines from dataset v1.1. On the 220-page M00638 test panel, the complete pipeline reaches 3.55% CER and 96.45% character accuracy. These pages were held out from the v1.1 continuation; however, the segmenter inherits older weights that had seen M00638 examples, so this is not a fully manuscript-unseen pipeline result.

Paper

The three-page research note, From Scholarly Transcriptions to 3.71% CER, documents the corpus construction, systems screened, final pipeline, and error analysis. Its LaTeX source and bibliography are included for revision and reuse.

Use

With Python 3.12 and uv:

uvx --from huggingface-hub hf download tadad/midf-sanskrit-ocr --revision main --local-dir sanskrit-ocr
uv run sanskrit-ocr/ocr.py page.jpg -o page.txt --json-output page.json

The command runs locally on CPU by default. --device cuda or --device mps selects a supported accelerator. JSON includes detected boundaries, toplines, reading order, recognized text, compact line-confidence summaries, and a reference-free review-risk assessment. The runner applies EXIF orientation and passes the full image to both models. Kraken normalizes the page to 1,000 pixels high for polygon tracing (scale=[1000, 0], topline=true) and returns polygons in original-image coordinates. PP-OCRv6 receives crops from the full-resolution page and normalizes them to its saved 96-pixel input height.

Reference-free review triage

JSON output includes review.risk_score (0–100), quality_confidence, the inverse review_priority, and human-readable reasons. The score combines PP-OCRv6 character confidence with observable failure signals such as tiny line fragments, unusually thin polygons, polygonization failures, sparse or unexpected output, repetition, and text-bearing pages on which no lines were found. Each line also includes a compact confidence summary.

This is a review-ranking heuristic, not a probability that the text is correct. In a targeted nine-page smoke panel, both pages below 1.4% known CER were high-confidence, the two catastrophic pages above 36% CER were high-priority, and every sampled page above 10% CER was at least medium-priority. See evaluation/review-risk-sample.json.

For archive-scale inference, sort pages by review_priority and risk_score, send the highest-risk pages to human review first, and audit a random sample of high-confidence pages for false negatives. Preserve the component signals and reasons so corrected pages can support later threshold calibration without repeating OCR.

To evaluate the pipeline on the dataset's nine-page legacy evaluation panel:

uv run sanskrit-ocr/evaluation/evaluate.py \
  /path/to/midf-egangotri-sanskrit/data/segmentation/evaluation/v0.1 \
  --output pipeline-evaluation

Complete pipeline evaluation

Page CER compares concatenated page text after NFC normalization and removal of Unicode whitespace.

Held-out panel	Pages	CER ↓	Character accuracy ↑
M00638 test panel	220	3.55%	96.45%
M00626/M00664 adaptation	9	8.88%	91.12%
Combined test set	229	3.71%	96.29%

The M00638 panel contains a single manuscript style and does not establish unrestricted Sanskrit manuscript OCR performance.

Aggregate evaluation metrics provide the evaluation record.

Recognition evaluation

These scores evaluate the recognizer on reviewed line geometry. They are diagnostic line-crop metrics, not complete-pipeline page CER.

Panel	Previous CER	PP-OCRv6 v1.0 CER
9,514 validation lines	10.28%	6.53%
2,620 recognizer-unseen M00638 lines	29.70%	18.27%
91 M00626/M00664 adaptation lines	19.21%	7.36%

Line-level evaluation penalizes text assigned to an adjacent crop. Complete page evaluation concatenates the lines and measures the OCR output users receive. Recognition details describe the checkpoint and metrics.

Training and selection

• BLLA v1.0 continues from v0.3 with learning rate 5e-5, augmentation, and plateau-based learning-rate reduction. A 60-page validation panel selects epoch 4 in one-based numbering. The run completes after 10 epochs through early stopping.
• PP-OCRv6 v1.0 continues from the strongest pre-v1.1 checkpoint with learning rate 1e-4, a cosine schedule, 200 warmup steps, batch size 8, augmentation, NFC targets, and no bidi reordering. The 9,514-line validation set selects epoch 8 in one-based numbering. The job evaluates the selected checkpoint after its 2.5-hour training budget.
• Both jobs use Kraken 7.1.1 and dataset revision fa66bc54. M00624, M00628, and M00668 supply validation. M00638 supplies the main test panel, while exact M00626 and M00664 test pages remain outside v1.1 training. The recognizer is manuscript-unseen on M00638 because its clean v0.9 parent began from the original PP-OCRv6 pretrain and excluded M00638. The segmenter resumes v0.3, whose earlier training included M00638 examples.

Segmentation provenance and recognition provenance record the full settings, checksums, jobs, and raw logs.

Limitations

The complete pipeline detects the exact reference line count on 142 of 229 test pages. Extra line detections often have little effect on page CER, but severe segmentation failures remain costly. M00664-p0017 produces 20 lines for a 9-line reference and reaches 38.0% CER. Margins, crossed-out writing, diagrams, water damage, slanted lines, and dense layouts require review. Body-text transcriptions do not consistently represent every visible mark.

The main evaluation covers M00638, but the complete pipeline is not fully manuscript-unseen because of the segmenter's inherited training history. Additional clean manuscript-held-out panels are required to estimate performance across other scripts, hands, inks, and layouts.

Repository layout

ocr.py runs the released pipeline using the weights declared in pipeline.json. Segmentation and recognition contain the current model weights, provenance, and component metrics. Training contains the final training runner, and evaluation contains the end-to-end evaluation runner and aggregate metrics. Historical checkpoints remain available through Git history and release tags.

Attribution and terms

The PP-OCRv6 base weights are by Benjamin Kiessling / Inria and released under Apache-2.0; see upstream attribution and the included license. This fine-tune modifies those weights for Sanskrit recognition. Kraken supplies segmentation, recognition, and training software under its upstream terms. MIDF/eGangotri images and scholarly transcriptions retain the provenance and usage terms in the dataset card, which lists its license as other. This release grants no additional source-data rights and is not an official MIDF, Inria, PaddlePaddle, Google, or Qwen release.