CardioSyntax v2: SYNTAX Score Prediction from Coronary Angiography

This repository contains code and pre-trained models for automated SYNTAX score prediction from coronary angiography videos. The SYNTAX score is a validated metric for quantifying the anatomical complexity of coronary artery disease.

Overview

CardioSyntax v2 is a large-scale dataset of 11,329 angiography studies with SYNTAX scores, designed for developing machine learning models for automated coronary lesion complexity assessment. This repository provides:

Dataset: 11,209 training studies (Philips Allura Clarity + Azurion 7) + 120 test studies (100 Philips + 20 Siemens) annotated by 6 independent interventional cardiologists
Baseline Models:
- Backbone network (R3D-18): single projection → SYNTAX contribution
- RNN head (LSTM/GRU/Transformer): multi-view aggregation → final SYNTAX score
Pre-trained Weights: 5-fold ensemble with post-calibration
Evaluation: Pearson correlation r ≈ 0.81, balanced accuracy ≈ 82% for SYNTAX > 22 classification

The dataset addresses inter-observer variability through multi-expert annotation and enables systematic investigation of domain shift effects across imaging systems.

Repository Structure

coronary-syntax-prediction/
├── backbone/                          # Single-frame backbone (3D ResNet)
│   ├── dataset.py                    # SyntaxDataset for single-view training
│   ├── pl_model.py                   # SyntaxLightningModule (Lightning wrapper)
│   ├── pl_train.py                   # Training script for backbone
│   └── __init__.py
├── full_model/                        # Multi-view RNN head + backbone
│   ├── rnn_dataset.py                # SyntaxDataset for multi-view (RNN) training
│   ├── rnn_model.py                  # SyntaxLightningModule with RNN variants
│   ├── rnn_train.py                  # Training script for full model
│   └── __init__.py
├── inference/                         # Inference & evaluation
│   ├── rnn_apply.py                  # Ensemble inference script
│   ├── metrics_visualization.py       # Plotly-based metrics & visualization
│   └── __init__.py
├── backbone_weights/                 # Backbone safetensors weights (5 folds × 2 arteries)
├── full_model_weights/               # Full model safetensors weights (5 folds × 2 arteries × variants)
├── classifier_weights/               # R3D-18 artery classifier (safetensors)
├── scaling_coeffs/                   # Calibration coefficients (a, b) per fold
├── requirements.txt
└── README.md

Installation

# Git LFS must be installed to download the model tensors
git lfs install

# Clone the repository
git clone https://huggingface.co/MesserMMP/coronary-syntax-prediction
cd coronary-syntax-prediction
git lfs pull

# Install dependencies
pip install -r requirements.txt

Requirements

Key packages:

PyTorch 2.0+ with CUDA support
Lightning 2.x
torchvision, pytorchvideo
numpy, scikit-learn 1.4+
safetensors for tensor-only model weights
plotly for visualization
click for CLI

See requirements.txt for full list.

Training

Run commands from the repository root with python -m .... JSON paths are relative to --dataset-root; each DICOM path is relative to its JSON's folder (absolute DICOM paths also work). Data folders are read-only inputs; logs and exported weights go to the explicitly selected output directories.

Expected files for fold 00 (no renaming is needed):

folds/fold00_train.json
folds/fold00_eval.json
rnn_folds/rnn_fold00_train.json
rnn_folds/rnn_fold00_eval.json

Backbone → full model

This sequence trains one artery and one fold. && starts the full model only after backbone training and export succeed. Both stages select the best checkpoint by validation RMSE and export tensor-only weights automatically.

DATA_ROOT=/mnt/s3-data2/home/mikhailmmp
OUTPUT_ROOT=/home/mikhailmmp/syntax-training

python -m backbone.pl_train \
  -r "$DATA_ROOT" --fold 0 -a right --num-classes 2 \
  -b 50 -f 32 -v 256 256 --pre-epochs 10 --max-epochs 10 \
  --num-workers 8 --devices 0 --precision bf16-mixed \
  --logdir "$OUTPUT_ROOT/logs/backbone" \
  --weights-dir "$OUTPUT_ROOT/weights/backbone" \
  --use-weighted-sampler && \
python -m full_model.rnn_train \
  -r "$DATA_ROOT" --fold 0 -a right --variant lstm_mean --num-classes 2 \
  -b 8 -f 32 -v 256 256 --pre-epochs 15 --max-epochs 15 \
  --num-workers 8 --devices 0 --precision bf16-mixed \
  --backbone-weights-dir "$OUTPUT_ROOT/weights/backbone" \
  --rnn-folds-dir rnn_folds \
  --logdir "$OUTPUT_ROOT/logs/rnn" \
  --weights-dir "$OUTPUT_ROOT/weights/full_model"

Adjust batch sizes to GPU memory. Use --devices 1 for GPU1; repeat the option for multiple GPUs (--devices 0 --devices 1). GPU indices are relative to CUDA_VISIBLE_DEVICES. With no CUDA device, training uses one CPU process.

The backbone first trains its output head, then fine-tunes the network. The full model first trains its aggregation head with frozen backbone parameters, then fine-tunes the complete model. Transitions load the selected pretraining weights from .safetensors. OneCycleLR advances on each optimizer step in both models. Training requires two outputs (classification and regression).

Outputs for this example:

$OUTPUT_ROOT/weights/backbone/rightBinSyntax_R3D_full_fold00.safetensors
$OUTPUT_ROOT/weights/full_model/RightBinSyntax_R3D_fold00_lstm_mean_post_best.safetensors

Repeat for left and folds 0–4 to build all ten models required by ensemble inference. Newly trained models need their own calibration; the supplied scaling coefficients belong to the supplied pretrained ensemble.

Training options and checkpoints

--pre-epochs: head-only epochs; backbone default 10, full-model default equals --max-epochs.
--max-epochs: fine-tuning epochs (default 10).
--weights-dir: export directory; defaults to ./trained_weights/backbone or ./trained_weights/rnn. Existing files with the same artery/fold name are replaced. Published weights are not overwritten by default.
--logdir: TensorBoard logs, intermediate .safetensors, and Lightning .ckpt files containing optimizer/scheduler state. These local training checkpoints are distinct from the tensor-only files intended for publication.
--backbone-weights-dir: full-model input directory, default backbone_weights; --backbone-pt-dir remains an alias. Safetensors is the default input format; no legacy format flag is needed.
--backbone-from-checkpoints --backbone-logdir PATH: legacy alternative selecting the most recently written backbone .ckpt from Lightning logs.
--pt-weights-format: retained for compatibility; safetensors transitions detect their format automatically and do not depend on this flag.
--use-weighted-sampler: sample training records by score-bin weights.
--seed: random seed, default 42.

The existing dataset filtering is retained: backbone validation and both full-model splits keep positive artery scores; full-model records also need at least one projection for the chosen artery. Each full-model training or validation item samples four projections, with replacement. Validation batch size is one. At least 2 * batch-size training records and one validation record must remain after filtering.

For a brief pipeline check, prepare separate small JSON subsets with the same filenames, point -r at that temporary directory, and use -b 1 --pre-epochs 1 --max-epochs 1. Keep the original DICOM paths absolute in these subsets.

Inference

Run ensemble inference on test datasets:

python -m inference.rnn_apply \
  -d "philips_100.json" -d "siemens_20.json" \
  -n "Philips Test" -n "Siemens Test" \
  -p "philips_100" -p "siemens_20" \
  -r /path/to/dataset_root \
  --model-dir full_model_weights \
  -v 256 256 \
  --frames-per-clip 32 \
  --num-workers 8 \
  --variant lstm_mean \
  --device cuda:0 \
  --use-scaling \
  --scaling-file "$PWD/scaling_coeffs/scaling_coeffs.json" \
  -e "Ensemble_lstm_mean" \
  -m metrics.json

Parameters:

-d, --dataset-paths: Multiple JSON test dataset paths (relative to --dataset-root)
-n, --dataset-names: Display names for each dataset (must match -d count)
-p, --postfixes: Suffixes for result files (must match -d count)
-r, --dataset-root: Dataset root (default: .)
--model-dir: Directory with full model .safetensors weights (default: full_model_weights)
-v, --video-size: Frame resolution H W (default: 256 256)
--frames-per-clip: Frames per clip (default: 32)
--device: cuda:0, cuda:1 or cpu; defaults to cuda:0 if available, otherwise CPU.
--num-workers: DataLoader workers (default: 8)
--variant: Head model variant (default: lstm_mean)
--pt-weights-format: Legacy .pt / .ckpt format flag; .safetensors files are detected automatically.
--use-scaling: Apply post-calibration scaling a*x+b from JSON (flag)
--scaling-file: Calibration coefficients file relative to --dataset-root
-e, --ensemble-name: Experiment name for metrics (e.g., "Ensemble_lstm_mean")
-m, --metrics-file: JSON output file path for all metrics
--seed: Random seed (default: 42)

Output:

Per-dataset predictions: results/{postfix}.json
Ensemble metrics: {metrics_file} with Pearson correlation, balanced accuracy, per-fold statistics
Plotly HTML plots: visualizations/Ensemble.html or Ensemble_scaled.html

Existing results/{postfix}.json files are reused. Use a new postfix or remove the old result file when changing weights, data or preprocessing. All five folds for both arteries must be present; missing weights raise an error.

Dataset

Link: https://huggingface.co/datasets/MesserMMP/coronary-angiography-syntax

Data Format

The dataset provides two different JSON annotation formats depending on the training stage:

Format 1: Backbone Training (Single-View/Per-Video)

Used for training the backbone model (backbone/ scripts). Each record represents one angiographic video projection.

File structure: folds/fold{fold_id}_{split}.json

[
  {
    "study_uid": "1.3.46.670589.28.26690171363123020190823114413193175",
    "series_uid": "1.3.46.670589.28.26690171363123020190823115102913895.2.2",
    "sop_uid": "1.3.46.670589.28.266901713631230201908231158145991852211512",
    "path": "../anon_data/chunk1/.../IM-2254-0039.dcm",
    "shape": [38, 512, 512],
    "artery": 0,
    "artery_prob": 1.9078343029832467e-05,
    "syntax": 0.0,
    "syntax_left": 0.0,
    "syntax_right": 0.0,
    "bin_syntax": 0,
    "manufacturer": "Philips",
    "device_model": "Allura Clarity"
  },
  {
    "study_uid": "1.3.46.670589.28.26690171363123020190823114413193175",
    "series_uid": "1.3.46.670589.28.26690171363123020190823115648342907.2.2",
    "sop_uid": "1.3.46.670589.28.266901713631230201908231158148282212211512",
    "path": "../anon_data/chunk1/.../IM-2260-0022.dcm",
    "shape": [15, 512, 512],
    "artery": 1,
    "artery_prob": 0.9559882879257202,
    "syntax": 0.0,
    "syntax_left": 0.0,
    "syntax_right": 0.0,
    "bin_syntax": 0,
    "manufacturer": "Philips",
    "device_model": "Allura Clarity"
  }
]

Field descriptions:

Field	Type	Description
`study_uid`	str	Unique study identifier (DICOM)
`series_uid`	str	Unique series identifier (DICOM)
`sop_uid`	str	Unique SOP instance UID (DICOM)
`path`	str	Relative path to DICOM video file
`shape`	list[int]	Video dimensions: [T, H, W] (frames, height, width)
`artery`	int	Coronary artery: 0 = left (LCA), 1 = right (RCA)
`artery_prob`	float	Model confidence for artery classification (0–1)
`syntax`	float	Total SYNTAX score (LCA + RCA)
`syntax_left`	float	SYNTAX score for left coronary artery
`syntax_right`	float	SYNTAX score for right coronary artery
`bin_syntax`	int	Binary classification: 0 = SYNTAX < threshold, 1 = SYNTAX ≥ threshold
`manufacturer`	str	Imaging equipment manufacturer (e.g., "Philips", "Siemens")
`device_model`	str	Equipment model (e.g., "Allura Clarity", "AXIOM-Artis")

Format 2: RNN Head Training (Multi-View/Per-Patient)

Used for training the full RNN model (full_model/ scripts). Each record represents one patient study with all angiographic projections grouped by artery.

File structure: rnn_folds/rnn_fold{fold_id}_{split}.json

[
  {
    "study_uid": "1.3.46.670589.28.26690171363123020190823114413193175",
    "manufacturer": "Philips",
    "device_model": "Allura Clarity",
    "syntax": 0.0,
    "syntax_left": 0.0,
    "syntax_right": 0.0,
    "bin_syntax": 0,
    "videos": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115102913895.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158145991852211512",
        "path": "../anon_data/chunk1/.../IM-2254-0039.dcm",
        "shape": [38, 512, 512],
        "artery": 0,
        "artery_prob": 1.9078343029832467e-05
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115648342907.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158148282212211512",
        "path": "../anon_data/chunk1/.../IM-2260-0022.dcm",
        "shape": [15, 512, 512],
        "artery": 1,
        "artery_prob": 0.9559882879257202
      }
    ],
    "videos_left": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115102913895.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158145991852211512",
        "path": "../anon_data/chunk1/.../IM-2254-0039.dcm",
        "shape": [38, 512, 512],
        "artery": 0,
        "artery_prob": 1.9078343029832467e-05
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115533239902.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158147562092211512",
        "path": "../anon_data/chunk1/.../IM-2258-0055.dcm",
        "shape": [50, 512, 512],
        "artery": 0,
        "artery_prob": 0.0003698925720527768
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115052517893.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158145581792211512",
        "path": "../anon_data/chunk1/.../IM-2253-0047.dcm",
        "shape": [47, 512, 512],
        "artery": 0,
        "artery_prob": 0.00018035581160802394
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115550658904.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158147942152211512",
        "path": "../anon_data/chunk1/.../IM-2259-0047.dcm",
        "shape": [41, 512, 512],
        "artery": 0,
        "artery_prob": 0.00022948876721784472
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115523054900.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158147162032211512",
        "path": "../anon_data/chunk1/.../IM-2257-0047.dcm",
        "shape": [43, 512, 512],
        "artery": 0,
        "artery_prob": 0.00034276593942195177
      }
    ],
    "videos_right": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115648342907.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158148282212211512",
        "path": "../anon_data/chunk1/.../IM-2260-0022.dcm",
        "shape": [15, 512, 512],
        "artery": 1,
        "artery_prob": 0.9559882879257202
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115657547909.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158149262272211512",
        "path": "../anon_data/chunk1/.../IM-2261-0070.dcm",
        "shape": [62, 512, 512],
        "artery": 1,
        "artery_prob": 0.9999938011169434
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115712699912.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158149652332211512",
        "path": "../anon_data/chunk1/.../IM-2262-0063.dcm",
        "shape": [54, 512, 512],
        "artery": 1,
        "artery_prob": 0.999855637550354
      }
    ],
    "videos_other": [
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115128658897.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158146391912211512",
        "path": "../anon_data/chunk1/.../IM-2255-0035.dcm",
        "shape": [33, 512, 512],
        "artery": 0,
        "artery_prob": 0.2866898477077484
      },
      {
        "series_uid": "1.3.46.670589.28.26690171363123020190823115132067898.2.2",
        "sop_uid": "1.3.46.670589.28.266901713631230201908231158146771972211512",
        "path": "../anon_data/chunk1/.../IM-2256-0035.dcm",
        "shape": [32, 512, 512],
        "artery": 0,
        "artery_prob": 0.3251875042915344
      }
    ]
  }
]

Field descriptions:

Field	Type	Description
`study_uid`	str	Unique study identifier (same across all videos for this patient)
`manufacturer`	str	Imaging equipment manufacturer
`device_model`	str	Equipment model
`syntax`	float	Total SYNTAX score (LCA + RCA)
`syntax_left`	float	SYNTAX score for left coronary artery
`syntax_right`	float	SYNTAX score for right coronary artery
`bin_syntax`	int	Binary classification label
`videos`	list[object]	All video projections (unfiltered)
`videos_left`	list[object]	Projections classified as left coronary (artery=0, high confidence)
`videos_right`	list[object]	Projections classified as right coronary (artery=1, high confidence)
`videos_other`	list[object]	Low-confidence projections (excluded from training); typically ~5–7% of total

Video object fields:

Field	Type	Description
`series_uid`	str	Unique series identifier (DICOM)
`sop_uid`	str	Unique SOP instance UID (DICOM)
`path`	str	Relative path to DICOM video file
`shape`	list[int]	Video dimensions: [T, H, W]
`artery`	int	Artery classification: 0 = left, 1 = right
`artery_prob`	float	Right-artery probability (0–1); high-confidence grouping: left <= 0.1, right >= 0.9

Test Set Format (Multi-Expert Annotations)

Test data (philips_100.json, siemens_20.json) follows Format 2 with additional expert annotations:

[
  {
    "study_uid": "...",
    "manufacturer": "Philips",
    "device_model": "Allura Clarity",
    "syntax": 18.5,
    "syntax_left": 10.0,
    "syntax_right": 8.5,
    "mean_syntax": 18.5,
    "expert0": 18,
    "expert1": 19,
    "expert2": 17,
    "expert3": 19,
    "expert4": 20,
    "expert5": 18,
    "videos_left": [...],
    "videos_right": [...],
    "videos_other": [...]
  }
]

Additional fields for test data:

Field	Type	Description
`mean_syntax`	float	Consensus SYNTAX score (mean of 6 experts)
`expert0`–`expert5`	float	Individual expert SYNTAX score (6 independent cardiologists)

Note: expert0 typically represents aggregated/consensus annotations, while expert1–expert5 are individual readers.

DICOM Video Format

Each DICOM file contains:

Dimensions: [T, H, W] = [frames, 512, 512] at 15 fps
Data type: uint8 (8-bit grayscale)
Frame count: Typically 15–60 frames per projection
Resolution: 512×512 pixels (standard for X-ray angiography)

Data Splits

Set	Files	Size	Equipment	Artery Classification
Backbone training	`folds/fold{00-04}_train.json`	Varies by fold	Philips	Single video
Backbone validation	`folds/fold{00-04}_eval.json`	Varies by fold	Philips	Single video
RNN training	`rnn_folds/rnn_fold{00-04}_train.json`	Varies by fold	Philips	Multi-view (patient)
RNN validation	`rnn_folds/rnn_fold{00-04}_eval.json`	Varies by fold	Philips	Multi-view (patient)
Test 1 (Philips)	`philips_100.json`	100 studies	Philips Allura Clarity	6 experts
Test 2 (Siemens)	`siemens_20.json`	20 studies	Siemens AXIOM-Artis	6 experts

SYNTAX > 0 prevalence: 39.26% in training data

Results

Verified inference (5-fold ensemble)

The supplied lstm_mean safetensors ensemble was run on October 15, 2026 with 32-frame, 256×256 clips and the supplied scaling coefficients:

Dataset	Studies	Pearson r	Balanced accuracy (SYNTAX >= 22)
Philips (`philips_100.json`)	100	0.8144	0.8182
Siemens (`siemens_20.json`)	20	0.3849	0.6429

These are measured results for these test files and settings, not metrics from the short training smoke test. Environment: NVIDIA A100, PyTorch 2.14.1+cu130, torchvision 0.29.1+cu130. Dataset composition and preprocessing must match when comparing runs.

Model Architecture

Backbone: R3D-18 (video ResNet) pre-trained on Kinetics-400
Head (default): LSTM with a two-output projection and mean pooling
Output: 2 channels (classification logit + log-transformed regression)
Loss: Weighted BCE (classification) + scaled MSE (regression)

Weights & Checkpoints

The repository includes 21 tensor-only .safetensors files:

backbone_weights/: five folds for each artery, e.g. rightBinSyntax_R3D_full_fold00.safetensors (lowercase artery name). These backbones have a one-output head. Use num_classes=1 to load a standalone backbone; the current two-output training loss expects a different head. The RNN loader discards this head for aggregation variants and strictly validates all feature-extractor weights. mean_out requires a two-output backbone; use a newly trained backbone for that variant.
full_model_weights/: five folds for each artery, e.g. RightBinSyntax_R3D_fold00_lstm_mean_post_best.safetensors. The supplied full-model variant is lstm_mean.
classifier_weights/r3d_art.safetensors: left/right artery classifier.
scaling_coeffs/: calibration parameters (a, b) per fold.

Safetensors files load directly as tensor dictionaries on CPU. Existing .pt and Lightning .ckpt loading remains available for trusted legacy checkpoints. Use --backbone-weights-dir for published or newly trained .safetensors backbones. The old --backbone-pt-dir and --backbone-from-pt names remain available for compatibility. --pt-weights-format only affects legacy files; safetensors is detected from the filename.

Artery classifier

classifier_weights/r3d_art.safetensors contains a torchvision R3D-18 classifier with a single-output Linear(512, 1) head. Apply sigmoid to its output to obtain the probability of the right coronary artery: class 0 is left, class 1 is right. The binary decision uses a 0.5 threshold (the reference uses round(p), so exactly 0.5 maps to class 0). For high-confidence grouping, probabilities <= 0.1 indicate left and >= 0.9 indicate right; intermediate values form the uncertain group.

The input is a float tensor (batch, 3, frames, 224, 224). Grayscale video is replicated into three channels, scaled from uint8 to [0, 1], resized to 224×224, and normalized using ImageNet mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225]. The classifier accepts a variable number of frames. Load its tensor-only weights as follows:

import torch
from torchvision.models.video import r3d_18
from safetensors.torch import load_file

model = r3d_18(weights=None)
model.fc = torch.nn.Linear(model.fc.in_features, 1)
model.load_state_dict(
    load_file("classifier_weights/r3d_art.safetensors", device="cpu"),
    strict=True,
)
model.eval()
# For a normalized video batch x: p_right = torch.sigmoid(model(x))

Citation

If you use this dataset or code, please cite:

@dataset{cardiosyntax_v2_2025,
  title={CardioSyntax v2: Angiographic dataset for SYNTAX score estimation},
  author={MesserMMP and collaborators},
  year={2025},
  url={https://huggingface.co/datasets/MesserMMP/coronary-angiography-syntax}
}

Related Work

SYNTAX Score Definition: Sianos et al., EuroIntervention (2005)
Original CardioSyntax: Ponomarchuk et al., arXiv:2407.19894 (2024)
Video Understanding: Hara et al., CVPR (2018) — R3D architecture

License

CC0 1.0 (Public Domain)

Contact & Support

For issues, questions, or contributions, please refer to the HuggingFace repository discussions.

Last updated: October 15, 2026

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Paper for MesserMMP/coronary-syntax-prediction

End-to-end SYNTAX score prediction: benchmark and methods

Paper • 2407.19894 • Published Jul 29, 2024 • 5