clockface

Reads an analog clock and returns the time. Finds the dial with a COCO-pretrained detector, then reads the crop.

263703 reads 51.5% of 200 held-out real photographs to the minute, against 49.0% for 263702. It is also wrong by more than half an hour more often -- 25.5% against 22.0% -- so this is a trade, not a clean win. Which one you want depends on whether a wrong answer costs you more than a missing one. Both are in this repository.

Version 263703, commit 1fce71d. Reader backbone: resnet50.

The two versions, same 200 photographs, same pipeline

	to the minute	median	within 5 min	MAE	worse than 30 min
263703	51.5%	1.00 min	69.0%	40.4 min	25.5%
263702	49.0%	1.25 min	73.0%	34.0 min	22.0%
guessing	~0%	180 min	3%	180 min	97%

The trade has a cause, and it is visible while training. Measured on real photographs held out from training, epoch by epoch: readings correct to the minute climb from 23.2% to 48.7% across all 22 epochs and never stop climbing, while mean error bottoms out at epoch 8 and then gets worse. Training longer makes the model sharper on the clocks it can read and less steady on the ones it cannot. There is no epoch that is best at both, so the choice is a choice.

What changed

The reader gained a third head. The other two each describe one hand, so a dial whose hands are hard to tell apart corrupts both at once -- on 263702, 27.5% of readings got the minute right and the hour wrong. The third head is asked for the time itself over the whole 12-hour ring, coarsely, with no notion of a hand: it cannot express a role confusion because it never assigns roles, and it only has to break the twelve-way tie. In a controlled comparison -- same data, same recipe, only the head differing -- it moved exactly what it was aimed at, hours missed with the minute correct falling from 12.0% to 8.5%.

The checkpoint is chosen on photographs, not renders. Renders stop predicting real accuracy about a third of the way through training; picking the epoch renders liked cost 3 to 7 minutes of real error. It is chosen on photographs held out from training and never in this test set, and by the to-the-minute rate, because MAE and the to-the-minute rate disagree about which epoch is best and only one of them is what this model is for.

When to trust a reading

The reader ranks its own answers, and the ranking is worth more than the headline. Declining the quarter it is least sure of takes it from 69.0% within five minutes to 96%.

200 readings. Keeping only the ones each signal is most sure of:

  hand agreement
       keep     n      MAE   median   <=5min
       100%   200    40.40     1.00    69.0%
        75%   150    26.06     1.00    80.0%
        50%   100    15.16     0.75    86.0%
        25%    50    10.31     0.50    90.0%
        10%    20     7.20     0.75    95.0%
    rank correlation with error +0.388 (useful)

  head sharpness
       keep     n      MAE   median   <=5min
       100%   200    40.40     1.00    69.0%
        75%   150    24.20     0.75    80.7%
        50%   100     7.80     0.75    92.0%
        25%    50     1.84     0.75    98.0%
        10%    20     3.55     0.50    95.0%
    rank correlation with error -0.504 (useful)

  search margin
       keep     n      MAE   median   <=5min
       100%   200    40.40     1.00    69.0%
        75%   150    21.12     0.75    82.0%
        50%   100    10.33     0.75    91.0%
        25%    50     5.75     0.50    96.0%
        10%    20     0.57     0.50   100.0%
    rank correlation with error -0.529 (useful)

  A caution on reading the low-coverage rows: at 25% of 200 photographs a row
  rests on 50 readings, so small differences between signals there are noise.

The signal this project set out to use for that was disagreement between the hands. On weaker readers it ranked almost nothing; it becomes useful here, which took a good enough model rather than a better signal.

Stage 1

fasterrcnn_mobilenet_v3_large_fpn with COCO weights, on CPU, not fine-tuned. It finds a clock in 93-96% of real photographs having never seen a render. A dial locator trained on renders scored no better than not cropping at all and was removed. On the 9 test photographs where stage 1 finds nothing, 1 is still read within five minutes.

Honest limits

The goal this was pointed at was reading the time to the minute 95% of the time. It does not get there: 51.5% across all photographs, and about 90% even when allowed to decline 85% of them.

The labels on these 200 photographs are third party. 106 were checked by hand against the image; the rest were not. There is no first-party test set, and a number measured on somebody else's labels is worth less than one measured on your own.

Licence

Apache 2.0. The synthetic dataset is CC BY 4.0.