Back to the section

Section 1 · Deep learning project

Industrial surface defect detection

From run-length-encoded labels to a tuned U-Net that scores 0.6486 on a held-out set.

Open the full lessons

The task

  • 11,393 training images, 3,000 test images
  • Each image is 800 × 128 pixels
  • Four defect types, defect_a to defect_d
  • Predict one binary mask per type
Surface images with coloured defect masks for each type.

Labels are run-length encoded

mask_rle = "7255 6 7377 12 7505 12 ..."
# start (1-based), length, start, length ...
# pixels are numbered column by column
  • Pixel 129 is the top of the second column
  • Empty string means the defect is absent
  • The submission uses the same format, with a single space for empty

The data is mostly empty

Bar charts of images per class and classes per image.
  • 5,516 clean images
  • defect_c 4,100, defect_d 1,707, defect_a 1,081, defect_b 313
  • 84.2% of image-class pairs are empty
  • Split 80/20, stratified on 15 label combinations

Masks are small

Histogram of mask area per positive image for four classes.
  • Median component: 239 px for defect_a, 2,696 px for defect_d
  • A 200-pixel filter would delete 15.2% of defect_a mask pixels

The score

Score = 0.70 × ForegroundDice + 0.20 × FalsePositiveScore + 0.10 × EmptyAccuracy

Stacked bars showing how the score splits into three weighted parts for four models.

Predict nothing and you score 0.30

The metric rewards recall

9.7e-5one positive pair, 0.7 / 7,201
2.6e-6one empty pair wrong, 0.1 / 38,371
~37×ratio
  • Predict a mask when the chance of a defect is above about 5%
  • Naive mean Dice pushes the other way: all-empty gets 0.8420

Model: a U-Net

  • segmentation_models_pytorch U-Net, ResNet-34 encoder, ImageNet weights
  • Native 128 × 800 input: both sizes divide by 32
  • Four output channels with a sigmoid each, because classes co-occur
  • Skip connections keep thin edges sharp
  • EfficientNet-B3 tried as a second encoder
cls(encoder_name="resnet34", encoder_weights="imagenet", classes=4)

Training recipe

  • Loss: BCE + multilabel Dice
  • AdamW, learning rate 3e-4, weight decay 1e-4
  • One-cycle schedule, 10 epochs, batch size 16
  • Augmentation: flips, brightness and contrast, small shift and scale
  • About 3.7 minutes per epoch on an Apple-silicon GPU
One image under original, horizontal flip, vertical flip, and brightness changes.

Training curves

Training loss and holdout score for ResNet-34 and EfficientNet-B3.
  • Loss 1.13 to 0.44 in 10 epochs
  • Epoch 2 dips, then the score climbs to 0.6284 at tau 0.3
  • EfficientNet-B3 is noisy: 0.6503 at epoch 6, 0.5928 at epoch 7

Training in 6 seconds

One global threshold

Score, foreground Dice, empty accuracy, and naive Dice against threshold.
  • Best raw score 0.6431 at tau 0.15 to 0.2
  • Naive Dice keeps rising with tau. The official score falls.

Per-class post-processing

Tuned thresholds per class with component and area minimums.
  • Three rules per class: threshold, component filter, total-area filter
  • Grid search, 336 combinations per class
  • Validated by 2-fold CV
  • 0.6420 to 0.6486

Threshold sweep, on real images

Inference and submission

  • Three views per model: none, horizontal flip, vertical flip
  • TTA lifts the score at tau 0.3 from 0.6284 to 0.6420
  • Average the probabilities across models, apply the tuned rules, RLE-encode
  • 12,000 rows: 3,000 images × 4 classes
uv run python src/predict.py r34_full effb3_full --params runs/ens/postproc.json

Lessons learned

Bar chart of the score rising from 0.30 to 0.6486 as steps are added.
  • Read the metric first. Predicting nothing scores 0.30.
  • Stratify on label combinations; tune per class; validate the tuning
  • Not shown: a measured ensemble, the r34_pos result, multiple seeds

Try it yourself

Back to the section overview