Section 1 · Lesson 1

The problem and the dataset

You get 11,393 long, thin photos of a surface and four defect types. For each photo and each type, you predict which pixels are defective.

You'll learn

The task

The project is a Kaggle-style course competition called "industrial surface defect detection". Each image is an 800 by 128 pixel picture of a manufactured surface. Four kinds of defect can appear, named defect_a to defect_d. An image can have none of them, one, or several.

You don't predict a label for the whole image. You predict a mask for each of the four types: a binary image the same size as the photo, where 1 means "this pixel is part of the defect". That task is called semantic segmentation.

Five long thin surface images. Four show coloured mask overlays for defect types a to d, and one shows a clean surface.
One training image for each defect type, with the ground-truth mask drawn on top, plus a clean image. Colours: a red, b green, c blue, d yellow.

What is in the data

11,393
training images
3,000
test images
45,572
image-class pairs (4 per image)
128 × 800
height × width, in pixels

The images are grayscale but stored as 3-channel JPEG files. The labels live in data/public/train.csv, with one row per image and defect type:

ColumnMeaning
image_idFile name without the .jpg extension.
defect_typeOne of defect_a, defect_b, defect_c, defect_d.
mask_rleThe mask as an RLE string. Empty when the image doesn't have that defect.
width, heightAlways 800 and 128.

Because every image has four rows, the code reshapes the table once so that each image has one row and four mask columns. This function is in src/common.py:

def load_train_wide():
    df = pd.read_csv(DATA / "train.csv")
    wide = df.pivot(index="image_id", columns="defect_type", values="mask_rle")
    wide = wide[CLASSES].reset_index()
    return wide

Run-length encoding

A 128 by 800 mask has 102,400 pixels, but most masks are mostly zeros. RLE stores only where each run of ones starts and how long it is. The string "7255 6 7377 12" means "6 ones starting at pixel 7255, then 12 ones starting at pixel 7377".

Two details trip people up:

def rle_decode(rle, shape=(H, W)):
    flat = np.zeros(shape[0] * shape[1], dtype=np.uint8)
    if not isinstance(rle, str) or not rle.strip():
        return flat.reshape(shape, order="F")
    s = np.asarray(rle.split(), dtype=np.int64)
    starts, lengths = s[0::2] - 1, s[1::2]
    for st, ln in zip(starts, lengths):
        flat[st : st + ln] = 1
    return flat.reshape(shape, order="F")

RLE playground

Draw on the grid, or edit the RLE text. This grid is 8 rows by 16 columns, so the pixel numbers run down each column first. Click and drag to paint; the first cell you touch decides whether you paint or erase.

Decode a real mask, run by run

This is a real defect_d mask from the training set (image ). Drag the slider to draw the first k runs and watch the mask grow.

The test set and the submission

The 3,000 test images have no masks. You submit one RLE string per image and defect type. When you predict nothing, you write a single space. The format of the submission file mirrors sample_submission.csv, with the columns ID, image_id, defect_type, and mask_rle. Lesson 7 covers how the project writes it.

Check your understanding of the encoding. The competition metric treats a missing defect and a false alarm differently. Lesson 3 explains how, and why that shapes every later decision in the project.

In this project's RLE, which pixel does the number 129 refer to in a 128-row mask?

  1. The first pixel of the second row.
  2. The last pixel of the first column.
  3. The first pixel of the second column.

Pixels are numbered column by column and start at 1. Pixels 1 to 128 fill the first column from top to bottom, so 129 is the top of the second column.

An image has no defect_b. What does its defect_b row contain?

  1. A mask string of all zeros.
  2. An empty mask_rle value.
  3. The row is missing from the file.

Every image has four rows. A defect that is absent has an empty mask_rle, and the code treats empty and missing strings as an all-zero mask.