Section 2 · Lesson 7

Blend models and write the submission

Blend probabilities from different model families and turn them into the final submission file.

Why blending works

Two models that make different mistakes can correct each other when you average their probabilities. Model quality isn't the only thing that matters: the ordinal LightGBM model scored only slightly higher than the multiclass one, but it added more to a CatBoost blend than another multiclass model did, because its errors differ.

The model experiments tested blends on the fold-0 holdout. Blend weights came from a 0.1-step grid that minimizes holdout log loss. The "cross-fitted weighted" column applies per-class weights fit on one half and scored on the other half.

Blends that include a mix of model families beat blends of near-identical models. Two CatBoost models averaged together gained less than a CatBoost plus an ordinal LightGBM.

The final recipe

The final notebook is deliberately simpler than the experiments. It trains two LightGBM models per fold and leaves CatBoost out. Both use the same parameters: learning rate 0.05, 127 leaves, 80% feature and row subsampling, and early stopping after 100 rounds without improvement on the fold's validation set.

for i, (fit, val) in enumerate(folds):
    Xf, Xv = X_train.iloc[fit], X_train.iloc[val]
    p_val, p_test = fit_predict({**PARAMS, "objective": "multiclass", "num_class": K},
                                Xf, y[fit], Xv, y[val], X_test)
    oof["multiclass"][val] = p_val
    pred["multiclass"] += p_test / FOLDS            # average test predictions over folds
    # ... three binary models for the ordinal version, same pattern ...

blend_oof = (oof["multiclass"] + oof["ordinal"]) / 2
weights = tune_class_weights(blend_oof)

Every fold model also predicts the test set. The notebook averages those five test predictions, which acts like a five-model ensemble for each model type.

The scores

The notebook's guide says that these CV scores understate the expected test score by about 0.004 (Lesson 6). The 0.7577 is also slightly optimistic, because the class weights are tuned on the same predictions that report the score. A cross-fitted check put the real gain from the weights at about +0.001.

Write the submission

The last cell applies the class weights to the averaged test probabilities and writes the CSV with the columns ID and price_tier:

submission = pd.DataFrame({"ID": test["space_id"],
                           "price_tier": (blend_test * weights).argmax(1) + 1})
submission.to_csv("submission.csv", index=False)

Remember the + 1: the models use labels 0 to 3, and the competition wants 1 to 4. The following table shows the predicted tier counts for the 114,538 test listings, next to the training shares.

The test mix differs from the training mix, with more tier 2 and fewer tier 4. That might reflect a different test population, in which about a third of listings have unseen operators, or a model bias. The notebook can't tell you which, so treat it as a prompt to check, not as a verdict.