(x, y)
Every training example includes an input and target. The loss can directly compare predictions with truth.
Learning path / Semi-supervised learning
Learn from a few answers—
and many examples.
Semi-supervised learning combines a small labeled set with a larger unlabeled set. Unlabeled data help only when the geometry or invariances of the inputs say something reliable about the missing labels.
01 / Start hereThree learning settings,
Three learning settings,
one prediction task.
The target is still supervised: predict y from x. What changes is which examples reveal y.
D = Dlabeled ∪ Dunlabeled
few (x, y) + many x
A supervised anchor plus an additional constraint or inferred target on unlabeled examples.
x only
No target labels. The objective discovers structure, density, compression, or representations.
Cluster assumption
Nearby points tend to share a label.
Low-density separation
A good boundary avoids dense groups of examples.
Consistency assumption
Small, label-preserving changes should not change a prediction.
02 / InteractivePredict, filter, retrain.
Predict, filter, retrain.
Watch the boundary spread.
Self-training converts confident predictions into temporary labels. Step through each round and lower the threshold to see coverage trade against confirmation error.
ŷ = arg max p(y|x), accept if max p(y|x) ≥ τStep 0 of 4Model is at its prior
Feature spacecolor = model label · ring = optional ground truth
human labelpseudo-labelunlabeledadded this round
1Fit
→Train on human and accepted pseudo-labels.
2Predict
→Score every remaining unlabeled example.
3Filter
→Keep predictions above confidence τ.
4Repeat
Add them as targets for the next round.
03 / Method choiceThe right method depends
The right method depends
on why unlabeled data help.
Choose the mechanism that matches the structure you actually believe—not whichever method is newest.
assumption → regularizer → failure modeMethodCore ideaUse whenMain risk
Pseudo-labelingRetrain on confident predictionsClasses already separate reasonably wellConfirmation bias
Consistency regularizationMatch predictions across perturbationsYou know label-preserving augmentationsInvalid augmentations
Graph propagationDiffuse labels through a similarity graphLocal neighborhoods reflect class identityBad graph geometry
Co-trainingTwo views teach one anotherFeatures split into sufficient, complementary viewsCorrelated errors
History & theory
Self-training and pattern classifiers using unlabeled examples have long histories. Blum and Mitchell formalized co-training in 1998 under assumptions about redundant views. Modern deep methods often combine pseudo-labels with consistency: FixMatch, for example, accepts confident predictions from weak augmentations and trains on strongly augmented versions.
Connect the idea
Unlabeled structure becomes a training objective.
Consistency regularization connects directly to neural-network objectives; confidence and uncertainty connect back to Bayesian decision-making.
Compare the losses