01
Data and independence
What produced each observation? Which samples share a person, source, time period or collection process? Decide what must be kept separate before choosing a split.
scikit-learn: cross-validation ↗Research notes
How I examine model evidence.
A practical framework for distinguishing a model score from evidence that supports an intended use.
02
A fair comparison
Choose a baseline, isolate the change being tested and keep final evaluation separate from model and preprocessing choices. Fit learned transformations on the appropriate training data.
scikit-learn: common pitfalls ↗03
Useful measures
Read aggregate scores alongside class-level errors, uncertainty and the cost of mistakes. Ask whether the metric actually represents the question the system is intended to answer.
scikit-learn: model evaluation ↗04
Transfer and scope
Check performance under relevant changes in data conditions. A good internal result does not by itself establish performance in a new environment; deployment claims require additional evidence.
WILDS: in-the-wild distribution shifts (ICML 2021) ↗From the projects
Different outputs need different evidence.
Visible expression is not internal emotion
A facial-expression classifier learns dataset labels. Claims about a person’s internal state need a different evidential basis.
Segmentation quality is not downstream value
The MSc work separates boundary quality, cell measurements and prediction. Improvement in one does not automatically establish improvement in another.