Intelligent Physics · Week 02 of 14

A model that fits your training data perfectly has usually memorized noise, not learned physics. Generalization is the only metric that matters.

Statistics First Code This Week Week 2

Objectives

By the end of this week, you will be able to:

  • Define bias and variance precisely, and explain why both are failure modes rather than opposites to be eliminated together
  • Explain why training error alone never tells you whether a model has actually learned something useful
  • Perform a train/test split and a k-fold cross-validation by hand on a small dataset, and explain what each estimates
  • Recognize data leakage caused by near-duplicate or grouped samples, and apply group-aware splitting to avoid it
  • Write and run your first scikit-learn code: loading data, splitting it, and computing a cross-validated error estimate

Prerequisites

This week assumes you have completed Week 1 and are comfortable with the following terms: feature, label, model, training, and the distinction between supervised and unsupervised learning. If any of these feel shaky, revisit the Week 1 Lesson before continuing.

From this week onward, you will write short Python scripts. No prior programming experience is assumed, but you will need a working Python environment with scikit-learn installed; setup instructions are given on the Practical Work page.

This week's pages

01 · Course Lesson Bias and variance defined and illustrated by hand; train/test splitting and k-fold cross-validation; the data-leakage trap. 02 · Directed Work Travaux Dirigรฉs Hand-computed cross-validation on a small dataset, and a data-leakage diagnosis exercise, with full solutions. 03 · Practical Work Travaux Pratiques Your first scikit-learn code: load a formation-energy dataset, split it two ways, and compare the resulting error estimates. 04 · Quiz Self-Check Quiz Ten questions on bias, variance, and validation — the foundation every later method in this course depends on.
Why this week matters more than it looks

Every method from Week 3 onward — regression, classification, neural networks, graph networks — will be evaluated using the validation tools introduced this week. If you do not trust your validation procedure, you cannot trust any accuracy number reported later in the course, no matter how sophisticated the model.

Looking ahead

Week 3 introduces linear and regularized regression, and the first real feature-engineering decisions for materials problems. The cross-validation procedure from this week will be used immediately to judge whether those models generalize.

← Week 1 Intelligent Physics — Master's Module Week 3 →

INTELLIGENT PHYSICS: MACHINE LEARNING METHODS FOR MATERIALS SCIENCE — WEEK 2 OF 14