Intelligent Physics · Week 03 of 14

Ridge and LASSO don't just prevent overfitting — they encode a belief about how many physical mechanisms actually matter.

Regression Feature Engineering Week 3

Objectives

By the end of this week, you will be able to:

  • Explain why ordinary least squares regression can fail when given many or correlated features, and how this connects to the variance problem from Week 2
  • Define Ridge and LASSO regression precisely, and explain what each penalty term does to a model's coefficients
  • Explain why LASSO can perform automatic feature selection while Ridge cannot, and predict which one is more appropriate for a given situation
  • Construct physically motivated descriptors for a materials problem, rather than feeding raw composition directly into a model
  • Fit Ridge and LASSO models in scikit-learn, compare which features each retains, and judge whether the surviving features make physical sense

Prerequisites

This week builds directly on Week 2. You should be comfortable with bias, variance, train/test splitting, and cross-validation — every regression model this week will be judged using exactly these tools. If anything from Week 2 feels shaky, revisit the Week 2 Lesson before continuing.

This week's pages

01 · Course Lesson Ordinary least squares and its failure modes; Ridge and LASSO defined and worked by hand; physically motivated descriptors. 02 · Directed Work Travaux Dirigés Guided problems on coefficient shrinkage, feature selection, and choosing descriptors for a real materials problem, with full solutions. 03 · Practical Work Travaux Pratiques Fit Ridge and LASSO in scikit-learn on a formation-energy dataset, and compare which features survive. 04 · Quiz Self-Check Quiz Ten questions on regression, regularization, and descriptor choice.
Why this week matters

Linear models are not just a stepping stone toward more advanced methods — they remain the right tool whenever the dataset is small, the relationship is plausibly close to linear, or interpretability matters more than squeezing out the last percent of accuracy. Materials datasets are very often exactly this size and this interpretable, which is why Ridge and LASSO will reappear as a baseline against which every more complex method in this course should be compared.

Looking ahead

Week 4 introduces classification and ensemble methods (decision trees, Random Forest, XGBoost), extending the supervised-learning toolkit to problems where the label is a category rather than a continuous number.

← Week 2 Intelligent Physics — Master's Module Week 4 →

INTELLIGENT PHYSICS: MACHINE LEARNING METHODS FOR MATERIALS SCIENCE — WEEK 3 OF 14