# Linear Regression — Machine Learning Basics

Source: https://www.skillbyai.com/en/machine-learning/r-linear

> A weighted sum of features.

## Fit a line (or plane) through the data

**Linear regression** predicts a number as a weighted sum of the features plus an intercept. Training finds the weights that minimise the squared differences between predictions and true values. It is fast, interpretable (each weight shows how the prediction moves with a feature, holding others fixed) and a strong baseline. Its limit: it captures only straight-line relationships unless you add transformed features, and correlated features make individual weights hard to interpret.

## Fit, measure, compare

Regression predicts numeric values; judge it against a baseline with clear error metrics.

![Three ideas: linear models, metrics, baselines.](assets/figures/machine-learning/section-3-map.svg) — Figure 3.1 — Linear models, metrics and baselines.

## Linear regression versus a mean baseline, run

I ran this with Python 3, numpy 2.5.3 and scikit-learn 1.9.1, using fixed random seeds. On the bundled diabetes progression dataset, predicting the training mean gives MAE 58.3 and R2 0.000; linear regression gives MAE 45.1, RMSE 56.4 and R2 0.359: clearly better than the baseline, but far from perfect.

```python
from sklearn.datasets import load_diabetes
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score
from sklearn.model_selection import train_test_split
X, y = load_diabetes(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0)
for name, m in [("baseline (predict mean)", DummyRegressor()), ("linear regression", LinearRegression())]:
    p = m.fit(X_tr, y_tr).predict(X_te)
    rmse = mean_squared_error(y_te, p) ** 0.5
    print(f"{name:<24} MAE {mean_absolute_error(y_te, p):6.1f}  RMSE {rmse:6.1f}  R2 {r2_score(y_te, p):6.3f}")
```

Output:

```
baseline (predict mean)  MAE   58.3  RMSE   70.5  R2 -0.000
linear regression        MAE   45.1  RMSE   56.4  R2  0.359
```

## Plot predictions against truth

A scatter of predicted versus actual values shows systematic errors (for example underpredicting high values) that a single metric hides.

**Quiz:** What does linear regression minimise during training?

- [x] The sum of squared differences between predictions and true values
- [ ] The number of features
- [ ] The training time
- [ ] The number of rows

*Answer:* The sum of squared differences between predictions and true values. Least squares is the standard objective.