Rules From Data Instead of Rules by Hand

Programs that learn their own logic

In ordinary programming you write the rules: if the email contains these words, mark it spam. In machine learning (ML) you provide examples (inputs with the correct answers) and an algorithm finds the rules that best map inputs to answers. This helps when the rules are too many, too subtle or keep changing: spam, fraud, image recognition, demand forecasting. The learned model is then used to make predictions on new inputs. ML is not magic: it needs representative data, it can only find patterns present in that data, and its errors must be measured.

Learning patterns from examples

Machine learning finds rules from data instead of having a programmer write them.

A hand rule versus a learned rule, run

I ran this with Python 3, numpy 2.5.3 and scikit-learn 1.9.1, using fixed random seeds. On ten made-up emails, the guess "contains free means spam" is right 60% of the time. A depth-1 decision tree learns from the labelled examples that more than 4 links predicts spam and gets all ten right (on the same ten emails, so this is not yet a fair test).

# Rules written by hand versus a rule learned from labelled examples
from sklearn.tree import DecisionTreeClassifier, export_text
emails = [  # (number of links, has the word "free", is spam)
    (0, 0, 0), (1, 0, 0), (2, 0, 0), (7, 1, 1), (9, 1, 1), (8, 0, 1), (1, 1, 0), (6, 1, 1), (0, 1, 0), (10, 0, 1)]
X = [[a, b] for a, b, _ in emails]; y = [c for *_, c in emails]
hand_rule = lambda links, free: int(free == 1)          # a guess: "free" means spam
print("hand rule accuracy   :", sum(hand_rule(*x) == t for x, t in zip(X, y)) / len(y))
tree = DecisionTreeClassifier(max_depth=1, random_state=0).fit(X, y)
print("learned rule accuracy:", tree.score(X, y))
print(export_text(tree, feature_names=["links", "has_free"]))

Output:

hand rule accuracy   : 0.6
learned rule accuracy: 1.0
|--- links <= 4.00
|   |--- class: 0
|--- links >  4.00
|   |--- class: 1

Try simple rules first

If a short rule solves the problem well, you may not need ML; ML pays off when rules get complicated or data shifts.

Quick check: What does a machine learning algorithm learn from?

• The programmer's intuition at runtime
• Only hand-written if statements
• Examples of inputs with their correct outputs (or structure in the inputs)
• Random numbers alone