Learning From Examples: The Perceptron

Adjust weights when wrong

Machine learning lets a system improve from examples instead of rules. The perceptron (1958) is the simplest learning classifier: it predicts 1 if a weighted sum of inputs plus a bias is positive, and whenever it makes a mistake it nudges the weights towards the correct answer. If the classes can be separated by a straight line, it is guaranteed to converge. Stacking many such units with non-linear activations and training them with gradient descent gives the neural networks behind modern deep learning (covered in depth in the Machine Learning and Deep Learning courses).

From examples and from rewards

Learning lets agents improve from data and experience instead of hand-written knowledge.

Training a perceptron by hand, run

I ran this with plain Python 3 (standard library only), with fixed random seeds where randomness is used. Six labelled points are separable, so the perceptron converges after 9 epochs with weights [-1, 4] and bias -7. It then classifies the new point (4, 4) as 1 and (1, 0) as 0.

data = [((2, 3), 1), ((3, 3), 1), ((4, 5), 1), ((1, 1), 0), ((2, 1), 0), ((1, 2), 0)]
w = [0.0, 0.0]; b = 0.0
for epoch in range(1, 11):
    errors = 0
    for (x1, x2), y in data:
        pred = 1 if w[0] * x1 + w[1] * x2 + b > 0 else 0
        if pred != y:
            errors += 1
            w[0] += (y - pred) * x1; w[1] += (y - pred) * x2; b += (y - pred)
    if errors == 0:
        print(f"converged after {epoch} epochs: w = {w}, b = {b}"); break
print("prediction for (4, 4):", 1 if w[0] * 4 + w[1] * 4 + b > 0 else 0)
print("prediction for (1, 0):", 1 if w[0] * 1 + w[1] * 0 + b > 0 else 0)

Output:

converged after 9 epochs: w = [-1.0, 4.0], b = -7.0
prediction for (4, 4): 1
prediction for (1, 0): 0

Know the limits of linear models

A single perceptron cannot learn XOR-like patterns; hidden layers are needed for that.

त्वरित जाँच: When is the perceptron guaranteed to converge?

• Only on images
• Always, on any data
• When the classes are linearly separable
• Only with one example