AI FUNDAMENTALS

From AI to ML to Deep Learning

Not synonyms. AI is the goal, ML is one method, deep learning is one technique. Understanding the hierarchy tells you which tool fits your problem.

1The Hierarchy — Three Nested Concepts

Artificial Intelligence is the broad goal: systems that exhibit intelligent behaviour. Machine Learning is one strategy to achieve it: algorithms that learn from data rather than following hand-coded rules. Deep Learning is one category of ML: multi-layer neural networks that learn hierarchical representations. Every deep learning model is an ML model; every ML model is an AI system; the reverse is false.

Artificial Intelligence
The broadest category. Any technique that enables a machine to exhibit goal-directed, intelligent behaviour. Includes rule-based systems, planning, search, and all of ML.
Machine Learning
A subset of AI. The algorithm learns patterns from data — you provide examples, it discovers the rules. Includes linear models, decision trees, SVMs, ensembles, and neural networks.
Deep Learning
A subset of ML. Multi-layer neural networks that learn feature representations automatically from raw inputs — images, text, audio — without manual feature engineering.

2From Rule-Based AI to Machine Learning

In rule-based AI, a programmer encodes every decision explicitly: if blood pressure > 140 and age > 60, flag as high risk. This is interpretable and requires no data — but it breaks when rules are too numerous, change over time, or interact in complex ways. The programmer becomes the bottleneck.

Machine learning inverts this: you provide labelled examples and the algorithm extracts the rules. A spam filter trained on millions of emails discovers patterns a human could never enumerate. The trade-off is a loss of full interpretability — the learned rules live in coefficient matrices, not if-else statements.

Key insight: ML is not magic — it is pattern matching at scale. It works when the signal exists in the data, when you have enough labelled examples to capture it, and when the future resembles the past. When those conditions fail, neither more data nor a bigger model fixes the problem.

Choose Rule-Based When
Rules are stable, auditable, and enumerable. Regulatory compliance requires full interpretability. Training data is unavailable. Decision rules must be explainable to non-technical stakeholders.
Choose ML When
Rules are too complex or numerous to encode. Large labelled datasets are available. The patterns change over time and the model can be retrained. Accuracy matters more than interpretability.

3What Deep Learning Adds Over Classical ML

Classical ML algorithms — logistic regression, decision trees, SVMs, random forests — require feature engineering: a human designs the input representation. For an image classifier, that might mean computing pixel gradients, colour histograms, or edge detectors. The quality of features caps the model's performance.

Deep learning removes this bottleneck. A convolutional neural network receives raw pixel values and learns its own features across hierarchical layers — edges in early layers, textures in mid layers, objects in later layers. For language, transformers learn token relationships from raw text. No manual feature design.

Deep Learning Advantages
End-to-end learning from raw inputs. Performance scales with data and compute — no plateau. Dominant on images, speech, text, and video. Transfer learning allows reuse across tasks.
Deep Learning Costs
Requires orders of magnitude more labelled data. Needs GPU compute — expensive to train and serve. Features are not human-interpretable. Debugging failures is hard. Overkill for tabular structured data.

4Choosing the Right Level for Your Problem

The wrong level wastes months. A practical decision: start at the simplest level that can solve your problem. Deep learning is not the default — it is the level you escalate to when simpler methods are insufficient. The code below trains a minimal linear model from scratch in pure Python — no libraries — to show what ML actually does: adjust parameters to reduce prediction error.

Related

Gradient descent from scratch — no libraries

Recall check

0 of …

Say the answer out loud before revealing — recalling is what makes it stick. ➜ Next: AI Agents and Uninformed Search

  1. What is the relationship between AI, ML, and deep learning?

  2. What is the key difference between rule-based AI and machine learning?

  3. What does deep learning add over classical ML, and what does it cost?

  4. When should you use deep learning vs classical ML vs rule-based AI?