How Did Rules Become Learning Systems?

1. The Workplace Automation Dilemma

Imagine a routine task: routing incoming customer support and operations emails to the right department.

In a traditional setup, an administrator writes explicit instructions:

• If the subject contains "Invoice", send it to Accounting.

• If the body contains "Error 500", send it to IT Support.

This works until real-world messiness arrives. A customer writes, "Urgent billing issue regarding server downtime." Does it go to Accounting or IT? Another writes, "I can't pay my statement because the login screen crashes." The rules misroute the message, or match nothing at all.

Organizations facing this problem historically had two options:

1. Keep writing hundreds of narrow, overlapping rules, or

2. Move to a system that learns patterns from historical data.

Understanding how computing moved from hand-crafted rules to adaptive, data-driven systems guards against a common mistake: assuming every modern automation problem needs a deep neural network. Simpler, deterministic methods are often cheaper, faster, and more reliable.

2. Preparation and Prerequisites

Before starting, review "What Are We Trying to Understand When We Say AI?". The key distinctions to recall:

This article opens the "AI box" to examine four paradigms that carried the field from basic programming to modern generative systems.

3. A Concrete Example: Explicit Rules vs. Statistical Categorization

To see the shift from traditional logic to statistical learning, compare how two systems categorize the same kind of email.

System A: A hand-crafted rule (deterministic)

IF sender CONTAINS "billing"
   OR subject CONTAINS "invoice"
THEN destination_folder = "Finance"
ELSE destination_folder = "General Inbox"

IF sender CONTAINS "billing"
   OR subject CONTAINS "invoice"
THEN destination_folder = "Finance"
ELSE destination_folder = "General Inbox"

IF sender CONTAINS "billing"
   OR subject CONTAINS "invoice"
THEN destination_folder = "Finance"
ELSE destination_folder = "General Inbox"

How it works: A developer explicitly programs every condition.

Strengths: Fully predictable, instant to run, needs no training data, and completely transparent.

Weaknesses: Brittle. It fails on typos ("invocie"), synonyms ("statement", "receipt"), and indirect phrasing ("Please stop charging my card").

System B: Example-based statistical categorization (probabilistic)

Instead of writing rules, we give the system 5,000 historical emails, each labeled Finance or Not Finance.

How it works: The algorithm estimates the probability of each category from how often words appear in each class. For a Naive Bayes classifier, the score for a class is its prior probability multiplied by the likelihood of each word given that class:

\text{Score}(\text{Finance}) = P(\text{Finance}) \times \prod_i P(\text{word}_i \mid \text{Finance})

\text{Score}(\text{Finance}) = P(\text{Finance}) \times \prod_i P(\text{word}_i \mid \text{Finance})

\text{Score}(\text{Finance}) = P(\text{Finance}) \times \prod_i P(\text{word}_i \mid \text{Finance})

The class with the higher score wins.

Outcome: The system discovers that words like "charge", "card", "overdue", and "subscription" correlate strongly with Finance, even in emails where "billing" never appears.

Strengths: Handles variation, typos, and evolving vocabulary without manual code changes.

Weaknesses: Needs labeled training data, produces probabilistic approximations (so false positives happen), and must be validated before it is trusted.

4. Mechanisms: The Four Eras of AI Approaches

AI did not start with deep learning. Over the decades, people have tried four main ways of making computers "smart". Think of four ways of training a new employee:

1. Hand them a rulebook and say "follow this exactly."

2. Give them a goal and let them try different options to find the best route.

3. Show them thousands of past cases so they spot patterns themselves.

4. Let them learn from raw experience (photos, sound, text) and work out what matters on their own.

These four ways match the four eras below. Modern systems often mix several of them.

The eras show when each approach became popular. A newer era does not replace the older ones.

Era 1: Rules and Symbols (Symbolic AI)

What it is: People write down what they know as clear IF-THEN rules, and the computer follows them. For example: IF a customer's order is over 5,000 rupees AND the address is new, THEN flag it for review. The computer learns nothing on its own. Everything it "knows" came from a person, so you can always trace exactly why it made a decision.

Three common building blocks:

• Expert system: A program that captures what a human specialist knows and uses it to answer questions, like having a doctor or tax accountant inside software. Developers interview the specialist, write their know-how as rules, and the program asks questions and reaches a conclusion. Famous early examples are MYCIN, which suggested antibiotics for blood infections, and DENDRAL, which helped chemists work out the structure of molecules.

• Production rule engine: The part of the software that stores all the rules and runs them. It looks at the facts of the current case, finds which rules apply, and acts on them. An insurance system, for instance, can check rules about age, past claims and policy type before approving a request. A "production rule" is simply an IF-THEN rule.

• Knowledge graph: A map of facts and how they connect. Things are dots and relationships are lines, such as "Paris is the capital of France" and "France is in Europe". By following the lines, a computer can answer questions like "Which European capitals are on a river?" Search engines and online product catalogs use this idea.

Best for: Areas with fixed boundaries and no room for guesswork, such as tax calculation and regulatory compliance checks.

Limitation: Experts must put everything into rules by hand. This is slow and costly, and real life always throws up cases nobody thought of. As the rules pile up, they also start to clash with each other. This is called the knowledge acquisition bottleneck.

When it rose: Mainly the 1960s to the 1980s, when expert systems were the face of AI in business and medicine.

Still used today: Tax software, loan eligibility checks, insurance claim rules, airline fare rules, and the "if this, then that" logic inside most business workflows.

On our email problem: A rule such as "subject contains invoice" is fast and clear, but it breaks on typos, synonyms and mixed messages like "billing issue with server downtime".

Era 2: Search and Shortcuts (Heuristics)

What it is: Some problems have no answer you can simply look up. You have to try moves and see where they lead, like finding the best delivery route or playing chess. The computer treats the problem as a map of options. Each choice leads to a new situation, and the computer explores the map to find a path to the goal.

What is a heuristic? A heuristic is a rule of thumb: an educated shortcut that points the computer to the most promising options, so it does not have to check everything. It works like driving to a city. You head in its general direction instead of trying every road. A heuristic is not guaranteed to be perfect, but it saves huge amounts of time.

Common techniques in plain words:

• A* search ("A-star"): Finds the shortest route, much like a GPS. It tracks how far you have already travelled and makes a smart guess of how far is left, then always explores the most promising road first.

• Alpha-beta pruning: Used in two-player games like chess. When the computer sees that a line of play is clearly worse than one it has already found, it drops it without exploring further.

• Constraint satisfaction: For puzzles with must-follow rules, such as Sudoku, exam timetables, or staff shifts. The computer fills in options and backs up whenever a rule is broken.

• Monte Carlo Tree Search: Instead of checking every move, the computer plays many quick, semi-random games from each option and picks the option that wins most often. It is used in games like Go.

Best for: Logistics routing, resource scheduling, and structured games such as chess and Sudoku.

Limitation: Too many possibilities. Every extra step multiplies the number of options, so the total can grow beyond what any computer can check. This is called combinatorial explosion. Shortcuts help, but people still have to design them for each problem.

When it rose: The 1960s onward. The A* method dates from 1968, and IBM's Deep Blue used a powerful form of game-tree search to beat world chess champion Garry Kasparov in 1997.

Still used today: GPS and map routing, airline crew and delivery scheduling, factory planning, and game-playing programs. Systems like AlphaGo combined Monte Carlo Tree Search with neural networks, which shows how the eras are often mixed.

On our email problem: Search is a poor tool for reading emails, but it is a good fit for the next step. Once an email is categorized, search can decide which agent should handle it so that workloads and deadlines stay balanced.

Era 3: Learning from Data (Classical Machine Learning)

What it is: Instead of people writing the rules, we give the computer many past examples with known outcomes, such as old loan applications marked "repaid" or "defaulted". The computer finds the patterns by itself. This works best on tidy, table-style data, where each row is one case and each column is one fact (age, income, number of late payments). The email example in Section 3 is exactly this idea.

Common methods in plain words:

• Linear regression: Predicts a number by weighing the facts, such as estimating a house price from its size and location.

• Logistic regression: Predicts the chance of a yes/no outcome, such as "Will this customer cancel?"

• Naive Bayes: Counts how often words or facts show up in each category, then uses those counts to guess. Many spam filters work this way.

• Decision tree: A flowchart of yes/no questions learned from data, such as "Income above a threshold? Any missed payments?"

• Random forest: Many decision trees that each give an answer and then vote. The group is more reliable than any single tree.

• Support vector machine (SVM): Draws the best dividing line between two groups, such as spam and not spam.

• Gradient boosting (e.g., XGBoost): Builds trees one after another, each one fixing the mistakes of the one before. It is very strong on table data and widely used in business.

Best for: Credit scoring, predicting which customers will leave, spam filtering, and fraud detection.

Limitation: People must prepare the data first. Raw things like email text or photos have to be turned into numbers and columns, such as the count of the word "refund" or the length of the email. This step is called feature engineering, and the model is only as good as the features people choose.

When it rose: Roughly the 1990s to the 2010s, as companies collected more digital records and computers became fast enough to learn from them. Gradient boosting libraries such as XGBoost (2014) became favorites for business data.

Still used today: Credit scoring, insurance pricing, demand forecasting, customer churn prediction, and fraud screening at banks and payment companies.

On our email problem: A Naive Bayes model counts which words appear in past Finance emails and learns that "charge", "card" and "overdue" signal Finance, even when "invoice" is missing. It still sees only word counts, so it does not grasp the meaning of a whole sentence.

Era 4: Deep Learning (Neural Networks)

What it is: Deep learning uses neural networks: software loosely inspired by the brain, made of many layers of tiny calculation units that pass results to the next layer. The big difference from Era 3 is that people no longer prepare the features. The network works them out itself from raw data. Given a photo, the first layers notice simple things like edges, the middle layers notice shapes like eyes or wheels, and the last layers recognize whole objects such as a face or a car. This needs a lot of example data and computing power.

Main kinds, and what they suit: You do not need to memorize these names yet; just notice that different designs suit different data.

• Networks for images and video: Good at spotting objects in pictures (technically called CNNs).

• Networks for ordered data: Older designs that read speech or text step by step (RNNs and LSTMs).

• Transformers: The newer design behind today's AI chat assistants and translation tools.

Best for: Recognizing images, understanding speech, translating languages, and AI that writes text.

Limitations: High computing cost, a need for large amounts of data, a tendency to hallucinate (confidently give wrong answers), and decisions that are hard to explain, often called the "black box" problem.

When it rose: The idea is old, but it took off in 2012 when a deep neural network (AlexNet) won a major image recognition contest by a wide margin. The Transformer design appeared in 2017 and led to today's large language models.

Still used today: Photo tagging and face unlock on phones, voice assistants, medical image analysis, machine translation, and AI chat assistants.

On our email problem: A neural model reads the whole message and picks up wording and context. It can tell that "I can't pay my statement because the login screen crashes" involves both billing and a technical fault, and it can flag the email for both teams.

The four eras side by side

5. Limits and Misconceptions

A frequent engineering mistake is tool misapplication: assuming newer paradigms completely replace older ones. A useful rule of thumb is to match the approach to how deterministic the task is and what kind of data it involves.

Common misconceptions

"Deep learning replaced traditional rules." Production systems routinely use rule engines at the perimeter. A banking app, for example, applies explicit rule checks (such as account balance limits) before calling a machine learning model to assess loan risk.

"Neural networks are always better than classical ML." On tabular data (spreadsheets and database tables), tree-based models like XGBoost frequently match or outperform deep neural networks, while also training faster, costing less, and being easier to interpret.

"Statistical models understand context the way humans do." ML models map inputs to output probabilities using correlations in training data. That can look like understanding, but it is not the same as human comprehension or reasoning.

Putting it together: one system, several eras

Real products rarely use just one approach. Picture a support desk that handles incoming emails:

1. Rules (Era 1) reject empty messages, spot known error codes, and enforce policies such as "refunds over a set amount need manager approval."

2. Classical ML (Era 3) scores each email for urgency or likelihood of churn, using the customer's past tickets and account data.

3. Deep learning (Era 4) reads the free-form text, summarizes it, and suggests a draft reply.

4. Search (Era 2) assigns the ticket to the best available agent, balancing skills, workload and deadlines.

The design choice is to use the simplest tool that does each job reliably, and to keep a human in the loop where mistakes are costly.