Spam Detection

Naive Bayes classifier

Prerequisites

graph TD
  Prob[Probability basics] --> Bayes[Bayes theorem]
  Bayes --> NB[Naive Bayes]
  Log[Log odds & underflow] --> NB
  NB --> Spam[Spam Detection]
  Text[Tokenization / NLP basics] --> Spam

When to use

• High-volume email or message filtering where hand-written rules cannot keep up with obfuscation (V1agra, F R E E).
• Personalized inboxes that learn from user “Report spam” feedback — each word’s $P(w\mid \text{spam})$ updates from counts.
• Lightweight edge classifiers that must score thousands of tokens per second without GPU inference.

When not to use

• You need semantic understanding of phishing intent, images, or attachments (use modern transformers + reputation feeds).
• Adversarial poisoning is a first-class threat without top-$k$ word trimming or robust training.
• Single-word decisions must be legally auditable — prefer explicit blocklists and policy engines alongside Bayes scores.

Lab

On the interactive spam detection lab, tune P(spam) prior, pick a sample email, and read per-word $P(w\mid \text{spam})$ vs $P(w\mid \text{ham})$ bars. Try Spam-heavy corpus, Balanced, or High prior presets, predict spam vs ham before classification, then reveal classifier prediction vs ground-truth label.

Simple Fundamental Explanation

Imagine you are trying to guess if an unmarked envelope contains a bill or a birthday card.

• If it has a shiny, colorful stamp, the probability it’s a birthday card goes up.
• If it has a window showing your account number, the probability it’s a bill goes up.
• If it has both, you weigh the evidence. An account number is very strong evidence of a bill, so it overrides the colorful stamp.

You didn’t open the letter, but you combined multiple probabilities to reach a highly confident conclusion.

This is how Naive Bayes Spam Filtering works. Instead of using a rigid list of rules (e.g., IF email CONTAINS "Viagra" THEN spam), the system looks at every single word in the email. It knows historically that the word “Click” appears in 80% of spam but only 5% of normal emails. It knows “Meeting” appears in 1% of spam and 40% of normal emails.

By combining the probabilistic weight of every word, the filter calculates the final probability that the entire email is spam. If it’s > 99%, it goes to the Junk folder.

---

Deep Dive: How It Works Under the Hood

1. Training (Building the Probabilities)

The filter requires a dataset of pre-labeled emails (e.g., 10,000 spam emails, 10,000 “ham” or normal emails). For every word in the vocabulary, the system counts frequencies to calculate two probabilities:

• $P(\text{Word}\mid \text{Spam})$: If an email is spam, what is the probability it contains this word?
• $P(\text{Word}\mid \text{Ham})$: If an email is normal, what is the probability it contains this word?

2. The “Naive” Assumption

The algorithm assumes that the occurrence of one word is completely independent of another word. (This is mathematically false—“Bank” and “Account” frequently appear together—but making this “naive” assumption dramatically simplifies the math and surprisingly still works incredibly well in practice).

3. Inference (Classifying a New Email)

When a new email arrives, it is split into words: ["Free", "Money", "Click", "Here"]. The system calculates the joint probability that the email is Spam given those words, versus the joint probability that it is Ham. Whichever probability is higher wins.

Visual Diagram

New Email arrives: "Urgent: Claim Free Prize"

Word Probabilities (from training):
Word     | P(Word | Spam) | P(Word | Ham)
----------------------------------------
Urgent   | 0.15           | 0.05
Claim    | 0.20           | 0.01
Free     | 0.30           | 0.02
Prize    | 0.10           | 0.001

Spam Score = P(Urgent|Spam) * P(Claim|Spam) * P(Free|Spam) * P(Prize|Spam)
Ham Score  = P(Urgent|Ham)  * P(Claim|Ham)  * P(Free|Ham)  * P(Prize|Ham)

Because Spam Score >>> Ham Score, the email is classified as SPAM.

---

Practical Example: Gmail’s Early Filters

Before deep learning and Transformer models took over, Naive Bayes was the gold standard for email providers like Yahoo and early Gmail (often popularized by the open-source tool SpamAssassin).

Spammers tried to defeat deterministic filters by obfuscating words (e.g., V1agra or F R E E). Deterministic IF statements failed instantly. However, Bayesian filters easily adapted. Once a few users marked V1agra emails as spam, the probability $P(\text{"V1agra"}\mid \text{Spam})$ skyrocketed. The filter automatically learned the new obfuscation without a human engineer needing to write a new rule.

Furthermore, spammers tried “Bayesian Poisoning” by adding giant paragraphs of classic literature (like Shakespeare) to the bottom of their spam emails, hoping the high quantity of “Ham” words would outweigh the “Spam” words. Filter designers countered this by only using the 10 most extreme words in the equation, ignoring the neutral Shakespearean text entirely.

---

The Mathematics: Equations and In-Depth Analysis

The algorithm relies entirely on Bayes’ Theorem:

$$P(A\mid B)=\frac{P(B\mid A)\cdot P(A)}{P(B)}$$

In the context of spam detection, we want to find $P(\text{Spam}\mid \text{Words})$: the probability the email is spam, given the words inside it.

1. Bayes’ Theorem for a Single Word

For a single word $W$:

$$P(\text{Spam}\mid W)=\frac{P(W\mid \text{Spam})\cdot P(\text{Spam})}{P(W)}$$

• $P(\text{Spam})$ is the prior probability (e.g., 80% of all internet email is spam).
• $P(W)=P(W\mid \text{Spam})P(\text{Spam})+P(W\mid \text{Ham})P(\text{Ham})$ (The overall probability of seeing the word anywhere).

2. Combining Multiple Words

To combine multiple words $W_1,W_2,\dots ,W_n$ under the Naive assumption (independence), we multiply the probabilities. However, multiplying many small fractions causes computer floating-point underflow (numbers get too small for the CPU to process). To fix this, we use the Logarithmic Sum.

Instead of comparing $P(\text{Spam})\prod P(W_i\mid \text{Spam})$, we take the natural logarithm $\ln ()$ of the equation. Because $\ln (a\cdot b)=\ln (a)+\ln (b)$, multiplication turns into simple addition!

The Final Scoring Equation:

$${\text{Score}}_{\text{Spam}}=\ln (P(\text{Spam}))+\sum _{i=1}^n\ln (P(W_i\mid \text{Spam}))$$ $${\text{Score}}_{\text{Ham}}=\ln (P(\text{Ham}))+\sum _{i=1}^n\ln (P(W_i\mid \text{Ham}))$$

If ${\text{Score}}_{\text{Spam}}>{\text{Score}}_{\text{Ham}}$, the email is classified as Spam. This logarithmic transformation allows the system to probabilistically score emails containing tens of thousands of words in microseconds.

Lab

Naive Bayes multiplies word likelihoods with a prior using log-scores to avoid underflow. Bars show P(w | spam) vs P(w | ham) from the training corpus — same pattern as the README classifier and early SpamAssassin filters.

Spam prior P(spam)40% prior spam

Spam-heavy corpusBalancedHigh prior

free win moneymeeting project teamurgent free moneyhello thanks schedulefree meeting

freespam

winspam

moneyspam

Red = P(w | spam), blue = P(w | ham). Corpus preset retunes training counts.

Email: free win money with prior 40% spam (balanced corpus). Will the classifier label this spam (posterior ≥ 50%)?

Yes — classified as spamNo — classified as ham

3 words · prior 40% · balanced corpus. Reveal above for prediction vs label.