Logistic Regression

 

    Logistic Regression is a classification algorithm used to predict binary outcomes, it is also called binary classifier.

• 0 or 1
• Yes or No
• True or False
• Pass or Fail

 

Step 1: Linear Equation (Top Left)

Where:

• b₀ = intercept
• b₁ = coefficient (weight)
• x = input feature
• y = linear output
• This is a linear model (straight line).
• It calculates a raw score .

This equation produces values from −∞ to +∞.

-        Important: This is just an intermediate value.
It is NOT the final prediction.

Step 2: Apply Sigmoid Function

Since classification needs probability (0 to 1), we apply the Sigmoid Function:

This function:

• Converts any number into a value between 0 and 1
• Produces an S-shaped curve
• Gives probability

So,

• If y is very large positive → p ≈ 1
• If y is very large negative → p ≈ 0
• If y = 0 → p = 0.5

Step 3: Classification Decision

After getting probability p, we use a threshold (usually 0.5):

• If p ≥ 0.5 → Class = 1
• If p < 0.5 → Class = 0

 

Logistic Regression with Example

Logistic Regression is used for binary classification (0 or 1).

It works in 3 steps:

Step 1: Linear Equation

First, we compute:

Suppose we are predicting:

-        Will a student pass (1) or fail (0) Based on hours studied (x)

Assume:

• b₀ = −3
• b₁ = 1
• x = 4 hours

Now calculate:

y = −3 + (1 × 4)
y = −3 + 4
y = 1

So the linear output is:

-         y = 1

Step 2: Apply Sigmoid Function

Now convert y into probability using:

Substitute y = 1:

We know:

e⁻¹ ≈ 0.367

So:

p ≈ 0.73

-         Probability = 73%

Step 3: Final Classification

Using threshold = 0.5

• If p ≥ 0.5 → Pass (1)
• If p < 0.5 → Fail (0)

Since 0.73 > 0.5

✅ Prediction = PASS