CHAPTER 06
Intrusion Detection and Prevention Systems
Updated: May 15, 2026
25 min read
# CHAPTER 6
Intrusion Detection and Prevention Systems
1. Introduction
A firewall is a gatekeeper; it checks if you are on the guest list (the allowed ports). But what if a hacker is on the guest list? If Port 80 (Web Traffic) is open, a hacker can send a malicious SQL Injection attack straight through the firewall. The firewall sees "Web Traffic on Port 80" and lets it pass. To stop this, we need a system that inspects the *behavior* and *content* of the traffic. In this chapter, we will explore Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS), understanding how they use signatures and anomaly detection to identify malicious activity.2. Learning Objectives
By the end of this chapter, you will be able to:- Define an Intrusion Detection System (IDS) and its passive nature.
- Define an Intrusion Prevention System (IPS) and its active nature.
- Differentiate between Signature-based and Anomaly-based detection.
- Understand the placement of IDS/IPS within a network architecture.
- Identify the risk of False Positives in an IPS.
3. Beginner-Friendly Explanation
Imagine an airport security checkpoint.- The Firewall: The agent who checks your boarding pass. If you don't have a ticket for a flight today (Port 80/443), you cannot enter.
- The IDS (Detection): A security guard standing near the x-ray machine. They watch the bags go through. If they see a knife, they pull out a walkie-talkie and yell, "Security breach!" But they don't actually stop the bag from going on the plane.
- The IPS (Prevention): The x-ray machine itself. If it detects a knife, it automatically locks the conveyor belt, traps the bag, and sounds an alarm. The threat is physically stopped.
4. IDS vs. IPS
- Intrusion Detection System (IDS): Passive. It sits "out of band" (like a camera on the wall) and copies network traffic. It analyzes the traffic and sends an alert to the SOC team if it sees an attack. It does not slow down the network, but it cannot stop the attack.
- Intrusion Prevention System (IPS): Active. It sits "inline" (the traffic must flow *through* it). If it detects an attack, it instantly drops the packet, resets the connection, and blocks the hacker's IP address.
5. Detection Methodologies
How do these systems know what a cyberattack looks like?- 1. Signature-Based Detection: The system has a massive dictionary of known attacks (Signatures), similar to Antivirus software.
' OR 1=1; -- (a known SQL Injection), it triggers an alert.
*Weakness:* It only catches *known* attacks. It cannot catch a brand new, zero-day exploit.
- 2. Anomaly-Based (Heuristic) Detection: The system uses Machine Learning to establish a "Baseline" of normal network traffic over a few weeks.
6. The Risk of False Positives
A False Positive occurs when the IPS thinks legitimate traffic is an attack. Because an IPS sits inline and actively blocks traffic, a False Positive is catastrophic for a business. If the IPS incorrectly flags the company's new payroll software as "malware" and blocks it, no one gets paid. For this reason, network engineers usually deploy an IPS in "Detection Only" (IDS) mode for the first month to tune out the false positives before turning on the "Prevention" blocking features.7. Mini Project: Simulate Intrusion Alert Workflow
Let's conceptualize writing a rule for Snort, one of the most famous open-source IDS/IPS engines.A Snort Rule Structure:
[Action] [Protocol] [Source IP] [Source Port] -> [Dest IP] [Dest Port] (Options)
Writing a Defensive Rule: We want to generate an alert if anyone tries to send an Nmap scan (which often contains a rapid sequence of SYN packets) to our web server.
text
*Translation:* "If any IP address sends more than 20 TCP SYN packets (flags:S) to our Web Server (192.168.1.100) on Port 80 within 10 seconds, trigger an alert that says 'Possible Nmap Scan Detected'."
8. Real-World Scenarios
A university experiences a massive influx of ransomware. Their traditional firewall allowed the initial phishing email through (because Port 80/443 was open). The university deploys an IPS with advanced Signature and Anomaly detection. A month later, a student downloads a new variant of a worm. The IPS doesn't recognize the signature, but its Anomaly engine notices the student's laptop is suddenly trying to initiate 5,000 SMB (Server Message Block) connections to every other computer on the network per second. The IPS instantly terminates the laptop's network connection, isolating the worm before it can spread.9. Best Practices
- Placement is Key: An IDS/IPS cannot inspect encrypted HTTPS traffic unless you give it the decryption keys. Therefore, an IPS is often placed just *behind* a Load Balancer or Web Application Firewall that terminates the SSL/TLS encryption, allowing the IPS to inspect the raw, unencrypted traffic before it reaches the internal servers.
10. Legal and Ethical Notes
Configuring an IPS to automatically "hack back" (e.g., launching a counter-attack against the source IP of a perceived threat) is illegal. The source IP might be spoofed, and you could inadvertently launch a cyberattack against an innocent third party. Active defense stops at blocking the traffic.11. Exercises
- 1. Compare and contrast an IDS with an IPS. In what scenario would an organization intentionally choose to deploy an IDS instead of an IPS?
- 2. Explain the fundamental difference between Signature-based detection and Anomaly-based detection.
12. FAQs
Q: Is a Next-Generation Firewall (NGFW) the same as an IPS? A: Functionally, yes. In the past, companies bought a Firewall box and a separate IPS box. Today, a NGFW combines the traditional port-blocking firewall and the deep-packet IPS engine into a single piece of hardware.13. Interview Questions
- Q: Describe the concept of a "False Positive" in the context of an Intrusion Prevention System. Why do False Positives carry a higher operational risk in an IPS compared to an IDS?
- Q: You are deploying a network-based IDS. The network utilizes complete end-to-end TLS 1.3 encryption. Detail the architectural challenges this poses for the IDS and propose a solution to ensure visibility.