Skip to main content
Shell Scripting – Complete Beginner to Advanced Guide
CHAPTER 20 Intermediate

Build Real-World Shell Automation Projects

Updated: May 16, 2026
45 min read

# CHAPTER 20

Build Real-World Shell Automation Projects

1. Introduction

You have completed the theoretical, tactical, and interview-prep journey of the Shell Scripting curriculum. You have mastered variables, loops, conditionals, text processing, network diagnostics, and defensive security. However, enterprise software engineering requires synthesis. In this final capstone chapter, you will transition from learning isolated commands to architecting complete, production-ready solutions. We will build three distinct, real-world tools that form the backbone of a standard DevOps toolkit: A Forensic Log Analyzer, an Autonomous System Health Monitor, and a Secure Backup Archiver.

2. Capstone Project 1: The Forensic Log Analyzer

The Requirement: A web server is currently under a DDoS attack. You must write a script that analyzes the massive NGINX access.log file, extracts all incoming IP addresses, identifies the top 5 most aggressive IPs, and outputs them to a clean, timestamped security report.

The Script (log_analyzer.sh):

sh
1234567891011121314151617181920212223242526272829
#!/bin/sh
set -e

LOG_FILE="/var/log/nginx/access.log"
REPORT_FILE="/tmp/security_report_$(date +%F).txt"

# Defensive Validation
if [ ! -f "$LOG_FILE" ]; then
    echo "ERROR: Log file missing!" >&2
    exit 1
fi

echo "Initiating Forensic Log Analysis..."

# Create the report header
echo "=== TOP 5 SUSPICIOUS IP ADDRESSES ===" > "$REPORT_FILE"
echo "Count | IP Address" >> "$REPORT_FILE"
echo "-----------------------------------" >> "$REPORT_FILE"

# The Text Processing Pipeline
# 1. awk prints the 1st column (IPs)
# 2. sort groups them alphabetically
# 3. uniq -c counts the occurrences
# 4. sort -nr ranks them highest to lowest numerically
# 5. head -n 5 grabs the top 5 lines
awk '{print $1}' "$LOG_FILE" | sort | uniq -c | sort -nr | head -n 5 >> "$REPORT_FILE"

echo "Analysis complete. Report successfully generated at $REPORT_FILE."
cat "$REPORT_FILE"

3. Capstone Project 2: The Autonomous System Monitor

The Requirement: You need a script that runs via cron every 5 minutes. It must check if the server's disk space exceeds a dangerous 90% threshold. If the server is dying, it must log the event and securely restart the crashing web application to self-heal the server.

The Script (health_monitor.sh):

sh
123456789101112131415161718192021222324252627
#!/bin/sh

# Configuration
DISK_THRESHOLD=90
SERVICE_TO_HEAL="apache2"
LOG_DEST="/var/log/system_health.log"

TIMESTAMP=$(date +"%Y-%m-%d %H:%M:%S")

# Extract the Disk Usage percentage using df, awk, and sed
CURRENT_USAGE=$(df -h / | awk '{print $5}' | tail -n 1 | sed 's/%//g')

if [ "$CURRENT_USAGE" -ge "$DISK_THRESHOLD" ]; then
    echo "[$TIMESTAMP] CRITICAL: Disk Usage is at ${CURRENT_USAGE}%. Executing self-healing protocol..." >> "$LOG_DEST"
    
    # Attempt to restart the service to clear temporary cache logs safely
    sudo systemctl restart "$SERVICE_TO_HEAL"
    
    if [ $? -eq 0 ]; then
        echo "[$TIMESTAMP] RECOVERY: $SERVICE_TO_HEAL restarted successfully to clear cache." >> "$LOG_DEST"
    else
        echo "[$TIMESTAMP] FATAL: Failed to restart service. Manual intervention absolutely required!" >> "$LOG_DEST"
    fi
else
    # Silent success for cron (Execute a colon which does nothing)
    : 
fi

4. Capstone Project 3: The Secure Backup Archiver

The Requirement: A daily cron job must compress the /var/www/html web directory into a .tar.gz file, append the exact timestamp to the filename to prevent overwriting, and store it in a secure backup directory, cleaning up partial files if the archival process crashes.

The Script (secure_backup.sh):

sh
123456789101112131415161718192021222324252627282930313233
#!/bin/sh
set -e
set -u

SOURCE_DIR="/var/www/html"
BACKUP_DIR="/archive/web_backups"
DATE=$(date +"%Y%m%d_%H%M%S")
BACKUP_FILE="${BACKUP_DIR}/website_backup_${DATE}.tar.gz"

# 1. Self-Healing Environment Setup
if [ ! -d "$BACKUP_DIR" ]; then
    mkdir -p "$BACKUP_DIR"
    chmod 700 "$BACKUP_DIR" # Restrict access to root only for security
fi

# 2. Trap for Cleanup on Failure
cleanup() {
    echo "ERROR: Backup failed unexpectedly! Cleaning up partial files..." >&2
    rm -f "$BACKUP_FILE"
}
# Catch Errors or Ctrl+C
trap cleanup ERR SIGINT

# 3. Execution
echo "Starting secure backup sequence..."

# Compress securely, discarding standard output to keep cron quiet, but allowing errors to trigger the trap
tar -czf "$BACKUP_FILE" "$SOURCE_DIR" > /dev/null

echo "SUCCESS: Backup securely stored at $BACKUP_FILE."

# Clear the trap since we succeeded naturally
trap - ERR SIGINT

5. Course Conclusion

You have reached the end of Shell Scripting – Complete Beginner to Advanced Guide. You have successfully evolved from a passive consumer of the operating system into an architect of automation. You have discarded the slow, error-prone paradigm of manual administration, opting instead for the programmatic, lightning-fast power of the Unix shell interpreter.

The Shell is not just a legacy language; it is the universal glue of the entire IT industry. Whether you are chaining together multi-million dollar cloud deployments in AWS, orchestrating complex Kubernetes CI/CD pipelines, or simply organizing files on your local macOS laptop—the shell script is the environment where raw computing power is harnessed and executed.

You are now equipped with the robust, fail-safe programming principles required to build enterprise-grade automation tools. Continue reading system logs, continue automating your daily routines, and remember the golden rule of DevOps: If you have to do it more than twice, write a script.

Congratulations on completing the course!

Finish this Chapter

Save your progress on your learning path and prepare for coding interview challenges.

Discussion

Join the discussion

Log in or create a free account to participate.

Sort: ·