Nginx + ModSecurity WAF: Protecting Web Apps with OWASP CRS

Introduction

Most SMBs and homelab operators put a lot of trust in Nginx as a reverse proxy in front of their web apps — but without a Web Application Firewall (WAF), that reverse proxy is transparent glass. SQL injection, cross-site scripting (XSS), remote file inclusion, and dozens of other OWASP Top 10 attack classes sail right through it.

ModSecurity is the industry-standard open-source WAF engine. Paired with the OWASP Core Rule Set (CRS), it gives you a maintained, community-vetted ruleset that blocks the majority of automated attack traffic. This guide walks you from a bare Nginx install to a fully operational WAF in detection mode, then hardens it to blocking mode and shows you how to tune rules so legitimate traffic doesn't get caught in the net.

By the end you will have:

• ModSecurity v3 compiled and loaded as an Nginx dynamic module
• OWASP CRS 4.x configured and active
• Detection mode → blocking mode workflow understood
• A test harness to verify SQL injection and XSS are blocked
• Log forwarding so WAF events surface in your SIEM

Prerequisites

Before starting, confirm these are in place:

# Confirm Nginx is running
nginx -v
systemctl is-active nginx
 
# Confirm build tools are present (we'll install the rest)
gcc --version
make --version

You will need approximately 1 GB of disk space for build dependencies and the compiled module. The entire process takes about 20–30 minutes on a modest VPS.

Step 1 — Install Build Dependencies

ModSecurity v3 (libmodsecurity3) must be compiled from source on most distributions to get the current stable release. Start by pulling in everything the build needs:

sudo apt update
sudo apt install -y \
  build-essential \
  git \
  libpcre3 libpcre3-dev \
  libssl-dev \
  libtool \
  libxml2-dev \
  libcurl4-openssl-dev \
  libgeoip-dev \
  liblmdb-dev \
  libmaxminddb-dev \
  libfuzzy-dev \
  ssdeep \
  pkg-config \
  zlib1g-dev \
  libyajl-dev

Step 2 — Build and Install libmodsecurity3

Clone the ModSecurity repository and build the library:

cd /opt
sudo git clone --depth 1 -b v3/master https://github.com/owasp-modsecurity/ModSecurity.git
cd ModSecurity
sudo git submodule init
sudo git submodule update
 
sudo ./build.sh
sudo ./configure
sudo make -j"$(nproc)"
sudo make install

The library installs to /usr/local/modsecurity/. Verify:

ls /usr/local/modsecurity/lib/libmodsecurity.so*
# Expected: libmodsecurity.so  libmodsecurity.so.3  libmodsecurity.so.3.x.x

Step 3 — Build the Nginx Connector Module

The Nginx connector wraps libmodsecurity3 as a dynamic module. You must build it against the exact same Nginx version currently installed:

NGINX_VERSION=$(nginx -v 2>&1 | grep -oP '(?<=nginx/)\S+')
echo "Building connector for Nginx $NGINX_VERSION"
 
cd /opt
sudo git clone --depth 1 https://github.com/owasp-modsecurity/ModSecurity-nginx.git
 
# Download matching Nginx source
sudo wget "http://nginx.org/download/nginx-${NGINX_VERSION}.tar.gz"
sudo tar -xzf "nginx-${NGINX_VERSION}.tar.gz"
 
cd "nginx-${NGINX_VERSION}"
 
# Capture the flags Nginx was compiled with
NGINX_FLAGS=$(nginx -V 2>&1 | grep "configure arguments" | sed 's/configure arguments: //')
 
# Build only the dynamic module — do NOT install, only copy the .so
sudo ./configure $NGINX_FLAGS --add-dynamic-module=/opt/ModSecurity-nginx
sudo make modules
sudo cp objs/ngx_http_modsecurity_module.so /etc/nginx/modules/

Step 4 — Load the Module in Nginx

Add the module load directive to the very top of /etc/nginx/nginx.conf (before any events or http blocks):

load_module modules/ngx_http_modsecurity_module.so;

Test the config, then reload:

sudo nginx -t
sudo systemctl reload nginx

If nginx -t fails with a missing symbol error, confirm the libmodsecurity.so is on the linker path:

echo '/usr/local/modsecurity/lib' | sudo tee /etc/ld.so.conf.d/modsecurity.conf
sudo ldconfig
sudo nginx -t

Step 5 — Configure ModSecurity Core Settings

Create the ModSecurity working directory and copy the recommended base config:

sudo mkdir -p /etc/nginx/modsecurity
sudo cp /opt/ModSecurity/modsecurity.conf-recommended /etc/nginx/modsecurity/modsecurity.conf
sudo cp /opt/ModSecurity/unicode.mapping /etc/nginx/modsecurity/

Open /etc/nginx/modsecurity/modsecurity.conf and make these key changes:

sudo sed -i 's/^SecRuleEngine DetectionOnly/SecRuleEngine DetectionOnly/' \
  /etc/nginx/modsecurity/modsecurity.conf
# (We'll switch to On after tuning — leave DetectionOnly for now)
 
# Enable audit logging with relevant parts
sudo sed -i 's/^SecAuditLogParts ABIJDEFHZ/SecAuditLogParts ABCFHIJZ/' \
  /etc/nginx/modsecurity/modsecurity.conf

Set the audit log path:

sudo mkdir -p /var/log/modsecurity
sudo chown www-data:www-data /var/log/modsecurity
 
# In modsecurity.conf, confirm or set:
# SecAuditLog /var/log/modsecurity/audit.log
# SecDebugLog /var/log/modsecurity/debug.log
# SecDebugLogLevel 0

Create a main include file that will pull in CRS rules:

sudo tee /etc/nginx/modsecurity/main.conf > /dev/null <<'EOF'
Include /etc/nginx/modsecurity/modsecurity.conf
Include /etc/nginx/modsecurity/crs/crs-setup.conf
Include /etc/nginx/modsecurity/crs/rules/*.conf
EOF

Step 6 — Install OWASP Core Rule Set

cd /opt
sudo git clone --depth 1 https://github.com/coreruleset/coreruleset.git
sudo cp -r coreruleset /etc/nginx/modsecurity/crs
 
# Copy the default CRS setup file
sudo cp /etc/nginx/modsecurity/crs/crs-setup.conf.example \
        /etc/nginx/modsecurity/crs/crs-setup.conf

The default crs-setup.conf uses Paranoia Level 1 (PL1), which has the lowest false-positive rate. This is the right starting point for most deployments. You can increase it later once you understand your traffic patterns.

Step 7 — Enable WAF in Your Nginx Virtual Host

Edit the Nginx server block for your site (e.g. /etc/nginx/sites-available/myapp):

server {
    listen 443 ssl http2;
    server_name myapp.example.com;
 
    # --- ModSecurity ---
    modsecurity on;
    modsecurity_rules_file /etc/nginx/modsecurity/main.conf;
    # -------------------
 
    ssl_certificate     /etc/ssl/certs/myapp.crt;
    ssl_certificate_key /etc/ssl/private/myapp.key;
 
    location / {
        proxy_pass http://127.0.0.1:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Reload:

sudo nginx -t && sudo systemctl reload nginx

Step 8 — Verify Detection Mode is Working

Send a classic SQL injection probe and confirm ModSecurity logs it:

# Simple SQLi test — should return 200 (detection only) but log an alert
curl -s -o /dev/null -w "%{http_code}" \
  "http://localhost/?id=1%27%20OR%20%271%27%3D%271"
# Expected: 200 (still passes through in DetectionOnly mode)
 
# Check the audit log for the triggered rule
sudo tail -f /var/log/modsecurity/audit.log | grep -A5 "SQLI\|942"

You should see an entry referencing rule IDs in the 942xxx range (SQL injection detection).

Step 9 — Switch to Blocking Mode

Once you have reviewed the audit log for a period (minimum 24–48 hours of real traffic) and addressed false positives, enable enforcement:

sudo sed -i \
  's/^SecRuleEngine DetectionOnly/SecRuleEngine On/' \
  /etc/nginx/modsecurity/modsecurity.conf
 
sudo nginx -t && sudo systemctl reload nginx

Now re-run the SQLi test:

curl -s -o /dev/null -w "%{http_code}" \
  "http://localhost/?id=1%27%20OR%20%271%27%3D%271"
# Expected: 403

Test XSS blocking:

curl -s -o /dev/null -w "%{http_code}" \
  "http://localhost/?q=<script>alert(1)</script>"
# Expected: 403

Step 10 — Tuning False Positives

False positives are inevitable, especially for apps that accept rich text input or have unusual URL patterns. Never disable entire rule ranges — use targeted exclusions.

Identify the noisy rule ID from the audit log:

sudo grep "id \"" /var/log/modsecurity/audit.log | \
  grep -oP 'id "\K[0-9]+' | sort | uniq -c | sort -rn | head -20

Disable a rule for a specific URI:

# In your Nginx server block or a separate inclusion file:
location /admin/upload {
    modsecurity_rules '
        SecRuleRemoveById 200002
        SecRuleRemoveById 920420
    ';
    proxy_pass http://127.0.0.1:8080;
}

Disable a rule globally (use sparingly):

# In /etc/nginx/modsecurity/crs/rules/EXCLUSION-RULES-AFTER-CRS.conf.example
# Copy to EXCLUSION-RULES-AFTER-CRS.conf and add:
SecRuleRemoveById 941100

Increase Paranoia Level after tuning:

# In /etc/nginx/modsecurity/crs/crs-setup.conf:
# Change: tx.blocking_paranoia_level=1
# To:     tx.blocking_paranoia_level=2

Verification and Testing

Run a quick battery of attack probes to confirm the WAF is working end-to-end:

#!/bin/bash
TARGET="http://localhost"
 
echo "=== WAF Validation Suite ==="
 
test_block() {
  local label="$1"
  local url="$2"
  local code
  code=$(curl -s -o /dev/null -w "%{http_code}" "$url")
  if [ "$code" = "403" ]; then
    echo "PASS [$label] — blocked (403)"
  else
    echo "FAIL [$label] — got $code (expected 403)"
  fi
}
 
# SQL injection
test_block "SQLi basic"       "$TARGET/?id=1' OR '1'='1"
test_block "SQLi UNION"       "$TARGET/?q=UNION+SELECT+1,2,3--"
 
# XSS
test_block "XSS script tag"   "$TARGET/?q=<script>alert(1)</script>"
test_block "XSS event attr"   "$TARGET/?q=<img+onerror=alert(1)+src=x>"
 
# Path traversal
test_block "Path traversal"   "$TARGET/?file=../../etc/passwd"
 
# Remote file inclusion
test_block "RFI"              "$TARGET/?page=http://evil.example.com/shell.php"
 
echo "=== Done ==="

Save as waf-test.sh, chmod +x waf-test.sh, and run it:

chmod +x waf-test.sh
./waf-test.sh

All six tests should print PASS.

Log Forwarding to SIEM

If you are running a SIEM like Wazuh or a log aggregator like Loki, forward ModSecurity events using promtail or filebeat:

# promtail scrape config addition
- job_name: modsecurity
  static_configs:
    - targets:
        - localhost
      labels:
        job: modsecurity
        __path__: /var/log/modsecurity/audit.log

For Wazuh, the default ossec.conf will pick up the audit log if you add a localfile stanza:

<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/modsecurity/audit.log</location>
</localfile>

Troubleshooting

Nginx fails to start after adding load_module: The .so version must exactly match the running Nginx version. Re-run the build in Step 3, ensuring $NGINX_VERSION matches nginx -v. Check ldd /etc/nginx/modules/ngx_http_modsecurity_module.so to confirm libmodsecurity.so.3 resolves.

All requests return 403 immediately: You likely have a misconfigured crs-setup.conf or a leftover rule from a previous install. Check /var/log/modsecurity/audit.log to see which rule ID is triggering, then trace it in /etc/nginx/modsecurity/crs/rules/.

ModSecurity engine shows as disabled: Run curl -sv http://localhost/ 2>&1 | grep -i modsecurity — some builds emit ModSecurity: not loaded. Confirm modsecurity on; is inside the correct server {} block and that modsecurity_rules_file points to a file that actually exists on disk.

Legitimate file uploads get blocked: Rule 200002 (request body too large) or 200003 (multipart parse error) often triggers on uploads. Increase the body limit in modsecurity.conf:

SecRequestBodyLimit 13107200
SecRequestBodyNoFilesLimit 131072

Or exclude the upload endpoint using SecRuleRemoveById scoped to that location block.

High CPU during large file transfers: ModSecurity inspects request bodies by default. For endpoints streaming large binary uploads, add SecRequestBodyAccess Off in a scoped location block to skip body inspection without disabling the WAF globally.

Summary

You now have a working ModSecurity v3 WAF in front of your Nginx reverse proxy, backed by the OWASP Core Rule Set. The key takeaways:

• Build from source — distro packages often lag by a major version or miss the Nginx connector
• Start in DetectionOnly — collect real traffic logs before enforcing blocks
• Tune with surgical exclusions — target specific rule IDs and URIs, never disable rule families wholesale
• Paranoia Level 1 → 2 → 3 is a progression, not a checkbox — each level adds rules and false-positive risk
• Watch the audit log continuously — ModSecurity events are your earliest signal of active exploitation attempts

From here, consider layering ModSecurity with CrowdSec for reputation-based IP blocking, giving you both behavioral detection (ModSecurity) and community threat intelligence (CrowdSec) in a single Nginx stack.