Runtime Security Monitoring with Falco: Detect Container Threats in Real Time

Overview

Most container security tools focus on what happens before a workload runs — image scanning, policy enforcement, secrets detection. Falco fills the gap they all leave open: what your containers actually do at runtime.

Falco is a CNCF-graduated open-source runtime security engine. It attaches to the Linux kernel via an eBPF probe or kernel module and intercepts system calls in real time — execve, open, connect, setuid, and hundreds more. Every syscall is evaluated against a rule set, and anything that matches fires a structured JSON alert. No agents inside containers, no instrumented base images, no performance cliff.

In this project you will:

Deploy Falco on a Docker host using the eBPF driver (works on kernels ≥ 5.8)
Add Falcosidekick to fan alerts out to Slack, InfluxDB, and a local web UI
Write custom rules targeting homelab workloads (databases, reverse proxies, n8n)
Expose Falco metrics to an existing Prometheus + Grafana stack
Run the official event generator to verify every rule fires correctly

Prerequisites:

Linux host (Ubuntu 22.04 or Debian 12) with Docker and Docker Compose installed
Kernel ≥ 5.8 (check with uname -r); kernel headers installed
Root or passwordless sudo on the Docker host
Optional: Slack incoming webhook URL

Architecture

┌─────────────────────────────────────────────────────────────┐
│  Docker Host (Linux ≥ 5.8)                                  │
│                                                             │
│  eBPF Probe (auto-loaded by Falco)                          │
│       │  raw syscall events                                 │
│       ▼                                                     │
│  ┌─────────────────┐   JSON alerts   ┌──────────────────┐  │
│  │  Falco daemon   │ ──────────────► │  Falcosidekick   │  │
│  │  (rule engine)  │                 │  (alert router)  │  │
│  └─────────────────┘                 └────────┬─────────┘  │
│       │ :8765/metrics                         │            │
│       ▼                               ┌───────┼───────┐   │
│  ┌─────────────┐              Slack   │  InfluxDB     │   │
│  │ Prometheus  │◄─────────    #alerts │  /metrics     │   │
│  └──────┬──────┘              alert   │  Webhook      │   │
│         ▼                     notify  └───────┬───────┘   │
│  ┌──────────────┐                             ▼           │
│  │   Grafana    │                    Falcosidekick UI      │
│  │  Dashboard   │                    :2802 (browser)       │
│  └──────────────┘                                         │
└─────────────────────────────────────────────────────────────┘

Falco sits between the kernel and userspace. The eBPF probe streams syscall events to the Falco userspace daemon, which evaluates each event against a compiled rule set. Matching events emit to configured outputs. Falcosidekick acts as an aggregation and routing layer, forwarding alerts to one or many destinations simultaneously without any changes to Falco's own config.

Step 1 — Install Kernel Headers

The Falco eBPF probe compiles against the running kernel's headers on first boot. Without them, the driver loader falls back to the kmod path or fails entirely.

# Ubuntu / Debian
sudo apt update
sudo apt install -y linux-headers-$(uname -r) build-essential
 
# Verify the headers directory exists
ls /usr/src/linux-headers-$(uname -r)

On Proxmox hosts, the kernel package name differs:

# Proxmox VE
sudo apt install -y pve-headers-$(uname -r)

Note for LXC containers: Falco cannot run inside an unprivileged LXC container — it requires direct kernel access. Install Falco on the Proxmox host itself, or use a full VM.

Step 2 — Project Layout

Create a directory to hold all Falco config and compose files:

mkdir -p ~/falco-stack/rules
cd ~/falco-stack

The final layout will be:

falco-stack/
├── docker-compose.yml
├── falco.yaml
├── falcosidekick.yaml
└── rules/
    └── homelab.yaml

Step 3 — Docker Compose

docker-compose.yml

services:
  falco:
    image: falcosecurity/falco-no-driver:latest
    container_name: falco
    privileged: true
    pid: host
    volumes:
      - /var/run/docker.sock:/host/var/run/docker.sock
      - /dev:/host/dev
      - /proc:/host/proc:ro
      - /boot:/host/boot:ro
      - /lib/modules:/host/lib/modules:ro
      - /usr:/host/usr:ro
      - /etc:/host/etc:ro
      - ./falco.yaml:/etc/falco/falco.yaml:ro
      - ./rules:/etc/falco/rules.d:ro
    environment:
      HOST_ROOT: /host
      FALCO_DRIVER_LOADER_OPTIONS: --ebpf
    restart: unless-stopped
    networks:
      - falco-net
 
  falcosidekick:
    image: falcosecurity/falcosidekick:latest
    container_name: falcosidekick
    ports:
      - "2801:2801"
    volumes:
      - ./falcosidekick.yaml:/etc/falcosidekick/config.yaml:ro
    restart: unless-stopped
    networks:
      - falco-net
 
  falcosidekick-ui:
    image: falcosecurity/falcosidekick-ui:latest
    container_name: falcosidekick-ui
    ports:
      - "2802:2802"
    environment:
      FALCOSIDEKICK_UI_REDIS_URL: redis://redis:6379
    restart: unless-stopped
    networks:
      - falco-net
 
  redis:
    image: redis:7-alpine
    container_name: falco-redis
    restart: unless-stopped
    networks:
      - falco-net
 
networks:
  falco-net:
    driver: bridge

The falco-no-driver image ships without the kernel module pre-compiled. On startup it auto-downloads and compiles the eBPF probe for your exact kernel version. The privileged: true and pid: host flags are required for kernel-level access.

Step 4 — Falco Configuration

falco.yaml

rules_file:
  - /etc/falco/falco_rules.yaml
  - /etc/falco/falco_rules.local.yaml
  - /etc/falco/rules.d/
 
json_output: true
json_include_output_property: true
json_include_tags_property: true
 
log_stderr: true
log_syslog: false
log_level: info
 
priority: debug
 
outputs:
  rate: 1
  max_burst: 1000
 
# Forward all alerts to Falcosidekick
http_output:
  enabled: true
  url: "http://falcosidekick:2801/"
  user_agent: "falco/0.40"
 
stdout_output:
  enabled: true
 
# Prometheus metrics endpoint
metrics:
  enabled: true
  interval: 1m
  output_rule: true
  output_rule_names: true
  resource_utilization_enabled: true
  kernel_event_counters_enabled: true

Step 5 — Falcosidekick Configuration

falcosidekick.yaml

listenaddress: ""
listenport: 2801
debug: false
 
customfields:
  environment: homelab
  datacenter: azu-docker01
 
slack:
  webhookurl: "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
  channel: "#security-alerts"
  username: "Falco"
  icon: ":falcon_bird:"
  minimumpriority: "warning"
  messageformat: "rule={{.Rule}} | priority={{.Priority}} | container={{.OutputFields.container_name}}"
 
influxdb:
  hostport: "http://influxdb:8086"
  database: "falco"
  minimumpriority: ""
 
webui:
  url: "http://falcosidekick-ui:2802/ui"

Replace YOUR/WEBHOOK/URL with your Slack incoming webhook. If you skip Slack, remove that block — alerts will still appear in the Falcosidekick web UI and in docker logs falco.

Step 6 — Custom Homelab Rules

Default Falco rules cover broad categories but generate noise in a homelab. Replace them with targeted rules for your actual workloads. Create rules/homelab.yaml:

# Detect interactive shell spawned inside any database container
- rule: Shell Spawned in Database Container
  desc: Interactive shell in a DB container — likely hands-on exploitation post-compromise
  condition: >
    spawned_process and container and
    container.image.repository in (postgres, mysql, mariadb, redis, mongodb) and
    proc.name in (bash, sh, zsh, dash, ash)
  output: >
    Shell spawned in DB container
    (user=%user.name container=%container.name image=%container.image.repository
     cmd=%proc.cmdline parent=%proc.pname pid=%proc.pid)
  priority: CRITICAL
  tags: [container, database, shell]
 
# Detect Traefik making unexpected outbound connections (SSRF signal)
- rule: Unexpected Outbound from Reverse Proxy
  desc: Traefik initiating connections not on expected ports — possible SSRF or implant
  condition: >
    outbound and container.name = "traefik" and
    not fd.sport in (80, 443, 8080, 8443, 53) and
    not fd.sip.name endswith "letsencrypt.org"
  output: >
    Unexpected outbound from Traefik
    (connection=%fd.name container=%container.name sport=%fd.sport)
  priority: WARNING
  tags: [network, traefik, ssrf]
 
# Detect writes to /etc inside any container
- rule: Write Below /etc in Container
  desc: Process wrote a file under /etc — potential credential or config tampering
  condition: >
    open_write and container and
    fd.name startswith /etc and
    not proc.name in (apt, apt-get, dpkg, rpm, yum, dnf, apk, update-alternatives)
  output: >
    File written below /etc in container
    (file=%fd.name user=%user.name proc=%proc.name container=%container.name)
  priority: HIGH
  tags: [filesystem, container]
 
# Detect reads of /etc/shadow or private SSH keys
- rule: Sensitive Credential File Read
  desc: Process read a sensitive credential file outside of expected system tooling
  condition: >
    open_read and
    fd.name in (/etc/shadow, /etc/gshadow, /root/.ssh/id_rsa,
                /root/.ssh/id_ed25519, /root/.ssh/authorized_keys) and
    not proc.name in (sshd, su, sudo, passwd, login, systemd, useradd, usermod)
  output: >
    Sensitive credential file accessed
    (file=%fd.name proc=%proc.name user=%user.name
     container=%container.name pid=%proc.pid)
  priority: CRITICAL
  tags: [filesystem, credentials]
 
# Detect unexpected privileged containers starting up
- rule: Unexpected Privileged Container Started
  desc: A privileged container launched that is not in the approved list
  condition: >
    container.privileged = true and evt.type = container and
    not container.name in (falco, traefik, watchtower, portainer-agent)
  output: >
    Privileged container started outside approved list
    (container=%container.name image=%container.image.repository)
  priority: WARNING
  tags: [container, privilege-escalation]
 
# Detect n8n executing system binaries (potential code-exec via workflow)
- rule: System Binary Executed from n8n
  desc: n8n spawned a system utility — possible remote code execution via workflow node
  condition: >
    spawned_process and container.name = "n8n" and
    proc.name in (bash, sh, curl, wget, nc, ncat, python3, perl, ruby, php)
  output: >
    System binary executed inside n8n
    (proc=%proc.name cmd=%proc.cmdline user=%user.name container=%container.name)
  priority: HIGH
  tags: [container, rce, n8n]

Step 7 — Start the Stack

cd ~/falco-stack
docker compose up -d
 
# Watch driver loading and rule compilation
docker logs falco -f --tail 60

You should see output similar to:

* Setting up /usr/src/falco-driver-loader-0.40.x...
* Trying to find prebuilt eBPF probe for kernel 6.8.0-117-generic...
* eBPF probe found and loaded, Falco will use it
Fri May 27 12:00:00 2026: Falco version: 0.40.x (x86_64)
Fri May 27 12:00:00 2026: Loading rules from /etc/falco/falco_rules.yaml
Fri May 27 12:00:00 2026: Loading rules from /etc/falco/rules.d/homelab.yaml
Fri May 27 12:00:00 2026: Starting internal webserver, listening on port 8765

Open the Falcosidekick UI in your browser: http://<host-ip>:2802

Testing Detections

Use the following commands to trigger each homelab rule and confirm alerts appear in the UI and in Slack.

Test 1 — Shell in database container

# Triggers: Shell Spawned in Database Container
docker exec -it postgres bash -c "echo test && exit"

Test 2 — Sensitive file read

# Triggers: Sensitive Credential File Read
docker run --rm alpine cat /etc/shadow

Test 3 — Write to /etc in container

# Triggers: Write Below /etc in Container
docker run --rm alpine sh -c "echo test > /etc/test_falco"

Test 4 — Official Falco event generator (comprehensive)

The event generator cycles through all built-in Falco test cases automatically:

docker run -it --rm \
  --privileged \
  falcosecurity/event-generator run syscall --loop

Stop it with Ctrl-C after one pass. Each test case maps to a named Falco rule. Check the Falcosidekick UI for incoming alerts — you should see several dozen fire within 30 seconds.

Verifying via logs:

# Tail Falco output for live JSON alerts
docker logs falco -f | grep '"priority"'
 
# Count alerts in Falcosidekick since startup
docker logs falcosidekick 2>&1 | grep "Alert sent" | wc -l

Grafana Integration

If you already run Prometheus and Grafana (see the Prometheus + Grafana monitoring stack project), adding Falco metrics takes two minutes.

Add Falco to your Prometheus scrape config:

# prometheus.yml — append to scrape_configs
- job_name: falco
  static_configs:
    - targets:
        - falco:8765

Reload Prometheus:

docker exec prometheus kill -HUP 1

Import the community dashboard:

In Grafana, go to Dashboards → Import and enter ID 11914. This dashboard shows:

Rule hit rate over time (per rule name)
Alert priority distribution (CRITICAL / HIGH / WARNING / INFO)
Top-10 offending containers
Kernel event throughput and eBPF probe CPU usage

Tuning and Reducing Noise

Day-one false positives are normal. Falco's exception mechanism lets you carve out approved combinations of container + process without disabling a rule entirely.

Adding exceptions to a rule:

- rule: Write Below /etc in Container
  exceptions:
    - name: trusted_config_writers
      fields: [container.name, proc.name]
      values:
        - [authentik-server, python3]
        - [n8n, node]
        - [homepage, node]

Changing rule priority without editing the rule file:

Override only the priority in your local rules file so upstream upgrades don't clobber your change:

# rules/overrides.yaml
- rule: Launch Privileged Container
  enabled: false

Useful Falco CLI commands:

# List all loaded rules with priority
docker exec falco falco -L
 
# Validate a rule file without restarting
docker exec falco falco --validate /etc/falco/rules.d/homelab.yaml
 
# Dump all active rule names matching a tag
docker exec falco falco -L --tags container

Deployment Considerations

eBPF vs kernel module: On kernels ≥ 5.8, always use eBPF. It loads without recompiling on kernel upgrades and has lower latency than the kmod path. Set FALCO_DRIVER_LOADER_OPTIONS: --ebpf in the compose environment block.

Performance overhead: Falco adds approximately 2–8% CPU overhead on a typical homelab host depending on container density and syscall volume. The eBPF path is lighter than kmod. Monitor via the Grafana dashboard's "Kernel Event Counters" panel.

Rule persistence and backups: Falco itself is stateless — rules and falco.yaml are the source of truth. Commit your rules/ directory and falco.yaml to your infrastructure git repo. On redeploy, the stack re-reads them automatically.

Proxmox hosts: Install Falco directly on the PVE host (not inside an LXC) using the official Debian package. It monitors all VMs and containers running on that node simultaneously.

Log retention: Falcosidekick writes structured JSON to InfluxDB if configured. For long-term SIEM correlation, add an Elasticsearch output and index Falco events alongside Traefik access logs and Wazuh alerts.

Extensions and Next Steps

Kubernetes DaemonSet deployment Deploy Falco as a DaemonSet across your k3s or k8s cluster using the official Helm chart. Each node gets its own eBPF probe; alerts aggregate into a central Falcosidekick instance.

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm install falco falcosecurity/falco \
  --namespace falco --create-namespace \
  --set falcosidekick.enabled=true \
  --set falcosidekick.webui.enabled=true

Automated container quarantine Wire Falcosidekick's webhook output to an n8n workflow. On CRITICAL-priority alerts, the workflow calls the Docker API to stop the offending container and post a detailed Slack message with the full alert context and container name.

Container drift detection Enable Falco's experimental drift detection to flag any binary written into a running container's filesystem layer — a reliable indicator of post-exploitation activity or a supply-chain backdoor that survived image scanning.

# falco.yaml
container_engines:
  docker:
    enabled: true
plugins:
  - name: k8saudit
    library_path: libk8saudit.so

FalcoFiddle for rule prototyping Use falcofiddle.io to write and test rule syntax interactively against synthetic events before deploying to production. Eliminates the edit-restart-tail cycle for rule development.

MITRE ATT&CK mapping Falco's default rules are pre-tagged with MITRE ATT&CK technique IDs. Add tags: [T1059.004] to your custom rules and filter the Grafana dashboard by technique — giving you a live view of which kill-chain stages are triggering most often across your homelab.

Runtime security fills the gap that image scanning and network firewalls leave open. Once Falco is running and tuned to your workloads, you have a persistent, low-overhead detector watching every container on the host — one that cannot be bypassed by an attacker who already has code execution inside a container.

Overview

In this project you will:

Deploy Falco on a Docker host using the eBPF driver (works on kernels ≥ 5.8)
Add Falcosidekick to fan alerts out to Slack, InfluxDB, and a local web UI
Write custom rules targeting homelab workloads (databases, reverse proxies, n8n)
Expose Falco metrics to an existing Prometheus + Grafana stack
Run the official event generator to verify every rule fires correctly

Prerequisites:

Linux host (Ubuntu 22.04 or Debian 12) with Docker and Docker Compose installed
Kernel ≥ 5.8 (check with uname -r); kernel headers installed
Root or passwordless sudo on the Docker host
Optional: Slack incoming webhook URL

Architecture

┌─────────────────────────────────────────────────────────────┐
│  Docker Host (Linux ≥ 5.8)                                  │
│                                                             │
│  eBPF Probe (auto-loaded by Falco)                          │
│       │  raw syscall events                                 │
│       ▼                                                     │
│  ┌─────────────────┐   JSON alerts   ┌──────────────────┐  │
│  │  Falco daemon   │ ──────────────► │  Falcosidekick   │  │
│  │  (rule engine)  │                 │  (alert router)  │  │
│  └─────────────────┘                 └────────┬─────────┘  │
│       │ :8765/metrics                         │            │
│       ▼                               ┌───────┼───────┐   │
│  ┌─────────────┐              Slack   │  InfluxDB     │   │
│  │ Prometheus  │◄─────────    #alerts │  /metrics     │   │
│  └──────┬──────┘              alert   │  Webhook      │   │
│         ▼                     notify  └───────┬───────┘   │
│  ┌──────────────┐                             ▼           │
│  │   Grafana    │                    Falcosidekick UI      │
│  │  Dashboard   │                    :2802 (browser)       │
│  └──────────────┘                                         │
└─────────────────────────────────────────────────────────────┘

Step 1 — Install Kernel Headers

The Falco eBPF probe compiles against the running kernel's headers on first boot. Without them, the driver loader falls back to the kmod path or fails entirely.

# Ubuntu / Debian
sudo apt update
sudo apt install -y linux-headers-$(uname -r) build-essential
 
# Verify the headers directory exists
ls /usr/src/linux-headers-$(uname -r)

On Proxmox hosts, the kernel package name differs:

# Proxmox VE
sudo apt install -y pve-headers-$(uname -r)

Note for LXC containers: Falco cannot run inside an unprivileged LXC container — it requires direct kernel access. Install Falco on the Proxmox host itself, or use a full VM.

Step 2 — Project Layout

Create a directory to hold all Falco config and compose files:

mkdir -p ~/falco-stack/rules
cd ~/falco-stack

The final layout will be:

falco-stack/
├── docker-compose.yml
├── falco.yaml
├── falcosidekick.yaml
└── rules/
    └── homelab.yaml

Step 3 — Docker Compose

docker-compose.yml

services:
  falco:
    image: falcosecurity/falco-no-driver:latest
    container_name: falco
    privileged: true
    pid: host
    volumes:
      - /var/run/docker.sock:/host/var/run/docker.sock
      - /dev:/host/dev
      - /proc:/host/proc:ro
      - /boot:/host/boot:ro
      - /lib/modules:/host/lib/modules:ro
      - /usr:/host/usr:ro
      - /etc:/host/etc:ro
      - ./falco.yaml:/etc/falco/falco.yaml:ro
      - ./rules:/etc/falco/rules.d:ro
    environment:
      HOST_ROOT: /host
      FALCO_DRIVER_LOADER_OPTIONS: --ebpf
    restart: unless-stopped
    networks:
      - falco-net
 
  falcosidekick:
    image: falcosecurity/falcosidekick:latest
    container_name: falcosidekick
    ports:
      - "2801:2801"
    volumes:
      - ./falcosidekick.yaml:/etc/falcosidekick/config.yaml:ro
    restart: unless-stopped
    networks:
      - falco-net
 
  falcosidekick-ui:
    image: falcosecurity/falcosidekick-ui:latest
    container_name: falcosidekick-ui
    ports:
      - "2802:2802"
    environment:
      FALCOSIDEKICK_UI_REDIS_URL: redis://redis:6379
    restart: unless-stopped
    networks:
      - falco-net
 
  redis:
    image: redis:7-alpine
    container_name: falco-redis
    restart: unless-stopped
    networks:
      - falco-net
 
networks:
  falco-net:
    driver: bridge

Step 4 — Falco Configuration

falco.yaml

rules_file:
  - /etc/falco/falco_rules.yaml
  - /etc/falco/falco_rules.local.yaml
  - /etc/falco/rules.d/
 
json_output: true
json_include_output_property: true
json_include_tags_property: true
 
log_stderr: true
log_syslog: false
log_level: info
 
priority: debug
 
outputs:
  rate: 1
  max_burst: 1000
 
# Forward all alerts to Falcosidekick
http_output:
  enabled: true
  url: "http://falcosidekick:2801/"
  user_agent: "falco/0.40"
 
stdout_output:
  enabled: true
 
# Prometheus metrics endpoint
metrics:
  enabled: true
  interval: 1m
  output_rule: true
  output_rule_names: true
  resource_utilization_enabled: true
  kernel_event_counters_enabled: true

Step 5 — Falcosidekick Configuration

falcosidekick.yaml

listenaddress: ""
listenport: 2801
debug: false
 
customfields:
  environment: homelab
  datacenter: azu-docker01
 
slack:
  webhookurl: "https://hooks.slack.com/services/YOUR/WEBHOOK/URL"
  channel: "#security-alerts"
  username: "Falco"
  icon: ":falcon_bird:"
  minimumpriority: "warning"
  messageformat: "rule={{.Rule}} | priority={{.Priority}} | container={{.OutputFields.container_name}}"
 
influxdb:
  hostport: "http://influxdb:8086"
  database: "falco"
  minimumpriority: ""
 
webui:
  url: "http://falcosidekick-ui:2802/ui"

Replace YOUR/WEBHOOK/URL with your Slack incoming webhook. If you skip Slack, remove that block — alerts will still appear in the Falcosidekick web UI and in docker logs falco.

Step 6 — Custom Homelab Rules

Default Falco rules cover broad categories but generate noise in a homelab. Replace them with targeted rules for your actual workloads. Create rules/homelab.yaml:

# Detect interactive shell spawned inside any database container
- rule: Shell Spawned in Database Container
  desc: Interactive shell in a DB container — likely hands-on exploitation post-compromise
  condition: >
    spawned_process and container and
    container.image.repository in (postgres, mysql, mariadb, redis, mongodb) and
    proc.name in (bash, sh, zsh, dash, ash)
  output: >
    Shell spawned in DB container
    (user=%user.name container=%container.name image=%container.image.repository
     cmd=%proc.cmdline parent=%proc.pname pid=%proc.pid)
  priority: CRITICAL
  tags: [container, database, shell]
 
# Detect Traefik making unexpected outbound connections (SSRF signal)
- rule: Unexpected Outbound from Reverse Proxy
  desc: Traefik initiating connections not on expected ports — possible SSRF or implant
  condition: >
    outbound and container.name = "traefik" and
    not fd.sport in (80, 443, 8080, 8443, 53) and
    not fd.sip.name endswith "letsencrypt.org"
  output: >
    Unexpected outbound from Traefik
    (connection=%fd.name container=%container.name sport=%fd.sport)
  priority: WARNING
  tags: [network, traefik, ssrf]
 
# Detect writes to /etc inside any container
- rule: Write Below /etc in Container
  desc: Process wrote a file under /etc — potential credential or config tampering
  condition: >
    open_write and container and
    fd.name startswith /etc and
    not proc.name in (apt, apt-get, dpkg, rpm, yum, dnf, apk, update-alternatives)
  output: >
    File written below /etc in container
    (file=%fd.name user=%user.name proc=%proc.name container=%container.name)
  priority: HIGH
  tags: [filesystem, container]
 
# Detect reads of /etc/shadow or private SSH keys
- rule: Sensitive Credential File Read
  desc: Process read a sensitive credential file outside of expected system tooling
  condition: >
    open_read and
    fd.name in (/etc/shadow, /etc/gshadow, /root/.ssh/id_rsa,
                /root/.ssh/id_ed25519, /root/.ssh/authorized_keys) and
    not proc.name in (sshd, su, sudo, passwd, login, systemd, useradd, usermod)
  output: >
    Sensitive credential file accessed
    (file=%fd.name proc=%proc.name user=%user.name
     container=%container.name pid=%proc.pid)
  priority: CRITICAL
  tags: [filesystem, credentials]
 
# Detect unexpected privileged containers starting up
- rule: Unexpected Privileged Container Started
  desc: A privileged container launched that is not in the approved list
  condition: >
    container.privileged = true and evt.type = container and
    not container.name in (falco, traefik, watchtower, portainer-agent)
  output: >
    Privileged container started outside approved list
    (container=%container.name image=%container.image.repository)
  priority: WARNING
  tags: [container, privilege-escalation]
 
# Detect n8n executing system binaries (potential code-exec via workflow)
- rule: System Binary Executed from n8n
  desc: n8n spawned a system utility — possible remote code execution via workflow node
  condition: >
    spawned_process and container.name = "n8n" and
    proc.name in (bash, sh, curl, wget, nc, ncat, python3, perl, ruby, php)
  output: >
    System binary executed inside n8n
    (proc=%proc.name cmd=%proc.cmdline user=%user.name container=%container.name)
  priority: HIGH
  tags: [container, rce, n8n]

Step 7 — Start the Stack

cd ~/falco-stack
docker compose up -d
 
# Watch driver loading and rule compilation
docker logs falco -f --tail 60

You should see output similar to:

* Setting up /usr/src/falco-driver-loader-0.40.x...
* Trying to find prebuilt eBPF probe for kernel 6.8.0-117-generic...
* eBPF probe found and loaded, Falco will use it
Fri May 27 12:00:00 2026: Falco version: 0.40.x (x86_64)
Fri May 27 12:00:00 2026: Loading rules from /etc/falco/falco_rules.yaml
Fri May 27 12:00:00 2026: Loading rules from /etc/falco/rules.d/homelab.yaml
Fri May 27 12:00:00 2026: Starting internal webserver, listening on port 8765

Open the Falcosidekick UI in your browser: http://<host-ip>:2802

Testing Detections

Use the following commands to trigger each homelab rule and confirm alerts appear in the UI and in Slack.

Test 1 — Shell in database container

# Triggers: Shell Spawned in Database Container
docker exec -it postgres bash -c "echo test && exit"

Test 2 — Sensitive file read

# Triggers: Sensitive Credential File Read
docker run --rm alpine cat /etc/shadow

Test 3 — Write to /etc in container

# Triggers: Write Below /etc in Container
docker run --rm alpine sh -c "echo test > /etc/test_falco"

Test 4 — Official Falco event generator (comprehensive)

The event generator cycles through all built-in Falco test cases automatically:

docker run -it --rm \
  --privileged \
  falcosecurity/event-generator run syscall --loop

Stop it with Ctrl-C after one pass. Each test case maps to a named Falco rule. Check the Falcosidekick UI for incoming alerts — you should see several dozen fire within 30 seconds.

Verifying via logs:

# Tail Falco output for live JSON alerts
docker logs falco -f | grep '"priority"'
 
# Count alerts in Falcosidekick since startup
docker logs falcosidekick 2>&1 | grep "Alert sent" | wc -l

Grafana Integration

If you already run Prometheus and Grafana (see the Prometheus + Grafana monitoring stack project), adding Falco metrics takes two minutes.

Add Falco to your Prometheus scrape config:

# prometheus.yml — append to scrape_configs
- job_name: falco
  static_configs:
    - targets:
        - falco:8765

Reload Prometheus:

docker exec prometheus kill -HUP 1

Import the community dashboard:

In Grafana, go to Dashboards → Import and enter ID 11914. This dashboard shows:

Rule hit rate over time (per rule name)
Alert priority distribution (CRITICAL / HIGH / WARNING / INFO)
Top-10 offending containers
Kernel event throughput and eBPF probe CPU usage

Tuning and Reducing Noise

Day-one false positives are normal. Falco's exception mechanism lets you carve out approved combinations of container + process without disabling a rule entirely.

Adding exceptions to a rule:

- rule: Write Below /etc in Container
  exceptions:
    - name: trusted_config_writers
      fields: [container.name, proc.name]
      values:
        - [authentik-server, python3]
        - [n8n, node]
        - [homepage, node]

Changing rule priority without editing the rule file:

Override only the priority in your local rules file so upstream upgrades don't clobber your change:

# rules/overrides.yaml
- rule: Launch Privileged Container
  enabled: false

Useful Falco CLI commands:

# List all loaded rules with priority
docker exec falco falco -L
 
# Validate a rule file without restarting
docker exec falco falco --validate /etc/falco/rules.d/homelab.yaml
 
# Dump all active rule names matching a tag
docker exec falco falco -L --tags container

Deployment Considerations

Proxmox hosts: Install Falco directly on the PVE host (not inside an LXC) using the official Debian package. It monitors all VMs and containers running on that node simultaneously.

Extensions and Next Steps

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm install falco falcosecurity/falco \
  --namespace falco --create-namespace \
  --set falcosidekick.enabled=true \
  --set falcosidekick.webui.enabled=true

# falco.yaml
container_engines:
  docker:
    enabled: true
plugins:
  - name: k8saudit
    library_path: libk8saudit.so

Runtime Security Monitoring with Falco: Detect Container Threats in Real Time

Tools & Technologies

Overview

Architecture

Step 1 — Install Kernel Headers

Step 2 — Project Layout

Step 3 — Docker Compose

Step 4 — Falco Configuration

Step 5 — Falcosidekick Configuration

Step 6 — Custom Homelab Rules

Step 7 — Start the Stack

Testing Detections

Grafana Integration

Tuning and Reducing Noise

Deployment Considerations

Extensions and Next Steps

Pi-hole v6 + Unbound: Network-Wide DNS Sinkhole with

Build a Production Monitoring Stack with Prometheus and

How CosmicBytez Labs Is Built: Next.js 16, velite, and

Runtime Security Monitoring with Falco: Detect Container Threats in Real Time

Tools & Technologies

Overview

Architecture

Step 1 — Install Kernel Headers

Step 2 — Project Layout

Step 3 — Docker Compose

Step 4 — Falco Configuration

Step 5 — Falcosidekick Configuration

Step 6 — Custom Homelab Rules

Step 7 — Start the Stack

Testing Detections

Grafana Integration

Tuning and Reducing Noise

Deployment Considerations

Extensions and Next Steps

Pi-hole v6 + Unbound: Network-Wide DNS Sinkhole with

Build a Production Monitoring Stack with Prometheus and

How CosmicBytez Labs Is Built: Next.js 16, velite, and