Wazuh NetFlow Monitoring Pipeline
Network flow visibility integrated into Wazuh SIEM using pmacctd, Python normalization, and 24 custom detection rules, validated against real internet traffic.
Wazuh NetFlow Monitoring Pipeline
Project Description
A complete network flow monitoring pipeline that integrates traffic metadata into Wazuh SIEM on a two VM architecture. The system captures raw flow data using pmacctd, normalizes it into a custom Python script, and forwards it to Wazuh for detection using 24 custom rules covering 10 threat categories.
Deployed on a live cloud VM, the pipeline detected automated RDP scanners, Telnet crawlers, MySQL and PostgreSQL port scanners, NetBIOS broadcasts, and C2 beaconing patterns within hours of going live.
Addresses, hostnames, and interface names are redacted so the pipeline can be published without exposing the host. The traffic samples and the detection results are real.
Goals & Objectives
Bring flow visibility into Wazuh
Add network flow metadata to an existing Wazuh deployment using only pmacctd, Python, and the built in decoder, no NetFlow appliance.
Normalize raw flow to JSON
Turn pmacctd output into flat structured JSON with an nf_ prefix that Wazuh parses natively, filtering noise before it reaches the manager.
Detect on real traffic
Fire 24 custom rules across 10 threat categories and validate them against live internet traffic rather than synthetic tests.
Two VM, reproducible pipeline
Keep the whole capture to dashboard flow on a two VM architecture that a home lab or cloud host can reproduce.
Architecture & Workflow
flowchart TD subgraph VM2["VM 2 - Linux Agent + NetFlow Collector"] A["Network Traffic"] --> B["pmacctd\n(Traffic Metadata Capture)"] B -->|"Raw JSON + timestamps"| C["Raw Flow Log\n/var/log/netflow/netflow_raw.json"] C --> D["Python Normalization Script\n(filter + flatten + normalize)"] D -->|"Flat normalized JSON"| E["Normalized Log\n/var/log/netflow/netflow_wazuh.json"] E --> F["Wazuh Agent\n(localfile monitor + forward)"] end
subgraph VM1["VM 1 - Wazuh All-in-One Server"]
G["Wazuh Manager\n(Log Ingestion)"]
G --> H["Built-in JSON Decoder\n(flat field extraction)"]
H --> I["Custom Rules\n(117001 – 117024)"]
I --> J["Wazuh Indexer\n(Alert Storage)"]
J --> K["Wazuh Dashboard\n(Alert Visualization)"]
end
F -- "Agent Connection\n(1514/TCP)" --> G
Implementation
The pipeline runs across two VMs. On the collector, pmacctd captures traffic metadata with real timestamps and writes raw JSON, flushing every 60 seconds. A Python normalization script runs once a minute from cron, reads the new lines since its last run, filters noise on either endpoint and drops flows where both ends are internal, then writes flat structured JSON with an nf_ prefix.
The Wazuh agent on the collector monitors that normalized file as a JSON localfile and forwards it to the manager, where the built in JSON decoder parses it and 24 custom rules (117001 to 117024) evaluate the fields and raise alerts across 10 threat categories. The alerts land in the indexer and render on a custom dashboard.
Two intervals set the pace: the pmacctd flush and the cron run, so an event reaches the dashboard roughly one to two minutes after the packet was seen. A marker file tracks the last processed line so the two schedules do not need to align and nothing is processed twice. The pipeline was validated against real internet traffic on a live cloud VM.
Tech Stack Used
Log ingestion, built in JSON decoding, 24 custom detection rules, and the alert dashboard.
Passive network flow metadata capture on the collector VM, flushing raw JSON every 60 seconds.
Reads raw flow, filters noise and internal to internal traffic, and writes flat nf_ prefixed JSON on a cron schedule.
Stores the alerts and renders the flow visualizations, including the threat category and traffic timeline panels.
Key Features & Deliverables
Agentless flow capture to Wazuh
pmacctd captures flow metadata with real timestamps and forwards it through the Wazuh agent, no NetFlow hardware required.
24 custom detection rules
Rules 117001 to 117024 across 10 threat categories, from port scans to C2 beaconing, matching flat JSON flow fields.
Validated on real traffic
4,472 firing events over 24 hours on a live cloud VM, with confirmed detections tied to real scanner source IPs.
Evidence based evaluation
A filter that would have dropped every inbound alert corrected, three conflicting alert totals reconciled to the dashboard.
Key Metrics
Results & Outcome
A working detection pipeline from traffic capture to dashboard alert, validated on a live cloud VM within hours of deployment.
Over a 24 hour window the dashboard recorded 4,472 firing events across 10 threat categories. Confirmed detections against real internet traffic include an automated RDP scanner (rule 117010), a Telnet scanner from 8 different source IPs (117013), MySQL 3306 and PostgreSQL 5432 port scans (117015), NetBIOS traffic at 2,470 hits (117020), a VNC 5900 scan (117022), and C2 beaconing patterns (117021).
The result is not only a working pipeline but a documented understanding of what the rules actually detect. An evidence based evaluation reconciled three conflicting alert totals against the dashboard, corrected a filter that would otherwise have dropped every inbound alert, and recorded which of the 24 rules have been observed firing and which have not.
Screenshots & Demo
NetFlow Dashboard Overview
A centralized Wazuh dashboard for monitoring NetFlow telemetry, showing traffic correlation, firing events, threat categories, alert severity, protocol distribution, source and destination activity, and representative network events for security analysis.
External Threat Detection Events
Filtered Wazuh Discover view showing NetFlow security events with external source traffic, focusing on medium to high severity alerts such as NetBIOS activity, suspicious port scanning, and potential network anomaly indicators.
High Severity NetFlow Alerts
Wazuh Discover view filtered to NetFlow alerts with severity level 9 and above, highlighting high-risk network events such as NetBIOS traffic detection, potential lateral movement, and suspicious external communication patterns.
Top Attacker IP Investigation
Wazuh Discover view filtered by NetFlow source IP 103.153.61.85, showing repeated security events related to NetBIOS traffic detection, network anomaly indicators, and potential lateral movement activity within the monitored traffic.
Wazuh Logtest Database Port Alert
Validation of a custom Wazuh NetFlow rule using wazuh-logtest, showing successful JSON decoding and rule matching for external access to database port 3306, generating a level 10 alert categorized as network anomaly and database activity.
Wazuh Logtest NetBIOS Traffic Alert
Validation of a custom Wazuh NetFlow rule using wazuh-logtest, showing successful JSON decoding and rule matching for NetBIOS UDP traffic on port 138, generating a level 9 alert categorized as network anomaly and potential lateral movement activity.
Wazuh Logtest RDP Flow Event
Validation of NetFlow JSON parsing using wazuh-logtest, showing successful decoding of TCP traffic to destination port 3389 and rule matching as a general NetFlow event with rule ID 117001 and severity level 3.
Wazuh Logtest Suspicious Port Alert
Validation of a custom Wazuh NetFlow rule using wazuh-logtest, showing successful JSON decoding and rule matching for TCP traffic to suspicious destination port 4444, generating a level 7 alert categorized as a network anomaly.
Wazuh Logtest Telnet Connection Alert
Validation of a custom Wazuh NetFlow rule using wazuh-logtest, showing successful JSON decoding and rule matching for TCP traffic to Telnet port 23, generating a level 10 alert categorized as a network anomaly and cleartext protocol activity.
Interested in This Project?
Discuss your security needs or explore collaboration opportunities.