Lightweight Sigma Rule Detection Engine for Small Teams Without a SIEM

The problem small teams have with log-based detection

A small software company, a school IT office or a clinic chain has servers and laptops producing logs that nobody watches. When something goes wrong, such as a brute-forced SSH account, a web shell uploaded through an old plugin, or a staff laptop running a suspicious PowerShell command, the evidence is sitting in the logs, unread.

The standard answer is a SIEM. Open source options exist, but a full stack with an indexer, dashboards and managers needs several gigabytes of memory and someone who understands it. Commercial SIEMs are priced by data volume. Small teams either never start or install one and abandon it.

What already exists and the gap

• Wazuh: open source security platform with agents, rules and dashboards. Capable, but the full deployment is heavy for a single small VPS, and there is no official Sigma backend.
• Elastic Security: powerful detection on top of Elasticsearch, with memory and operational demands to match.
• Chainsaw and similar tools: run Sigma rules against Windows event logs quickly, mainly for hunting in collected logs after the fact.
• SigmaHQ rule repository: thousands of community detection rules in a vendor-neutral format, which is the reason this project is possible.

The gap is a small, always-on engine that takes community Sigma rules as they are, runs them in real time on a modest machine across Linux, web and Windows logs, and produces alerts a non-specialist can act on.

What you build

Roles: admin (installs, chooses rule packs), responder (receives alerts).

1. Lightweight shippers forward logs from servers and laptops to the engine over TLS.
2. The engine normalises events to a common schema and maps Sigma field names to it.
3. Compiled rules match events as they arrive. Correlation rules track short sequences per key (IP, user, host).
4. Alerts are grouped, deduplicated and sent with an explanation, the matching event, and suggested next steps.
5. A small dashboard shows recent alerts, rule health (rules that never fire, rules that fire too often) and ingestion rate.

The technically interesting part

This is a compiler and data structures project in disguise. Compiling Sigma rules into a fast matcher means parsing YAML, handling modifiers such as contains, startswith, endswith and regular expressions, and evaluating boolean conditions over named selections. Naively checking every rule against every event is slow. Index rules by log source and by required field values, prefilter with string search structures (for example Aho-Corasick over literal substrings), and only then evaluate full conditions. Measure the speed-up.

The second hard problem is field mapping. The same idea appears as different field names in different log sources. Build a clear mapping layer and report which rules you could not map.

How to evaluate it

• Throughput: events per second on a 2 vCPU, 4 GB VPS with 150 rules, with and without your prefilter.
• Correctness: a test corpus of events where the expected rule matches are known; compare with the official Sigma tooling where possible.
• Detection in the lab: run scripted, safe test activities on your own machines and report which were detected, missed, and why.
• Alert quality: over two weeks on your own lab or a consenting small organisation's test server, count alerts per day and the share that were actionable.
• Footprint: memory and setup time compared with a heavier stack running the same rules.

Suggested plan

1. Weeks 1 to 4: study the Sigma specification and rule repository; build the lab.
2. Weeks 5 to 10: parser, compiler and a naive matcher with tests.
3. Weeks 11 to 15: ingestion, normalisation and field mapping.
4. Weeks 16 to 20: prefilter optimisation, correlation, alerting.
5. Weeks 21 to 26: lab detection tests, benchmarks, comparison, write-up.

Path to a product

Release the engine as open source to build trust, which matters for security tools. Sell a hosted version or a managed rule pack with tuning for common small-business setups. The first customer is a small managed service provider who wants to offer "we watch your logs" to clients without running a full SIEM for each.

Scope warning

Do not build a log search and analytics platform; store alerts and a short window of raw events, not everything forever. Do not write your own agent for Windows; use Sysmon and an existing shipper. Only run attack simulations on machines you own in your lab.

Learn what you need on Sefism

• System design roadmap : ingestion pipelines, backpressure and storage choices.
• SigmaHQ Sigma Rules : the rule format and community rules your engine runs.
• Sysmon : the Windows telemetry source for endpoint rules.
• Wazuh : the heavier open source stack to compare against.
• LetsDefend : practice reading alerts the way a responder will.
• Go Documentation : a good fit for a single-binary, concurrent engine.

Get help with this idea

Scoping a security project is mostly about deciding what you will not defend against and how you will prove the rest works. If you want a second opinion on scope, stack or your evaluation plan, a mentor can review it with you; you still build it yourselves. New to Sefism? Create a free account first.

• Discuss this idea with a mentor
• Send a query to Tauseef
• Book a 1:1 session