SIEM use cases: how to develop bespoke threat detection rules

Security Information and Event Management (SIEM) platforms play a vital part in enabling organisations to defend against cyber threats.

But, as we explored in the first of this two-part series, there are limitations to using the out-of-the-box rules which form part of the technology.

In this blog post, we explore how to customise rules, the rule development process and the role of Sigma.

The SIEM rule development life cycle

Writing or customising threat detection rules ensures that organisations maximise the performance of their SIEM. This requires a defined process or lifecycle:

1. Design

The design stage is frequently determined by the source of the rule requirement. It is often intelligence-led in order to respond to a new exploit being published. Threat intelligence is a valuable source of information as it contains behavioural indicators which can easily be converted into a set of logical conditions for your SIEM.

At other times, rule development may be data source-led. One example scenario could be when an organisation has bought a new application and needs to know which threats can be detected or mitigated through log analysis.

If log sources are already supported by your SIEM, studying the parser will save a lot of time. If it’s unsupported, you will need to read the log source documentation in detail and understand the event IDs and the key-value pairs within them.

Resource efficiency and false positives are also important considerations at the design stage.

2. Implementation

Implementation should be focused on configuring the rule in your test deployment. The challenge for many engineers is building a test deployment with a suitably large dataset.

Test log sources often lack the breadth and depth of data to effectively test a new SIEM rule. Forwarding data from production to test can solve this problem but create parsing issues and duplicate ingestion costs.

In Azure Sentinel, rules in a test deployment can be configured to query data in a production workspace, with no additional ingestion cost. Alerts in this scenario only exist in the test deployment so we can implement the rule without impacting our colleagues in the SOC.

3. Testing

Now that you’ve configured the rule, how is it performing? Is it resource-intensive? If it is, you need to consider logical changes to make it more efficient.

Validation is another key objective during testing. Has the rule actually triggered? If relevant events haven’t taken place, attack emulation is required.

If you have access to pen testers, they can help to trigger and validate your SIEM rules. This is an area in which red and blue teams should work together. The red team emulates the attack and the blue team observes the data in the SIEM.

If the data (or normalisation of the data) is different to what was anticipated, you’ll need to tune the rule until it fires reliably.

Lastly, SIEM rules should be tested regularly to ensure they remain effective.

4. Tuning

This is typically the most challenging stage. Environments are always changing, and it can take time to get the answers required to shape the rule effectively. Whitelisting false positives is the most common form of tuning. Blacklisting is an alternative approach where the rule aims to detect a narrower set of known bads.

Once tuning is done, you should repeat stages 3 and 4 until you have a well-performing rule.

5. Promotion

This is the stage where you go through change management to ensure the rule is implemented effectively within your business. In most organisations, the SOC is a key stakeholder in this process.

Why use Sigma for developing use case rules?

Sigma is, as described on its GitHub page, “a generic and open signature format that allows you to describe relevant log events in a straightforward manner”.

Sigma provides a way of writing detections or rules that are focused on what you’re trying to detect, rather than on the syntax of a particular platform. It creates a layer of abstraction above the native query languages of your SIEM and Endpoint Detection & Response (EDR) technologies. Converters exist to transform your Sigma rules into the structure and syntax understood by your detection technologies.

Having a ‘write once, run everywhere’ capability is valuable for any size organisation running multiple detection technologies. You can also write a rule to share with others who may be using a completely different set of technologies.

Assuming that your SIEM or EDR technology has an appropriate API, you can automate the deployment of your rule set and implement version control.

Using Sigma enables you to be vendor-agnostic, rather than having to change your Infosec Evaluation Methodology (IEM) and repeating rule development. It also provides a level of future proofing: if and when you adopt another SIEM platform, you can take the detection rules with you.

Recommendations for developing rules

Development of bespoke SIEM rules is important for detecting threat activity in your environment. The key things to remember are:

• Out-of-the-box SIEM rules are not adversary-oriented​, responsive to new threats or tuned to your environment
• Use an open-source format like Sigma to develop rules​
• Ensure that the rules you create don’t create a burden of false positives​
• Don’t rely on ‘set it and forget it’ – continuous improvement is key for effective long-term cyber security

Getting external help to manage use cases

There are many elements to consider when developing SIEM rules. As it is often a complex, costly and time-consuming process, outsourcing it can be more effective. Consider using a managed SIEM provider to benefit from the latest rules and the most up-to-date insight and threat intelligence to support them, or a Managed Detection and Response provider for a turnkey service to take the burden away from your in-house teams.

Learn more about our ThreatDetect MDR service