Migrate Splunk detection rules to Microsoft Sentinel

Splunk detection rules are security information and event management (SIEM) components that compare to analytics rules in Microsoft Sentinel. This article describes the concepts to identify, compare, and migrate them to Microsoft Sentinel. The best way to migrate Splunk detection rules to Microsoft Sentinel is to start with the SIEM migration experience, which identifies out-of-the-box (OOTB) analytics rules for automatic translation.

If you want to migrate your Splunk Observability deployment, learn more about how to migrate from Splunk to Azure Monitor Logs.

Audit rules

Microsoft Sentinel uses machine learning analytics to create high-fidelity and actionable incidents. Some of your existing Splunk detections may be redundant in Microsoft Sentinel, so don't migrate them all blindly. Review the following rule-auditing considerations as you identify your existing detection rules.

• Make sure to select use cases that justify rule migration, considering business priority and efficiency.
• Check that you understand Microsoft Sentinel rule types.
• Check that you understand the rule terminology.
• Review outdated rules that don't have alerts for the past 6-12 months, and determine whether they're still relevant.
• Eliminate low-level threats or alerts that you routinely ignore.
• Microsoft Sentinel Analytics require that the data type is present in the Log Analytics workspace before a rule is enabled. Confirm that your required data sources are connected and review your data connection methods. Revisit data collection conversations to ensure data depth and breadth across the use cases you plan to detect. Then use the SIEM migration experience to ensure the data sources are mapped appropriately.

Migrate rules

After you identify the Splunk detections to migrate, review the following migration-process considerations:

• Compare the existing functionality of Microsoft Sentinel's OOTB analytics rules with your current use cases. Use the SIEM migration experience to see which Splunk detections are automatically converted to OOTB templates.
• Translate detections that don't align to OOTB analytics rules. The best way to translate Splunk detections automatically is with the SIEM migration experience.
• Discover more algorithms for your use cases by exploring community resources such as the SOC Prime Threat Detection Marketplace.
• Manually translate detections if built-in rules aren't available or aren't automatically translated. Create the new KQL queries and review the rules mapping.

For more information, see best practices for migrating detection rules.

Rule migration steps

Follow these steps to validate, translate, and test each Splunk detection rule during migration to Microsoft Sentinel.

1. Verify that you have a testing system in place for each rule you want to migrate.

   1. Prepare a validation process for your migrated rules, including full test scenarios and scripts.

   2. Ensure that your team has useful resources to test your migrated rules.

   3. Confirm that you have the required data sources connected, and review your data connection methods.

2. Verify whether your detections are available as OOTB templates in Microsoft Sentinel:

   • Use the SIEM migration experience to automate translation and installation of the OOTB templates.

     For more information, see Use the SIEM migration experience.

   • If you have use cases not reflected in the detections, create rules for your own workspace with OOTB rule templates.

     In Microsoft Sentinel, go to the Content hub.

     Filter Content type for Analytics rule templates.

     Find and Install/Update each corresponding Content hub solution or standalone analytics rule template.

     For more information, see Detect threats out-of-the-box.

   • If you have detections that aren't covered by Microsoft Sentinel's OOTB rules, first try the SIEM migration experience for automatic translation.

   • If neither the OOTB rules nor the SIEM migration completely translate the detection, create the rule manually. Use the following steps to create your rule:

     1. Identify the data sources you want to use in your rule. Identify the Microsoft Sentinel tables you want to query by creating a mapping table between data sources and data tables.

     2. Identify any attributes, fields, or entities in your data that you want to use in your rules.

     3. Identify your rule criteria and logic. When identifying your rule criteria and logic, consider using rule templates as samples for how to construct your KQL queries.

        Consider filters, correlation rules, active lists, reference sets, watchlists, detection anomalies, aggregations, and so on. You might use references provided by your legacy SIEM to understand how to best map your query syntax.

     4. Identify the trigger condition and rule action, and then construct and review your KQL query. When reviewing your query, consider KQL optimization guidance resources.

3. Test the rule with each of your relevant use cases. If it doesn't provide expected results, review and edit the KQL and test it again.

4. When you're satisfied, consider the rule migrated. Create a playbook for your rule action as needed. For more information, see Automate threat response with playbooks in Microsoft Sentinel.

Learn more about analytics rules:

• Create custom analytics rules to detect threats. Use alert grouping to reduce alert fatigue by grouping alerts that occur within a given timeframe.
• Map data fields to entities in Microsoft Sentinel to enable SOC engineers to define entities as part of the evidence to track during an investigation. Entity mapping also makes it possible for SOC analysts to take advantage of an intuitive [investigation graph] (investigate-cases.md#use-the-investigation-graph-to-deep-dive) that can help reduce time and effort.
• Investigate incidents with UEBA data, as an example of how to use evidence to surface events, alerts, and any bookmarks associated with a particular incident in the incident preview pane.
• Kusto Query Language (KQL), which you can use to send read-only requests to your Log Analytics database to process data and return results. KQL is also used across other Microsoft services, such as Microsoft Defender for Endpoint and Application Insights.

Compare rule terminology

The following comparison table clarifies how a KQL-based rule in Microsoft Sentinel corresponds to an SPL-based detection in Splunk.

Map and compare rule samples

Use the following SPL-to-KQL samples to compare and map rules from Splunk to Microsoft Sentinel in various scenarios.

Common search commands

The following table maps common SPL search commands to their KQL equivalents in Microsoft Sentinel.

lookup command: KQL example

The following KQL example uses externaldata to replicate the SPL lookup pattern by filtering users against an external data source.

Users 
| where UserID in ((externaldata (UserID:string) [
@"https://storageaccount.blob.core.windows.net/storagecontainer/users.txt" 
h@"?...SAS..." // Secret token to access the blob 
])) | ... 

stats command: KQL example

The following KQL example uses summarize to count transactions and sum totals, grouped by fruit and month.

Sales 
| summarize NumTransactions=count(), 
Total=sum(UnitPrice * NumUnits) by Fruit, 
StartOfMonth=startofmonth(SellDateTime) 

mstats command: KQL example

The following KQL example bins metric values into price ranges and counts the records in each range.

T | summarize count() by price_range=bin(price, 10.0) 

transaction command: SPL example

The following SPL example groups related start and stop events into a single transaction by activity ID.

sourcetype=MyLogTable type=Event
| transaction ActivityId startswith="Start" endswith="Stop"
| Rename timestamp as StartTime
| Table City, ActivityId, StartTime, Duration

transaction command: KQL example

The following KQL example replicates SPL transaction grouping by joining start and stop events on a shared activity ID and calculating the duration.

let Events = MyLogTable | where type=="Event";
Events
| where Name == "Start"
| project Name, City, ActivityId, StartTime=timestamp
| join (Events
| where Name == "Stop"
| project StopTime=timestamp, ActivityId)
on ActivityId
| project City, ActivityId, StartTime, 
Duration = StopTime – StartTime

Use row_window_session() to calculate session start values for a column in a serialized row set.

...| extend SessionStarted = row_window_session(
Timestamp, 1h, 5m, ID != prev(ID))

eventstats command: SPL example

The following SPL example calculates per-category event counts in one-minute bins and then appends a per-time-bin total using eventstats.

… | bin span=1m _time
|stats count AS count_i by _time, category
| eventstats sum(count_i) as count_total by _time

eventstats command: KQL example

Here's an example with the join statement:

let binSize = 1h;
let detail = SecurityEvent 
| summarize detail_count = count() by EventID,
tbin = bin(TimeGenerated, binSize);
let summary = SecurityEvent
| summarize sum_count = count() by 
tbin = bin(TimeGenerated, binSize);
detail 
| join kind=leftouter (summary) on tbin 
| project-away tbin1

Here's an example with the make_list statement:

let binSize = 1m;
SecurityEvent
| where TimeGenerated >= ago(24h)
| summarize TotalEvents = count() by EventID, 
groupBin =bin(TimeGenerated, binSize)
|summarize make_list(EventID), make_list(TotalEvents), 
sum(TotalEvents) by groupBin
| mvexpand list_EventID, list_TotalEvents

anomalydetection command: SPL example

The following SPL example detects anomalies in historical closing price data over a 10-year period.

sourcetype=nasdaq earliest=-10y
| anomalydetection Close _ Price

anomalydetection command: KQL example

The following KQL example uses time-series analysis functions to detect anomalies in disabled-account sign-in trends over a seven-day lookback period.

let LookBackPeriod= 7d;
let disableAccountLogon=SignIn
| where ResultType == "50057"
| where ResultDescription has "account is disabled";
disableAccountLogon
| make-series Trend=count() default=0 on TimeGenerated 
in range(startofday(ago(LookBackPeriod)), now(), 1d)
| extend (RSquare,Slope,Variance,RVariance,Interception,
LineFit)=series_fit_line(Trend)
| extend (anomalies,score) = 
series_decompose_anomalies(Trend)

Common eval commands

The following table maps common SPL eval functions to their KQL equivalents in Microsoft Sentinel.

case(X,"Y",…) SPL example

The following SPL example returns a different message string based on the HTTP error code.

case(error == 404, "Not found",
error == 500,"Internal Server Error",
error == 200, "OK")

case(X,"Y",…) KQL example

The following KQL example uses case() to assign a message based on the error code value.

T
| extend Message = case(error == 404, "Not found", 
error == 500,"Internal Server Error", "OK") 

if(X,Y,Z) KQL example

The following KQL example uses iif() to label whether a timestamp falls on the current day.

iif(floor(Timestamp, 1d)==floor(now(), 1d), 
"today", "anotherday")

isint(X) KQL example

The following KQL example checks whether X has an integer-compatible type by using gettype and iif.

iif(gettype(X) =="long","TRUE","FALSE")

isstr(X) KQL example

The following KQL example checks whether X is a string value by using gettype and iif.

iif(gettype(X) =="string","TRUE","FALSE")

like(X,"y") example

The following KQL examples show several string-matching operators you can use to replicate SPL like pattern matching.

… | where field has "addr"

… | where field contains "addr"

… | where field startswith "addr"

… | where field matches regex "^addr.*"

min(X,…) KQL example

The following KQL examples show different ways to compute minimum values using min_of, min, and arg_min.

min_of (expr_1, expr_2 ...)

…|summarize min(expr)

…| summarize arg_min(Price,*) by Product

mvfilter(X) KQL example

The following KQL example uses mv-apply to filter and reduce multi-valued content, similar to the SPL mvfilter function.

T | mv-apply Metric to typeof(real) on 
(
 top 2 by Metric desc
)

mvjoin(X,Y) KQL example

The following KQL example uses strcat_array to join array elements into a single string with a custom separator.

strcat_array(dynamic([1, 2, 3]), "->")

relative time(X,Y) KQL example

The following KQL example converts a datetime value to Unix epoch time for relative-time calculations.

let toUnixTime = (dt:datetime)
{
(dt - datetime(1970-01-01))/1s 
};

replace(X,Y,Z) KQL example

The following KQL example uses replace() with a regex pattern to swap the month and day parts of a date string.

replace( @'^(\d{1,2})/(\d{1,2})/', @'\2/\1/',date)

strptime(X,Y) KQL example

The following KQL example uses format_datetime to display only the hours and minutes from a datetime value.

format_datetime(datetime('2017-08-16 11:25:10'),
'HH:mm')

time() KQL example

The following KQL example formats a datetime value as a time string showing hours, minutes, and seconds.

format_datetime(datetime(2015-12-14 02:03:04),
'h:m:s')

tostring(X,Y) description

Returns a field value of X as a string.

• If the value of X is a number, X is reformatted to a string value.
• If X is a boolean value, X is reformatted to TRUE or FALSE.
• If X is a number, the second argument Y is optional and can either be hex (converts X to a hexadecimal), commas (formats X with commas and two decimal places), or duration (converts X from a time format in seconds to a readable time format: HH:MM:SS).

tostring(X,Y) SPL example

The following tostring SPL example returns:

foo=615 and foo2=00:10:15:

… | eval foo=615 | eval foo2 = tostring(
foo, "duration")

urldecode(X) SPL example

The following SPL example decodes a URL-encoded string back to its original form.

urldecode("http%3A%2F%2Fwww.splunk.com%2Fdownload%3Fr%3Dheader")

Common stats commands KQL example

The following table maps common SPL stats aggregation functions to their KQL equivalents in Microsoft Sentinel.

Next steps

You learned how to identify, compare, and map Splunk detection rules to Microsoft Sentinel analytics rules. Continue migrating your SOAR automation.