In this article, I will present a detailed overview of Microsoft Sentinel, demonstrating its capabilities in investigating security incidents within your environment. Further, I’ll illustrate how leveraging automation and AI can substantially augment your security operations, enabling more efficient and enhanced security operations.

Microsoft Sentinel is a cloud-native Security Information and Event Management (SIEM) platform designed to assist organizations in detecting, investigating, and responding to security threats efficiently. It operates by systematically collecting and analyzing security data from diverse sources, including cloud services, devices, and on-premises systems.

Sentinel utilizes sophisticated machine learning algorithms to pinpoint potential security threats, providing timely alerts to security teams. It offers a centralized interface for thorough investigation and swift response to incidents, enabling automated actions like quarantining devices or blocking malicious IPs when necessary.

The platform seamlessly integrates with various Microsoft security products and other third-party tools, enhancing the organization’s ability to respond to threats more rapidly and reducing the manual effort required for incident response.

Moreover, Sentinel provides comprehensive visualization tools and reporting capabilities, aiding security teams in maintaining awareness of the organization’s security status and ensuring adherence to security policies.

While several SIEMs are available in the market, Sentinel stands out due to its cloud-native architecture, facilitating easy integration with cloud solutions without the need for additional infrastructure. Its scalability and the inclusion of artificial intelligence solutions, integrated natively, offer a seamless and efficient user experience

Setting up Sentinel

The setup process for Microsoft Sentinel is straightforward and primarily involves establishing a foundational structure for data storage and event management.

Create a Resource Group: Start by creating a designated resource group for Sentinel.
Establish a Log Analytics Workspace: This will act as the central repository for data, events, and other information.
Deploy Sentinel: Following the creation of a workspace, deploy Sentinel to the desired resource group.\

During this setup, you’ll be prompted to designate a Log Analytics workspace for Sentinel to use. After confirming your selection and adding the resource, the setup for Microsoft Sentinel is essentially complete.

You will then be redirected to a welcome page that provides an overview of your newly created Sentinel environment.

General Overview

Before we delve into constructing our Sentinel artifacts, it’s crucial to familiarize ourselves with the various tabs available, each offering insights into different aspects of our security environment.Let’s start with the general section and click on overview.

The Overview tab is instrumental in providing a comprehensive summary of your organization’s security posture. It grants insights into active threats, ongoing security incidents, and vital security metrics, acting as a barometer for your security environment. Initially, this section may appear blank, but it will dynamically populate with relevant information as we integrate data sources and configure analytics rules. This enables a real-time reflection of the security landscape, allowing for prompt identification and response to any discrepancies or threats detected.

Next, we turn our attention to the Logs tab.This tab provide access to raw log data aggregated from various data sources within your operational environment. It employs Kusto Query Language, or KQL, a robust language designed to retrieve specific data sets from pre-defined tables effectively. Microsoft offers an array of custom KQL queries to facilitate users in initiating data presentation from their respective data sources, aiding in a smooth start to data exploration and analysis.

The incident tab provides a detailed view of security incidents that have been detected in your environment, including information about the incident, severity, status and related entities.

The Workbooks tab is equipped with pre-constructed dashboards and reports, designed to facilitate the visualization and analysis of security data within your Sentinel environment. Users are afforded a variety of templates to select from, aligning with diverse analytical needs. For those seeking a more tailored approach, the platform also supports the creation of custom workbooks, allowing users to mold their analytical space according to their specific requirements and preferences.

Hunting is used to proactively search for security threats in your environment with the ability to create custom queries and rules. You can also select what you want to look for in your environment. Based on MITRE framework, Microsoft already gives you many predefined hunting queries for you to leverage.

The Notebook tab is a sophisticated feature of Microsoft Sentinel, providing an interactive environment designed for crafting and sharing code, foundational analysis, and visual representations of security data.

Following this, we explore Entity Behavior. In Azure, entities serve as unique identifiers—these can be account names, hostnames, IP addresses, file hashes, or other identifiable markers. This tab is integral for offering in-depth insights about your entities and any alerts correlated with them, allowing for a more meticulous understanding of the behavioral patterns within your environment

Another integral tab is Threat Intelligence. Threat Intelligence is pivotal for every security operation, serving as a repository of information that details both existing and potential threats to systems and users. Microsoft offers integration with numerous threat intelligence feeds, while also providing the flexibility to construct your own customized feeds. Cyber threat intelligence (CTI) is information describing existing or potential threats to systems and users. This intelligence takes many forms, from written reports detailing a particular threat actor’s motivations, infrastructure, and techniques, to specific observations of IP addresses, domains, file hashes, and other artifacts associated with known cyber threats.

The final tab under our review is MITRE AT&CK. This feature allows users to link analytic rules directly to the MITRE AT&CK framework, a comprehensive compilation of known tactics and techniques employed by threat actors. The framework serves as a rich source of information for detecting and mitigating threats in your environment, and this tab can help you identify the coverage of your analytical rules and discern areas that might be lacking in coverage.

Another vital section is dedicated to content management, starting with Content Hub. Content Hub is a highly potent feature of Microsoft Sentinel and significantly contributes to the efficacy and user-friendly nature of Sentinel.

Within this hub, users are presented with hundreds of diverse solutions, each capable of being seamlessly integrated into your SIEM with mere clicks. When delved into, each solution reveals more intricate details on the right, offering insights into pricing and, more crucially, the components encompassed in the package.

Visible are sub-application packages with analytics rules for book watch lists, data connectors and parsers, all ready to be installed with a single click

Another noteworthy feature is Repositories. This represents a more advanced functionality within Microsoft Sentinel, offering automation in the deployment of Sentinel components directly from the code. It’s versatile, supporting various types of content including, but not limited to, analytics rules, hunting queries, playbooks, automation rules, and parser workbooks, each contributing to a richer, more adaptable security management environment.

The last tab is all about configuration.

First, our data connectors, which are pre-built integrations that allow organizations to assimilate data from a diverse array of sources directly into their Sentinel environment. These connectors have been engineered with user-friendliness in mind, allowing for straightforward setup and configuration. They offer a standardized methodology for harvesting and normalizing data from varied origins, ensuring consistency and reliability in data management.

Next one is analytics. Here you can create analytics rule designed to identify potential threats.They run periodically and they query your data for threats based on provided criteria. There are hundreds of analytics rule templates already provided by Microsoft. You can access them by clicking on rule templates.

Next on the list are watchlists. Watchlists are collection of data from external sources used for correlation.

Once you have your analytic group set up, implementing some degree of automation is beneficial. This could involve data enrichment tasks, like verifying if an IP address is malicious, or perhaps setting alerts for certain critical activities. Automation in Sentinel can be divided into parts. Automation Rules and Playbooks.

Automation rules are straightforward rules to orchestrate incident process. For instance, they can modify the severity of an incident, auto-resolve incidents, or allocate playbooks..

Playbooks, on the other hand, have the capability to integrate with external systems, enabling a broad range of actions. These include but are not limited to blocking IP addresses, disabling user accounts, creating tickets, and enriching alert information with a multitude of options readily available for utilization.

These include but are not limited to blocking IP addresses, disabling user accounts, creating tickets, and enriching alert information with a multitude of options readily available for utilization.

This structured approach to automation optimizes incident response and management, enhancing the overall efficiency and effectiveness of the security operations.

Setting up data connectors

To begin, we need to ingest some data by configuring our data connectors. At the moment the data connector page are empty, so our next step is to navigate to the content hub to incorporate the desired solutions.

For now I will select the Azure Active Directory and the Azure Activity solutions.

After installing these solutions,revisiting the data connector tab reveals the newly added data connectors. Opening the data connector page displays several prerequisites, including the necessity to be the subscription owner and to possess read and write permissions to the workspace.

Within the data connector wizard, we specify a scope and name, select the primary log workspace, and activate the managed identity.

That essentially concludes the setup, and data should start populating after some time.

Another solution I will install is the Training lab:

Incident Investigation

Following the integration of the Training Lab, we observe three incidents illustrated in the incident tab, each paired with comprehensive details on the right.:

Selecting ‘view full details’ unveils a more expansive overview:

From the events in the incident timeline I can dig even deeper in the incident by investigating the logs from the log analytics workspace.

Above, the analytic rule that initiated the incident is visible, and below, the raw logs derived from the query of the analytic rule are displayed. From here I can start new queries to pivot to other potentially interesting events in the logs.

We also have the investigation view for a more visual representation of the incident:

The incident map in Sentinel is a visual representation of the relationships between different entities and events related to an incident. It provides a high level view of the incident, allowing analysts to quickly identify key details and potential areas of concern.

We only have one entitity in this incident. Checking the IP address in abuseipdb.com we can see that it is registered in North Korea.

Sign-ins from north korea should definitely not happen so this if this was a real incidents it is something we would want to investigate further.

Threat Hunting

In Microsoft Sentinel, threat hunting involves the proactive search for security anomalies and threats across an organization’s network, endpoints, and various data sources, which includes logs, events, and alerts. This process is crucial for identifying signs, patterns, and indicators of compromise that may have bypassed automated detection systems.

Looking at out incident page we have a high severity incident we have not looked at.

From the description this incident tells us that the analytic rule: “Identifies a match across various data feeds for domains IOCs related to the Solorigate incident.” We can see that the domain name is avsvmcloud.com/ with the IP address of **17.81.146.1**

Lets use threat hunting to find out more information about this incident. Searching for solorigate we see a fitting query for finding more information:

It says that on the system where the malicious solarwind dss is running, it is known that the attacker used a hardcoded name pipe. This activity can be detected if we are collecting sysmon event ID 1718 or security event ID 5145. Running the query we see 3 systems that are potentially infected. We can save the results as a bookmark and add them to our incident:

Going back to our incident we can see that entities have been updated with the infected systems:

At this point we may be ready to initiate the containment phase, isolating the systems and removing malicious artifacts.

You can also run other queries not related to an incident or write free form queries based on an a hypothesis of possible threats in your environements.

Automation

Automation is a critical aspect of security operations, facilitating swift responses to security incidents and minimizing the manual effort involved in managing security events.

In Microsoft Sentinel, there are two primary methods of automation. Automation rules and playbooks.

One way of quickly making an automation rule is directly from the incident tab, clicking an incident, clicking action and selecting create automation rules. The fields will then automatically be filled out. Another way would be creating an automation rule in the automation tab to the left. This is also one way of tuning out false positive incidents. Another way would be editing the query directly in the analytic rule.

If we move to the automation page and click on the playbook tab I can see that we already have two playbooks.

That is because playbooks are nased on Azure Logic Apps and I have already made a logic app in this blog post: https://ohenrik.com/projects/OnboardAutomator

You also use the same process through the logic app designer to create a playbook. Lets create another playbooks using Microsofts template of playbooks. I will be using a playbook that will enrich incidents with information from Virustotal, a useful tool for security analysts during incident investigations. I first go to the content hub and install the Virustotal solution.

After installation I now have 9 possible Virustotal templates I can install.

I select the IP Enrichment Entity Trigger playbook, create the playbook and continue to the logic app designer for the playbook. In the logic app designer I can see that the connection to Virustotal is invalid:

To get a working connection I will need to sign in to Virustotal and get my api key. From the logic app, i go to api connections, choose the logic app, click on api connections and paste in my API key.

The last thing we need to do is give the playbook the necessary permissions. From the logic app page, I click on identity, azure role assignments, add role assignment, and choose the Microsoft Sentinel Responder role.

I can now go back to one of the previous incidents, click on an IP address entity, select action and run the new playbook. And can see the results from the playbook in the activity log:

To make it run automatically, you can use automation rules.

Making a ChatGPT playbook

ChatGPT, since its release in November last year, has swiftly risen to prominence, amassing millions of users and poised to reshape both the industry and the landscape of cybersecurity. In this section, we will construct a playbook leveraging ChatGPT’s functionalities.

From the automation page in Sentinel, I create a new playbook with incident as its trigger. This option will trigger automation when the incident is created. I name the playbook ChatGPTIncidentEnrichment and are directed to the Logic App Designer

Microsoft Sentinel is already the playbook trigger. I choose the GPT3 Completes your prompt as the next step of the logic app. For this I need an API key from OpenAI. I create a key from this location: https://platform.openai.com/account/api-keys and paste it into the step. I now get the following options:

I start with the prompt “How can I remediate” and input some dynamic content. Dynamic content refers to data that is generated or retrieved at runtime based on the context of the workflow. Simply put, you can insert unique information from the previous building blocks. In this case, we will use our Microsoft Sentinel incident building block.

Our final prompt will be How can I remediate @{triggerBody()?[‘object’]?[‘properties’]?[‘title’]} with description @{triggerBody()?[‘object’]?[‘properties’]?[‘description’]}

Finally, we want to put the response from the ChatGPT inside our Microsoft Sentinel incident. I use the add comment to incident for this step:

After this I need to give the playbook the necessary permissions. It needs to write to incidents so I give it the Micrsoft Sentinel Responder role. After this it is time to test the playbook.

I run the playbook on the Incident: Malicious Inbox Rule, affected user AdeleV@contoso

When checking out the incident it never recieved an incident. Checking out the logic app I saw that it was still running, eventually hitting a timeout and the error “You exceeded your current quota, please check your plan and billing details”

So after some research I discovered that the I need to pay for the api access as my free credits for my account was already expired. I enter my credit card information and add the 5 dollar minimum amount for use of the API. Lets try again.

After running the playbook ChatGPT only answered with this comment:

That was dissapointingly short and not helpful. Checking the same prompt in https://platform.openai.com/playground gives a much better answer so something is wrong here.

I saw that someone has been doing something very similar here https://techcommunity.microsoft.com/t5/manufacturing/introduction-to-openai-and-microsoft-sentinel/ba-p/3761907 where they used a custom value for the engine prompt while I used the default one. I used the same engine value as he did, text-davinci-003 and got a much more complete answer.

This looks like it could be very useful for analysts as starting tips during incident investigations. In addition there are already developed playbooks from others, for example this one that you can easily deploy that provides incident tasks and example kql queries to run. This looks much more advanced then my prototype: https://github.com/format81/MicrosoftSentinel-ChatGPT-playbook

Conclusion

I have in this article given an overview of Microsoft Sentinel, its automation capabilities and how to integrate it with AI. Given Microsoft’s substantial investments in cybersecurity and AI—amounting to billions, this is an area that will see much growth and learning opportunities in the coming years.