Professional Services Customer Success Stories: Improving Mean Time to Resolution

An educational service company that provides an array of support services in the post-secondary education sector was in the process of reassessing their security posture when they identified some network visibility gaps. This coincided with an executive mandate to increase their security profile as fast as possible. To strengthen security and add visibility, they evaluated and eventually implemented ExtraHop Reveal(x) network detection and response.

War Room Challenges

As they were in the midst of change, the educational services provider faced several challenges. Almost every time there was an incident or outage, multiple staff were involved in the investigation and resolution process. These frequent war rooms had a negative impact on their mean time to respond (MTTR).

The organization began using the Reveal(x) integration capabilities with Splunk, their SIEM tool. This out-of-the-box integration created automated workflows that could improve MTTR, but their SIEM administrator was using custom coding to parse data.

The educational provider viewed security as spanning IT operations, not a siloed function. To improve response times, rich data sets from both operational and security triggers had to be considered and then intelligently parsed to the respective IT or security incident queue for evaluation and remediation.

To decrease the need for war rooms, the organization expanded Reveal(x) throughout the organization. This provided visibility into the same traffic systems, across diverse teams. These actions would result in a significant improvement in their response times.

Help From ExtraHop Professional Services

To address their challenges and expedite the process of improving their operational and security maturity, the team engaged ExtraHop Professional Services.

Out of the box, Reveal(x) supports the syslog data format, but the Professional Services Team helped the educational services provider convert log data from Reveal(x) to JSON format. This allowed more detailed data to be sent to Splunk which would eliminate the manual, interim process of parsing data and give the administrator a richer data set to create queries.

Enhanced contextual data would allow for a better setup of workflows that were triggered from detections which determined whether to tag an incident as an IT event or a security event. As a result, the organization's SIEM would know to create tickets for either the IT support queue or the security incident review queue.

The ExtraHop Solution Architect started by creating a script to convert the log data into JSON format. The Solution Architect then set a custom risk score level with filters on detection titles specific to the organization. This leveraged machine learning to prioritize triggers considered high value and critical. Having the ability to enhance the log data, now in JSON format, additional information could be added and easily correlated.

Correlated data, added to the detection, could now include information about the number of users potentially impacted by the incident. For example, "Does this issue or event impact 3 users or 1,000 users?" Richer data sets reduced investigation times, aided prioritization and improved the decision process involved in remediation.

Solving Investigation Challenges

The work from home VPN surge due to the pandemic in 2020 created more challenges for the educational services provider. When an incident was detected, they now needed to validate and investigate users or devices connected through the VPN.

When a user logs on to the network through the VPN, the IP address (of that user/device) is dynamically allocated—which means it is constantly changing. This presents a problem when you get an alert specific to a particular IP address, as the actual device or user that generated that alert was needed for triage or investigation.

With help from ExtraHop Professional services, when an original detection was generated, the organization hard coded that IP address to Splunk, then correlated the Active Directory log to the original detection and was able to validate the source. All of these actions were significant in helping to improve their MTTR.

Response Time Success

As a result of engaging ExtraHop Professional Services, the educational services provider was able to achieve their objective of exponentially growing their security profile in as short a time as possible.

Enhanced data sets with greater detail and correlated data enabled automated workflows with intelligent prioritizations, less investigation time, and reduced alerts which resulted in faster incident validation and shorter time to resolution.

Training, custom dashboards, and triggers enabled diverse groups throughout the organization such as DBAs (database analysts) to start using Reveal(x) to analyze outages and disruptions. This also had a significant impact on reducing MTTR.

Finally, as the organization was forced to shift to VoIP from PSDN (public switches) during the pandemic, the work done by Professional Services helped them with faster remediation of their Cisco VoIP infrastructure, which reduced the frequency and duration of outages.