SANS SOC Survey Trends Over the Past Four Years

The Security Operations Center (SOC) increasingly sits on the front line in the battle against spiraling cyber threats. As organizations have shifted over the years from a prevention-first mindset to one of detection and response, security operations (SecOps) teams have emerged as a critical function to mitigate cyber-related risk. The question is how well they can perform their roles amid technology challenges and serious skills shortages.

To find out more, the SANS Institute runs an annual survey of SecOps staff and managers to better understand their challenges and broader industry trends. They report the findings in highly detailed reports, co-authored by principal SANS instructor Christopher Crowley and SANS director of emerging technologies, John Pescatore. While the recently released 2021 report offers some great insight, we've taken the time to run through some of the key findings in order to illuminate the broader SOC trends over the past few years.

How are SOCs adapting to changes in the threat landscape and technology innovation to keep their organizations safe? It's clear from the past few SANS reports that skills shortages are a perennial challenge, as is a lack of effective automation and orchestration capabilities. The latter is often seen as a potentially useful way to mitigate the problems posed by the former.

Technology trends have also ebbed and flowed over the years. For example, AI tooling was criticized in 2019 as overhyped but widely adopted by 2021. Let's take a closer look.

Skill and Visibility Gaps: SOC Survey 2018

ExtraHop noted three key findings from the 2018 SANS SOC Survey, which helped to identify why so many organizations were still struggling to extract value from their SecOps deployments.

A skills gap was the biggest challenge for SOCs, cited by 62% of respondents. At the time, a lot was being asked of Tier 1 analysts, work which automated tooling can arguably help with today—that is, derive actionable insights from large volumes of data. Considering that the burden was on the least experienced SOC staffers, there was a clear need even in 2018 for enhanced automation and orchestration to take the pressure off human analysts.

Too few organizations were using SOC metrics. Just 54% reported collecting key metrics, meaning many SecOps teams were unable to prove the value of their SOC to the business. Correlated stats around the number of incidents handled, time from detection to eradication, and the number of incidents closed in a single shift could tell a powerful story of the importance of SecOps work.

Asset discovery and inventory tools had the lowest satisfaction rates. Only 54% of respondents said good things about the technology, compared to the 75% who gave a positive rating of behavioral analysis tools. This may have been due to the relatively immature state of the market for asset discovery/inventory at the time.

SOC Challenges Persist: SOC Survey 2019

A year later, some things had changed, but several 2019 SOC trends remained the same. We found that:

Skills and automation shortages were the top challenges. The latter may certainly have been exacerbating the former challenge. But respondents also complained that too many tools weren't properly integrated. Better integration could also take the pressure off of stretched teams by reducing the need to swivel-chair between screens and tools.

Organizations embraced cloud-based SOC services. In fact, the number of respondents who claimed they were doing so doubled since 2018.

Network-based tools came top for detection. They outshone UEBA, endpoint, and log-based detection tools. However, AI/ML tools got the worst satisfaction rating, perhaps reflecting the disconnect between marketing hype and reality at the time.

What changed? SANS SOC Survey 2018-2019

In fact, it wasn't a case of what changed between 2018 and 2019 but what remained the same. When asked what barriers prevented SOCs from becoming fully integrated within the organization, most challenges were cited by a similar percentage of respondents in both years. A lack of skilled staff was the most popular, although tools not being integrated and failing to automate routine work were also popular answers. A lack of visibility into endpoints was another key technology failing highlighted in both years, with production OT systems and IoT devices proving especially problematic.

Keeping the Lights On: SOC Survey 2020

The SANS SOC Survey of 2020 was conducted as SOC teams were adjusting to work-from-home models, causing fresh challenges to bubble up mid-survey. Despite the rapid shift that altered responses, some key themes stood out:

Organizations were still not using metrics effectively. In fact, a quarter (24%) said they didn't provide metrics to management, making it difficult for business leaders to make well-informed decisions about the effectiveness of their SOC.

Many staff only stuck around 1-3 years. This was the most commonly reported average tenure, cited by two-fifths (41%) of respondents. Training, pay, and career development were seen as most important to retention, although mounting job stress may also have played a part.

Skills shortages persisted. A lack of skilled staff was cited as the most common "hindrance" to the SOC (30%), followed by a lack of management support. The latter may have been exacerbated by the failure to share suitable metrics. Finally, organizations struggled to turn to automation to help skills shortages—this was cited as a challenge for nearly a quarter (22%).

Just keeping the lights on was a job for many SecOps staff. In fact, general-purpose staff were the most common team members, with no dedicated monitoring, incident response, threat intelligence, or support staff working in responding organizations.

Outsourcing Takes Hold: SOC Survey 2021

By 2021, the industry had matured somewhat, but some old challenges and trends persisted. We found that:

Skills remained the top SOC challenge. It was cited just ahead of automation and orchestration, indicating persistent issues in these two critical and related areas. Automation of mundane tasks has become increasingly key to alleviating the skills problem. The report also uncovered a continued need for "deep technical knowledge" to drive up productivity, as well as soft skills like analytical thinking and customer service. Automation was mainly seen as a means to reduce detection and response times.

A third of organizations suffered an intrusion over the previous year. However, 12% didn't know whether they had or not—a sign of resource-starved SOCs.

Outsourcing became more popular. Over 50% of respondents outsourced skills such as pen testing, red teaming, purple teaming, threat intelligence (attribution), threat intelligence (production), threat research, and "other."

Most SOCs worked around the clock. Only 16% said they didn't operate 24/7, highlighting the growing pressures facing SOC teams and the need to outsource. Some 87% of respondents allowed their staff to work from home.

What's Changed? SANS SOC Survey 2018-2021

While many challenges have remained the same, there have been some broad shifts over the past five years. Between 2020 and 2021, lack of automation and orchestration overtook lack of management support as the second leading challenge, hinting that the pressures of security tool bloat and alert overload made a more urgent case for better automation and alert correlation.

A shortage of skills appears to have been a constant from 2018 to the present day, with many teams lacking specialists, and grappling with staff retention challenges. Organizations have also been consistent in their lack of metrics—metrics that may help make the business case for SOC investment—or at least improve transparency about the value it offers an organization. By 2021, over three-quarters were using metrics but only 67% said they were happy with the type of data used.

In terms of technology, by 2021, respondents favored their next-gen firewalls the most and were most dissatisfied with deception technology (such as honeypotting). However, a more indicative trend of SOC progress is how AI has evolved from being "over-hyped" in 2019 to a technology that 69% of respondents said they were implementing, had purchased, or were planning to buy two years later.