Sunburst Origin Story Part 2: A Forensic Examination of SUNBURST After Detection

The SolarWinds Orion SUNBURST supply chain attack has rocked the confidence of many security teams across industries. This two-part blog series is an examination of the attack by Todd Kemmerling, Director of Data Science at ExtraHop, to reconstruct the timeline of the attack over the past 9+ months and provide insights about how to improve threat detection in the future. (See Part 1.)

Continuing my analysis of the days after SUNBURST, I share my interpretation of the data that we collected. The primary takeaway: subtle indicators can reveal really large problems.

SUNBURST Rising

The following chart shows the threat activity detected—with anonymized, aggregate data from the many environments ExtraHop secures—between January 1, 2020 to December 19, 2020. Between late March and early October there is a steady increase in the detected threats—approximately a 150% increase.

A detected threat is data that shows a significant change in behavior from what is considered normative. For example, the following types of behavior can be considered unusual:

• Todd logged in at 3:30 AM and he normally does not
• Todd is a data scientist and his laptop activities are suddenly different from the laptop activities of other data scientists
• Todd is a data scientist but his laptop activities suddenly match the laptop activities of accountants

We also see a somewhat quick decline of detected threats from late October to mid-December. We now know that SUNBURST modified its activity or ceased operation when it was actively hunted or analyzed. The trend suggests that the attackers were aware that SUNBURST had been detected in late October and they began to modify the operational tempo.

Strategic Misdirection

ExtraHop, along with many other security researchers, began compiling a comprehensive set of indicators of compromise (IOCs). These IOCs are a list of domain names and IP addresses that are involved with the SUNBURST attack. We are no longer updating this list because the SUNBURST part of the attack has been shut down. (View our list of IOCs and associated scripts.)

The bulk of the IOCs in the set are expected: IP addresses in subnets that are associated with major cloud providers that are no longer active. However, there are also IP addresses associated with legitimate domains and sites—and that leads to many questions and a good deal of confusion about these types of IOC lists.

A closer look at the legitimate sites reveals a common thread. These sites are either domain resellers, small cloud service providers, email providers, proxy services, or large commercial sites. These sites are how the attackers operated, enhanced, and evolved SUNBURST over time.

Step 1\: Find and purchase dormant domains with no negative threat intelligence history. Purchases could be made from prepaid cards, which are hard to trace. These domains are now part of an inventory of assets, such as deftsecurity[.]com, thedoccloud[.]com, and incomeupdate[.]com

Step 2\: Stage exfiltrated data on servers hosted by smaller cloud service providers, such as chunkhost.com, liquidtelecom.com, or mivocloud.com. Stolen data could be moved to one of these providers for analysis, then reduced or discarded. The valuable data would ultimately be moved further upstream.

Step 3\: Hide the location of external connections through proxy services that do not keep logs. Without logs, these connections become nearly impossible to trace. Some of these services have been taken down recently for alleged criminal activity, such as insorg[.]org, safe-inet[.]net, and safe-inet[.]com.

Step 4\: Detect sandboxing by periodically requesting the homepage of a large commercial site. Large commercial sites seldom go down, are not normally blocked, and are not unusual requests in enterprise traffic. If a request returns an error, SUNBURST can surmise that its activities were detected and the malware has been isolated for analysis.

Proxy services can be difficult to trace.

Attack Epilogue

SUNBURST infiltrated networks through a SolarWinds update and then stayed dormant for at least two weeks. There was nothing to detect at first. After SUNBURST became active, the attack began with reconnaissance activities that could be detected by network detection and response (NDR) systems in the following ways:

• A programmatic listing of all network drive shares can indicate an attacker performing reconnaissance on a network.
• A programmatic listing of all network device aliases can indicate an attacker performing reconnaissance on a network.
• When a network device increases its level of privilege or its span of control it can indicate an attempt to exploit a victim network device.
• When Windows Management Interface activity has increased beyond normal, or shows new activity, it can indicate an attempt to exploit a victim network device.
• A programmatic listing of all shared network devices can indicate an attacker performing reconnaissance on a network.

Further along the attack chain, SUNBURST contacted one of the IP addresses on the IOC list. These IOCs were unknown at that time, but it should be considered unusual when a highly-privileged device (such as a SolarWinds node) contacts any unfamiliar external host for the first time.

This might seem overly simplistic, but given that the associated FQDN addresses appear to be a web service hosted in Amazon Web Services (such as ifd927kt23hm24mwu30b2st.appsync-api.eu-west-1.avsvmcloud[.]com) it increases the likelihood that the hostname might pass a quick visual inspection.

A few days after this external communication, further suspicious activity could be detected:

• Collection of data in preparation for exfiltration is a strong indication that data is in the process of being stolen.
• Network file reads that are outside of normal activity are an activity that indicates data is being analyzed or collected.
• Indications that data has been collected, packaged, and is being exfiltrated or has been exfiltrated.
• Communications with a command & control server indicates that SUNBURST is sending status or receiving commands.
• Unexpected or increased SSH activity indicates possible remote access by an attacker, data transfer, or remote execution of commands.

Because these events occurred over a week to ten days and between extended dormant periods, they are harder to connect to a coordinated attack outside of NDR systems. NDR systems have the advantage of complete network visibility, deep history, and the ability to connect what appears to be random noise to an accurate and detailed view of an attack.

In short, SUNBURST activity was detectable and ExtraHop detected these types of activities on a number of systems. The extent of damage (and whether a network with SolarWinds was affected at all) depended on how long SUNBURST had access to the network, which permissions were configured on SolarWinds, and whether or not the data accessed was considered valuable.

Here are some best practices to follow:

• Monitor critical assets by their network activity, not just endpoint and logs
• Restrict outbound activity
• Secure build systems and infrastructure that generate or store your configurations or code
• Set up and authenticate through two-factor authentication, especially for critical infrastructure, even within your perimeter

For more on SUNBURST and other emerging threats, visit our security alerts page.

sunburst-origin-story-part-2.md Displaying sunburst-origin-story-part-2.md.