See Cynet’s Autonomous
Breach Protection in Action

Prefer a one-on-one demo? Click here

ATTACK TECHNIQUES – HANDS ON

Brought to you by Cynet's CyOps Center

Cynet's 24/7 MDR with the latest security updates and reports

Learn more about CyOps

SUNBURST Backdoor C2 Communication Protocol

By: Igor Lahav

A lot has been written following the supply chain attack which targeted SolarWinds Orion software- and which according to most sources was carried out by APT29 also known as Cozy Bear, yet the backdoor is not yet fully understood. Spanning almost 3500 lines of code, “obfuscated” with casual naming, trying to evade shallow review, it has quite of subtleties yet to uncover.

Our contribution in this article is a better understanding of the command-and-control communication channel, its’ various stages and conditions required for execution. Our findings explain many of the observations seen so far regarding the SUNBURST backdoor which was implanted in the Orion attack. The main goal of this article is to understand how we can find infected beaconing machines. We will be focusing mostly on anything that is passed on the wire, while touching upon a few of the evasion techniques employed by the SUNBURST backdoor.

Introduction

Following recent news regarding a supply chain attack targeting SolarWinds Orion software in order to infiltrate its customers, with emphasis on high-profile targets, such as multiple US companies and government networks, it was discovered that the threat actor behind this attack has maliciously infected a legitimate SolarWinds Orion DLL, basically transforming it into an extremely sophisticated, digitally-signed backdoor.

The backdoor begins it execution in the legitimate SolarWinds.Orion.Core.BusinessLayer.BackgroundInventory.InventoryManager.RefreshInternal function.

Cleverly masqueraded as OrionImprovementBusinessLayer this function call does not draw any attention at first glance.

Furthermore, as it runs, it checks first if it is executed as part of the legitimate “solarwinds.businesslayerhost” process. Following by staying dormant between 12 to 14 days, after first arriving or an update. Afterwards it confirms it is the only instance of the backdoor currently running on the machine using named pipe existence as a synchronization mechanism.

Diving deeper to the communication of the backdoor, we make a distinction between 3 stages in the communication:

  • Stage 0: Ends with the internet connectivity check, resolving api.solarwinds.com
  • Stage 1: DNS communicating with *.appsync-api.*.avsvmcloud.com
  • Stage 2: HTTPS communicating with CNAME from Stage 1

Stage 0

The main function of this stage is OrionImprovementBusinessLayer.DnsHelper.CheckServerConnection. The function receives “api.solarwinds.com” as an input.

All classification of IP is done inside OrionImprovementBusinessLayer.IPAddressesHelper.GetAddressFamily and the most detailed classification is based on a list of ranges called “nList”. The actual logic of this stage can be summarized with the following diagram:

Figure 1: Actual logic behind of stage 0

This logic allows it to stop execution and evade detection on many sandboxes and hostile environments where there is some sort of a “catch-all” domain resolver.

 

Stage 1 – DNS-Based Communication

The code for this stage resides in the OrionImprovementBusinessLayer.Update function. While we consider the call to UpdateNotification to be part of Stage 0, all the following code, up to the call to HttpHelper.Initialize,is in Stage 1.

Stage 1, Part 1 – FQDN Generation and resolving

In the first part of this stage, a hostname is generated using one of the following 4 functions:

  • GetCurrentString
  • GetPreviousString
  • GetNextStringEx
  • GetNextString

At the end of this part, hostName will hold the FQDN to contact and resolve. No matter which method is chosen, the subdomain will be appended with one of the following 4 suffixes generated from the GetStatus function:

  • .appsync-api.eu-west-1.avsvmcloud.com
  • .appsync-api.us-west-2.avsvmcloud.com
  • .appsync-api.us-east-1.avsvmcloud.com
  • .appsync-api.us-east-2.avsvmcloud.com

addressFamilyEx, in turn, will hold the resolved IP’s classification mostly based on the mentioned “nList” of ranges.

Stage 1, Part 2 – Acting based on DNS answer for the subdomain

In the second part of this stage, actions will be taken according to the classification of returned IP for this subdomain.

Any IP classification besides AddressFamilyEx.NetBios will ultimately cause failure to move to the next stage and a local config ReportWatcherRetry to be updated, but only if it were set beforehand to any value between 3 and 5.

A classification of AddressFamilyEx.Atm (anything that would have caused stage 0 to fail – meaning IP is a loopback device or in the local network or multicast IP) will cause the backdoor to call the OrionImprovementBusinessLayer.ProcessTracker.SetAutomaticMode function which will try to turn back on services in the OrionImprovementBusinessLayer.svcList list, which were previously disabled by the backdoor in OrionImprovementBusinessLayer.ProcessTracker.SetManualMode. These attempts are to shut down software which might be interfering with clear communication with the C2 DNS server. A list of products attempted to be manipulated can be found in Appendix A.

Decoded nList is shown below for reference:

To move to Stage 2 (HTTPS communication channel) we need 2 conditions:

  • appsync-api subdomain IP resolution with ext=true (“ext”/“dnssec”)
    • Also implies NetBios classification
    • Seen as true at the end of line in the screenshot above. Also present in the table below.
    • Without this condition, we will not get NetBios classification when we get CNAME later.
  • CNAME response for the appsync-api subdomain (“cname”)
    • Without this condition, the backdoor will not be started

Classification of AddressFamilyEx.NetBios is given to the following ranges:

Furthermore, the last 2 bytes of the IP resolved will have implications on the communication structure and will be discussed at the next stage.

Upon closer inspection of public partial list of passive DNS resolution for subdomains of *.appsync-api.*.avsvmcloud.com among 1460 subdomain, we found only 1 resolution that would truly activate the next stage of the backdoor. This is a testament to the pinpoint precision and OPSEC which the malicious actor was practicing.

Stage 2 – HTTPS-Based Communication

After getting through all the conditions of Stage 0 and Stage 1, we finally arrive to the last stage, Stage 2. This stage starts with the execution of OrionImprovementBusinessLayer.HttpHelper.Initialize function. Here we start to work with a Job-related state machine, in a loop.

Each round in the loop begins with some random delay (IsSynchronized). Then, if a minute passed since last time, some defense evasion checks are implemented (TrackEvent and subsequent ProcessTracker.TrackProcesses). Any intimidating process (from assemblyTimeStamps list or svcList[*].timeStamps lists) will cause this stage to abort and exit.

CreateUploadRequest is responsible for delivering results and errors of previous execution, as well as fetching the next command to execute.

ParseServiceResponse is responsible for parsing the received command from CreateUploadRequest to a Job and arguments.

Finally, ExecuteEngine executes received commands with arguments and prepares any errors and results to be sent back later.

Stage 2 – HTTPs C2 URL

The analysis of this section is based of GetBaseUriImpl as well as relevant contextual information. The URL of the 2nd Stage is largely dependent on the resolved IP of the subdomain at Stage 1. As a rule, the backdoor tries to make these URLs unique, but will give up eventually. Although HTTPS URLs are used, any certificate errors are discarded and connection continues as usual. The URL begins with https://<cname>/ and afterwards there are 3 distinct cases (“pki”, “fonts” and “swip”):

 

swip/Upload.ashx – last command had big output ( >10,000  bytes after compression )

swip/Events           – last command had small  output ( <= 10,000 bytes after compression )

swip/upd/              – see options in figure below.

Figure 2:  URL Structure for “swip” case, percentage represent probability of each part to appear in the final URL.

Figure  3: URL structure for “fonts” case, no matter if response exists or not.

Figure  4: URL structure for “pki” case, no matter if response exists or not.

Stage 2 – Sending Back Previous Errors and Results

If there is a result (“response”) to be sent back from previous command, it will be concatenated with the error (“err”) and the generated GUID (“customerId”). Afterwards it is compressed and XORed with the checksum. If the response is short, it is further encoded as a JSON object.

Either way, regarding the HTTP method, when there is a result (inData in CreateUploadRequestImpl), a POST method will be used, otherwise a GET method will be used.

Figure  5: First transformation to the result of previous command

Figure  6: Model of result of last command encoded as JSON object

Stage 2 – Command Parsing

The analysis in this part is based on ParseServiceResponse function. First of all, to extract anything meaningful 200 OK status code for the HTTP response required. Parsing of the HTTP response to the above-described HTTP request depends on the case we are working in, which in turn depends on the last number in the IP returned for the subdomain (Stage 1).

If the response is shorter than 4 bytes, no command is attempted to be recovered. For the first part of the parsing (“body”) can be as simple as skipping 48 bytes (in the “fonts” case) or 12 bytes (in the “pki” case). In the remaining “swip” case a more elaborate scheme employed:

  • First the following regex is matched, which looks for hex encoded bytes
    • “\{[0-9a-f-]{36}\}”|”[0-9a-f]{32}”|”[0-9a-f]{16}”
  • Further cleaned, and hex decoded

After these steps, for all cases the following steps follow (which is almost the opposite of how the previous’ command result was encoded):

Figure 7: Transformation of incoming command

 

The rest of the command parsing is left as an exercise to the reader.

 

Stage 2 – Supported Command Execution

The analysis in this part is based on the ExecuteEngine function. Ultimately the back supported the following command:

Closing Notes

We have covered many aspects of the backdoor such as defense evasion mechanisms, but most notably the various stages the backdoor communicates in and tries to blend in. Future work can be directed towards figuring all the different subdomain encoding schemes, which in turn will hopefully allow to decode many of the C2 subdomains. We hope the information provided will be helpful to the Cyber Defense Community in their efforts with identifying and remediating any active beacons of this SUNBURST backdoor, from Cozy Bear actors, or who ever it may be.

.

Cynet Protection against SUNBURST backdoor

Rest assured:Cynet 360 customers are fully protected against any attempts of abusing the SUNBURST backdoor implanted in the Orion attack. Customers who utilize the SolarWinds Orion software have been notified by our CyOps team.

Cynet XDR has released relevant detections for the compromised software and the
malicious backdoor, based on multiple detection vectors. We will continue to add additional
IOCs as new ones emerge.

Additional security endpoint protection recommendations:
– Isolate the vulnerable endpoint using the Cynet Dashboard, and ready for triage.
– Reset any passwords used in the SolarWinds platform, treat them as compromised
passwords.
– Enable all detection and remediation mechanisms at Cynet 360 UI.

 

Appendix A – List of services attempted to be shut down and reset later

Thanks to the Hashcat Team for cracking the hashes of the malware.

Below are listed service names which are also keys that can be found in the registry at HKLM\SYSTEM\CurrentControlSet\services and the corresponding vendor.

  • Windows Defender
    • 917638920165491138 – windefend
  • Windows Defender ATP
    • 16335643316870329598 – sense
  • Carbon Black
    • 11385275378891906608 – carbonblack
    • 13693525876560827283 – carbonblackk
    • 17849680105131524334 – cbcomms
    • 18246404330670877335 – cbstream
  • Crowdstrike
    • 11771945869106552231 – csagent
    • 9234894663364701749 – csdevicecontrol
    • 8698326794961817906 – csfalconservice
  • FireEye
    • 15695338751700748390 – xagt
    • 9384605490088500348 – fe_avk
    • 6274014997237900919 – fekern
    • 15092207615430402812 – feelam
    • 3320767229281015341 – fewscservice
  • ESET
    • 15587050164583443069 – eamonm
    • 9559632696372799208 – eelam
    • 4931721628717906635 – ehdrv
    • 3200333496547938354 – ekrn
    • 2589926981877829912 – ekrnepfw
    • 17997967489723066537 – epfwwfp
    • 14079676299181301772 – ekbdflt
    • 17939405613729073960 – epfw
  • F-Secure
    • 17624147599670377042 – f-secure gatekeeper handler starter
    • 16066651430762394116 – f-secure network request broker
    • 13655261125244647696 – f-secure webui daemon
    • 12445177985737237804 – fsaua
    • 3421213182954201407 – fsma
    • 14243671177281069512 – fsorspclient
    • 16112751343173365533 – f-secure gatekeeper
    • 3425260965299690882 – f-secure hips
    • 9333057603143916814 – fsbts
    • 3413886037471417852 – fsni
    • 7315838824213522000 – fsvista
    • 13783346438774742614 – f-secure filter
    • 2380224015317016190 – f-secure recognizer
    • 3413052607651207697 – fses
    • 3407972863931386250 – fsfw
    • 10393903804869831898 – fsdfw
    • 12445232961318634374 – fsaus
    • 3421197789791424393 – fsms
    • 541172992193764396 – fsdevcon

Dive In

Ebook Free Download

Securing Your Organization’s Network on a Shoestring

How to protect your resource-constrained organization’s endpoints, networks, files and users without going bankrupt or losing sleep.

DOWNLOAD NOW
Ebook Free Download

Securing Your Organization’s Network on a Shoestring

How to protect your resource-constrained organization’s endpoints, networks, files and users without going bankrupt or losing sleep.

DOWNLOAD NOW
SOLUTION BRIEF

Automated Threat Discovery & Mitigation

Secure your all organizational assets with a single platform. Cynet 360 protects across all threat vectors, across all attack stages.

DOWNLOAD NOW
SOLUTION BRIEF

Automated Threat Discovery & Mitigation

Secure your all organizational assets with a single platform. Cynet 360 protects across all threat vectors, across all attack stages.

DOWNLOAD NOW
FREE TRIAL

Deploy Cynet in Minutes and Try it for 14 Days

Try Cynet’s easy-to-launch prevention, detection and response platform across your entire organization - free for 14 days!

START YOUR TRIAL