Achieved 100% detection in 2023
Stop advanced cyber
threats with one solution
Cynet’s All-In-One Security Platform
- Full-Featured EDR and NGAV
- Anti-Ransomware & Threat Hunting
- 24/7 Managed Detection and Response
Key Malware Detection Techniques
February 17, 2022
Last Updated:
November 27, 2024
Share on:
Threat actors employ malicious software, known as malware, to perform certain malicious activities. There are numerous types of malware, each deployed to achieve different objectives. For example, spyware aims to gather information from computing devices, trojans aim to gain a persistent hold on a computer system, and ransomware aims to encrypt information and extort its owners.
Malware detection involves using techniques and tools to identify, block, alert, and respond to malware threats. Basic malware detection techniques can help identify and restrict known threats and include signature-based detection, checksumming, and application allowlisting. Advanced malware detection tools employ artificial intelligence (AI) and machine learning to proactively look for and identify new and unknown malware threats.
This is part of an extensive series of guides about malware protection.
Malware Detection Techniques
Stop advanced cyber
Rated 4.8/5
2024 Leader
Signature-Based Detection
Signature-based detection uses the unique digital footprint, known as a signature, of software programs running on a protected system. Antivirus programs scan software, identifies their signature and compares it to signatures of known malware.
Antivirus products use a large database of known malware signatures, typically maintained by a security research team operated by the antivirus vendor. This database is frequently updated and the latest version is synchronized with protected devices.
When an antivirus program identifies software that meets a known signature, it stops the process and either quarantines or deletes it. This is a simple and effective approach to malware detection and is important as the first line of defense. However, as attackers become more sophisticated, the signature-based approach cannot detect a wide variety of newer threats.
Checksumming
This method is a type of signature analysis that involves calculating cyclic redundancy check (CRC) checksums. Checksumming helps verify that files are uncorrupted. The main drawback of signature-based detection is creating a massive database generating false positives, which checksumming aims to address.
Hackers often use polymorphic malicious advertisements to avoid detection by signature-based identification methods. Polymorphic viruses can change themselves when replicating, eliminating consistent search strings – usually, the hacker encrypts random decryption command sets in the form of non-constant keys in the virus code.
Thus, when the security team identifies a malicious signature, the malware no longer contains the code fragment and cannot be found. The absence of a detectable signature in the variable code requires other malicious code detection techniques, such as:
- Statistical analysis – analyzes the frequency of processor commands to determine if a file is infected.
- Cryptanalysis – known-plaintext cryptanalysis decodes encrypted viruses using an equation system (like the classic cryptographic technique of decoding text without a decryption key). The cryptanalysis system reconstructs the decryption program’s algorithm and keys, applying the algorithm to encoded fragments to decode the overall body of the encrypted virus.
- Heuristics – a malware detection team scans and analyses behavioral data to identify anomalous activity. The team must search for malicious code associated with suspicious behavior, such as a code served to thousands of users within a few minutes. The security team can then prioritize and further investigate suspicious incidents.
- Reduced masks – the malware detection team can use elements within the encrypted virus body to circumvent the need for an encryption key when obtaining static code. The static code produced can reveal the malware’s signature or mask.
Tips From the Expert
In my experience, here are tips that can help you optimize malware detection in your organization:
- Use deception-based malware detection Implement deception technologies, such as decoy files or honeypots, to lure malware into revealing itself. These traps can trigger alerts when malware attempts to interact with seemingly valuable files, exposing threats that bypass traditional detection.
- Deploy memory-based malware detection Since advanced malware can evade file-based detection by operating solely in memory, using tools that monitor in-memory processes is crucial. This ensures that fileless malware, which does not leave a footprint on disk, is caught and neutralized.
- Employ threat intelligence integration into detection tools Incorporate real-time threat intelligence feeds directly into EPP and EDR solutions to continuously update your detection capabilities with the latest indicators of compromise (IoCs) for emerging malware strains and zero-day attacks.
- Focus on lateral movement detection Many malware infections start at a single endpoint but aim to spread laterally across the network. Implement lateral movement detection strategies, such as monitoring internal traffic and using EDR’s threat-hunting capabilities to identify unusual patterns.
- Use multifactor entropy analysis on suspicious files To detect malware using polymorphic techniques, apply entropy analysis to assess the randomness of file structures. Malware often uses encryption or obfuscation to hide its true nature, and entropy analysis can uncover these hidden threats.
Application Allowlisting
Application allowlists (aka whitelists) are the opposite of the attack signature approach. Instead of defining which software the antivirus program should block, it maintains a list of approved applications and blocks everything else.
This solution is not perfect but can be highly effective, especially in high-security environments. It is quite common for legitimate applications to have security vulnerabilities, or introduce unneeded features that increase the attack surface. In some cases, the application itself is benign, but its use could expose the device to threats – for example, in some environments, there may be a need to block web browsing and email.
Application allow listing works best with devices that are strictly task-focused, such as web servers and internet of things (IoT) devices.
Machine Learning Behavioral Analysis
The above techniques are known as “static” detection techniques because they rely on binary rules that either match or do not match a process running in the environment. Static malware detection cannot learn, it can only add more rules or fine-tune its rules over time to increase coverage.
By contrast, new dynamic techniques, based on artificial intelligence and machine learning (AI/ML), can help security tools learn to differentiate between legitimate and malicious files and processes, even if they do not match any known pattern or signature. They do this by observing file behavior, network traffic, frequency of processes, deployment patterns, and more. Over time, these algorithms can learn what “bad” files look like, making it possible to detect new and unknown malware.
AI/ML malware detection is known as “behavioral” detection because it is based on an analysis of the behavior of suspect processes. These algorithms have a threshold for malicious behavior, and if a file or process exhibits unusual behavior that crosses the threshold, they determine it to be malicious.
Behavioral analysis is powerful, but can sometimes miss malicious processes or incorrectly classify legitimate processes as malicious. In addition, attackers can manipulate AI/ML training processes. In several cases, attackers fed specially-crafted artifacts to a behavioral analysis mechanism, to train it to recognize malicious software as safe.
Related content: Read our guide to advanced malware detection (coming soon)
Stop advanced cyber
threats with one solution
Cynet’s All-In-One Security Platform
- Full-Featured EDR and NGAV
- Anti-Ransomware & Threat Hunting
- 24/7 Managed Detection and Response
Achieved 100% detection in 2023
Rated 4.8/5
2024 Leader
Advanced Malware Detection Technologies
While many organizations rely on legacy antivirus as their malware detection strategy, mature security organizations typically use two types of advanced solutions to defend against malware – endpoint protection platforms and endpoint detection and response solutions.
Endpoint Protection Platforms (EPP)
EPPs are deployed on endpoints such as employee workstations, servers, and cloud-based resources. They serve as the first line of defense that can identify threats and block them before they cause damage to sensitive assets.
EPPs use multiple techniques to detect and block malware:
- Static analysis – EPPs leverage traditional static analysis methods to identify known malware strains and allow/deny applications flagged by administrators.
- Behavioral analysis – EPPs add behavioral analysis to detect unknown threats or known malware that uses evasion tactics like mutation or encryption.
- Sandboxed inspection – EPPs can run suspicious content in a sandbox, isolated from the main operating system. This makes it possible to “detonate” a file, observe its behavior and confirm if it is really malicious or not.
- Content Disarm and Reconstruction (CDR) – EPPs make it possible to remove malicious elements of legitimate content, and allow the user to access the content itself. For example, if a Word document has a malicious macro, CDR can remove the macro and allow the user to access the file, instead of blocking it entirely.
In addition to these techniques, once the malware is detected, EPPs can actively protect the environment, for example by isolating the endpoint from the network.
Endpoint Detection and Response (EDR)
EDR solutions complement EPP solutions by allowing security teams to identify and respond to attacks on endpoint devices. If EPP failed to contain a threat, EDR makes it possible to:
- Triage and investigate alerts – EDR provides rich data from endpoints that allows security analysts to identify signs of an attack, and investigate them to confirm a security incident.
- Threat hunting – EDR makes it possible to proactively search endpoints and explore relevant data for signs of a breach.
When an analyst confirms a threat on an endpoint, they can use the EDR platform for incident response. For example, analysts can quarantine all devices affected by malware, wipe and reimage infected endpoints, and run automated security playbooks. Security playbooks can be used to coordinate a response to a malware threat across multiple security tools – including firewalls and network segmentation, intrusion prevention systems (IPS), and email security. Many EDR solutions incorporate EPP capabilities.
Advanced Malware Protection with Cynet
The Cynet 360 Advanced Threat Detection and Response platform provides protection against threats including zero-day attacks, advanced persistent threats (APT), advanced malware, and trojans that can evade traditional signature-based security measures.
Block exploit-like behavior
Cynet monitors endpoints memory to discover behavioral patterns that are typical to exploit such as an unusual process handling request. These patterns are common to the vast majority of exploits, whether known or new, and provide effective protection even from zero-day exploits.
Block exploit-derived malware
Cynet employs multi-layered malware protection that includes ML-based static analysis, sandboxing and process behavior monitoring. In addition, they provide fuzzy hashing and threat intelligence. This ensures that even if a successful zero-day exploit establishes a connection with the attacker and downloads additional malware, Cynet will prevent this malware from running so no harm can be done.
Uncover hidden threats
Cynet uses an adversary-centric methodology to accurately detect threats throughout the attack chain. Cynet thinks like an adversary, detecting behaviors and indicators across endpoints, files, users, and networks. They provide a holistic account of the operation of an attack, irrespective of where the attack may try to penetrate.
Accurate and precise
Cynet uses a powerful correlation engine and provides its attack findings with near-zero false positives and is free from excessive noise. This simplifies the response for security teams so they can react to important incidents.
You can carry out automatic or manual remediation, so your security teams have a highly effective yet straightforward way to detect, disrupt, and respond to advanced threats before they have a chance to do damage.
Learn more about Cynet’s Next-Generation Antivirus (NGAV) Solution.
See Our Additional Guides on Key Cybersecurity Topics
Together with our content partners, we have authored in-depth guides on several other topics that can also be useful as you explore the world of Cybersecurity
Network Attacks
Learn about the main cyber attacks that threaten security of modern networks.
- Cobalt Strike: White Hat Hacker Powerhouse in the Wrong Hands
- Mimikatz: World’s Most Dangerous Password-Stealing Platform
- Understanding Privilege Escalation and 5 Common Attack Techniques
Zero-Day Attack
Learn about the risk of unknown threats that can hit organizations before they are discovered by vendors and researchers.
- 5 Ways to Defend Against Zero-Day Malware
- Zero-Day Attack Prevention: 4 Ways to Prepare
- Zero-Day Exploit: Recent Examples and 4 Detection Strategie
XDR
Learn how extended detection and response (XDR) solutions provide a single platform for responding to endpoint, cloud, email, and network-based threats.
- XDR Security Solutions: Get to Know the Top 8
- Cortex XDR by Palo Alto: Architecture & Capabilities Overview
- Cisco XDR: SecureX Suite at a Glance
Advanced Persistent Threat
Learn how organized crime groups and nation state attackers wage coordinated cyber attacks against organizations.
- Advanced Persistent Threat (APT) Attacks
- APT Security: Warning Signs and 6 Ways to Secure Your Network
- How and Why You Need to Protect
EDR
Learn how endpoint detection and response (EDR) solutions can help immediately contain breaches on endpoint devices.
