Anti-Analysis Techniques: How Malware Detects Your Sandbox

Comprehensive catalog of VM detection, sandbox evasion, debugger detection, and timing-based anti-analysis techniques - with concrete code, the rationale behind each check, and counter-measures for analysts hardening their lab.

Why Anti-Analysis Exists

Modern malware authors care about a single asymmetry: their sample only has to work on a real victim. If it can recognise that it’s running inside a sandbox, an analyst’s VM, or under a debugger, the cheapest possible response is to do nothing - exit cleanly, sleep for an hour and exit, deliberately throw a benign exception. From the defender’s automated pipeline this looks like “no malicious behaviour observed”, and the sample sails through with a clean verdict.

Knowing the catalogue of techniques is essential whether you’re writing implants (so you can choose evasion that suits your engagement) or analysing them (so you can pre-emptively patch your sandbox to defeat each check). Most modern commodity loaders chain dozens of these - dropping out at the first failed check, in random order, so an incomplete patch list still loses.

This post is the working catalogue I keep next to my analysis VM. Every technique includes a snippet, the signal it’s reading, and what to do about it on the defender’s side.

VM Detection

Registry Checks

// VMware
RegOpenKeyEx(HKLM, "SOFTWARE\VMware, Inc.\VMware Tools", ...);
// VirtualBox
RegOpenKeyEx(HKLM, "SOFTWARE\Oracle\VirtualBox Guest Additions", ...);

Other registry keys worth scrubbing on a hardened lab:

Key	Reveals
`HKLM\SYSTEM\ControlSet001\Services\VBoxGuest`	VirtualBox guest service
`HKLM\SYSTEM\ControlSet001\Services\VBoxMouse`	VirtualBox mouse driver
`HKLM\SYSTEM\ControlSet001\Services\VMTools`	VMware Tools service
`HKLM\HARDWARE\DESCRIPTION\System\BIOS\SystemManufacturer`	“VMware, Inc.”, “innotek GmbH”
`HKLM\HARDWARE\DESCRIPTION\System\BIOS\SystemProductName`	“VMware Virtual Platform”, “VirtualBox”
`HKLM\HARDWARE\DESCRIPTION\System\VideoBiosVersion`	VirtualBox / VMware video string
`HKLM\SYSTEM\CurrentControlSet\Enum\IDE` / `\SCSI`	“VBOX_HARDDISK”, “VMware_Virtual_…”
`HKLM\SYSTEM\CurrentControlSet\Enum\PCI\VEN_15AD`	VMware PCI vendor (0x15AD)
`HKLM\SYSTEM\CurrentControlSet\Enum\PCI\VEN_80EE`	VirtualBox vendor (0x80EE)

Hardening tip: an analysis VM should rename or remove the VBox/VMware services, scrub BIOS strings (VirtualBox supports this via VBoxManage setextradata "VM" "VBoxInternal/..."), and present generic Microsoft Hyper-V device IDs instead.

WMI Queries

SELECT * FROM Win32_ComputerSystem WHERE Model LIKE '%Virtual%'
SELECT * FROM Win32_BIOS WHERE SerialNumber LIKE '%VMware%'

Practical WMI queries malware uses regularly:

-- BIOS / system manufacturer
SELECT Manufacturer, Model FROM Win32_ComputerSystem
SELECT Manufacturer, SerialNumber, Version FROM Win32_BIOS
SELECT Caption FROM Win32_VideoController         -- "VMware SVGA II", "VirtualBox Graphics Adapter"

-- Storage devices
SELECT Caption, Model FROM Win32_DiskDrive
SELECT Description FROM Win32_PnPEntity WHERE Caption LIKE '%VMware%' OR Caption LIKE '%VBOX%'

-- Network adapters
SELECT MACAddress, Description FROM Win32_NetworkAdapter

-- Process list (look for analysis tools)
SELECT Name FROM Win32_Process

Hardware Artifacts

MAC address prefixes: 00:0C:29 (VMware), 08:00:27 (VirtualBox)
CPU brand string containing “hypervisor”
CPUID leaf 0x40000000 (hypervisor vendor)
Red Pill: SIDT/SGDT instruction return different values in VMs

The CPUID checks deserve concrete examples:

// CPUID leaf 1, ECX bit 31 = "hypervisor present"
int regs[4];
__cpuid(regs, 1);
if (regs[2] & (1 << 31)) {
    // running under SOME hypervisor (any modern one sets this)
}

// CPUID leaf 0x40000000 - hypervisor vendor signature
__cpuid(regs, 0x40000000);
char vendor[13] = {0};
memcpy(vendor + 0, &regs[1], 4);   // EBX
memcpy(vendor + 4, &regs[2], 4);   // ECX
memcpy(vendor + 8, &regs[3], 4);   // EDX
// "VMwareVMware"   → VMware
// "VBoxVBoxVBox"   → VirtualBox
// "Microsoft Hv"   → Hyper-V
// "KVMKVMKVM"      → KVM/QEMU
// "XenVMMXenVMM"   → Xen

Red Pill is the historical (Joanna Rutkowska, 2004) trick using SIDT (Store Interrupt Descriptor Table register). On bare metal the IDT lives at a low address; on a VM, the guest’s IDT typically lives at a much higher address (0xFFXXXXXX style). Modern VMs with VT-x / AMD-V largely defeat this on x64, but variations like STR, SLDT, and timing-based equivalents persist.

MAC OUI vendor blocks:

Prefix	Vendor
`00:05:69`, `00:0C:29`, `00:1C:14`, `00:50:56`	VMware
`08:00:27`	VirtualBox
`00:03:FF`, `00:15:5D`	Microsoft Hyper-V
`00:1C:42`	Parallels
`52:54:00`	QEMU/KVM

Memory and CPU Footprints

// Cores
SYSTEM_INFO si; GetSystemInfo(&si);
if (si.dwNumberOfProcessors < 2) sandbox = TRUE;

// RAM (in MB)
MEMORYSTATUSEX ms = { sizeof(ms) };
GlobalMemoryStatusEx(&ms);
if (ms.ullTotalPhys < 2ULL * 1024 * 1024 * 1024) sandbox = TRUE;

VirtualBox/VMware default templates often have 2 cores and 2-4 GB RAM. A real workstation is closer to 8 cores and 16+ GB. Hardening: configure your analysis VM with 4+ cores and 8+ GB RAM even if it’s overkill for the sample.

Sandbox Evasion

Resource Checks

Less than 2 CPU cores → sandbox
Less than 4GB RAM → sandbox
Disk size < 60GB → sandbox
Screen resolution 1024x768 → default sandbox

// Disk size
ULARGE_INTEGER total;
GetDiskFreeSpaceEx("C:\\", NULL, &total, NULL);
if (total.QuadPart < 60ULL * 1024 * 1024 * 1024) sandbox = TRUE;

// Screen resolution
int x = GetSystemMetrics(SM_CXSCREEN);
int y = GetSystemMetrics(SM_CYSCREEN);
if (x == 1024 && y == 768) sandbox = TRUE;

The 1024×768 default is characteristic of headless analysis sandboxes (Cuckoo, ANY.RUN, Hatching Triage in their default configs). Real users almost never run at this resolution any more.

Behavioral Checks

No recent files in Downloads/Documents
No browser history
Uptime < 20 minutes
Username is “admin”, “sandbox”, “analyst”, “malware”

// Uptime
DWORD ms = GetTickCount();
if (ms < 20 * 60 * 1000) sandbox = TRUE;

// Recent files
WIN32_FIND_DATAA fd;
HANDLE h = FindFirstFileA("C:\\Users\\*\\Documents\\*.docx", &fd);
int count = 0; if (h != INVALID_HANDLE_VALUE) { do { count++; } while (FindNextFileA(h, &fd)); FindClose(h); }
if (count < 5) sandbox = TRUE;

// Username blacklist
char user[256]; DWORD sz = sizeof(user); GetUserNameA(user, &sz);
const char* black[] = {"sandbox","analyst","malware","virus","sample","test","lab","john","admin","user"};
for (auto b : black) if (_stricmp(user, b) == 0) sandbox = TRUE;

Other behavioural tells some loaders look for:

Number of visible top-level windows (EnumWindows) - humans usually have 5+; clean sandboxes often have 1-2.
Mouse movement / cursor position changes over a 30-second window - GetCursorPos, sleep, GetCursorPos again. Identical position = no human.
Recent process activity - EnumProcesses for browser, office, slack, etc.
Domain join state - corporate machines are domain-joined; sandboxes rarely are.
Installed software footprint - HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall should have dozens of entries on a real workstation.

Hardening tip: bake your analysis image with a year-old user profile, hundreds of .docx and .pdf files generated programmatically, populated browser history (ChromeProfileTool), and an aged uptime baseline before snapshotting.

Timing Attacks

DWORD start = GetTickCount();
Sleep(10000);  // 10 seconds
DWORD elapsed = GetTickCount() - start;
if (elapsed < 9000) exit(0);  // Sleep was accelerated = sandbox

The reason this works: many sandboxes (Cuckoo, FireEye AX, FortiSandbox) hook Sleep/SleepEx/NtDelayExecution and either accelerate the wait or skip it entirely, because waiting through every malware’s defensive sleep would slow the analysis pipeline to a crawl.

More sophisticated timing variants:

// RDTSC sandwich - measures CPU cycles, hard to fake without full CPU emulation
unsigned long long t1 = __rdtsc();
volatile int x; for (int i = 0; i < 100000; i++) x = i;
unsigned long long t2 = __rdtsc();
if ((t2 - t1) > 0x100000000ULL) sandbox = TRUE;   // way too slow → emulator

// Dual-clock divergence - Sleep + RDTSC + GetTickCount, all should agree
QueryPerformanceCounter(&q1);
ULONGLONG g1 = GetTickCount64();
Sleep(5000);
QueryPerformanceCounter(&q2);
ULONGLONG g2 = GetTickCount64();
double qpc_secs = (q2.QuadPart - q1.QuadPart) / qpc_freq;
double tick_secs = (g2 - g1) / 1000.0;
if (fabs(qpc_secs - tick_secs) > 1.0) sandbox = TRUE;

// Stalling code - millions of useless arithmetic operations,
// designed to take ~10 minutes on real hardware. Sandboxes time out.
volatile uint64_t k = 0;
for (uint64_t i = 0; i < 0x100000000ULL; i++) k += i * i;

Debugger Detection

IsDebuggerPresent() - checks PEB.BeingDebugged
NtQueryInformationProcess with ProcessDebugPort
Hardware breakpoint detection via GetThreadContext
INT 2D / INT 3 exception-based checks

The full anti-debug toolkit goes much further. The standard categories:

PEB-Based

The Process Environment Block has multiple fields a debugger touches:

// PEB.BeingDebugged (offset 0x02)
BOOL beingDebugged = ((PPEB)__readgsqword(0x60))->BeingDebugged;

// PEB.NtGlobalFlag (offset 0xBC) - heap flags set under debugger
DWORD ntGlobalFlag = *(DWORD*)((BYTE*)__readgsqword(0x60) + 0xBC);
if (ntGlobalFlag & (FLG_HEAP_ENABLE_TAIL_CHECK | FLG_HEAP_ENABLE_FREE_CHECK | FLG_HEAP_VALIDATE_PARAMETERS))
    debugger = TRUE;

// Heap.Flags & Heap.ForceFlags - also tweaked by ntdll under a debugger
PVOID heap = *(PVOID*)((BYTE*)__readgsqword(0x60) + 0x30);
DWORD heapFlags = *(DWORD*)((BYTE*)heap + 0x70);
DWORD forceFlags = *(DWORD*)((BYTE*)heap + 0x74);
if (heapFlags & ~HEAP_GROWABLE) debugger = TRUE;

NT Query

// ProcessDebugPort (info class 7)
DWORD_PTR debugPort = 0;
NtQueryInformationProcess(GetCurrentProcess(), 7, &debugPort, sizeof(debugPort), NULL);
if (debugPort != 0) debugger = TRUE;

// ProcessDebugObjectHandle (info class 0x1E) - present even with anti-anti-debug plugins
HANDLE debugObj = NULL;
NtQueryInformationProcess(GetCurrentProcess(), 0x1E, &debugObj, sizeof(debugObj), NULL);

// ProcessDebugFlags (info class 0x1F) - inverted: 0 means debugged
DWORD debugFlags = 0;
NtQueryInformationProcess(GetCurrentProcess(), 0x1F, &debugFlags, sizeof(debugFlags), NULL);
if (debugFlags == 0) debugger = TRUE;

Hardware Breakpoint Detection

CONTEXT ctx = { 0 };
ctx.ContextFlags = CONTEXT_DEBUG_REGISTERS;
GetThreadContext(GetCurrentThread(), &ctx);
if (ctx.Dr0 || ctx.Dr1 || ctx.Dr2 || ctx.Dr3) debugger = TRUE;

DR0-DR3 store hardware breakpoint addresses; if any are non-zero, hardware breakpoints are armed.

Exception-Based

// INT 2D - debug-only instruction. With a debugger attached, the debugger
// catches the exception and skips it; without one, the program crashes.
__try {
    __asm int 0x2d
    __asm nop
    debugger = TRUE;     // unreachable on bare metal
} __except (EXCEPTION_EXECUTE_HANDLER) {
    debugger = FALSE;
}

// SEH-handler chain - a debugger may add its own filter, lengthening the chain

Timing-Based Anti-Debug

ULONGLONG t1 = __rdtsc();
__asm { nop; nop; nop; }
ULONGLONG t2 = __rdtsc();
if ((t2 - t1) > 100000) debugger = TRUE;   // single-step debugger inflates timings

Defensive Allies

For analysts: ScyllaHide patches PEB fields and hooks NtQueryInformationProcess to lie about debug state. TitanHide is the kernel-driver counterpart. x64dbg’s “Anti-Anti-Debug” plugin chains both with hardware-breakpoint hiding. None of these are silver bullets - modern packers chain unique checks specifically to defeat each plugin.

Process and File-System Tells

const wchar_t* analysis_processes[] = {
    L"ollydbg.exe", L"x64dbg.exe", L"x32dbg.exe", L"ida.exe", L"ida64.exe",
    L"idaq.exe", L"idaq64.exe", L"radare2.exe", L"ghidra.exe",
    L"wireshark.exe", L"tshark.exe", L"fiddler.exe", L"procmon.exe",
    L"procexp.exe", L"procexp64.exe", L"tcpview.exe", L"autoruns.exe",
    L"sysinternals.exe", L"vmtoolsd.exe", L"VBoxService.exe",
    L"vmware-tray.exe", L"VBoxTray.exe",
};

// Snapshot via CreateToolhelp32Snapshot or via NtQuerySystemInformation

File-system equivalents: C:\Windows\System32\drivers\VBoxMouse.sys, C:\Program Files\VMware\, C:\Program Files\Oracle\VirtualBox Guest Additions\. Renaming and pruning these on the analysis image is part of standard hardening.

Behavioural Stealth (the Other Half of Anti-Analysis)

Detection isn’t the only anti-analysis play. Loaders also actively delay analysis:

Long initial sleep (with detected-acceleration check) - defeats automated sandboxes that timeout at 5 min.
User-interaction gating - wait for GetForegroundWindow to change, scroll wheel events, or a real keyboard input.
Domain-joined gating - only execute on domain-joined machines, optionally only when a specific domain matches.
Geo-fencing - refuse to execute outside a specific country (resolved by IP geolocation).
Time-of-day gating - only execute during business hours in the target timezone.
Decoy execution - run a benign payload first; only download the malicious second-stage after the sandbox has ended.

Each of these is a single if statement, but stacked together they slash sandbox detection rates dramatically.

A Realistic Anti-Analysis Chain

A modern loader’s first few hundred milliseconds typically look like:

Resolve APIs by hash (no kernel32!GetProcAddress imports)
Verify CPUID hypervisor bit, vendor signature
Verify >= 4 cores, >= 4 GB RAM, >= 100 GB disk
Verify >= 30 min uptime
Verify username not in blacklist
Verify >= 5 .docx in user's Documents
Verify domain-joined (or specific domain match)
Sleep 5 min, RDTSC sandwich the sleep
Sleep 5 min more if first sleep was accelerated
Decrypt second-stage payload
Reflective-load second stage

Each step takes microseconds; missing any single check terminates the whole chain. From an analyst’s perspective, every one of those ifs is a memory write you can patch - but you need to find them first under whatever obfuscation the author bundled in.

Understanding anti-analysis is essential for both offense (implementing evasion) and defense (building resilient sandboxes). Each technique has a counter-technique. The cat-and-mouse game keeps going because the costs are wildly asymmetric - adding a check is one line, defeating it is hours of analysis.