Malware Sandboxing the Xandora Way

MALWARE SANDBOXING – THE XANDORA WAY Lau Kai Jern, Chief Development Officer – kj@xandora.net INTRODUCTION Most of the Eme SomeEmes Once a year Working in Panda Security since year 2005 Running xandora.net project. Member of vnsecurity.net Crew > 
> 
Runing the technical team In charge of APAC malware incidents > 
The coder > 
> 
The administrator > 
> 
The everything Good friends Can’t really recall what I did for my good friends > 
Yet to be define KAIJERN @ XWINGS 9am – 6pm Weekday Malware Analysis 101 Define: Sandbox What is xandora Architecture Infrastructure Technical Problems idenXficaXon Global Partnership Sector Roadmap References & Acknowledgements AGENDA IntroducXon MALWARE ANALYSIS 101 • 
Reading the binary • 
Understanding the binary • 
Become crazy STATIC/DYNAMIC ANALYSIS StaEc Analysis Dynamic Analysis • 
Virtual Machines • 
Analysis tools/ debugger • 
Human Analysis DEFINE: SANDBOX AUTOMATED SANDBOX Sandbox •  Isolated environment to run untrusted code •  Run a suspicious file within a locked down environment •  “Locked” but not overly restricXve. Eg: Sandbox must comes with network access •  Provide file behavioral report PROBLEMS OBJECTIVES ObjecEves OTHER MALWARE SANDBOX •  Too many malware sandboxes out there •  Most the the sandbox design have only one objecXve, which is to provide complete analysis report for a file being processed. This will lead to: i. 
ii. 
iii. 
iv. 
v. 
Lengthy report, 40-­‐60 pages Too much informaXon Too Enterprise Takes too much resources to process Process files in-­‐Xme. 24 hours malware Performance Scoring System How to solve this problem and why this is important Network Malware samples received at PandaLabs
Data: May 2009
20 M.
Process Scoring Engine X10
X2
X2
X2
File 2003
2004
Source: PandaLabs
2005
2006
2007
2008
Registry WHAT IS XANDORA WHAT IS XANDORA Online Global Collaboration
Partnership
Possible Malware
Virtualization Management
Automated Malware Analysis
Platform
ARCHITECTURE Pre Analysis Post Analysis Xandbox Monitor File System Checksum Registry Appearance FTP Mail Vendor Memory Dump Copy-­‐on-­‐Write Image Web Malware Scanner Portal XANDBOX Virtual Network Adapters Goodware Scanner InteracEons Response Kernel Level AcceleraEon RAM Disk AcceleraEon Public Network Digital InvesEgaEon VM AcceleraEon Unpacker Government Connectors VM Control Center Screen Capture AdministraEon Private Connectors Vendor ARCHITECTURE OVERVIEW Source Checksum Checksum •  Common unique idenXficaXon •  Generate SHA1 and MD5 Appearance Appearance •  Check against database •  Update the last Xme file being received Malware Goodware Scanner Scanner Unpacker Malware Scanner Goodware Scanner •  Compare against all anXvirus vendor listed in VirusTotal •  Private access to VirusTotal database •  Check file belongs to which Company and Product •  VirusTotal •  shaodowserver.org XANDBOX Post Analysis File System Registry Network Digital InvesEgaEon PRE ANALYSIS Pre Analysis Screen Dump Unpacker •  Unpack binary for staXc analysis •  Able to automaXcally unpack ASPack, NSPack, UPX and PE_Compact VM IO Access VM IO Dump VM Monitor Access •  Capture screen dump when there is a screen change •  Issue specific command such as mouse movement and key stroke Monitor Fork virtual disk image •  Able to accept VM dump for analysis Full access to RAM Memory Dump Copy-­‐on-­‐Write Image •  Dump full RAM snapshot from VM InteracEons Response Allow file to access internet Kernel Level AcceleraEon OpXmized RAM Disk AcceleraEon VM AcceleraEon Queue CPU Usage •  Suspicious file able to access network from VM •  Monitor request from the suspicious file to create a new file or made changes in the registry Monitor every acXon •  Both images master and running images are stored in RAM •  Gain full access to RAM XANDBOX Virtual Network Adapters Xanbox •  Fork disk image from master VM Image Network Usage VM Control Center AcceleraXon •  Use different kinds of hardware and soiware acceleraXon to make sure all the VM fork by Xandora is being opXmized VM Control Center •  Monitor process and process queue •  Ensure CPU usage is not overloaded •  Only one network adapter for one VM XANDBOX VM Screen Dump Checksum Appearance Malware Goodware Scanner Scanner POST ANALYSIS Pre Analysis Unpacker XANDBOX Post Analysis File System File System •  Look for newly generated file •  Store newly generated executable file Registry •  Dump VM registry •  Look for newly generated, edited and deleted registry entries Registry Network Digital InvesEgaEon Network •  Analyze network traffic •  DesXnaXon host and port •  DesXnaXon URL •  Extract downloaded file Screen Dump Digital InvesXgaXon •  Full memory dump from VM •  Analyze acXve and suspicious process Screen Dump •  Capture screenshot from VM •  Do not store if there are no acXviXes •  Do not store if screen is duplicated IN ACTION PORTAL Landing Page PORTAL Dashboard File Header PORTAL File InformaEon Process Network PORTAL File system and Registry Screen Shot INFRASTRUCTURE AnXvirus Result Receive 25,000 files/day Process 400 files/hour 70% non detected by AV FUTURE 1 x Server Capacity CURRENT Files Scanned Receive 150,000 files/day Process 1,000 files/hour 90% non detected by AV CollaboraEon Call Malware Analysis Infrastructure VirtualizaEon Hardware VirtualizaXon •  Requires specific Xme •  Improvement only in the number of virtual machines in each server •  FoundaXon File Analysis File Analysis File InjecEon File InjecEon VirtualizaEon VirtualizaEon Algorithm Sandboxing •  Threat evoluXon will increase the amount of analysis and processing requirements •  Fundamental CollaboraEon CollaboraXon •  Request for informaXon from our analysis partners •  ExperXse and focus PERFORMANCE AND CAPACITY TECHNICAL PROBLEMS •  Find soluXons for malware to run under actual machine ExecuXon Timing •  Requires specific Xme •  Improvement only in the number of virtual machines in each server •  FoundaXon Volume •  Increase in numbers Hiding Client •  Hiding sensors •  Increase in variants •  Kernel driver Report •  Demand for more informaXon snapshots •  Delay in processing •  Hidden process •  Demand for more detailed analysis •  Vendors process files without sandbox File System •  Reduce mount and umount at preprocess Concurrent VM •  How many VMs •  Post process qcow
+NTFS problems •  Which process with highest CPU load •  How to check Input/Output •  Base image protecXon •  Faster read/write for VM •  Faster read write for post processing SANDBOX PROBLEMS Detect VM ENV •  Binary that do not run under virtual machine •  Find soluXons for malware to run under actual machine ExecuXon Timing •  Requires specific Xme •  Improvement only in the number of virtual machines in each server •  FoundaXon Volume •  Increase in numbers Hiding Client •  Hiding sensors •  Increase in variants •  Kernel driver Report •  Demand for more informaXon snapshots •  Delay in processing •  Hidden process •  Demand for more detailed analysis •  Vendors process files without sandbox File System •  Reducing mount and umount at preprocess Concurrent VM •  How many VMs •  Post process qcow
+NTFS problems •  Which process with highest CPU load •  How to check Input/Output •  Base image protecXon •  Faster read/write for VM •  Faster read write for post processing SANDBOX PROBLEMS Detect VM ENV •  Binary that do not run under virtual machine •  Find soluXons for malware to run under actual machine ExecuXon Timing •  Requires specific Xme •  Improvement only in the number of virtual machines in each server •  FoundaXon Detect VM ENV •  Detect samples not able to run under VM ExecuXon Timing •  Fixed Xme between 3 to 5 minutes •  Possible Malware •  ExecuXon and no response from binary •  Possible Malware Volume •  Increase in numbers Hiding Client •  Hiding sensors •  Increase in variants •  Kernel driver Report •  Demand for more informaXon snapshots •  Delay in processing •  Hidden process •  Demand for more detailed analysis •  Vendors process files without sandbox Volume •  Small scale Windows Hiding Client •  No client required Report •  Simple •  VM monitoring and queuing engine •  Possible malware scoring algorithm •  Ensure readability •  Task allocaXon THE XANDORA WAY Detect VM ENV •  Binary that do not run under virtual machine •  Find soluXons for malware to run under actual machine ExecuXon Timing •  Requires specific Xme •  Improvement only in the number of virtual machines in each server •  FoundaXon Volume •  Increase in numbers Hiding Client •  Hiding sensors •  Increase in variants •  Kernel driver Report •  Demand for more informaXon snapshots •  Delay in processing •  Hidden process •  Demand for more detailed analysis •  Vendors process files without sandbox File System •  Reducing mount and umount at preprocess Concurrent VM •  How many VMs •  Post process qcow
+NTFS problems •  Which process with highest CPU load •  How to check Input/Output •  Base image protecXon •  Faster read/write for VM •  Faster read write for post processing SANDBOX PROBLEMS Detect VM ENV •  Binary that do not run under virtual machine Preprocessing i. 
i. 
Too many mount/umount kill the system – Kernel Panic Group all required files in to a ISO, using mkisofs Sandbox i. 
ii. 
Start VM with ISO image as ISO Run the ISO while VM boots up i.  Register runonce ii.  Autorun.inf Post Processing i.  Mount nqs over tcpip ii.  Mount nqs over ramfs iii.  Modding nqs-­‐3g a.  Disable checking b.  Force read only c.  Fix to one NTFS version FILE SYSTEM Problems •  Find soluXons for malware to run under actual machine ExecuXon Timing •  Requires specific Xme •  Improvement only in the number of virtual machines in each server •  FoundaXon Volume •  Increase in numbers Hiding Client •  Hiding sensors •  Increase in variants •  Kernel driver Report •  Demand for more informaXon snapshots •  Delay in processing •  Hidden process •  Demand for more detailed analysis •  Vendors process files without sandbox File System •  Reducing mount and umount at preprocess Concurrent VM •  How many VMs •  Post process qcow
+NTFS problems •  Which process with highest CPU load •  How to check Input/Output •  Base image protecXon •  Faster read/write for VM •  Faster read write for post processing SANDBOX PROBLEMS Detect VM ENV •  Binary that do not run under virtual machine Preprocessing i. 
i. 
Too many mount/umount kill the system – Kernel Panic File queue a.  Priority b.  Balanced for mulXple sandbox Sandbox i. 
ii. 
Pick up files and insert into VM VM monitoring a.  Total running VMs b.  Heavy process – RAM Dump iii.  Process RAM Dump. Post Processing i. 
Process output files VIRTUAL MACHINE Problems •  Find soluXons for malware to run under actual machine ExecuXon Timing •  Requires specific Xme •  Improvement only in the number of virtual machines in each server •  FoundaXon Volume •  Increase in numbers Hiding Client •  Hiding sensors •  Increase in variants •  Kernel driver Report •  Demand for more informaXon snapshots •  Delay in processing •  Hidden process •  Demand for more detailed analysis •  Vendors process files without sandbox File System •  Reducing mount and umount at preprocess Concurrent VM •  How many VMs •  Post process qcow
+NTFS problems •  What the most heave process •  How to check Input/Output •  Base image protecXon •  Faster read/write for VM •  Faster read write for post processing SANDBOX PROBLEMS Detect VM ENV •  Binary that do not run under virtual machine Preprocessing i. 
ii. 
i. 
ii. 
So far the only problem is slow No disk error yet Move required file to RAM Disk SSD saves the world Sandbox i. 
ii. 
Protect Master Image a.  charr +I SSD saves the world Post Processing i. 
ii. 
Move required files to RAM disk SSD saves the world INPUT OUTPUT Problems IDENTIFICATION Registry •  How to know changes in registry is good or malicious Process •  Good or malicious process Networking •  IdenXfy good and malicious traffic SANDBOX PROBLEMS File System •  What is good, what is bad. Clean and easy to idenXfy a bad file i.  Compare old and new file system change ii.  Malicious change a.  Dropping exe b.  Dropping dll c.  Dropping sys iii.  Dropped locaXon a.  c:\windows\fonts FILE SYSTEM File System Registry •  How to know changes in registry is good or malicious Process •  Good or malicious process Networking •  IdenXfy good and malicious traffic SANDBOX PROBLEMS File System •  What is good, what is bad. i. 
Registry change a.  Disable anXvirus b.  Add in autorun at startup REGISTRY Registry Registry •  How to know changes in registry is good or malicious Process •  Good or malicious process Networking •  IdenXfy good and malicious traffic SANDBOX PROBLEMS File System •  What is good, what is bad. Process PROCESS How to hunt for a malicious process i.  List down all processes ii.  Full process path iii.  Process file name (svch0st.exe) iv.  File MD5 or SHA1 for comparison Registry •  How to know changes in registry is good or malicious Process •  Good or malicious process Networking •  IdenXfy good and malicious traffic SANDBOX PROBLEMS File System •  What is good, what is bad. None of these being implemented yet. i.  IP BlacklisXng ii.  Domain blacklisXng NETWORKING Networking STATISTICS 15 DAYS RECEIVED FILES 25000 20000 15000 10000 5000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 15 DAYS RECEIVED SAMPLES – BY VENDORS 25000 20000 15000 10000 5000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 VENDORS DETECTION 90000 80000 70000 60000 50000 40000 30000 20000 10000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 URL – 15 DAYS 40000 35000 30000 25000 20000 15000 10000 5000 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 GLOBAL PARTNERSHIPS GLOBAL PARTNERSHIP SECTOR Collaborate with experts globally to outnumber and outsmart cybercriminals. Even cybercriminals collaborate. > 
> 
From security experts to endless possibiliXes of collaboraXon Xandora plaqorm enables global collaboraXon Government Country CERTs should have operaXons to monitor targeted aracks which can affect its economy and security. > 
> 
> 
Monitors country-­‐wide acXviXes Enterprise EducaEon Enable large enterprises to monitors its security. Establish long-­‐term working partnership with universiXes to train future experts in CERT. > 
CollaboraXon between all Government departments ProacXve effort towards security > 
> 
Business disasters such as downXme, data leakage, etc. widely affected large enterprises in recent Xmes. Corporate espionage Shareholders must be proacXvely protected > 
> 
Providing Xandora for FREE UniversiXes can be collaborators and contributors. MEETING FUTURE NEEDS Community ROADMAP ROADMAP Jan -­‐ March New ReporXng Interface Nov/Dec PDF/Office/
APK File Analysis Oct ExecuXve ReporXng 2012 April MulX Vendor Profiling May/June Binary Profiling Aug/Sept NoSQL Conversion July/August TranslaXon REFERENCES & ACKNOWLEDGEMENTS 1. 
2. 
3. 
4. 
5. 
6. 
Nguyen Anh Quynh, Virt-­‐ICE: next generaXon debugger for malware analysis Nguyen Anh Quynh, eKimono: A Malware Scanner for Virtual Machines Georg Wicherski, dirtbox, A x86/Windows Emulator Daniel Raygoza, Automated Malware Similarity Analysis Project: Cuckoo Book: Malware analysis cookbook Acknowledgements 1. 
2. 
3. 
4. 
Very good friends from vnsecurity.net Rodrigo Rubira Branco Meling Mudin PandaLabs REFERENCES & ACKNOWLEDGEMENTS References THANK YOU kj@xandora.net | h]p://xandora.net | h]p://www.facebook.com/xandora | @kaijern | @susPEciousfile