Stephan Neuhaus Saarland University Thomas Zimmermann
Transcription
Stephan Neuhaus Saarland University Thomas Zimmermann
Predicting Vulnerable Software Components Stephan Neuhaus Saarland University Thomas Zimmermann Andreas Zeller 0 Vulnerabilities Security Advisory 2005-12 Title: Livefeed bookmarks can steal cookies Impact: High Products: Firefox Description: Earlier versions of Firefox allowed javascript: and data: URLs as Livefeed bookmarks. When they updated the URL would be run in the context of the current page and could be used to steal cookies or data displayed on the page. If the user were on a page with elevated privileges (for example, about:config) when the Livefeed was updated, the feed URL could potentially run arbitrary code on the user's machine. 0 Vulnerabilities Security Advisory 2005-12 Title: Livefeed bookmarks can steal cookies Impact: High Products: Firefox Description: Earlier versions of Firefox allowed javascript: and data: URLs as Livefeed bookmarks. When they updated the URL would be run in the context of the current page and could be used to steal cookies or data displayed on the page. If the user were on a page with elevated privileges (for example, about:config) when the Livefeed was updated, the feed URL could potentially run arbitrary code on the user's machine. 0 Vulnerabilities 0 Vulnerabilities Security Advisory 2005-13 Title: Window Injection Spoofing Severity: Low Products: Firefox, Mozilla Suite Description: A website can inject content into a popup opened by another site if the target name of the popup window is known. An attacker who knows you are going to visit that other site could spoof the contents of the popup. 0 Vulnerabilities Security Advisory 2005-13 Title: Window Injection Spoofing Severity: Low Products: Firefox, Mozilla Suite Description: A website can inject content into a popup opened by another site if the target name of the popup window is known. An attacker who knows you are going to visit that other site could spoof the contents of the popup. 0 Vulnerabilities Security Advisory 2006-76 2005-15 2005-16 2005-41 2005-14 Title: Heap Spoofing Privilege using overflow escalation download outer window's via andDOM security in UTF8 Function property dialogs to object Title: XSS SSL "secure site"possible indicator spoofing Unicode with overrides Impact: overlapping conversion High windows Severity: Moderate Severity: Products: High Firefox Products:Critical Firefox, 2.0 Mozilla Suite Products: Description: Firefox, moz_bug_r_a4 Thunderbird, Mozilla Suite demonstrated Mozilla Suitethat Description:Various schemes were reported Description: the prototype ItMichael moz_bug_r_a4 is possible regression Kraxfor demonstrates reported a UTF8 described string several that inwith thatFunction could cause the "secure site" lock icon to invalid the exploits bug download 355161 sequences giving could dialog an certificate attacker to betrigger and exploited security the adetails heap ability todialogs bypass overflow tothe install can the of be appear and show for wrong converted spoofed malicious protections bycode Unicode partially against orbesteal cross data. covering Exploitability sitephishers requiring them scriptwith (XSS) would an that site. These could useddata, by toonly make depend overlapping the injection, on do which the window. commonplace attackers could Some belegitimate, ability used actions users to tomay steal get likenot the click credentials notice string onina theiruser spoofs look more particularly into the link or sensitive OS or theopen window buggy data the converter. border context from and menu. General browser The sites web common or statusbar content perform windows that hide the arbitrary address bar showing the is converted bisecting cause destructive in each what elsewhere actions case appears was onbut behalf to privileged we be can't aofsingle a UI logged-in rule code dialog, out the user. and be true location. possibility of convinced ("chrome") by being athe successful spoofing overly trusting attack. text ofofthe DOM top-most nodes window from theto content click on window. the "Allow" or "Open" button of the window below. 0 Vulnerabilities Security Advisory 2006-76 2005-15 2005-16 2005-41 2005-14 Title: Heap Spoofing Privilege using overflow escalation download outer window's via andDOM security in UTF8 Function property dialogs to object Title: XSS SSL "secure site"possible indicator spoofing Unicode with overrides Impact: overlapping conversion High windows Severity: Moderate Severity: Products: High Firefox Products:Critical Firefox, 2.0 Mozilla Suite Products: Description: Firefox, moz_bug_r_a4 Thunderbird, Mozilla Suite demonstrated Mozilla Suitethat Description:Various schemes were reported Description: the prototype ItMichael moz_bug_r_a4 is possible regression Kraxfor demonstrates reported a UTF8 described string several that inwith thatFunction could cause the "secure site" lock icon to invalid the exploits bug download 355161 sequences giving could dialog an certificate attacker to betrigger and exploited security the adetails heap ability todialogs bypass overflow tothe install can the of be appear and show for wrong converted spoofed malicious protections bycode Unicode partially against orbesteal cross data. covering Exploitability sitephishers requiring them scriptwith (XSS) would an that site. These could useddata, by toonly make depend overlapping the injection, on do which the window. commonplace attackers could Some belegitimate, ability used actions users to tomay steal get likenot the click credentials notice string onina theiruser spoofs look more particularly into the link or sensitive OS or theopen window buggy data the converter. border context from and menu. General browser The sites web common or statusbar content perform windows that hide the arbitrary address bar showing the is converted bisecting cause destructive in each what elsewhere actions case appears was onbut behalf to privileged we be can't aofsingle a UI logged-in rule code dialog, out the user. and be true location. possibility of convinced ("chrome") by being athe successful spoofing overly trusting attack. text ofofthe DOM top-most nodes window from theto content click on window. the "Allow" or "Open" button of the window below. What other components are vulnerable? 0 Vulnerabilities Security Advisory 2006-76 2005-15 2005-16 2005-41 2005-14 Title: Heap Spoofing Privilege using overflow escalation download outer window's via andDOM security in UTF8 Function property dialogs to object Title: XSS SSL "secure site"possible indicator spoofing Unicode with overrides Impact: overlapping conversion High windows Severity: Moderate Severity: Products: High Firefox Products:Critical Firefox, 2.0 Mozilla Suite Products: Description: Firefox, moz_bug_r_a4 Thunderbird, Mozilla Suite demonstrated Mozilla Suitethat Description:Various schemes were reported Description: the prototype ItMichael moz_bug_r_a4 is possible regression Kraxfor demonstrates reported a UTF8 described string several that inwith thatFunction could cause the "secure site" lock icon to invalid the exploits bug download 355161 sequences giving could dialog an certificate attacker to betrigger and exploited security the adetails heap ability todialogs bypass overflow tothe install can the of be appear and show for wrong converted spoofed malicious protections bycode Unicode partially against orbesteal cross data. covering Exploitability sitephishers requiring them scriptwith (XSS) would an that site. These could useddata, by toonly make depend overlapping the injection, on do which the window. commonplace attackers could Some belegitimate, ability used actions users to tomay steal get likenot the click credentials notice string onina theiruser spoofs look more particularly into the link or sensitive OS or theopen window buggy data the converter. border context from and menu. General browser The sites web common or statusbar content perform windows that hide the arbitrary address bar showing the is converted bisecting cause destructive in each what elsewhere actions case appears was onbut behalf to privileged we be can't aofsingle a UI logged-in rule code dialog, out the user. and be true location. possibility of convinced ("chrome") by being athe successful spoofing overly trusting attack. text ofofthe DOM top-most nodes window from theto content click on window. the "Allow" or "Open" button of the window below. 0 Vulnerabilities Security Advisory 2006-76 2005-15 2005-16 2005-41 2005-14 Title: Heap Spoofing Privilege using overflow escalation download outer window's via andDOM security in UTF8 Function property dialogs to object Title: XSS SSL "secure site"possible indicator spoofing Unicode with overrides Impact: overlapping conversion High windows Severity: Moderate Severity: Products: High Firefox Products:Critical Firefox, 2.0 Mozilla Suite Products: Description: Firefox, moz_bug_r_a4 Thunderbird, Mozilla Suite demonstrated Mozilla Suitethat Description:Various schemes were reported Description: the prototype ItMichael moz_bug_r_a4 is possible regression Kraxfor demonstrates reported a UTF8 described string several that inwith thatFunction could cause the "secure site" lock icon to invalid the exploits bug download 355161 sequences giving could dialog an certificate attacker to betrigger and exploited security the adetails heap ability todialogs bypass overflow tothe install can the of be appear and show for wrong converted spoofed malicious protections bycode Unicode partially against orbesteal cross data. covering Exploitability sitephishers requiring them scriptwith (XSS) would an that site. These could useddata, by toonly make depend overlapping the injection, on do which the window. commonplace attackers could Some belegitimate, ability used actions users to tomay steal get likenot the click credentials notice string onina theiruser spoofs look more particularly into the link or sensitive OS or theopen window buggy data the converter. border context from and menu. General browser The sites web common or statusbar content perform windows that hide the arbitrary address bar showing the is converted bisecting cause destructive in each what elsewhere actions case appears was onbut behalf to privileged we be can't aofsingle a UI logged-in rule code dialog, out the user. and be true location. possibility of convinced ("chrome") by being athe successful spoofing overly trusting attack. text ofofthe DOM top-most nodes window from theto content click on window. the "Allow" or "Open" button of the window below. Is this new component likely to be vulnerable? 0 Vulnerabilities Vulnerability Database Version Archive Code Code Code Code Vulnerability Database Version Archive Vulture Code Code Code Code Vulnerability Database Version Archive Code Code Code Code Vulture Component Component Component Vulnerability Database Version Archive Code Code Code Code Vulture Component Component Predictor Component Vulnerability Database Version Archive Code Code Code Code Vulture Component Component Code Predictor Component Vulnerability Database Version Archive Code Code Code Code Vulture Component Component Code Predictor Component Code Programmer Code Programmer Code Complexity Code Programmer Code Complexity Code Language Programmer Code Complexity Language Programmer Code Complexity Language Programmer Code Complexity Language Code Complexity Language Code Complexity Language Language Look for features that are invariant under evolution Language GUI Database Certificates OS Imports GUI Database Certificates OS Imports GUI Database Certificates OS Imports GUI Database Certificates OS nsIContent.h nsIContentUtils.h nsIScriptSecurityManager.h nsIContent.h nsIContentUtils.h nsIScriptSecurityManager.h ✘ ✘ nsIContent.h ✘ ✘ nsIScriptSecurityManager.h ✘ ✘ ✘ ✘ ✘ nsIContentUtils.h ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ nsIContent.h ✘ ✘ nsIScriptSecurityManager.h ✘ ✘ ✘ ✘ ✘ nsIContentUtils.h ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✔ ✘ ✘ nsIPrivateDOMEvent.h nsReadableUtils.h nsIPrivateDOMEvent.h nsReadableUtils.h ✘ ✘ ✘ ✘ nsIPrivateDOMEvent.h ✘ nsReadableUtils.h ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ Research Questions • How well do imports predict vulnerabilities? • Can imports be used for classification (vulnerable or not) and for regression (number of vulnerabilities)? Case Study: Mozilla • CVS from January 4, 2007 • 14,368 C/C++ files • 134 Security Advisories since January 2005 • Only 424 vulnerable components (4.05%) Prediction is challenging libpkix pkix_pl_nss modu pki sy security nss lib freebl mpi ecl pkix incl top uti r ssl mailnews base src util search src util pk11wr ns certd smim pki pkcs12 pki1 jar cry compose src import outl src eud local src mime src bas asn cmd lib m pk si fips pk ce c crlu blt S oex news exten src palm db b modules jss org layout style s xul base src tr oji tests src JNI C Arr A C Ac libimg png plugin tools sam test s s def sr pu html content src svg content src doc src xpcom io glue bs mapi old ma t manager generic addrbook src base src S libfont libpr0n zlib jmcge dec s src reflect string typelib xptcal x pu sr xpi x src s base tests ds obsolete c build compo MoreFi thr pr s widget src mac mathml forms base src base svg base src xpconnect liveco src test rdf src re prin in ht bu js tamarin core libre libp libb softupd src src src src xml s li libjar pro ps gtk gfx src xlib mac windo l fdlib pcre code MM shell pl e jsd os2 gtk2 os2 beos xpwi qt ph gtk xlib co g g theb xlib qt phot be xp sh f x11sh xpr cairo cairo src windows glitz src libpixma src thebe src publi publ embedding browser activex gtk phot src src web co co powerp plu pl qt compon qa printin teste win fi web events src extensions webservice python spellch soap pro xpco src w i walle univ sche src src src java xpcom met pre ins typ sr aut w s p xmlterm sql coo s base line b xm l sche xml d tests mfc w xforms xtf xbl src can directory c-sdk ldap libraries clie i libldap exa suncsdk c-sdk ldap libraries cli i xp b intl uconv ucvlat src uti ucv ucvcn src tables canvas3d src cont pkcs de certhig zlib imap src softoken crmf ckfw builtins ca Mozilla Vulnerabilities content base xslt xul src p src temp doc xslt xpath src src mork src editor libeditor html base u u t text u unichar locale ctl src src src chardet l s src netwerk base protocol src http ftp src streamco test co cache dns bui s java webclient pluggab src_moz wf xpcom te ja u db sqlite3 src do plu jni calendar libical src libical libic au test base js2 src re browser components places migrat src boo s nsprpub pr tests o b misc pthre thr cp io Runtim gc Syste sr i md C N C Tools include md p li li ef Compiler Utilitie Code Front Gener md Primi zli qa x c tri sr li t src md wi uni ma Pack xpinstall wizard windows libxpne setup uni GUSI mac os2 setup co D unix src Ex D toolkit xpfe components airbag compone bootstra place his s airbag sear boo app com src do pa xre m hi txtsv appshel st txm parser tools htmlparser expa trace- codes re src p lib li re d f p accessible src atk bas ht xu ms le jp dom src base msgsdk C protocol plugin oji other-license MRJ MRJ 7zst libart_ plu pl src rdf 7zi atk-1. base chro src jpeg d cck expat muc driver ib gc boehm c uriloader camino extha b src lib mac mston view src src ipc ipcd e mail com profile buil dbm sun web win s i stu w sr docshell sto gcon mini config caps base s src sr chro 1 3 5 7 9 11 Number of MFSAs 13 20 50 5 1 2 Number of Components 20 50 5 1 2 Number of Components 300 Distribution of Bug Reports 300 Distribution of MFSAs 1 3 5 7 9 13 17 Number of Bug Reports 24 Experiments random splits • 40 6,968 rows in training set, 3,484 rows in validation set • Classification Train SVM, compute recall and precision • Regression Train SVM, compute rank correlation on top 1% linear kernel with default parameters • SVM: R implementation (up to 10GB of main memory) (b) Rank Correlation 2/3 of all vulnerable components detected 1.0 0.55 (a) Precision and Recall 0.40 ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.8 0.6 ● ●●● 0.4 0.45 ● 0.0 0.2 ● ● ● ● 0.35 Precision 0.50 ● Cumulative Distribution ● 0.55 0.60 0.65 Recall 0.70 0.75 ● 0.2 ● ● 0.3 ● ● ● ● ● ● ● ● ● 0.4 ● ● ● ● ● ● ● ● 0.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.6 Rank Correlation ● ● ● ● ● ● 0.7 (b) Rank Correlation 2/3 of all vulnerable components detected 1.0 0.55 (a) Precision and Recall 0.40 ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.8 0.6 ● ●●● 0.4 0.45 ● 0.0 0.2 ● ● ● ● 0.35 Precision 0.50 ● Cumulative Distribution ● 0.55 0.60 0.65 Recall 0.70 0.75 ● 0.2 ● ● 0.3 ● ● ● ● ● ● ● ● ● 0.4 ● ● ● ● ● ● ● ● 0.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.6 Rank Correlation ● ● ● ● ● ● 0.7 (b) Rank Correlation 2/3 of all vulnerable components detected 1.0 0.55 (a) Precision and Recall 0.40 ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.8 0.6 ● ●●● 0.4 0.45 ● 0.0 0.2 ● ● ● ● 0.35 Precision 0.50 ● Cumulative Distribution ● 0.55 0.60 0.65 0.70 0.75 ● 0.2 ● ● 0.3 Recall 45% (about 1/2) of predictions correct ● ● ● ● ● ● ● ● ● 0.4 ● ● ● ● ● ● ● ● 0.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.6 Rank Correlation ● ● ● ● ● ● 0.7 (b) Rank Correlation 2/3 of all vulnerable components detected 1.0 0.55 (a) Precision and Recall 0.40 ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.8 0.6 ● ●●● 0.4 0.45 ● 0.0 0.2 ● ● ● ● 0.35 Precision 0.50 ● Cumulative Distribution ● 0.55 0.60 0.65 Recall 0.70 0.75 ● 0.2 ● ● 0.3 ● ● ● ● ● ● ● ● ● 0.4 ● ● ● ● ● ● ● ● 0.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.6 ● ● ● ● ● ● 0.7 Rank Correlation 2/3 of all vulnerable components detected 45% (about 1/2) of predictions correct (b) Rank Correlation 2/3 of all vulnerable components detected 1.0 0.55 (a) Precision and Recall 0.40 ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.8 0.6 ● ●●● 0.4 0.45 ● 0.0 0.2 ● ● ● ● 0.35 Precision 0.50 ● Cumulative Distribution ● 0.55 0.60 0.65 Recall 0.70 0.75 ● 0.2 ● ● 0.3 ● ● ● ● ● ● ● ● ● 0.4 ● ● ● ● ● ● ● ● 0.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.6 ● ● ● ● ● ● 0.7 Rank Correlation 2/3 of all vulnerable components detected 45% (about 1/2) of predictions correct moderately strong correlation (mostly significant at p < 0.01) (b) Rank Correlation 2/3 of all vulnerable components detected 1.0 0.55 (a) Precision and Recall 0.40 ● ●● ● ● ● ● ● ● ●●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.8 0.6 ● ●●● 0.4 0.45 ● 0.0 0.2 ● ● ● ● 0.35 Precision 0.50 ● Cumulative Distribution ● 0.55 0.60 0.65 Recall 0.70 0.75 ● 0.2 ● ● 0.3 ● ● ● ● ● ● ● ● ● 0.4 ● ● ● ● ● ● ● ● 0.5 ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0.6 ● ● ● ● ● ● 0.7 Rank Correlation 2/3 of all vulnerable components detected 45% (about 1/2) of predictions correct Predicted Rank Component Actual Rank 1 nsDOMClassInfo 3 2 SGridRowLayout 95 3 xpcprivate 6 4 jsxml 2 5 nsGenericHTMLElement 8 6 jsgc 3 7 nsISEnvironment 12 8 jsfun 1 9 nsHTMLLabelElement 18 10 nsHttpTransaction 35 Predicted Rank Component Actual Rank 1 nsDOMClassInfo 3 2 SGridRowLayout 95 3 xpcprivate 6 4 jsxml 2 5 nsGenericHTMLElement 8 6 jsgc 3 7 nsISEnvironment 12 8 jsfun 1 9 nsHTMLLabelElement 18 10 nsHttpTransaction 35 Predicted Rank Component Actual Rank 1 nsDOMClassInfo 3 2 SGridRowLayout 95 3 xpcprivate 6 4 jsxml 2 5 nsGenericHTMLElement 8 6 jsgc 3 7 nsISEnvironment 12 8 jsfun 1 9 nsHTMLLabelElement 18 10 nsHttpTransaction 35 foo.h #ifndef _FOO_H_ # define _FOO_H_ extern int foo(); extern void bar(); extern struct z* baz(); #endif /* _FOO_H_ */ foo.h #ifndef _FOO_H_ # define _FOO_H_ quux.c #include “foo.h” /* ... */ extern int foo(); extern void bar(); extern struct z* baz(); int f() { int a = foo(); struct z* z = baz(); #endif /* _FOO_H_ */ bar(); return z != 0; } foo.h #ifndef _FOO_H_ # define _FOO_H_ quux.c #include “foo.h” /* ... */ extern int foo(); extern void bar(); extern struct z* baz(); int f() { int a = foo(); struct z* z = baz(); #endif /* _FOO_H_ */ bar(); return z != 0; } foo.h #ifndef _FOO_H_ # define _FOO_H_ quux.c #include “foo.h” /* ... */ extern int foo(); extern void bar(); extern struct z* baz(); int f() { int a = foo(); struct z* z = baz(); #endif /* _FOO_H_ */ bar(); return z != 0; } Results for Functions Hot Off The Press! • Precision: Median 45% • Recall: Median 70% • Highest values: Precision 60%, Recall 80% • Rank correlation on top 10: Median 75%