Scarica la presentazione - CAST Software: On Quality Blog

Scarica la presentazione - CAST Software: On Quality Blog

New Standards for
Automating Source Code
Analysis of Structural Quality
Dr. Bill Curtis
Executive Director, CISQ
1
CISQ Founders and Sponsors
Co-founders
IT
Executives
OMG
Special
Interest
Group
CISQ
Develop standard,
automatable measures
for evaluating software
size and quality with an
ecosystem supporting
policy and deployment
2
Technical
experts
CISQ Sponsors
CISQ/OMG Standards Process
CISQ Work Groups
Automated
Function Points
Defined
Measures
Reliability
CISQ
Exec
Forum
Performance
Efficiency
OMG
ISO
Fasttrack
Security
Deployment
Workshops
Maintainability
3
Automated FP Specification
• OMG Supported Specification for
Automated Function Points
• Mirrors IFPUG counting guidelines,
but automatable
• Specification developed by
international team led by David
Herron of David Consulting Group
4
CISQ Measures and ISO 25010
•
•
•
•
ISO 25010 defines quality characteristics and sub-characteristics
ISO 25023 defines measure elements for each sub-characteristic
ISO 25023 does not define measures at the source code level
CISQ supplements ISO 25023 by defining code level measures
ISO/IEC 25010 Quality
Characteristic Hierarchy
CISQ defining automatable measures for characteristics highlighted in orange
5
CISQ Quality Measure Structure
Structure of ISO 25023 Measures
Structure of CISQ Security Measure
Software Quality Characteristics
Security
Quality Sub-Characteristics
Confidentiality, Authenticity,
Integrity, Accountability, etc.
Software Quality Attributes
Quality Measure Elements
Quality Rule Violations
•
•
•
•
•
•
ISO structure
Examples from CISQ measures
6
Cross-site scripting
SQL injection
Buffer overflow
OS command injection
Unvalidated array
Etc.
CWEs in CISQ Security Measure
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
CWE-22
CWE-78
CWE-79
CWE-89
CWE-120
CWE-129
CWE-134
CWE-252
CWE-327
CWE-396
CWE-397
CWE-434
CWE-456
CWE-606
CWE-667
CWE-672
CWE-681
CWE-706
CWE-772
CWE-789
CWE-798
CWE-835
Path Traversal Improper Input Neutralization
OS Command Injection Improper Input Neutralization
Cross-site Scripting Improper Input Neutralization
SQL Injection Improper Input Neutralization
Common
Buffer Copy without Checking Size of Input
Weakness
Array Index Improper Input Neutralization
Enumeration
Format String Improper Input Neutralization
Unchecked Return Parameter of Control Element Accessing Resource
Broken or Risky Cryptographic Algorithm Usage
Declaration of Catch for Generic Exception
Declaration of Throws for Generic Exception
File Upload Improper Input Neutralization
Storable and Member Data Element Missing Initialization
Unchecked Input for Loop Condition
Shared Resource Improper Locking
Expired or Released Resource Usage
Numeric Types Incorrect Conversion
Name or Reference Resolution Improper Input Neutralization Robert
Martin
Missing Release of Resource after Effective Lifetime
MITRE
Uncontrolled Memory Allocation
Hard-Coded Credentials Usage for Remote Authentication
Loop with Unreachable Exit Condition ('Infinite Loop')
7
Focus Shift in Diagnostic Metrics
Traditional
metrics
measure program elements such as
tokens, objects, or control structures
These elements correlate with
the potential for defects
These elements are defects
Violation
metrics
measure violations of good
architectural and coding practice
Violations can be analyzed as patterns
8
IT Quality Challenge  System Level
Architectural Compliance
1
Unit Level
Code style & layout
Expression complexity
Code documentation
Class or program design
Basic coding standards
Developer level
APIs
JSP
ASP.NET
Java
Java
Java
Web
Services
Java
Java
Java
2 Technology Level
Single language/technology layer
Intra-technology architecture
Intra-layer dependencies
Inter-program invocation
Security vulnerabilities
Development team level
Java
Hibernate
Messaging
Struts
.NET
Sprin
g
COBOL
EJB
Oracle
PL/SQ
L
T/SQL
3
SQL
Server
Integration quality
Architectural
compliance
Risk propagation
Application security
Resiliency checks
Transaction integrity
DB2
Sybase
System Level
IMS
Technology Stack
9
Function point,
Effort estimation
Data access control
SDK versioning
Calibration across
technologies
IT organization level
Architecturally Complex Defects
Architecturally
Complex Defect
A structural flaw involving interactions
among multiple components that
reside in different application layers
% of total
repair effort
% of total
app defects
Componentlevel violations
Architecturally
Complex Defects
48%
92
%
52%
20x as
many
fixes to
correct
8%
Architectural hotspots provide a
roadmap for remediating the worst
risk, rework, and cost drivers
80% of architecturally complex defects
touch an Architectural Hotspot—a badly
designed component causing problems
10
Data Layer
Logic Layer
UI Layer
Detecting Architectural Hotspots
Contributor to architecturally complex defect
Architectural hotspot
11
Structural Quality Measure Uses
Acquisition
Managers
IT Executives
FIN
HR
CRM
ERP
App / Project
Managers
Portfolio
insight
Deliverables
insight
Developers
Application
insight
Remedial
insight
12
Using CISQ Measures with Vendors
1. Set compliance targets
2. FEnforce a measurement process
3. Evaluate contract deliverables
4. Use rewards and penalties wisely
13
1  Set Quality Compliance Targets
Quality Characteristic Score
Quality Score Target by Release
3,9
3,7
3,5
3,3
3,1
Reliability
Performance
Security
Maintainability
2,9
2,7
2,5
14
2  Enforce a Measurement Process
Establish a
Baseline
Set thresholds
and targets
Monitor and
review results
Determine
remediation
Update SLAs
15
3  Evaluate Contract Deliverables
Use CISQ measures in contracts to
establish objective, measureable
agreements on quality priorities that
comply with industry standards
16
4  Use Rewards and Penalties Wisely
Partnership
Common goals
Negotiation
Rewards
Vendor
Profit goals
Contract terms
Penalties
Low maturity
Maturity
shift
17
New CISQ Measurement Specifications
Must account
for size of
maintenance
activities
Automated
Enhancement
Function
Points
Productivity
Estimation
Must add
future effort to
fix bugs into
productivity
Quality
Adjusted
Productivity
Effort
& Cost
Benchmarks
Value & ROI
Must estimate
the corrective
costs in cost
of ownership
Structural
Technical
Debt
Etc.
18
Join CISQ Free  www.it-cisq.org
19