View
223
Download
2
Category
Preview:
Citation preview
Microsoft Australia Security Summit
Tools for Quality CodeTools for Quality Code
Nigel Watson, Microsoft AustraliaSean Salisbury, Compuware CorpNigel Watson, Microsoft Australia
Sean Salisbury, Compuware Corp
Microsoft Australia Security Summit
AgendaAgenda
Testing – so what?
Testing in Visual Studio Team System
Extending VSTS – Compuware DevPartner
Summary
Testing – so what?
Testing in Visual Studio Team System
Extending VSTS – Compuware DevPartner
Summary
Microsoft Australia Security Summit
Projects and TestingProjects and Testing
Often an expensive afterthought
Strategies for minimising impact
Often an expensive afterthought
Strategies for minimising impact
RequirementsCoding
IntegrationBeta Test
Post-Release
5
10
15
20
25
30
Relative CostTo Fix Bugs...
Microsoft Australia Security Summit
Problems...Problems...
It is expensive to find and fix bugs that get past daily development practices
Potential security flaws need to be caught early
It is hard to diagnose errors at runtime
Why does an application run slowly?
Individual Developers and Testers need to know if they are on track
Test and development are often out of synch
Final test phase for shipping is often ad-hoc
How much testing is enough?
It is expensive to find and fix bugs that get past daily development practices
Potential security flaws need to be caught early
It is hard to diagnose errors at runtime
Why does an application run slowly?
Individual Developers and Testers need to know if they are on track
Test and development are often out of synch
Final test phase for shipping is often ad-hoc
How much testing is enough?
Microsoft Australia Security Summit
Defense In DepthDefense In Depth
Microsoft uses a 'defense in depth' strategyUnit testing
Code reviews
Frequent builds
Catch bugs earlyStatic checks
Runtime checks
Microsoft uses a 'defense in depth' strategyUnit testing
Code reviews
Frequent builds
Catch bugs earlyStatic checks
Runtime checks
Microsoft Australia Security Summit
Testing in VSTSTesting in VSTS
Change Management
Work Item Tracking
Reporting
Project Site
Visual Studio
Team Foundation Project Management
Visual Studio
Team Architect
Visio and UML Modeling
VS Pro
Class Modeling
Application Modeling
Logical Infra. Modeling
Deployment Modeling
Visual Studio
Team DeveloperVisual Studio
Team Test
Project SiteWork Item Tracking
Reporting
Project Management
Integration Services
Load Testing
Manual Testing
Test Case Management
Unit Testing
Code Coverage
Dynamic Code Analyzer
Static Code Analyzer
Code Profiler
Team Foundation Client
Microsoft Australia Security Summit
Testing in VSTSTesting in VSTS
Change Management
Work Item Tracking
Reporting
Project Site
Visual Studio
Team Foundation Project Management
Visual Studio
Team Architect
Visio and UML Modeling
VS Pro
Class Modeling
Application Modeling
Logical Infra. Modeling
Deployment Modeling
Visual Studio
Team DeveloperVisual Studio
Team Test
Project SiteWork Item Tracking
Reporting
Project Management
Integration Services
Load Testing
Manual Testing
Test Case Management
Unit Testing
Code Coverage
Dynamic Code Analyzer
Static Code Analyzer
Code Profiler
Team Foundation Client
Microsoft Australia Security Summit
Test-Driven DevelopmentTest-Driven Development
Integrate testing into the development process
Tests define what code will doTests come from specifications
Write code to pass tests
Don't write code that doesn't contribute to passing a test...
Integrate testing into the development process
Tests define what code will doTests come from specifications
Write code to pass tests
Don't write code that doesn't contribute to passing a test...
CodeCode
Microsoft Australia Security Summit
VSTS Unit TestingVSTS Unit Testing
Integrated into VS
Automatic generation of test classes
Comprehensive test management
Code coverage testing
Integrated into VS
Automatic generation of test classes
Comprehensive test management
Code coverage testing[TestMethod()][TestMethod()]public void public void GetValueTestGetValueTest()() {{ double d = myObject.double d = myObject.getValuegetValue();(); if (d < 10.0)if (d < 10.0) Assert.Fail("Bad return value");Assert.Fail("Bad return value"); }}
Microsoft Australia Security Summit
Unit TestingUnit Testing
Microsoft Australia Security Summit
Code ReviewsCode Reviews
For the Visual Studio 7.0 product cycle86% of bugs occurred in reviewed code
60% of all bugs were coding errors
Static analysis helps catch bugsSource code analysis
PREfast for C and C++
FxCop for .NET
For the Visual Studio 7.0 product cycle86% of bugs occurred in reviewed code
60% of all bugs were coding errors
Static analysis helps catch bugsSource code analysis
PREfast for C and C++
FxCop for .NET
Microsoft Australia Security Summit
PREFastPREFast
Static analysis for C/C++ codeManaged and unmanaged C++
Catches common bugsBuffer overruns, uninitialized memory
Memory leaks, null pointer dereference
Reported as compiler warningsDisplay path to problem
Use #pragma to turn off
Static analysis for C/C++ codeManaged and unmanaged C++
Catches common bugsBuffer overruns, uninitialized memory
Memory leaks, null pointer dereference
Reported as compiler warningsDisplay path to problem
Use #pragma to turn off
Microsoft Australia Security Summit
FxCopFxCop
Static analysis for .NET assembliesNot just C++
Uses design guidelines(including many in the .NET Class Design Guidelines)
CustomizableWhich checks to include
Whether to report as error or warning
Create custom rules
Static analysis for .NET assembliesNot just C++
Uses design guidelines(including many in the .NET Class Design Guidelines)
CustomizableWhich checks to include
Whether to report as error or warning
Create custom rules
Microsoft Australia Security Summit
Static code analysisStatic code analysis
Microsoft Australia Security Summit
Integrating Dev and TestIntegrating Dev and Test
Tests are just another form of source code:Stored in source code control
Versioned with the product
“Test Complete”Test writing is scheduled along with development work
Tracked by work items
Testers are notified when bugs are fixed
Tests are just another form of source code:Stored in source code control
Versioned with the product
“Test Complete”Test writing is scheduled along with development work
Tracked by work items
Testers are notified when bugs are fixed
Microsoft Australia Security Summit
VSTS Test TypesVSTS Test Types
Unit TestsTest class methods
Web TestsRecord and playback interactions
Load TestsSimulate multiple users
Manual TestsProvide scripts for manual tasks
Third-party TestsIntegrated into VSTS
Unit TestsTest class methods
Web TestsRecord and playback interactions
Load TestsSimulate multiple users
Manual TestsProvide scripts for manual tasks
Third-party TestsIntegrated into VSTS
Microsoft Australia Security Summit
Application QualityApplication Quality
Best Practices and ToolsBest Practices and Tools
Sean SalisburySenior Regional Tech SpecialistCompuware Corporationsean.salisbury@compuware.com
Microsoft Australia Security Summit
Integrated development and test automation tools
Rich process management Detailed and relevant
project information
Microsoft and CompuwareMicrosoft and Compuware
Production Readiness
Automated Software Quality
Development & Integration
Performance & Availability
Management
QACenterExtends quality assurance testing
DevPartnerExtends quality in development
VS & Team Systemintegration platform, base tools
Microsoft Australia Security Summit
Compuware DevPartner Studioenhance and extend Visual Studio
Compuware DevPartner Studioenhance and extend Visual StudioNative and Managed Code Analysis
Local and Remote Data Collection:Performance Analysis
.NET Memory Analysis
Code Coverage Analysis
Distributed Application Analysis
VB, VB.NET, ASP.Net and C# Source Code Review with >600 Rules
C/C++ Memory Error & Thread Deadlock Detection
Native and Managed Code Analysis
Local and Remote Data Collection:Performance Analysis
.NET Memory Analysis
Code Coverage Analysis
Distributed Application Analysis
VB, VB.NET, ASP.Net and C# Source Code Review with >600 Rules
C/C++ Memory Error & Thread Deadlock Detection
Microsoft Australia Security Summit
Code AnalysisCode Analysis
600+ Rules enhance problem resolution
Supports VS6/2002/2003/2005
Accelerates learning curves
Improves code quality and maintainability
Supports Visual Basic, VB.NET, C#, ASP.Net
600+ Rules enhance problem resolution
Supports VS6/2002/2003/2005
Accelerates learning curves
Improves code quality and maintainability
Supports Visual Basic, VB.NET, C#, ASP.Net
Microsoft Australia Security Summit
Microsoft Australia Security Summit
Memory AnalysisMemory Analysis
Optimize Local or Remote Memory Use
View allocations/deallocations over time: get an overall feel for memory use
Identify Objects That:
Consume a lot of memory
Create a lot of temporary objects
Stay around longer than they need to, including leaks
Compare Runs- Did Code Changes Help?
Tune Garbage Collection
Optimize Local or Remote Memory Use
View allocations/deallocations over time: get an overall feel for memory use
Identify Objects That:
Consume a lot of memory
Create a lot of temporary objects
Stay around longer than they need to, including leaks
Compare Runs- Did Code Changes Help?
Tune Garbage Collection
Microsoft Australia Security Summit
Memory Analysis at Run TimeMemory Analysis at Run Time
Real-Time
trace of memory usage
System Allocations
Your Code
RAM usage
Time
Microsoft Australia Security Summit
Memory AnalysisMemory Analysis
Many Different Data Views with Details Available
Many Different Data Views with Details Available
Microsoft Australia Security Summit
Automatic Error DetectionAutomatic Error Detection
Memory/Resource/ Interface Leaks
API Errors
Threading Issues
Event Debugging
C/C++/VC++
Memory/Resource/ Interface Leaks
API Errors
Threading Issues
Event Debugging
C/C++/VC++
Microsoft Australia Security Summit
Microsoft Australia Security Summit
Thread Deadlock Detection Thread Deadlock Detection
Locate Actual or Potential Thread Deadlocks or Other Synchronization Issues
Deadlock: 2 or more code paths running at the same time, contending for the same resource(s)
BenefitsThread deadlock are difficult to detect: automating detection is very useful
Locate Actual or Potential Thread Deadlocks or Other Synchronization Issues
Deadlock: 2 or more code paths running at the same time, contending for the same resource(s)
BenefitsThread deadlock are difficult to detect: automating detection is very useful
Microsoft Australia Security Summit
Performance ProfilingPerformance Profiling
Pinpoint bottlenecks across app Tiers/Versions
Optimize application performance
Increase usability
Pinpoint bottlenecks across app Tiers/Versions
Optimize application performance
Increase usability
Microsoft Australia Security Summit
Microsoft Australia Security Summit
Compare Performance Runs Compare Performance Runs
Microsoft Australia Security Summit
Code CoverageCode Coverage
Quickly identify untested code across tiers & VS6/02/03/05
Ensure test coverage during unit testing
More reliable components and applications
Quickly identify untested code across tiers & VS6/02/03/05
Ensure test coverage during unit testing
More reliable components and applications
Microsoft Australia Security Summit
Microsoft Australia Security Summit
Distributed AnalysisDistributed Analysis
Microsoft Australia Security Summit
What’s New….What’s New….
Microsoft Australia Security Summit
IT ChallengesIT Challenges
Identifying what errors can occur & when
Tools lacking for error simulation and analysis
Errors corrupt the debugging environment
Impossible to trace error handling execution
Difficult to create repeatable tests
Time-consuming, manual process
Identifying what errors can occur & when
Tools lacking for error simulation and analysis
Errors corrupt the debugging environment
Impossible to trace error handling execution
Difficult to create repeatable tests
Time-consuming, manual process
Microsoft Australia Security Summit
What If You Could…What If You Could…
Quickly determine what errors could occur at any point in your application?
Ensure you have error handlers in place to cope
Simulate errors safely and efficiently?With no impact on the OS, .NET framework or any other running application
Observe and debug your error handlers
Build reusable fault test libraries?Create repeatable tests that are reusable by development & QA
Quickly determine what errors could occur at any point in your application?
Ensure you have error handlers in place to cope
Simulate errors safely and efficiently?With no impact on the OS, .NET framework or any other running application
Observe and debug your error handlers
Build reusable fault test libraries?Create repeatable tests that are reusable by development & QA
Microsoft Australia Security Summit
DevPartner Fault SimulatorDevPartner Fault Simulator
Developer Insight
What errors can occur at what point in the code
Integrated with Visual Studio debugging features to monitor error handling execution
Break at fault occurrence
Developer Insight
What errors can occur at what point in the code
Integrated with Visual Studio debugging features to monitor error handling execution
Break at fault occurrence
Microsoft Australia Security Summit
DevPartner Fault SimulatorDevPartner Fault Simulator
Error handling validation
Simulate Environmental and .NET Framework faults
Simple method of selection of errors to validate, with user defined conditions
Reusable Fault Sets for repeat and QA testing
VS 2003/05 IDE integrated, standalone and command line operation
Error handling validation
Simulate Environmental and .NET Framework faults
Simple method of selection of errors to validate, with user defined conditions
Reusable Fault Sets for repeat and QA testing
VS 2003/05 IDE integrated, standalone and command line operation
Microsoft Australia Security Summit
DevPartner Fault SimulatorDevPartner Fault Simulator
Results analysis
Simulate Stack tracing & error details
“Go to source” linking for detailed analysis
Live view and summary of fault execution
Saved Results files for later review
Results analysis
Simulate Stack tracing & error details
“Go to source” linking for detailed analysis
Live view and summary of fault execution
Saved Results files for later review
Microsoft Australia Security Summit
DevPartner Fault SimulatorDevPartner Fault Simulator
DemonstrationDemonstration
Microsoft Australia Security Summit
Securing ASP.Net ApplicationsSecuring ASP.Net Applications
Microsoft Australia Security Summit
Security VulnerabilitySecurity Vulnerability
“Today over 70% of attacks against a company’s network come at the Application Layer, not the Network or System Layer”
John Pescatore, Gartner chief security analyst
The responsibility for application security is shifting to the development organization
How do they address this aspect of application quality?
How do they gain the skills they need to assess and correct security vulnerabilities?
“Today over 70% of attacks against a company’s network come at the Application Layer, not the Network or System Layer”
John Pescatore, Gartner chief security analyst
The responsibility for application security is shifting to the development organization
How do they address this aspect of application quality?
How do they gain the skills they need to assess and correct security vulnerabilities?
Microsoft Australia Security Summit
What If You Could…What If You Could…
Quickly locate security vulnerabilities in your application during development?
Minimize the cost and mean-time-to-repair
Improve the quality/reliability of your application
Have a wealth of security expertise and advice at your fingertips?
Have the information you need, when you need it
Quickly locate security vulnerabilities in your application during development?
Minimize the cost and mean-time-to-repair
Improve the quality/reliability of your application
Have a wealth of security expertise and advice at your fingertips?
Have the information you need, when you need it
Microsoft Australia Security Summit
DevPartner SecurityCheckerDevPartner SecurityChecker
A vulnerability assessment scanner that locates security vulnerabilities in ASP.NET (C# or VB.NET)
Locates complex & hard-to-find security problems
Organizes results by priority and category Pinpoints vulnerabilities to the line of source code
Explains why it is an issue
Suggests steps to repair each vulnerability
Provides links to additional technical information
A vulnerability assessment scanner that locates security vulnerabilities in ASP.NET (C# or VB.NET)
Locates complex & hard-to-find security problems
Organizes results by priority and category Pinpoints vulnerabilities to the line of source code
Explains why it is an issue
Suggests steps to repair each vulnerability
Provides links to additional technical information
Microsoft Australia Security Summit
DevPartner SecurityCheckerDevPartner SecurityChecker
Integrity Analysis(attach simulation)
Replays a series of known security attacks against the application
Secures the interface to the application
Compile-time Analysis
Scans source code for known security problems
Test while coding
Run-time Analysis Monitors execution of the application
Observes interior/hidden facets, beyond the external interface
Expert Advisor
Go to line of source code
Detailed assistance
Allows the developer to quickly: Find & fix the vulnerability
Become more knowledgeable about security
Accelerates secure application development
Microsoft Australia Security Summit
DevPartner SecurityCheckerDevPartner SecurityChecker
DemonstrationDemonstration
Microsoft Australia Security Summit
Quality Continues in TestingQuality Continues in Testing
Automate functional testing and validationManage test plans and execution
Comparison of complex data results
Seamlessly capture defect information
Simulate application under loadSimulate load conditions ‘000,000’s of users
Determine application scalability
Compuware QACenter Enterprise Wide
Compuware Vantage - Network and Server monitoring
Automate functional testing and validationManage test plans and execution
Comparison of complex data results
Seamlessly capture defect information
Simulate application under loadSimulate load conditions ‘000,000’s of users
Determine application scalability
Compuware QACenter Enterprise Wide
Compuware Vantage - Network and Server monitoring
Microsoft Australia Security Summit
Microsoft & CompuwareMicrosoft & Compuware
Tools to:
Improve application reliability & performance
Increase team productivity
Lower costs
Deliver better applications to the market faster
Tools to:
Improve application reliability & performance
Increase team productivity
Lower costs
Deliver better applications to the market faster
Production Readiness
Automated Software Quality
Development & Integration
Performance & Availability
Management
Microsoft Australia Security Summit
SummarySummary
Appreciated the importance of testing to the development process
Had a quick look at some of the testing tools in Visual Studio Team System
Sean showed us how Compuware DevPartner Studio uses the integration capabilities of Visual Studio to extend the power of the IDE
Appreciated the importance of testing to the development process
Had a quick look at some of the testing tools in Visual Studio Team System
Sean showed us how Compuware DevPartner Studio uses the integration capabilities of Visual Studio to extend the power of the IDE
Microsoft Australia Security Summit
© 2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
Recommended