Building quality software on target Marcel de Vries [email protected]

Building quality software on target Marcel de [email protected]://blogs.infosupport.com/marcelv

Building quality software on target Marcel de [email protected]://blogs.infosupport.com/marcelv

AgendaAgenda

• Team System OverviewTeam System Overview• Team FoundationTeam Foundation• Project ManagementProject Management• Distributed systems designDistributed systems design• Team Foundation Source ControlTeam Foundation Source Control• Class designClass design• Unit testingUnit testing• Dynamic Code analysisDynamic Code analysis• Team buildTeam build• Web testingWeb testing• Load testingLoad testing• Team System ExtensibilityTeam System Extensibility

The Software LifecycleThe Software LifecycleWhat We’ve HeardWhat We’ve Heard

The Software LifecycleThe Software LifecycleWhat We’ve HeardWhat We’ve Heard

• ““Building software today is very difficult”Building software today is very difficult”

• ““My team is spread out and specialized”My team is spread out and specialized”

• ““Tools today don’t work well together”Tools today don’t work well together”

• ““I need to be able to predict the success of I need to be able to predict the success of my projects”my projects”

• ““My organization needs customizable My organization needs customizable process guidance”process guidance”

Source: THE STANDISH GROUP 2003

90%90%Delivered

LateDelivered

Late66%66%

Were not Considered Successful

Were not Considered Successful

Source: THE STANDISH GROUP 2003Source: THE STANDISH GROUP 2003

54%54%Delivered

over BudgetDelivered

over Budget

Cancelled prior to Completion

Cancelled prior to Completion

30%30%

Project success ratesProject success rates

Software delivery is still an art, not a science.

It has never been more difficult to manage delivery of software in a predictable and reliable manner!

Project failure statistics are scary…

50

60

70

80

90

100

%

0

10

20

30

40

Overall Need :

Increase the predictability of quality delivery, on time, and within budget

Development TeamsDevelopment Teams

InfrastructureInfrastructureArchitectArchitect

SolutionSolutionArchitectArchitect

Project ManagerProject Manager

DeveloperDeveloper

TesterTester

End UserEnd User

Communication BarriersCommunication Barriers

• Pockets of information within disciplinesPockets of information within disciplines• Unclear delineation of responsibilitiesUnclear delineation of responsibilities• Conflicting best practices and architecturesConflicting best practices and architectures• Conflicting strategic goals and objectivesConflicting strategic goals and objectives

SolutionSolutionArchitectArchitectInfrastructureInfrastructure

ArchitectArchitect

IT Operations doesn’tunderstand applicationstructure

Developers don’tunderstand operationspolicy

Reduce complexityReduce complexitythrough operationsthrough operationsknowledgeknowledge

Increase communication Increase communication and collaborationand collaboration

via product integrationvia product integration

Core Tenets to SuccessCore Tenets to Success

• Tools need to…Tools need to…– ……reduce the complexity of delivering modern reduce the complexity of delivering modern

service-oriented solutionsservice-oriented solutions– ……be tightly integrated and facilitate better be tightly integrated and facilitate better

team collaborationteam collaboration– ……enable customization and extensibility by enable customization and extensibility by

organizations and ISVsorganizations and ISVs

Challenges of Building IT SolutionsChallenges of Building IT Solutions

• Communication across diverse teams – Communication across diverse teams – (Testers, Architects, Developers, etc.)(Testers, Architects, Developers, etc.)

• Bundled tools, as opposed to integrated Bundled tools, as opposed to integrated toolstools– Defect Tracking Defect Tracking – DevelopmentDevelopment– Test ManagementTest Management– Task AssignmentTask Assignment

Challenges of Building IT SolutionsChallenges of Building IT Solutions

• Ill-fitting processIll-fitting process– Little automatic guidanceLittle automatic guidance– Heavy burdenHeavy burden

• Disappointing ROIDisappointing ROI– Lack of VisibilityLack of Visibility– Lack of PredictabilityLack of Predictability

Process QuestionsProcess Questions

• What is the process?What is the process?• What document should be filled out next?What document should be filled out next?• What are the requirements?What are the requirements?• What development tasks relate to what What development tasks relate to what

requirements?requirements?• What tests relate to what requirements?What tests relate to what requirements?• What code relates to what test, that relates to what What code relates to what test, that relates to what

requirement?requirement?• Are we on schedule?Are we on schedule?

ManagementManagementQuestionsQuestions

ListsLists(Bugs, Requirements, etc)(Bugs, Requirements, etc)

Source CodeSource CodeAssetsAssets

TeamTeamCommunicationCommunication

ProcessProcessGuidanceGuidance

ReleasesReleases

Development QuestionsDevelopment Questions

• Did we do design?Did we do design?• What all was considered in the design?What all was considered in the design?• Did we account for the production Did we account for the production

environment?environment?• Do we have coding standards?Do we have coding standards?• Are developers following standards?Are developers following standards?• Are developers testing their code?Are developers testing their code?

– Are you sure?Are you sure?

Developer Needs?Developer Needs?

• Standard version controlStandard version control

• Historical traces from code to work itemHistorical traces from code to work item– What was the requirement?What was the requirement?– What bug was being fixed?What bug was being fixed?

• Parallel developmentParallel development

• Tools for interruptible workflowTools for interruptible workflow

• Remote version controlRemote version control

Tester Needs?Tester Needs?

• Version control for test artifactsVersion control for test artifacts

• Notification of available bug fixesNotification of available bug fixes

• Release notes for individual buildsRelease notes for individual builds

Project Lead Needs?Project Lead Needs?

• Reports on code churnReports on code churn

• Status updates derived from check-insStatus updates derived from check-ins

• Rapid start-up for new team membersRapid start-up for new team members

• Change Management assistanceChange Management assistance– Simple change migration between branchesSimple change migration between branches– Identify fixes needing migrationIdentify fixes needing migration– Easily back-out a fixEasily back-out a fix

Improving the Software Lifecycle


• Better communication across rolesBetter communication across roles– Simplified reportingSimplified reporting– MS Project integration for Project managersMS Project integration for Project managers– Distributed Systems Designers for architect to Distributed Systems Designers for architect to

developer communicationdeveloper communication



• Reporting and automated collection of dataReporting and automated collection of data

• Integrated ToolsIntegrated Tools

• Integrated Workflow and ProcessIntegrated Workflow and Process

• Improved ROIImproved ROI

Expanding Visual StudioExpanding Visual Studio

IncreasedIncreasedReliabilityReliability

QualityQualityEarly & OftenEarly & Often

PredictabilityPredictability& Visibility& Visibility

Design forDesign forOperationsOperations

ProjectProjectManagerManager

SolutionSolutionArchitectArchitect

DeveloperDeveloperTesterTester

InfrastructureInfrastructureArchitectArchitect

Visual Studio 2005“Personalized productivity”

Visual Studio 2005“Personalized productivity”

Novices

Enthusiasts

Students

Hobbyists

Part-Timers

Professionals

Consultants

Enterprise Devs

Architects

Testers

VB6 Devs

Visual Studio Team SystemVisual Studio Team System

Change Management

Work Item Tracking

Reporting

Project Site

Visual Studio

Team Foundation Integration Services

Project Management

Pro

cess

an

d A

rch

itect

ure

Pro

cess

an

d A

rch

itect

ure

G

uid

an

ceG

uid

an

ce

Dynamic Code Analyzer

Visual Studio

Team Architect

Static Code Analyzer

Code Profiler

Unit Testing

Code Coverage

Visio and UML Modeling

Team Foundation Client (includes CAL)

Visual Studio Professional Edition

Class Designer

Load Testing

Manual Testing

Test Case Management

Application Designer

Logical Infra. Designer

Deployment Designer

Visual Studio

Team DeveloperVisual Studio

Team Test

Vis

ual S

tud

io In

du

stry

V

isu

al S

tud

io In

du

stry

Part

ners

Part

ners

Big Build

AgendaAgenda


Team FoundationProblem Space

Team FoundationProblem Space

• Fragmented informationFragmented information– Can’t relate and query work items, work, processes and artifactsCan’t relate and query work items, work, processes and artifacts– Tools sets are incomplete or poorly integratedTools sets are incomplete or poorly integrated– Difficulty integrating tools with existing infrastructureDifficulty integrating tools with existing infrastructure

• Communication is manualCommunication is manual– People use the phone or face-to-facePeople use the phone or face-to-face– Information is not capturedInformation is not captured– Distributed development aggravates the problemDistributed development aggravates the problem

• Life cycle tools are expensive and complexLife cycle tools are expensive and complex– Tools are difficult to learn and useTools are difficult to learn and use– Developers view tools as “overhead”Developers view tools as “overhead”– Administration costs are highAdministration costs are high

What is VS Team Foundation?What is VS Team Foundation?

Integrated Platform for Collaborating on Software Integrated Platform for Collaborating on Software Development Projects.Development Projects.

Version Version ControlControl

Work Item Work Item TrackingTracking

Build Build AutomationAutomation

Team Team CommunicationsCommunications

ReportingReporting

Guiding Principles of VSTSGuiding Principles of VSTS

ProductiveProductive

IntegratedIntegrated

ExtensibleExtensible

CapableCapable

Short learning curveShort learning curveMinimal administrative overheadMinimal administrative overhead

Tools integrated tightlyTools integrated tightlyAutomates common tasksAutomates common tasks

Customizable for your processCustomizable for your processIntegrates with 3Integrates with 3rdrd party tools party toolsRemotely accessibleRemotely accessibleRobust, secure, scalableRobust, secure, scalable

Team Foundation CapabilitiesTeam Foundation Capabilities

• Build automationBuild automation– Scripted “nightly” buildScripted “nightly” build– ReportingReporting– Integrates many of the Team Integrates many of the Team

System toolsSystem tools

• Metrics warehouseMetrics warehouse– Automatic data collectionAutomatic data collection– OLAP reportingOLAP reporting– Trending, aggregation & Trending, aggregation &

drilldowndrilldown

• Project portalProject portal– Single place to go for high-Single place to go for high-

level project informationlevel project information– Web access work itemsWeb access work items– WSS basedWSS based

• Work item trackingWork item tracking– Easy, Integrated into Easy, Integrated into

development processdevelopment process– Design your own process – Design your own process –

Fields, Forms, States, RulesFields, Forms, States, Rules– Extensive linking – bugs, Extensive linking – bugs,

reports, artifactsreports, artifacts– NotificationsNotifications

• Source Code ControlSource Code Control– Easy, integrated into Easy, integrated into

development environmentdevelopment environment– Integrated check in experienceIntegrated check in experience– Scalable and robustScalable and robust– Parallel developmentParallel development– NotificationsNotifications

Configuring rights in TFSConfiguring rights in TFS

• Team system uses ADAM to store team system Team system uses ADAM to store team system groupsgroups

• Makes it possible to administer team system Makes it possible to administer team system groups independent of the ADgroups independent of the AD– No need to let AD administrator create groups for the No need to let AD administrator create groups for the

development environmentdevelopment environment– Just make a mapping between AD and ADAM groupsJust make a mapping between AD and ADAM groups

• Configuration done from the VS.NET IDEConfiguration done from the VS.NET IDE• At this moment (B2) only one domain supported!At this moment (B2) only one domain supported!• Remote access only trough VPN because user Remote access only trough VPN because user

must be member of the domain.must be member of the domain.

Configuring Team Foundation Server

Configuring Team Foundation Server

Team ProjectTeam Project

• A “bucket” on a TFSA “bucket” on a TFS– Work itemsWork items– Source codeSource code– Build dataBuild data– Test resultsTest results– Reference to project portal documentsReference to project portal documents

Demo Create and examine a Team Project

Demo Create and examine a Team Project

AgendaAgenda


Project Management And VSTSProject Management And VSTS

• Keep track of work to doKeep track of work to do– Work itemsWork items

• Let every body work with the tools he likesLet every body work with the tools he likes– Support for Excel, Project and Visual StudioSupport for Excel, Project and Visual Studio

• Get reports out of the boxGet reports out of the box

• Zero friction information gatheringZero friction information gathering

Projects have lots of listsProjects have lots of lists• What kind of lists do you have?What kind of lists do you have?

– RequirementsRequirements– Programming TasksProgramming Tasks– Documentation TasksDocumentation Tasks– Bug ListsBug Lists– RisksRisks– Etc.Etc.

Managing ListsManaging Lists

• Using Microsoft ProjectUsing Microsoft Project– Integrated with Team Foundation ServerIntegrated with Team Foundation Server– Data ValidationData Validation

Managing ListsManaging Lists

• Using Microsoft ExcelUsing Microsoft Excel– Integrated with Team Foundation ServerIntegrated with Team Foundation Server– Data ValidationData Validation

Team Foundation – Work Item TrackingTeam Foundation – Work Item Tracking

Source Code Source Code ControlControl




ReportingReporting

Work ItemsWork Items

• Very similar to task list in MS ProjectVery similar to task list in MS Project

• Work should be done based off tasksWork should be done based off tasks

• Different types of tasksDifferent types of tasks– RequirementRequirement– BugBug– XXXX– YYYY

Working with Work ItemsWorking with Work Items

• Project managers can use MS Project or Project managers can use MS Project or Excel and synchronize with the work item Excel and synchronize with the work item databasedatabase

• Developers will see work items and interact Developers will see work items and interact with them directly in Visual Studiowith them directly in Visual Studio

• Testers will interact with work items using Testers will interact with work items using Visual Studio Team Test Visual Studio Team Test

• Project Managers could also use Team Project Managers could also use Team Foundation ClientFoundation Client

DemoDemo• Synchronize MS Project/Excel with Work Synchronize MS Project/Excel with Work

ItemsItems

AgendaAgenda


Team Architect Problem Space

Team Architect Problem Space

• Distributed systems and SOA increase Distributed systems and SOA increase development and deployment complexitydevelopment and deployment complexity

• Communication problems exist betweenCommunication problems exist between– Architects and DevelopersArchitects and Developers– Development and Infrastructure Development and Infrastructure

ArchitectsArchitects

Source: David M. Anderson, Design for Manufacturability: Optimizing Cost, Quality, and Time-to-Market, Second Edition (2001), CIM Press 805-924-0200Source: David M. Anderson, Design for Manufacturability: Optimizing Cost, Quality, and Time-to-Market, Second Edition (2001), CIM Press 805-924-0200

Lessons from Car ManufacturingLessons from Car Manufacturing

Modeling as the ApproachModeling as the Approach

• Modeling provides a common languageModeling provides a common language– Focus on key abstractionsFocus on key abstractions– Enables design and communicationEnables design and communication– Enables analysis and problem solvingEnables analysis and problem solving

• ChallengesChallenges– Keeping models synchronized with the Keeping models synchronized with the

real worldreal world– Designing/choosing the modeling Designing/choosing the modeling

language and toolslanguage and tools

Merc.Merc. StageStage ShuttleShuttleJobJob

CacheCache

HTTPHTTP

SQL Feed Feed StoreStore DSSDSS

FTPFTP

System Definition ModelCreating a living blueprint of a system

System Definition ModelCreating a living blueprint of a system

ResourcesResourcesRequiredRequired

OperationalOperationalPoliciesPolicies

OperationsOperationsCapabilitiesCapabilities

Create a Create a definition of definition of

a new or a new or existing existing SystemSystem

Operate the Operate the System based System based

on its on its definitiondefinition

Automatically allocate and Automatically allocate and configure resources using its configure resources using its

definitiondefinition

STORAGESTORAGESERVERSSERVERS NETWORKINGNETWORKINGSDM DocumentSDM Document

Design for Operations

Dynamic Data

Center

http://eue/whitehorse/pdc2003/WHPDC/DSD.bmp

System Definition ModelLayers

System Definition ModelLayers

Applications

ApplicationHosting

Logical Machines & Network Topology

Hardware

SystemDefinition

Model

Application Modeling DefinitionsApplication Modeling Definitions• An An ApplicationApplication is an individually deployable is an individually deployable

unit of functionunit of function– Which may provide or consume services via Which may provide or consume services via

endpointsendpoints

• A A SystemSystem is a configured set of one or is a configured set of one or more applicationsmore applications– Can be composed to create more complex Can be composed to create more complex

systemssystems– Is the unit of deployment and managementIs the unit of deployment and management

Application DesignerApplication Designer• Used to define or visualize applications in a VS Used to define or visualize applications in a VS

Solution and how they are connected Solution and how they are connected

System DesignerSystem Designer• Used to define a specific configuration of Used to define a specific configuration of

applications and other systemsapplications and other systems

• Physical network diagrams are not aimed at Physical network diagrams are not aimed at developers!developers!

Logical Datacenter ModelingLogical Datacenter Modeling

Logical Datacenter Modeling Definitions

Logical Datacenter Modeling Definitions

• A Logical Server describes the application A Logical Server describes the application hosting characteristics of a type of serverhosting characteristics of a type of server

• A Logical Datacenter is a configured set of A Logical Datacenter is a configured set of logical serverslogical servers

• A Zone is a logical datacenter boundary A Zone is a logical datacenter boundary (physical, security, communication etc.) or (physical, security, communication etc.) or securitysecurity

• A Deployment Definition describes the A Deployment Definition describes the deployment of a system to a logical datacenterdeployment of a system to a logical datacenter

Logical Datacenter DesignerLogical Datacenter Designer• Used to define a logical model of a datacenterUsed to define a logical model of a datacenter• Provides a developer’s perspectiveProvides a developer’s perspective

Deployment DesignerDeployment Designer• Defines how applications in a system will Defines how applications in a system will

be deployed in a datacenterbe deployed in a datacenter

DemoDemo• Application DesignersApplication Designers

AgendaAgenda


Team Foundation – Source Code ControlTeam Foundation – Source Code Control

Source Code Source Code ControlControl




ReportingReporting

Visual SourceSafeVisual SourceSafe

• Considered industry’s easiest to use Considered industry’s easiest to use version control toolversion control tool

• Often first SCM tool encountered by Often first SCM tool encountered by developersdevelopers

• Not considered a full featured SCM toolNot considered a full featured SCM tool

• Not appropriate for large scale software Not appropriate for large scale software developmentdevelopment

Team Foundation Source Code ControlTeam Foundation Source Code Control

• 3-tier, web service architecture3-tier, web service architecture• Built on SQL Server for robust, transaction-Built on SQL Server for robust, transaction-

based version controlbased version control• Integration with other VSTS tools for Integration with other VSTS tools for

automated workflow managementautomated workflow management• Implementation of advanced SCM conceptsImplementation of advanced SCM concepts• Migrate fromMigrate from

– VSS, ClearCase, Source DepotVSS, ClearCase, Source Depot

SourceSafe vs Team Foundation

SourceSafe vs Team Foundation

Visual SourceSafe 2005 Visual SourceSafe 2005 Visual Studio 2005 Team Visual Studio 2005 Team Foundation Foundation

Description Description Version Control Version Control Integrated Software Integrated Software Lifecycle Suite Lifecycle Suite

Team Size Team Size Individuals and Small Individuals and Small Teams Teams

Scalable Scalable

Storage Storage File System File System SQL Server 2005 SQL Server 2005

Security Security Application specific Application specific Windows Integrated Windows Integrated

Remote Access Remote Access New Web Service for VS New Web Service for VS Integration Integration

Optimized Web Service Optimized Web Service

Source Code ControlFeatures

Source Code ControlFeatures

• Complete version control feature setComplete version control feature set– Uses Windows identities (rather than source Uses Windows identities (rather than source

control specific) for access.control specific) for access.

• Innovative new SCC featuresInnovative new SCC features– Integrated checkinIntegrated checkin– Parallel developmentParallel development– Atomic checkinsAtomic checkins– Remote developmentRemote development

• Integration with other VS Team System Integration with other VS Team System toolstools

Source Code ControlDifferences compared to VSS

Source Code ControlDifferences compared to VSS

• No sharingNo sharing

• No pinningNo pinning

• No Archive and restore featuresNo Archive and restore features

Validate Policy DetailValidate Policy Detail

SCC Policy EngineSCC Policy EngineSCC Policy EngineSCC Policy Engine

Unit

Test

ing

U

nit

Test

ing

U

nit

Test

ing

U

nit

Test

ing

WI A

ssoci

ati

on

WI A

ssoci

ati

on

WI A

ssoci

ati

on

WI A

ssoci

ati

on

Sta

tic

An

aly

sis

Sta

tic

An

aly

sis

Sta

tic

An

aly

sis

Sta

tic

An

aly

sis

Rele

ase

Note

sR

ele

ase

Note

sR

ele

ase

Note

sR

ele

ase

Note

s

Cust

om

Polic

yC

ust

om

Polic

yC

ust

om

Polic

yC

ust

om

Polic

yPolicy Policy DefinitionsDefinitions•.NET Assemblies.NET Assemblies•Return Pass or Return Pass or Fail and messageFail and message•Customer Customer ExtensibleExtensible•User Over-ridableUser Over-ridable

Source Code ControlParallel Development

Source Code ControlParallel Development

• Multiple ReleasesMultiple Releases– BranchingBranching– MergingMerging

• Multiple CheckoutsMultiple Checkouts

Source Code ControlShelving

Source Code ControlShelving

• Interrupted workflowInterrupted workflow

• Transfer changes without checkinTransfer changes without checkin

• Checkpoint or share work in progressCheckpoint or share work in progress

Source Code ControlRemote Development

Source Code ControlRemote Development

• Remote development is a realityRemote development is a reality– Distributed teams, at-home, offshoreDistributed teams, at-home, offshore

• A system built for the InternetA system built for the Internet– Web service protocols Web service protocols – Compatible with proxies and firewallsCompatible with proxies and firewalls– Optimized for high-latency networksOptimized for high-latency networks

DemoDemo• Source code ControlSource code Control

AgendaAgenda


Class DesignerClass Designer• Reflects the underlying source fileReflects the underlying source file

– Diagram is a view of the codeDiagram is a view of the code

• Stored at a project levelStored at a project level• Supports inheritanceSupports inheritance• Supports associationsSupports associations• Supports enums and structsSupports enums and structs

Use Class Diagrams to…Use Class Diagrams to…• Understand existing codeUnderstand existing code

• Initial class designInitial class design

• Review and refactor codeReview and refactor code

• Diagrams for documentationDiagrams for documentation

DemoDemo• Class DesignerClass Designer

AgendaAgenda


Projects and TestingProjects and Testing

• Often an expensive afterthoughtOften an expensive afterthought

• Strategies for minimizing impactStrategies for minimizing impact

RequirementsCoding

IntegrationBeta Test

Post-Release

5

10

15

20

25

30

Relative CostTo Fix Bugs...

National Institute of Software and Technology(www.nist.gov/director/prog-ofc/report02-3.pdf)

Problems...Problems...• It is expensive to find and fix bugs that get past It is expensive to find and fix bugs that get past

daily development practicesdaily development practices• It is hard to diagnose errors at runtimeIt is hard to diagnose errors at runtime• Why does an application run slowly?Why does an application run slowly?• Individual Developers and Testers need to know Individual Developers and Testers need to know

if they are on trackif they are on track• Test and development are often out of synchTest and development are often out of synch• Final test phase for shipping is often ad-hocFinal test phase for shipping is often ad-hoc• How much testing is enough?How much testing is enough?

Defence In DepthDefence In Depth• Microsoft uses a 'defence in depth' strategyMicrosoft uses a 'defence in depth' strategy

– Unit testingUnit testing– Code reviewsCode reviews– Frequent buildsFrequent builds

• Catch bugs earlyCatch bugs early– Static checksStatic checks– Runtime checksRuntime checks

Unit TestingUnit Testing• Diagnostic checks during developmentDiagnostic checks during development

– Test script for all public methods on a typeTest script for all public methods on a type– Basic sanity checkingBasic sanity checking– Useful for regression testingUseful for regression testing

public double public double MethodA()MethodA() { ... } { ... }

public void public void TestMethodA()TestMethodA(){{ double d = double d = MethodA()MethodA();; // is value correct?// is value correct?}}

Each method has a Each method has a corresponding test methodcorresponding test method

Code ReviewsCode Reviews• For the Visual Studio 7.0 product cycleFor the Visual Studio 7.0 product cycle

– 86% of bugs occurred in reviewed code86% of bugs occurred in reviewed code– 60% of all bugs were coding errors60% of all bugs were coding errors

• Static analysis helps catch bugsStatic analysis helps catch bugs– Source code analysisSource code analysis– PREfast for C and C++PREfast for C and C++– FxCop for .NETFxCop for .NET

Static AnalysisStatic Analysis• Find errors in your code before you run or Find errors in your code before you run or

deploy itdeploy it• A range of checksA range of checks

– From style to code correctness to security issuesFrom style to code correctness to security issues

• Integrated into the Team System Build Integrated into the Team System Build EnvironmentEnvironment

PREFastPREFast• Static analysis for C/C++ codeStatic analysis for C/C++ code

– Managed and unmanaged C++Managed and unmanaged C++

• Catches common bugsCatches common bugs– Buffer overruns, uninitialized memoryBuffer overruns, uninitialized memory– Memory leaks, null pointer dereferenceMemory leaks, null pointer dereference

• Reported as compiler warningsReported as compiler warnings– Display path to problemDisplay path to problem– Use #pragma to turn offUse #pragma to turn off

FxCopFxCop• Static analysis for .NET assembliesStatic analysis for .NET assemblies

– Not just C++Not just C++

• Uses design guidelinesUses design guidelines

• CustomizableCustomizable– Which checks to includeWhich checks to include– Whether to report as error or warningWhether to report as error or warning– Create custom rulesCreate custom rules

Frequent BuildsFrequent Builds• Build often to check consistencyBuild often to check consistency

– Nightly build scriptNightly build script– 'Buddy building''Buddy building'

• Microsoft enforces checksMicrosoft enforces checks– Unit testing and source code analysis at Unit testing and source code analysis at

check-incheck-in– Check-in policy for team foundation!Check-in policy for team foundation!

Testing in VSTSTesting in VSTS

Change Management

Work Item Tracking

Reporting

Project Site

Visual Studio

Team Foundation Project Management

Visual Studio

Team Architect

Visio and UML Modeling

VS Pro

Class Modeling

Application Modeling

Logical Infra. Modeling

Deployment Modeling

Visual Studio

Team DeveloperVisual Studio

Team TestLoad Testing

Manual Testing

Test Case Management

Unit Testing

Code Coverage

Dynamic Code Analyzer

Static Code Analyzer

Code Profiler

Team Foundation Client

Integration Services

Team SystemTeam System

Dev Writes Unit TestsDev Writes Unit Tests

Dev Writes And Tests CodeDev Writes And Tests Code

Dev Reviews WorkDev Reviews Work

Dev Runs Code AnalysisDev Runs Code Analysis

Dev Writes Load TestsDev Writes Load Tests

Dev Checks In WorkDev Checks In Work

Dev Diagnoses & FixesDev Diagnoses & Fixes

Dev Checks In WorkDev Checks In Work

Tester Checks Build StatusTester Checks Build Status

Tester Runs Load TestTester Runs Load Test

Tester Reports BugTester Reports Bug

DeveloperDeveloper

TesterTester

Integrating Dev and TestIntegrating Dev and Test• Tests are just another form of source code:Tests are just another form of source code:

– Stored in source code controlStored in source code control– Versioned with the productVersioned with the product

• ““Test Complete”Test Complete”– Test writing is scheduled along with Test writing is scheduled along with

development workdevelopment work– Tracked by work itemsTracked by work items

• Testers are notified when bugs are fixedTesters are notified when bugs are fixed

VSTS Test TypesVSTS Test Types• Unit TestsUnit Tests

– Test class methodsTest class methods

• Web TestsWeb Tests– Record and playback interactionsRecord and playback interactions

• Load TestsLoad Tests– Simulate multiple usersSimulate multiple users

• Manual TestsManual Tests– Provide scripts for manual tasksProvide scripts for manual tasks

• Third-party TestsThird-party Tests– Integrated into VSTSIntegrated into VSTS

Additional Test TypesAdditional Test Types

• OrderedOrdered– Execute a set of existing tests in orderExecute a set of existing tests in order

• GenericGeneric– A wrapper for an existing programA wrapper for an existing program

Test Manager and Test ViewTest Manager and Test View• Test management toolsTest management tools

• Test ViewTest View– Flat list of all testsFlat list of all tests

• Text ManagerText Manager– View and run all tests for a projectView and run all tests for a project

DemoDemo• Manual TestingManual Testing

Test-Driven DevelopmentTest-Driven Development

• Integrate testing into the development Integrate testing into the development processprocess

• Tests define what code will doTests define what code will do– Tests come from specificationsTests come from specifications– Write code to pass testsWrite code to pass tests– Don't write code that doesn't contribute to Don't write code that doesn't contribute to

passing a test...passing a test...

CodeCode

Test-Driven DevelopmentTest-Driven Development

Write a testWrite a test

Refactor ifRefactor ifneededneeded

Compile testCompile test

Fix compile errorsFix compile errors

Run testRun test

Fix runtime errorsFix runtime errors

VSTS Unit TestingVSTS Unit Testing

• Integrated into VSIntegrated into VS

• Automatic generation of test classesAutomatic generation of test classes

• Comprehensive test managementComprehensive test management

• Code coverage reportingCode coverage reporting[TestMethod()][TestMethod()]public void public void GetValueTestGetValueTest()() {{ double d = myObject.double d = myObject.getValuegetValue();(); if (d < 10.0)if (d < 10.0) Assert.Fail("Bad return value");Assert.Fail("Bad return value"); }}

Code CoverageCode Coverage

• How much of your class How much of your class have you tested?have you tested?– Percentage or graphicalPercentage or graphical

DemoDemo• Test driven development using VSTSTest driven development using VSTS

AgendaAgenda


Dynamic code analysisDynamic code analysis• Gather information on application timing and Gather information on application timing and

performanceperformance• Integrated into VSTSIntegrated into VSTS• Used by many internal teams and on Used by many internal teams and on

customer engagementscustomer engagements• Two measurement methodsTwo measurement methods

– Sampling gathers information at fixed intervals Sampling gathers information at fixed intervals on whatever is executingon whatever is executing

– Instrumentation records every callInstrumentation records every call

Sampling And InstrumentationSampling And Instrumentation• Sampling has low overheadSampling has low overhead

– Stack pollingStack polling– Non-invasiveNon-invasive– Some methods could be missedSome methods could be missed

• Instrumentation inserts entry/exit probes Instrumentation inserts entry/exit probes into methodsinto methods– Exact resultsExact results– High overheadHigh overhead

ProfilingInstrumentation

ProfilingInstrumentation

mainmain AMethodAMethod BMethodBMethod

main main 001001AMethodAMethod 00230023BMethodBMethod 00980098

• Adds instructions to your code to monitorAdds instructions to your code to monitor

ProfilingSampling

ProfilingSampling

mainmain AMethodAMethod BMethodBMethod

• Analyze without changing your appAnalyze without changing your app

main 3main 3AMethod 6AMethod 6

BMethod 9BMethod 9

22

Analyzing Performance DataAnalyzing Performance Data• Visualize performance data within Visual Visualize performance data within Visual

StudioStudio– SummarySummary– Function and caller/calleeFunction and caller/callee– Call stackCall stack

• Also export as XMLAlso export as XML

DemoDemo• Instrumentation profilingInstrumentation profiling

AgendaAgenda


Team Build GoalsTeam Build Goals• Make it easier to produce daily buildsMake it easier to produce daily builds

– Good build process is hard to achieveGood build process is hard to achieve

• Support end-to-end build scenario Support end-to-end build scenario – Tight integration with team system toolsTight integration with team system tools

• Provide flexibility and extensibilityProvide flexibility and extensibility– Support custom steps in the Build processSupport custom steps in the Build process

Build ScenariosBuild Scenarios• Public build lab Public build lab

• Desktop buildsDesktop builds

• Resuming builds after failuresResuming builds after failures

• Publishing build results and build dropsPublishing build results and build drops

FeaturesFeatures• Public builds and desktop builds Public builds and desktop builds

– Multiple build configurationsMultiple build configurations– Scheduled recurring builds or on-demand buildsScheduled recurring builds or on-demand builds– Resuming builds after failuresResuming builds after failures

• Build status Build status – Email notificationsEmail notifications– Publishing builds and resultsPublishing builds and results

• Build reportsBuild reports– Storing build results in SQL server Storing build results in SQL server – Historical trend reports using warehouseHistorical trend reports using warehouse

• Separate build serversSeparate build servers• Build customizationBuild customization

Team Build ArchitectureTeam Build Architecture• Team Build is an end-to-end build solution Team Build is an end-to-end build solution

for Team Projectfor Team Project• Team Build is built upon MSBuildTeam Build is built upon MSBuild

– Not a new build engine. Not a new build engine. – Adds bunch or tasks, targets, and project files Adds bunch or tasks, targets, and project files

for the end-to-end scenariofor the end-to-end scenario

Build customizationBuild customization• What steps comprise a build? Do you What steps comprise a build? Do you

introduce additional steps in the build?introduce additional steps in the build?

• CustomizationCustomization– Introduce additional build steps in the buildIntroduce additional build steps in the build– Integrate with third party toolsIntegrate with third party tools– Override properties Override properties – Through editing a separate XML fileThrough editing a separate XML file

Build ExecutionBuild Execution

Build Build initiated from initiated from

TFSTFS

Prepare build Prepare build machine & machine &

Create a build IDCreate a build ID

Sync Sync sources sources

Calculate Calculate code churncode churn

Produce Produce build reportbuild report

Publish Publish buildbuild

UpdateUpdatework itemswork items

Calculate Calculate code code

coveragecoverage

Compile and Compile and analyzeanalyze

ExecuteExecuteteststests

DemoDemo• Team BuildTeam Build

AgendaAgenda


Testing Web ApplicationsTesting Web Applications• Web applications need testing just like Web applications need testing just like

applications...applications...– What is response time for common tasks?What is response time for common tasks?– Has the upgrade broken basic functionality?Has the upgrade broken basic functionality?– Do searches still work?Do searches still work?

• Many external, third-party solutionsMany external, third-party solutions– Variations in capability and flexibilityVariations in capability and flexibility

Web TestsWeb Tests• Integrated into Visual StudioIntegrated into Visual Studio

– Just Another TestJust Another Test

• Recorded or code based testingRecorded or code based testing– With validationWith validation

• Not only for ASP.NETNot only for ASP.NET– HTTP-basedHTTP-based– Any web pageAny web page

• Can be used to test Web ServicesCan be used to test Web Services

Test TypesTest Types• Recorded testsRecorded tests

– XML formatXML format– List of URLs and parametersList of URLs and parameters– Validation and extractionValidation and extraction

• Coded testsCoded tests– Extend via program logicExtend via program logic– Convert from recorded testsConvert from recorded tests

Recording and Playing TestsRecording and Playing Tests• Simple recording via Internet ExplorerSimple recording via Internet Explorer

• Web test file captures script detailsWeb test file captures script details– VS interface allows editingVS interface allows editing

• Player within VS replays scriptsPlayer within VS replays scripts

Handy FeaturesHandy Features• Forms Authentication awareForms Authentication aware

• Viewstate TrackingViewstate Tracking

• Cookie TrackingCookie Tracking

Validation and Extraction RulesValidation and Extraction Rules• Validate web page accessValidate web page access

– Check content or responseCheck content or response– Create custom rulesCreate custom rules

• Validation can be data-drivenValidation can be data-driven

• Extract attributes / parametersExtract attributes / parameters– For use in subsequent requestsFor use in subsequent requests

Coded TestsCoded Tests• Full custom testing solutionFull custom testing solution

// Create a test// Create a testWebTestRequest request1 = WebTestRequest request1 = new WebTestRequest("http://localhost/StoreCSVS"); new WebTestRequest("http://localhost/StoreCSVS");request1.ThinkTime = 8;request1.ThinkTime = 8;

// Add a rule// Add a ruleValidationRuleFindText rule1 = ValidationRuleFindText rule1 = new ValidationRuleFindText(); new ValidationRuleFindText();rule1.FailIfTextFound = false;rule1.FailIfTextFound = false;rule1.TextToFind = "IBuySpy";rule1.TextToFind = "IBuySpy";request1.ValidateResponse += request1.ValidateResponse += new ValidateResponseEventHandler(rule1.Validate); new ValidateResponseEventHandler(rule1.Validate);

DemoDemo• Web Application TestingWeb Application Testing

AgendaAgenda


ASP.NET Performance Problems

ASP.NET Performance Problems

• How does your application work under How does your application work under load?load?

• What operations have the greatest impact?What operations have the greatest impact?

• Processor / disk utilization?Processor / disk utilization?

• Do you need to deploy to a production Do you need to deploy to a production server to test?server to test?

ASP.NET Load TestingASP.NET Load Testing• Integrated witIntegrated within VS.NEThin VS.NET

– Uses testing scripts for scenariosUses testing scripts for scenarios– Browser / user load / server control (?)Browser / user load / server control (?)

• Result visualizationResult visualization

Login script - 10%

Browse script - 60%

Search script - 30%

WebWebSiteSite

GraphsGraphs

TablesTables

XMLXML

DemoDemo• Web Load TestingWeb Load Testing

AgendaAgenda


Visual Studio as a PlatformVisual Studio as a Platform

User Customization

Macros

Add-Ins

SDK

Ext

ensi

bil

ity

Ext

ensi

bil

ity

Cu

sto

miz

atio

nC

ust

om

izat

ion

Team Foundation Extensibility?Team Foundation Extensibility?

Version Version ControlControl




ReportingReporting

Custom workitem typesCustom workitem typesCustom work flowsCustom work flowsCustom FieldsCustom Fields

Custom Web partsCustom Web partsSite Creation extensibilitySite Creation extensibility

Custom Build StepsCustom Build StepsMS Build extensibilityMS Build extensibility

Custom ReportsCustom Reports

Custom policiesCustom policiesCustom file typesCustom file types

Extensibility in SchemaExtensibility in Schema• Work Item TypesWork Item Types

– Fields, form, state transitionsFields, form, state transitions– Defined throughDefined through

• Administration UIAdministration UI• Import/export between serversImport/export between servers

• ScenariosScenarios– Add enterprise specific work item typesAdd enterprise specific work item types– Modify existing work item type forms and rulesModify existing work item type forms and rules– Off the shelf work item typesOff the shelf work item types

• TFS will ship with pre-defined typesTFS will ship with pre-defined types– Bug, task, risk, work packageBug, task, risk, work package

Customizable Methodology TemplatesXML File

Customizable Methodology TemplatesXML File

• Project LifecycleProject Lifecycle– Phases and Iterations Phases and Iterations – Exit CriteriaExit Criteria

• Process Guidance Process Guidance – Document TemplatesDocument Templates– Work Item Types and RulesWork Item Types and Rules

• Bug, Task, RiskBug, Task, Risk• Work Packages (Scenario, Feature, Requirement)Work Packages (Scenario, Feature, Requirement)

• ReportsReports

• Project GroupsProject Groups

DemoDemo• Team System Extensibility & customizationTeam System Extensibility & customization

Boek Boek

• Iedere bezoeker krijgt 1 boek Iedere bezoeker krijgt 1 boek • Alleen te krijgen door je badge in te leveren bij Alleen te krijgen door je badge in te leveren bij

de uitgang de uitgang

Where to find more infoWhere to find more info• Articles in .Net magazineArticles in .Net magazine

– http://www.microsoft.comhttp://www.microsoft.com– http://msdn.microsoft.comhttp://msdn.microsoft.com– http://www.gotdotnet.comhttp://www.gotdotnet.com

• Training and consultancyTraining and consultancy– http://www.infosupport.nl/training/Default.aspxhttp://www.infosupport.nl/training/Default.aspx– Keywords: Keywords: team systemteam system– 3 day training event on team system (end of March begin of June)3 day training event on team system (end of March begin of June)

• Dutch NewsgroupsDutch Newsgroups– http://www.infosupport.com/nieuwsgroepenhttp://www.infosupport.com/nieuwsgroepen– MS DotnetMS Dotnet

• My BlogMy Blog– Blogs.infosupport.com/marcelvBlogs.infosupport.com/marcelv

• Contains lots of information on stuff I do with Team System and things I discover or Contains lots of information on stuff I do with Team System and things I discover or learn about the productlearn about the product..

Documents

Building quality software on target Marcel de Vries [email protected]