Security for Modern Engineering

Information Security & Risk Management Microsoft ITPublished: 2016

Copyright InformationThe information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication.Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoftcannot guarantee the accuracy of any information presented after the date of publication.This white paper is for informational purposes only. Microsoft makes no warranties, express or implied, in this document.© 2016 Microsoft Corporation. All rights reserved.

1

Contents1 Acknowledgements...........................................................................5

2 Forward............................................................................................6

2.1 Bret Arsenault................................................................................................62.2 Sue Barsamian...............................................................................................7

3 Introduction.....................................................................................8

3.1 Setting the scope...........................................................................................83.2 The SDL is our foundation..............................................................................83.3 The challenge of modern engineering...........................................................8

3.3.1 The modern engineer..............................................................................93.3.2 The Microsoft IT model............................................................................9

3.4 Our journey..................................................................................................104 A closer look at the challenges........................................................10

4.1 DevOps culture............................................................................................104.2 DevOps and security....................................................................................114.3 Additional requirements..............................................................................12

4.3.1 Continuous assurance...........................................................................124.3.2 Intelligent automation...........................................................................13

5 Our approach..................................................................................135.1 Knowledge management.............................................................................14

5.1.1 CALM board...........................................................................................145.1.2 Technical Control Procedures................................................................145.1.3 Guidance factory...................................................................................17

5.2 Automation..................................................................................................185.2.1 Static security analysis..........................................................................185.2.2 Dynamic security analysis.....................................................................195.2.3 Runtime detection and prevention........................................................20

5.3 Implementation............................................................................................21

2

5.3.1 Static analysis........................................................................................215.3.2 Fortify SCA and intelligent automation..................................................225.3.3 Fortify SCA implementation process......................................................225.3.4 Fortify SCA deployment architecture.....................................................235.3.5 Shortfalls and opportunities...................................................................245.3.6 VSTS integration....................................................................................255.3.7 Dynamic analysis...................................................................................265.3.8 WebInspect deployment architecture....................................................275.3.9 Runtime detection and protection.........................................................285.3.10 Automation factory................................................................................30

5.4 Metrics focused on driving the right behavior..............................................315.5 User experience...........................................................................................36

5.5.2 Taking security to engineers.................................................................376 Future of application security..........................................................397 Lessons learned..............................................................................39

7.1 Partner with engineers.................................................................................397.2 Focus on the willing.....................................................................................407.3 Be thoughtful about selecting technology...................................................407.4 Build your process first, then focus on tools................................................407.5 Integrate your tools into the engineers’ world.............................................407.6 Build a relationship with your vendor..........................................................417.7 Be mindful of business impact.....................................................................417.8 Keep up with changing technology..............................................................41

8 Conclusion......................................................................................419 Appendix A: Resources....................................................................43

9.1.1 SDL........................................................................................................439.1.2 Modern engineering and DevOps..........................................................43

3

1 AcknowledgementsAuthorsAnmol Malhotra Talhah MirContributorsAaron ClarkGlenn LeifheitJonathan GriggsManish Prabhu Shoham DasguptaReviewersAndrew MarshallBrijesh DesaiBruce JenkinsDave ChristiansenKaren LueckingMichael HowardRalph HoodRob Polly

4

2 Forward2.1 Bret Arsenault

The pace at which business is moving today requires that technology be more agile, to keep up with the rapidly evolving needs of companies and organizations around the world. Technology companies need to ensure that security is keeping pace with the speed of software, and address the security gaps created by moving to agile workflows. While security has always been a primary focus for us at Microsoft, today’s threat landscape demands that we adapt the way we address security as a business. We work constantly to ensure that security is top-of-mind for everyone at the company. It’s clear that to build a strong security posture, we must engage everyone from our engineering teams all the way through to our senior leadership. Facing new pressures, modern engineering teams are leading the transformation to agile development and are delivering what customers need, as they need it. With an agile methodology, Microsoft IT provides the flexibility and speed with which solutions are released in as short a time as operationally feasible. To properly land the value of these accelerated development cycles, companies need to ensure that they have the right security processes and automated tools in-place to address new risk exposure that is created by a high-speed development environment. Importantly, leadership must also make sure we are creating a security culture and driving the right behavior with engineers - enabling them to succeed, while delivering the best possible products to our customers.Microsoft’s Information Security and Risk Management team (ISRM) has been fortunate to partner closely with Hewlett Packard Enterprise (HPE) to accelerate some of our emerging modern engineering security plans. Using HPE Fortify SCA to conduct static security analysis of our applications, and HPE WebInspect for dynamic web application security testing, we are taking the right steps to protect our development environment effectively and efficiently, as we stay agile for business success.

5

Corporate Vice President andChief Information Security OfficerMicrosoft

6

2.2 Sue BarsamianThe rapid growth of the app economy and the increasing pressure to innovate has put the software developer in the driver’s seat in modern IT. Developers are now deeply involved in every part of the software development lifecycle as the boundaries between software and hardware continue to blur and infrastructure moves to the cloud. Developers are now responsible for driving innovation and keeping up with the increasing need for a faster time-to-market. This notion has challenged the traditional development lifecycle, pushing for more agile processes and greater collaboration across development, QA, security, and operations. Securing the software development process has never been easy, but in the midst of such seismic shifts in software development, application security is more challenging than ever. In this faster-paced new development lifecycle, security organizations must adapt to becoming a natural part of the development process or they risk getting in the way. Even worse, they could be left behind as applications become more complex and more vulnerable than ever. The Microsoft ISRM team has taken a unique and aggressive approach to this challenge by partnering with development organizations to build security into the process while staying true to the discipline of the acclaimed Microsoft Security Development Lifecycle (SDL). By teaming with us in HPE Security Fortify, Microsoft has enabled effective, unobtrusive application security automation at scale that provably secures their applications and saves time and money during development. We are excited to share the experience and lessons learned by bringing together the world’s largest software company and the leading application security solution. Together we have built a world class Application Security program that could provide a model for helping you secure the applications that run your business.

7

Senior Vice President and General Manager, Security ProductsHewlett Packard Enterprise

3 Introduction  3.1 Setting the scopeMicrosoft’s ISRM organization, which is part of Microsoft IT, has a mission to ensure that all of the company's information and services are protected, secured, and available for appropriate use through innovation and a robust risk management framework. Microsoft is committed to building and implementing best-in-class security programs and processes and is constantly working to reduce exposure to cybersecurity risks. ISRM supports Microsoft’s overall security mission by providing key security services that help to protect Microsoft’s corporate systems, services, data, and users. The service lines through which we deliver these services include risk management, threat and vulnerability management, identity and access management, security and incident management, and security monitoring.Across Microsoft IT and throughout the company, the ISRM team is continuously evolving the security strategy and taking actions to protect key assets and the data for our organization. One primary focus for the team is to protect line-of-business (LOB) applications for Microsoft IT. ISRM drives the SDL for IT applications.

3.2 The SDL is our foundationThe SDL is a foundational framework for Microsoft, and it defines the basis for how we drive security in our software engineering processes. This whitepaper will not delve into the details of a software security assurance process such as the SDL, but instead, this paper will showcase how we approach enhancing the SDL process in response to the rapidly shifting challenges that security organizations face in today’s modern engineering landscape. For more detailed resources related to the SDL model, including books and websites, see Appendix A. The SDL defines the standards and best practices for providing security and privacy for new and existing LOB applications currently under development or being planned for development. IT LOB applications are a set of applications that are vital to running an enterprise organization including accounting, legal, finance, human resources, payroll, supply chain management, and resource planning applications, among others.

3.3 The challenge of modern engineeringSoftware engineering teams in the modern world are under tremendous pressure. Continuous customer demand for new capabilities and competitive pressures for differentiation necessitate significantly shorter time-to-market schedules while maintaining the highest quality in software applications. To address this demand, modern engineering teams often adopt agile development methodologies, embrace

8

DevOps (a merging of development and operations), and maintain development infrastructure that support continuous integration/continuous delivery (CI/CD).

3.3.1 The modern engineerEngineers in the modern engineering world must play multiple roles. Everything from gathering customer feedback and requirements, design, coding, testing, deploying to production, and even support, are all under the purview of a modern engineer.Just as the SDL is agnostic to any specific development methodology, practice, or tool, the concepts in this showcase whitepaper apply to this modern engineering world, broadly speaking. Our goal is to empower modern engineers with a set of tools, guidance, and processes to empower them to write, deliver, and maintain more secure applications and services.

3.3.2 The Microsoft IT modelMicrosoft IT has been on a journey to adopt a modern engineering model. Because business customers are demanding faster and faster turnaround on solutions and feature requests, gone are the days when a business waited for a quarter or longer for new features, solutions, or bugs fixed in their applications. To respond to this growing need for efficiency and quicker delivery, Microsoft IT has been transitioning to a modern engineering model. This transition includes merging development and operations roles (DevOps) and using agile development principles, practices, and tools to shorten release cycles. With an agile methodology, Microsoft IT provides the flexibility and speed with which solutions are released in as short a time as operationally feasible. Agile teams are receiving faster customer feedback though an iterative design and feature approach, and mature agile teams often release every day or even multiple times a day. While this is great for business enablement, this poses a huge challenge for security in terms of how to effectively and efficiently drive security and privacy in these CI/CD scenarios. For example, consider a security process that takes two weeks to complete sign off on a release. This model plainly fails when applied to an agile application which may take, for example, a single week to ideate, create, and be ready for release. Additionally, the more traditional security approach – to review every application release – worked well when release cycles spanned months, but this approach is highly inefficient against modern engineering practices where schedules are much more condensed.Given the ubiquity of customer data and critical data, security and privacy are of utmost importance to consumers. For example, would you feel comfortable using a banking application on your mobile phone if security and privacy aspects were overlooked by the engineers? Security can be friction, but it can’t be completely ignored either.

So, this is our challenge:

9

“How can we make security low friction (efficient) while maintaining its effectiveness in this new world of modern

engineering?”

This challenge demands that the security culture and approach are modernized and adapted for shorter release cycles and sprints. Security teams must support decentralized security processes, but they must also drive greater automation and move beyond point-in-time assessment practices. Under these modern engineering challenges, they need to adopt a solution that can scale and that can provide continuous assurance.

3.4 Our journeyThe ISRM team has been on a journey to evolve and enhance our approach to the SDL so that we are more aligned to DevOps and to modern engineering practices. The intent of this whitepaper is to share some of our lessons learned to date and to hopefully spark a dialogue in the security community. We recognize that there is no perfect solution – every business has its own unique circumstances and factors that impact its security requirements. The journey to align to this changing engineering culture keeps us motivated to address the challenges it throws our way. We will also discuss some of the key trends we see in application security that have started to redefine not just how we look at application security but how application security processes such as the SDL are completely re-scoped. For example, with development roles merging with operations, application security processes also need to evolve to effectively secure the Ops in DevOps. Finally, we’ll close by sharing some of the lessons we learned in this journey of driving security into modern engineering practices.

4 A closer look at the challenges4.1 DevOps cultureAn emerging aspect of IT culture, DevOps is defined in several ways across the industry. The following is one example:

Gartner “DevOps represents a change in IT culture, focusing on rapid IT service delivery through the adoption of agile, lean practices in the context of a system-oriented approach. DevOps emphasizes people (and culture), and seeks to improve collaboration between operations and development teams. DevOps implementations utilize technology — especially automation tools that can leverage an increasingly programmable and dynamic infrastructure from a life cycle perspective.”

We don't want to single out DevOps, or say that the concepts discussed in this whitepaper work only with DevOps, because it can be difficult to identify a common definition of DevOps. For the purpose of this discussion, the important aspect of the

10

DevOps movement is to recognize certain fundamental principles that define for us what we consider “Modern Engineering.” For more information, see Appendix A.

The principle focus is around people and culture. Development roles and operational roles have merged, and the expectation from a service or software engineer is not just to develop and test code, but also to be able to deploy and operate the code effectively. Engineers have full control over the runtime environment so they can build with predictability. This avoids the "throw it over the wall" mentality that can occur when development hands off to operations.

4.2 DevOps and securityOne of the single biggest reasons teams adopt a DevOps model is to enable CI/CD to address business demand. The key principles of focus are:

Speeding up the pace of innovation by shortening the release cycles using agile methodology, and maintaining control over the entire technology stack (from code through to the infrastructure and operational practices.)

Enabling faster feedback loops from customers which can result in application features and bug fixes.

Considering these principles as key components of modern engineering, anything that gets in the way of this process is friction. More traditional approaches to software security assurance, that rely on gates, are seen as friction in modern engineering practices. Here are a few examples: 4.2.1.1 Manual security assessmentsTypical white box code or black box security assessments last anywhere from few days to few weeks depending on the size and complexity of the application. With application development sprint cycles shrinking to days, just engaging security

11

FIGURE 1 COMPONENTS OF DEVOPS

teams and scheduling code reviews is a challenge, let alone reviewing anything on time. This time-consuming process is counterproductive for the needs of business and engineering teams because it is clearly a speed bump for fast-paced release cycles.4.2.1.2 Security compliance/Attestation processesAny security attestation processes with lengthy questionnaires may seem complete, but they can also be seen as merely a compliance checkbox exercise with little or no impact on the security of the system being engineered. This not only adds friction but adds limited value to the engineering teams, especially if the process mandates attestation for every release. At the very least, self-attestations help convey a set of baseline expectations and drive awareness. However, self-attestation for every release is not an effective security control within the fast pace of modern engineering release cycles.

To better understand this challenge within Microsoft, we decided to do multiple feedback sessions with our engineering teams. We did this through Voice of Customer (VoC) sessions to understand the pain points and challenges the engineering teams experienced.

4.3 Additional requirementsWhen you look a little deeper at these more traditional software security practices that require manual review and at the demands of modern engineering, two fundamental requirements emerge: continuous assurance and intelligent automation. Continuous assurance is needed to maintain a secure posture and intelligent automation can help scale and keep pace with faster release cycles.

4.3.1 Continuous assuranceWith old protracted development lifecycles, doing point-in-time assessments to help define a security assurance level of software may have been sufficient, especially when a software application went through infrequent changes in production. But with continuous releases, point-in-time assessments don't work as effectively. We have to provide continuous assurance that is embedded in the process. Software can be viewed as a living entity that requires continuous assurance after release, particularly in light of CI/CD. DevOps and security cannot be disparate silos any longer (a fact now understood in the industry and coined as DevSecOps.) We need to start thinking about DevOps as a culture that is not only about merging development and operations, but also about how security responsibilities are merged and shared over time. Security teams will become the provider of these continuous assurance services ranging from network, host, and application security that engineering teams will consume to maintain secure applications.

4.3.2 Intelligent automationIt's natural for any security team to look at some of the challenges articulated thus far and assume that automation is the solution. It’s correct that automation is a big part of any solution. However, automation is very easy to get wrong and very hard to get right. A common response is for security teams to find a wide range of

12

automation tools and “throw them over the wall” at engineers. In our experience, this doesn't always work for two primary reasons:

Running tools isn’t enoughIf tools are pushed as a quick fix, using them can turn into a compliance activity rather than a way to reduce risk. Teams may run the tools, but perform little action based on the output of the tools. Thus, we’ve lost the end goal of automation. To do it right, it’s important to carefully drive selected metrics to help engineers take action effectively. The tools alone aren’t a quick fix, but should instead be a part of a well-thought-through solution to advance security goals.

Tool fatigue is common The feedback we received from VoC sessions revealed that engineers can start to experience tool fatigue if too many tools are thrown at them for compliance. Careful planning and deployment of tools is key to ensure this does not become a pain point.

It’s important to understand that the solution to the challenge of maintaining CI/CD and security compliance is not just automation. We needed intelligent automation that can yield the kind of information that engineers can act on to drive positive impact.

5 Our approachOur approach to the challenges is itself an agile one in which we are evolving the solution through incremental feedback and updates. We are on this journey to evolve and to enhance how we integrate security with modern engineering and how we enable our engineering teams to succeed. The following are the four core tenets around which we are building our solution:

FIGURE 2 FOUR CORE TENETS

5.1 Knowledge managementHaving a sound knowledge management process is one of the key aspects of any effective security solution. Underneath any SDL process are the requirements, which we call Technical Control Procedures (TCPs). Defining clear technical

13

requirements for engineering teams is the foundation of an SDL program, and over the years our Control Assessment Library and Methodology (CALM) board has continuously refined our requirements.

5.1.1 CALM boardA successful knowledge management solution ensures that there is a governance process to constantly refine and refresh the requirement set. We do this by keeping our TCPs up-to-date through our CALM board. Subject matter experts (SMEs) from across the security teams sit on this board. The CALM board meets monthly and makes updates to specific TCPs so that engineers have the most up-to-date controls. They continuously evolve the library, and they make sure controls are relevant with the changing security landscape.

5.1.2 Technical Control ProceduresTCPs are the actionable security controls that engineering teams must implement during the development of applications. TCPs are the minimum bar for security standards that all LOB applications must meet. TCPs are technical or process-oriented and are defined to be agnostic of technology. The following are a few examples of areas these requirements cover:

14

TCPs are built around positive instead of negative attributes and are focused towards engineering teams. We identify what the known positive controls are that a system should implement to be secure. In our taxonomy of a TCP, we define (among other attributes):

How to implement Implement the control or safeguard in an application. How to verify Verify the technical control procedure to measure if it is

correctly implemented or needs improvement. Why to implement Align to company security standard(s) and policy for

compliance measurements.The following are few example categories from the TCP library and their underlying procedures:

15

FIGURE 3 AREAS OF COVERAGE FOR TCPS

TABLE 1 TCP CATEGORIES

TCPs are the backbone of everything we drive with the engineering teams. The following are areas of the SDL to which TCPs are integral:

Core guidance to our development engineers TCPs are the avenue by which we deliver core guidance to our engineers about how controls must be implemented and about their applicability. TCPs are the super set of security controls, and we hold our engineering teams accountable to them through the SDL process.

Security control assessment criteria When an application is selected for a security assessment, all the applicable TCPs based on the context of the application are tested and verified for correct implementation. Any identified failures result in risk findings for the engineering teams to fix.

Tool evaluation Security tools evaluations are fundamentally focused around which TCPs the tools will help automate (either for detection, correction, or both). This mapping to TCPs not only allows us to optimize our manual process, but also helps reveal where we have adequate tool coverage.

Compliance alignment to company security policyTCPs must be aligned to the company’s standards and security policy. This provides the rationale behind “Why.” Alignment helps business and risk stakeholders determine the risk in omitting the implementation of TCPs. The alignment function’s biggest payoff is simplifying the engineering team’s interaction with standards. For example, multiple standards may have a requirement for encryption, and the TCPs allow us to combine each of those requirements into a single actionable control for the engineering team.

16

FIGURE 4 ALIGNMENT TO COMPANY POLICY

5.1.2.1 TCP maintenanceAs described earlier, TCPs are maintained by our CALM board and are authored in a technology agnostic fashion. We review our full set at least quarterly – and also the methodology by which the assessment occurs – and if we have carefully defined them, there should be few changes. For example, TCPs that define requirements for input validation should rarely change. However, when it comes to implementation details that are technology-specific, such as implementing security controls in Azure, we update the implementation guidance that supports the given TCP more often. Because this implementation guidance can change more often, we needed to find a method to address areas in which engineers have frequent questions. The need is also amplified by the fact that development platforms such as .NET and Azure are updating more frequently. To keep up with such a pace, we are experimenting with the concept of a “guidance factory.”

5.1.3 Guidance factoryTo keep pace with modern engineering, the old form of "guidance documents" just aren't as efficient. We don't have the luxury to spend several months (or even weeks) to publish an essay about "Writing secure code for [fill in your platform here]." TCPs are the next step and provide an excellent minimum bar, but sometimes, engineers need bite-sized snacks of guidance. We created a guidance factory where we could take requests on demand and turn them around into bite-sized chunks in a timely fashion. We started piloting a guidance factory approach in 2015. In this pilot, an analyst posts quick guidance in response to questions from the engineering team to a SharePoint site. We leveraged that site as a key input to our TCP maintenance

17

process and continue to evolve them over time. With this agile approach, engineers receive up-to-date guidance that delves deeper than our currently published TCPs in a timely fashion, and in the long term we leverage the guidance produced into the TCPs.

5.2 AutomationTo enable teams for CI/CD and continuous assurance for security, automation must be a focal point in the solution. In support of this, our focus is to develop low-friction security services. We have identified the following three classes of services as key building blocks for our solution to deploy and maintain:

FIGURE 5 THREE CLASSES OF SECURITY SERVICES

5.2.1 Static security analysisStatic security analysis discovers security issues by scanning the code against a set of rules. It is one of the most valued activities in any mature SDL process. When static analysis is run in an Integrated Development Environment (IDE), engineers are exposed to potential security issues while they are coding the application. This instant feedback to engineers makes static analysis highly effective. The following foundational security principles (that static analysis promotes) are why we have focused so much on this security service capability:5.2.1.1 Code hygiene Known security issues in the applications should not be present in the code we write and deploy. Granted, this is easier said than done, but this is our aspiration. Engineers are accountable to ensure code hygiene is addressed, and by leveraging static analysis they can do this effectively and efficiently. This is not to say that static analysis tools are perfect or that they are guaranteed to catch all issues. However, having a clean scan that reports zero issues – from a static analysis tool that addresses all known true positives – is what we consider basic hygiene.5.2.1.2 Code coverageManual line-by-line security assessment is a tiring activity (the human eye must examine lines of code for days), and it is also time-consuming and expensive. Additionally, manual inspection can vary greatly based on the experience and knowledge of the security analyst who looks at the code, which affects the consistency of the code reviews. Static analysis tooling, however, is consistent in terms of their rules and can cover millions of lines of code in a short period of time.

18

This type of security service can provide the scale and code coverage that we needed. In fact, combining static analysis with human code review is one of the best ways to ensure all aspects of the application have been vetted. By leaving the common and tactical issues to the automated tool, we found that our most experienced security SMEs can focus their time on other important aspects such as design issues and business logic issues in higher-risk applications. 5.2.1.3 Transparent security We have found that static analysis is most effective when it is integrated into the build process for the application. This makes the security control a part of the engineering process and makes it transparent to the end user. 5.2.1.4 Moving security upstreamThe traditional security model verified code security after a piece of software was code complete, which is costly to fix. Static analysis allows us to move the code verification into the engineers’ IDEs so that security issues can be caught closer to the time code is written instead of waiting for the security team to catch all issues after the application is code complete.5.2.1.5 Security educationAs engineers get used to running static analysis for their day-to-day code development, they are exposed to security concepts and rules as they triage and learn from regular scan results. This helps to raise the security awareness within the engineering teams.

5.2.2 Dynamic security analysis Dynamic security analysis examines how code runs during the execution of a program. By reviewing how the code responds to and interacts with other system components, a dynamic security analysis tool can uncover security issues within the runtime. Dynamic security analysis discovers security issues by mimicking an attacker’s behavior on the actual application. We believe this service complements static analysis, and a combination of the two yields the best results in terms of completenessThe following are security principles that dynamic analysis addresses and reasons why we think it is an essential part of the overall security automation strategy: 5.2.2.1 HygieneDynamic security analysis helps engineers to test an application during runtime. As they test their application workflow, dynamic security analysis is another tool or service in their arsenal to leverage and verify that the application has fixed security defects that only surface when you run the application.

5.2.2.2 CoverageDynamic security analysis can uncover different types of issues that may be non-discoverable to static analysis or can help validate issues discovered during static analysis. For example, the impact of environment and server-level configuration issues are easier to detect by running dynamic security scans. Potential information

19

leaks that can occur as a result of interaction with the application can also be easier to discover through dynamic analysis versus static analysis. These issues can have severe impact and may be overlooked without effective dynamic scanning.5.2.2.3 Security educationSimilar to static analysis, dynamic security analysis also helps to raise security awareness within the engineering teams.

5.2.3 Runtime detection and prevention A significant focus of the SDL is on proactive security controls and processes during development to ensure that the LOB applications we develop are secure from the ground up. There is a growing need to protect applications in a production environment as the threat landscape continues to evolve. Relying on static and dynamic analysis alone may not be sufficient. Code deployed to production may have gone through the SDL and static and dynamic analysis. However, due to “operational drifts” (where, for example, configurations of a runtime environment are altered), bug fixes, evolving threat landscape, and the fact that agile teams are always evolving code, there must be another layer of protection in a production environment. Runtime detection and protection for application – a technology that we think is still maturing in the industry – provides continuous assurance during application runtime. This is a nascent space, but it is a must-have service capability for the future and will become a crucial part of the overall application security strategy.We have identified the following risk areas that will benefit from a runtime detection service:5.2.3.1 DevOps and cloudEngineering teams continue to move toward a DevOps model, and also towards the cloud. This further shortens the end-to-end time to market (TTM) for a solution. It’s important that engineers use a detection service that hooks into the runtime and provides continuous security assurance and telemetry. This monitoring data enables the engineering teams and security teams to identify potential bad actors on the application layer. We believe this is a crucial component of securing the Ops in DevOps for modern engineers.5.2.3.2 Plugging the gap for SOCCurrently, Security Operations Center (SOC) for most organizations have visibility into host, network, and the end points from a monitoring standpoint. If an attack occurs on the application layer, very little is detected. A view from just the network is not enough anymore.5.2.3.3 Legacy applicationsRuntime detection can be an incredibly powerful tool to protect legacy applications which may no longer have any engineering support and are “sunset” applications. Such applications tend to use older technology stack which were less secure and easy targets for attacks. There could be cases where some applications are so old

20

that they might not have gone through any software assurance processes whatsoever. 5.2.3.4 Compensatory controlDue to time and resource constraints that agile teams face, even when application vulnerabilities are known, changing code to address new and existing threats could take time – often weeks or months – especially under non-agile methodologies. In such cases, applications that have known security issues may choose to implement runtime detection as a compensatory control for the time they need to fix the issues and beyond.

5.3 Implementation 5.3.1 Static analysisThere is no such thing as perfect security, but the pursuit of thoroughness is important because one vulnerability is all it takes to compromise an application. In ISRM, we use HPE Fortify SCA to conduct automated static security analysis of our applications. After evaluating many static analysis tools, we chose Fortify SCA because of its thoroughness, its ability to facilitate collaboration, and most importantly, its coverage against our TCPs. Fortify SCA is a static application security solution. It traverses your code, identifies potential security issues, and compiles a list for you to review. A human determines which issues are serious flaws that must be fixed and which issues are false positives or low priority and can be deprioritized or ignored.Fortify offers two desktop IDEs to view and aggregate issues. Our engineers use the Visual Studio plugin to view Fortify SCA results inside their development environment, while most of our analysts use the Fortify proprietary IDE, Audit Workbench. Everyone accesses scan results from the same Fortify Software Security Center (SSC) server, which ensures that all comments and reviews are preserved from one scan to the next. Build engineers will configure regular scans which will automatically upload results to the Fortify SSC server, ensuring everyone has recent scans to work from. We started the journey with Fortify in 2014, and we use the following three capabilities with Fortify for our needs in ISRM:

Build integrationThis is our most preferred method. We’ve integrated Fortify SCA static analysis with the application build processes. Based on the schedule and release cadence, static analysis runs as well. All the results from the scan are then automatically uploaded to the Software Security Center (SSC) server.

Self-hosted scans Self-hosted scans are one-time scans. This provides a scanning method for those applications that either do not have a build environment for integration or that are one-time applications. Examples are marketing campaign applications that focus on a specific event or applications with a shelf life of less than six months, for example. These applications leverage self-hosted

21

scanning to perform Fortify SCA static security scanning. Results are then uploaded to the SSC server. Engineers are encouraged to utilize this feature so they can see scan results within their IDE before their code is uploaded to build servers.

Static security analysis on-demandOn-demand static security analysis provides engineering teams with white glove treatment. A central security team scans the code on behalf of the engineering teams and helps the team triage the issues. This is similar to HPE’s Fortify on Demand service, but it is run within the organization by our central security team. We developed this capability primarily to handle venture integration scenarios. For example, this capability comes in handy when your company inherits LOB applications as a result of an acquisition and the engineering teams managing those applications have little or no exposure to static analysis. You will need to hand hold them to get them started and slowly move to self-hosted scans and finally to build integration.

5.3.2 Fortify SCA and intelligent automationVetting Fortify SCA and enabling our development teams to use it is our best example of intelligent automation. Introducing automation into a development environment reduces the burden on engineers of using security tools and processes. Throwing a tool over the wall and expecting that a team will not only use the tool, but benefit from the data, may not set a development team up for success. Time is a precious commodity to engineers and onboarding a new tool can be far outside their scope. In this sense, a security team needs to become a “team enabler,” and must work with development teams to find the best methods to fit the tool into their workflow. It must become a part of their environment and must work within their established processes. To that end, and focusing on team enablement, we partnered with our engineering teams to help integrate the tool into their build processes.

5.3.3 Fortify SCA implementation process We enabled our engineering teams to succeed with Fortify SCA during implementation by designing two process tracks: one for the build teams and one for development teams. Error: Reference source not found illustrates the steps we’ve followed for each track, including the phases that are utilized for training. 5.3.3.1 Build trackThe build track is where the heavy lifting occurs. We work with the build owner to ensure that Fortify SCA is installed on their build environment and is configured and scheduled to run based on their release cadence. We then run the first scan and hand off to the development track.5.3.3.2 Development trackOnce the build automation is complete, and we receive the first baseline results, we triage the results together with the engineering teams. We meet with all stakeholders and help them understand the review process so that they can begin to own the process in their environment.

22

5.3.4 Fortify SCA deployment architecture Our Fortify SCA deployment currently has nine Fortify SSC servers running with their own database on a shared database server. Each Fortify SSC server is reachable with a distinct URL and is independent of the others. One of the Fortify SSC servers is dedicated to our one-time scan activity to ensure that we have good separation before teams fully onboard the tool. We use views in each database to collect data for reporting, which is then migrated at regular intervals to the reporting warehouse. Based on our experience, we chose to use three separate User Acceptance Test (UAT) environments:

Environment we use for testing the current version of Fortify SSC. Environment we use for all the patches Fortify provides us to test that our

issues are fixed before they are rolled into a major release. Environment we use for testing underlying operational changes or updates.

FIGURE 6 UAT ENVIRONMENTS

5.3.5 Shortfalls and opportunitiesWe faced several challenges when we began to deploy Fortify SCA in our large enterprise space. The following are some of the shortfalls, including ways that Fortify SCA is helping us address or will be addressing these shortfalls in the future.5.3.5.1 SSC hierarchyFortify SSC does not currently allow for load balancing and horizontal scaling. Because of the size of our organization, and because we were unable to balance the load of our Fortify SSC servers, we knew we would have to manually scale the solution. We currently manage nine Fortify SSCs in our environment for static

23

analysis and two for dynamic analysis. Each group in Microsoft IT is so large (from an application portfolio perspective) that initially we chose to separate by group, which was an easy distinction to keep the load manageable. We further addressed load issues by identifying which group uses which Fortify SSC server in reports. Even through reorganizations and group name changes, we have been able to manage the load consistently by using this approach.Achieving better horizontal scaling is still an opportunity. However, the performance of the Fortify SSC servers has improved greatly over the past several releases and made this scaling less necessary.5.3.5.2 Differential scanningCurrently Fortify SCA scans the entire code base with each scan to determine code changes and any security issues in the code. With Agile and DevOps picking up pace of development, security teams are looking for ways to deliver results in a shortened timeframe. The ability to perform differential scanning on only the code that has changed since the last scan is key to making scans run faster and more efficiently.While using Fortify SCA as our analysis tool, we found an opportunity to openly communicate to the Fortify team about our needs as an enterprise organization along with our feedback. In doing so, we’ve found tremendous value in moving beyond a customer/vendor relationship. The partnership we’ve built with the Fortify team in our journey has yielded important advances in our ability to secure our code. We believe our relationship with the Fortify team, and continuously sharing information, has uncovered valuable new insights that not only helped us, but helped Fortify as well. 5.3.6 VSTS integrationMuch like the rest of the industry, Microsoft is moving many of our key resources into the cloud, including moving software engineering to Visual Studio Team Services (VSTS). This platform has become a focal point for our engineering teams. As a security organization, ISRM knew that it was necessary to find ways to integrate our programs and the scan results into this platform to help enable the engineering teams. With the support of the Microsoft Engineering group behind VSTS, we worked together with the Fortify team to design and build a VSTS extension that allows our engineering teams to remain in compliance with our internal security policies without slowing them down. This integration of Fortify within Microsoft Visual Studio Team Services highlights one of the most beneficial achievements to come out of our relationship with the Fortify team. The goal with VSTS integration was not merely the basic use of Fortify in VSTS like you would use any source repository system. This was about building a solution that allows minimal input from the user and that would enable the teams to use Fortify with less friction in a deployment phase. We wanted to make the Fortify scan feel more like doing a hosted build in VSTS. Another significant aspect is the ability to enable Fortify scanning through a virtual machine, which is preconfigured and ready for teams to use directly in Microsoft

24

Azure. Teams can pick the Fortify build machine out of the Azure store and it will have Fortify ready to go. Scanning machines will spin up on demand instead of requiring dedicated machines that are only used periodically for scans. This on demand service improves the efficiency of both the engineering and security teams.

FIGURE 7 FORTIFY SCA ON VSTS

5.3.7 Dynamic analysis In ISRM we use HPE Fortify WebInspect to conduct dynamic web application security testing. It complements static code analysis performed through Fortify SCA, and both use a common Fortify SSC server for management and reporting.Our approach for deploying dynamic security analysis was pragmatic. We rolled out WebInspect initially as an SDL requirement to help teams discover issues during application testing. We were very aware of the tool fatigue issue our engineers raised, so we focused on static security analysis first and then moved to dynamic analysis once we achieved the sufficient effective adoption of static analysis.

25

FIGURE 8 WEBINSPECT

In the first year, we encouraged all applications (with web interfaces) to run WebInspect by self-scanning and uploading the results for compliance. Multiple application teams used this model, and some lessons we learned included the following:

Low quality scans The quality of scans submitted to Fortify SSC were low. Either they were incomplete scans or misconfigured scans. We realized that engineers must be trained in the tool to run effective scans. For example, running a basic crawl without configuring macros can yield incomplete or duplicative scans of the same resource.

Resources for scans Applications teams complained that there was a lack of dedicated servers from which to conduct self-scans, particularly for larger applications and services.

These lessons paved the way to make changes in our dynamic security scanning strategy. We decided to onboard higher risk applications to a dynamic security scanning service. This service uses a security team to engage with the engineering team once and provide that team with automated, scheduled macro-based scans. Because the scans are led and configured by a security team, we are better assured of the quality of the scans and their results. We are also evaluating use of APIs to further streamline scans scheduling and kick off process.

5.3.8 WebInspect deployment architecture The diagram below describes the two main implementation methodologies for WebInspect at ISRM.

The right side of the diagram illustrates continuous dynamic scanning for high-risk applications. These scans are run as periodic scheduled scans on the WebInspect Enterprise sensors controlled from the WebInspect Enterprise server.

26

The left side illustrates dynamic scanning for all other applications. These scans are run by engineering teams on their own infrastructure and are uploaded to the WebInspect Enterprise server.

FIGURE 9 WEBINSPECT DEPLOYMENT ARCHITECTURE

5.3.9 Runtime detection and protectionCurrently there are two technology solution types to tackle application runtime security detection and protection challenges. These are Web Application Firewall (WAF) and Runtime Application Self-Protection (RASP). ISRM looked into both these technology solution types to address the runtime detection and prevention challenge. We believe RASP is the more promising of the two, although RASP technology is evolving and more work is needed before it becomes the mature solution we need. 5.3.9.1 WAFA WAF works by sitting in front of an organization’s web application stack. It analyzes all incoming traffic for attack patterns and blocks any suspected malicious requests from reaching the web application itself.However, typical WAFs suffer from a number of limitations, including – but not limited to – the following:

Utilizes signature-based pattern matching and blacklist filtering, which can be bypassed as attacks become more sophisticated.

Lacks insight into application logic, data, and event flows because it does not have application context.

Has high false positive rate from list of detected malicious activity. Has relatively limited vulnerability category coverage. Requires extensive coverage and maintenance for firewall rules to stay in

sync with application changes.

27

Even with these limitations, WAFs can provide a level of protection that justify their cost (including operational maintenance) and can be an effective control in many scenarios. 5.3.9.2 RASP RASP is a fairly new security technology. It is hooked into an application or application runtime environment and can be capable of controlling application execution to prevent real-time attacks as well as detection.We have been evaluating a RASP technology from HPE Fortify called Application Defender to provide runtime detection and protection. Our evaluation for RASP has been focused around the following key features:

Enterprise management How does the technology scale for an enterprise deployment?

Performance and scalability How does the technology impact the application performance?

Agent installation and maintenance How easy is it to install and manage agent updates as new rules for attacks roll out? Does it require server restart?

Platform support Does the technology works seamlessly for on premise applications and cloud applications, particularly Platform as a Service (PaaS)?

Ruleset quality What is the current ruleset coverage when mapped to our TCPs, and what is the update frequency of the ruleset?

It’s clear that the deployment and evaluation of any tool that sits in a production environment needs to be carefully vetted from all aspects listed above. We have been working closely with the Fortify team to vet solution details and to work on bridging the gaps to make sure this technology is ready for our needs. The following are two key areas we see that require enhancements:

Azure PaaS support Applications running in Infrastructure as a Service (IaaS) and PaaS (Web Roles) environments can be supported, but the deployment automation is yet to be determined.App Defender also has challenges with uniquely identifying the Azure PaaS hosts which prohibits effective monitoring.

Seamless agent upgrades An update to the agent installed on the host application can require IIS and application server restart, depending on the application type. This can be a significant operational burden for engineering teams.

We plan to test the on-premise solution for App Defender. We are testing it against different scenarios and applications to identify what it would take to deploy this technology in the least disruptive way for the engineering teams. We intend to deploy an end-to-end scenario where App Defender is configured in “Detection Mode Only” at first, and then integrate the feed into a security information and event management (SIEM) technology such as HPE Security ArcSight. Our goal is to

28

directly enable our engineering teams with the detection capabilities that can be incorporated into their telemetry systems to not only gauge the operational state of their application but also the security posture.

A solution like App Defender can provide very advanced application monitoring capability. It is a key component of enabling continuous assurance, and also helps share the responsibility of security with the engineering teams which enables our modern engineers to gain the appropriate security insight into their applications.

5.3.10 Automation factoryHaving the ability to license third-party enterprise tools like Fortify SCA and WebInspect help security teams like ours focus on operational aspects of the service while ignoring the costs associated with engineering and support. But this isn’t to say that internally developed custom built tools or scripts aren’t useful. Much like the rest of the industry, Microsoft IT is moving its workloads and application portfolio to Azure. Azure has enabled DevOps to really flourish by effectively democratizing the infrastructure. Engineers can now control virtual networks and host configuration through code changes. This not only opens up tremendous opportunities for engineering teams but, if leveraged correctly, it enables security teams to drive more effective continuous assurance. To address this shift, we’ve made it a priority to develop what we call the “Secure DevOps Kit for Azure” which contains a set of tools, extensions, code snippets, and other automations. Most specifically, the Secure DevOps Kit for Azure contains the following components:

A package of scripts and programs that ensure more secure provisioning, configuration, and administration of an Azure subscription.

A set of extensions, plug-ins, templates, and PowerShell modules that empower a developer to create, check-in, and deploy code with greater security from the beginning.

A tool that can capture snapshots of “secure” state for a subscription, for the application resources, watch for drift, and that can enable operational security compliance.

A package that would grant the visibility to enterprise IT teams about the shape and form of all of the above along with extensions that can provide application layer events and alerts for individual service line teams.

29

FIGURE 10 SECURE DEVOPS KIT In the spirit of agile development, this kit is a work in progress. We are hoping to refine many of the ideas as we iterate and enhance it over sprints with regular feedback from our engineering community.

5.4 Metrics focused on driving the right behaviorWith a foundation of robust knowledge management in place, together with intelligent automation using industry tools and internally developed custom tools and scripts, we need to make sure we are driving the right behavior with the engineers and enabling them to succeed. We do this by defining carefully selected metrics which are not only actionable but also drive positive reinforcement. It’s easy to come up with metrics such as number of bugs or number of fixes to measure application quality (security or otherwise). However, that can be a rather antiquated model that worked better with traditional engineering practices. In the modern engineering model, where the pace of innovation and continuous assurance are the core drivers, we focused on mean-time based metrics instead such as mean-time-to-triage (MTTT) and mean-time-to-fix (MTTF). These metrics are driven by mean-time and measure how fast the engineers respond to security issues by triaging them first and then by fixing them in their release cycles. Enabling our engineers with automation is critical, but we also need to measure how fast they are leveraging the automation to drive continuous assurance. Automation metrics is one dimension we considered while developing an all up Application Security (AppSec) Health Index for an application or for a portfolio of applications managed by a team. We’ll discuss the details of our application

30

security health index or AppSec Health Index later in the paper, but first we want to discuss the concept of the “four stages of competence.” This forms the backdrop for why we developed the AppSec Health Index in the form we did. The four stages of competence model (developed by psychologist Noel Birch in the 1970s) is a helpful illustration of the stages that an individual goes through to learn a new skill. We’ve applied it to the process that an engineer progresses through as they learn how to achieve security hygiene with the AppSec Health Index. When learning a new skill, individuals or teams progress through each phase from “unconscious incompetence,” (an unskilled state) to “unconscious competence” (a state of expertise).

We think this illustration of the progression of skills is a great concept and is a basis for what we wanted to achieve when creating a health index for our AppSec program. The AppSec Health Index measures the impact of our SDL in maintaining a secure posture of an application portfolio. It is a qualitative metric that is used to indicate the application security health of a team or organization. The index is derived from four underlying quantitative metrics: 

31

TABLE 2 APPSEC HEALTH INDEX QUALITATIVE METRICS

It’s important to note that the targets for each metric should be based on an organization’s maturity and needs. For example, in some organizations, having SDL compliance for as little as 50% of the portfolio may be a reasonable target. In other organization, aiming for 100% compliance may seem realistic. Determining the right target is not trivial and can often take many iterations to converge. However, it’s important to stress that the above numbers are examples only and should not be treated as targets for any company that wants to achieve the most ideal application security posture. If we apply the four stages of competence to our AppSec program, and factor in the AppSec Health index metrics, the progression of skill might look like the following:

32

Finally, to calculate the single qualitative AppSec Health Index, we analyze the four quantitative metrics to determine the conditions that set its value. One such example is to define the overall AppSec Health Index into three tiers of Green (on target), Yellow (close to target) and Red (off target) and set the value based on this:

FIGURE 12 APPSEC HEALTH INDEX

33

FIGURE 11 STAGES PROGRESSION

5.4.1.1 Driving competition The concept of using metrics to drive competition or gamification is not new in security. Microsoft engineering teams love to compete to be at the top and to be the best among their peers. We approached this by publishing the AppSec Health Index quarterly on a scorecard for all engineering teams in Microsoft IT. This drove a sense of competition and encouraged teams to mimic the best practices of the teams who are on the top of the scorecard. 5.4.1.2 Positive reinforcement Security teams traditionally point out negative metrics and gaps in the scorecard for the engineering teams. Although these are required at times to drive action by the teams and to reduce risk, consistently calling out only negative metrics does not promote the positive behavior we want to ultimately instill. We used the natural alternative approach where the metric also reflected positive reinforcement for the teams who were doing better. We weren’t surprised to learn that showcasing positive behavior goes even further if highlighted in the right way. Our AppSec Health Index helps us surface and drive positive reinforcement. For example, we started publishing the AppSec Health Index by using a quarterly risk scorecard across all our business groups. This initiative not only drove a sense of competition and positive behavior, but it also increased health indexes across those groups.5.4.1.3 Auxiliary metricsFinally, we also developed an auxiliary metric specifically around trending information for “Fortify Top 10.” As more and more applications began using automated static security scanning, we wanted to identify what the top trending Fortify issues were across our engineering community and within specific engineering teams. This metric helps uncover potential pain points such as a high number of unfixed issues in a certain category or in a certain pocket of the business which may require special attention. We have driven multiple targeted campaigns and programs to deal with the pain points this metric revealed.

FIGURE 13 SAMPLE - FORTIFY TOP 10 VULNERABILITIES

34

We also found “Bug density data” valuable to determine an engineering team’s security maturity and to enable the conversations with the teams to figure out corrective actions.

FIGURE 14 SAMPLE - BUG DENSITY (PER 10K LOC)

5.5 User experienceSo far we have discussed having the right knowledge management, deploying in-place intelligent automation, and finally capturing it all with effective metrics to drive the right behavior within our engineering community. All of this could be in vain if we don’t take a very explicit look at overall user experience. Users in this context are our software engineers who interact with and use this solution. Oftentimes, security teams undermine the importance of end-to-end user experience while creating and deploying security solutions. Although as an industry, we’ve made significant progress, we think the security industry as a whole has been behind on this – just look at the plethora of security solutions in the industry that treat user experience as an afterthought! Security is critical, but it doesn’t mean we have an excuse to ignore user experience. We focused on simplicity and user experience when thinking about our end-to-end solution. Some of the concepts that helped orient our work on this included: 5.5.1.1 If it compiles, it compliesHow can we integrate security seamlessly with the engineering experience so it does not feel like a bolted-on solution but is instead part of the experience? For example, using security static analysis in build environments, we can define the rules that automate TCP verification as part of the compile and build processes. If all controls have been implemented accurately, the code just complies cleanly. It’s important to note that this is an aspiration state. We don’t believe we are yet at a point where all known security issues can be effectively detected during build in an automated capacity.

35

5.5.1.2 The secure way should be an easy wayAlways look for opportunities to define the secure way as an easy way. For example, using automation is the easy way. Running Fortify in build environments instead of employing a manual assessment saves weeks of time. And instead of hiring pen testers to test an app over several weeks, periodic WebInspect scans can save time and resources. User experience is a journey and we are still working toward achieving the optimal end-to-end user experience for our process. If engineers are treated like customers and if they are engaged during tool evaluations, process redesign, and overall user experience discussions, we believe this can yield very powerful results.

5.5.2 Taking security to engineers5.5.2.1 Fortify integration with IDEOne of the reasons engineers value Fortify is that it is in their Integrated Development Experience (IDE); namely, Visual Studio (VS). Visual Studio is where engineers spend most of their time, and having security plugged right into the environment is very convenient for them to use and interact with. They can view the results of the scan in their familiar interface and can triage and collaborate by responding to the issues within the same UI. The following is a screenshot taken from our training material which we use to articulate the value of Fortify to our engineering teams.

FIGURE 15 FORTIFY PLUGIN FOR VISUAL STUDIO

36

5.5.2.2 Fortify build-integrationBy integrating Fortify in the build process, we took another step to take security to engineers directly. This integration promotes transparency and helps security become even more a part of the overall engineering experience. 5.5.2.3 Reducing unnecessary security questionnaireWe looked closely at the SDL process and the underlying security attestation questionnaire that engineering teams complete for their application. We uncovered many duplicate and unnecessary questions and saw an opportunity to reduce noise by keeping only the questions we really need. For example, when we evaluated our initial risk assessment process, it had grown to an alarming 93 questions just to determine the initial risk score. Those 93 questions spanning two questionnaires were reduced to two questions in one questionnaire. The end result was an efficient and quicker security process which reduces friction even further.

We went a step further by also making the questionnaire intelligent. When the engineering team opts to run static and dynamic security tools, certain control questions (that validate the implementation of TCPs) are automatically hidden, which makes the questionnaire shorter and smarter, and therefore, enhances the user experience.5.5.2.4 Reducing unnecessary security and privacy

requirementsSimilar to reducing security friction from a process point of view, we continuously look for ways to ensure that our TCP set (security and privacy requirements) are refreshed and streamlined on an ongoing basis. During the last process enhancement we did in ISRM, we were able to reduce 106 varying technical requirements (TCPs) to 73 requirements.5.5.2.5 Reducing time to engage securityHow easy is it to engage your security team? What is the average time it takes to engage the security team for the organization? These were the questions we asked when we looked at the very first step of our process: engagement. We realized that our initial intake process, from discovering a new application to giving them SDL guidance, could take several days. It was obvious that this had to drastically change. We discarded the interim processes and a tool set that weren’t needed or that didn’t add any value and reduced the overall time to initiate the SDL process by 83%. We enabled our engineers to self-provision the SDL process without waiting for the central team to provision one for them.

There is much more work we need to do as we integrate security more closely to engineering systems and processes. One of the big initiatives we are driving is evaluating how we can use Visual Studio Team Systems (VSTS) work items to drive security activities more effectively and efficiently.

6 Future of application security The scope of application security has to be redefined. Traditionally, application security was seen as the practice of protecting the application code. In the modern engineering era, this is an extremely limited view. The same set of engineers who

37

write the code have to be concerned with operational practices, host configuration, and network in addition to the code. In light of this, we are repositioning ourselves from virtual silos of “application security,” “infrastructure security,” and “network security,” to a single unit focused on “Secure Engineering.” The details of the strategy behind this shift is outside the scope of this paper, but it’s important to emphasize that we believe modern engineering practices such as those described in this paper require a shift toward holistic thinking about security engineering.

7 Lessons learnedWe have learned so much from our engineering teams and from our partnerships – and we have a lot more yet to learn. The following are some of the more important lessons we’ve learned so far.

7.1 Partner with engineers It’s critical to partner with your engineers. They need to be stakeholders in the process and not just consumers of the process. When you evaluate your tools, don’t evaluate just within your security team but extend it to the engineers. Pay special attention to what is working for them and what isn’t. They can offer important feedback on not only your process and requirements but also can identify gaps to share with your tool vendor to make the tool work even better for your engineers. Engineers need to love the tools even more than security does.

7.2 Focus on the willingAny security process relies on its participants to be successful. Adopting new methods and processes is not an easy endeavor, and it’s important to focus on the willing. Listen to your engineers who are positive and willing to adopt robust strategies for your security program and focus on their needs. As more engineers engage with the process and become stakeholders, others will follow.

7.3 Be thoughtful about selecting technologyBecause automation is such a significant part of our strategy to secure modern engineering, we learned key lessons along the way about evaluating and licensing third-party technology that are worth sharing. First, technology is not a one-size-fits-all solution, nor is it a quick fix in terms of creating a security process. Second, be mindful about customizing any tool to work for your engineering experience and don’t merely throw it over the wall to your engineers. Involve them in your vetting of any new third-party tool and listen to what works for them and what doesn’t before you make any decisions about onboarding a tool.

7.4 Build your process first, then focus on toolsIt’s a common mistake to jump right into technology. It’s tempting to think that a tool can solve all your security problems. The reality is that you need to have your process built first and then evaluate tools to support that process with the right metrics. The main reason is that it’s rare that you’ll need just one piece of

38

technology. If you are not conscientious about how you bring in technology to support your process, every new technology you introduce could end up corrupting your process and leave you with a Frankenstein-like end result.Avoid an overly architected, rigid process. In fact, the nimbler the process the better. For example, have at least your knowledge base well defined so you know exactly what you are evaluating and how it will help you streamline your process in driving software security assurance.

7.5 Integrate your tools into the engineers’ worldIf it complies, it compiles. If you start with this as your North Star, you quickly discover that you must bring security right into the engineers’ world – not the security world! This not only helps to reduce friction for your engineers, it also enables them to move faster by integrating your tools into the world in which they live and operate. If they don’t need to exit their IDE to use the tool, they are more likely to use the tool and tool fatigue is less of a road block. This is our aspiration because there are security issues, such as poor authorization controls or design issues, that tools have a very hard time detecting. Nevertheless, it’s an aspiration worth driving towards – automating the detection of all those security issues that can be automated.

7.6 Build a relationship with your vendor A tool that meets your needs today may not be able to keep pace with the needs of tomorrow. We didn’t want to just license a tool for our security needs, but instead wanted to develop a relationship our tool vendor. To this day, we maintain a very open dialogue with HPE Fortify not only about how we use their security products, but also about what we would like to see to meet the ever-changing needs of our engineers. These could include obvious needs such as supporting newer technologies or frameworks. Another advantage of this relationship has been the opportunity to test features or new technology that HPE Fortify has developed in answer to other customer’s requirements.

7.7 Be mindful of business impactKeeping business impact always in perspective may seem obvious, but it’s not always the easiest to practice. Security solutions exist to enable business to achieve its goals in a secure and trusted manner. If security does not affect the business in a positive meaningful way, then it’s going to be an irritating speed bump that runs the risk of becoming a secondary consideration. The easiest approach we found was to invite your business stakeholders (the engineering teams) into the discussion. This discussion should include not just the processes and technology, but the development of metrics such as mean-time-to-triage and mean-time-to-fix to measure success. If the business can align to your metrics and agree that driving to defined targets will help create an impact, you’ve got a natural win-win situation.

39

7.8 Keep up with changing technologyTechnology is incredibly fickle. Not only is it important to maintain your vendor relationships to continue to drive your requirements, but it’s important to constantly look at new players coming into the field who may have more novel solutions. Additionally, once you’ve built your security processes, take care not to become complacent with respect to changing technology. As important as a robust security process is, it’s equally important to keep an eye on the horizon of emerging technology, and to be aware that your security model may need to continually adapt to new innovations in the industry.

8 Conclusion The modern engineering world presents ongoing challenges for us as we adapt to a DevOps model while also navigating the changing security landscape. Our approach has been to enable our engineers with security solutions that reduce friction as much as possible. We want to improve and enhance the engineering experience for our engineers with respect to software security assurance. We feel very comfortable with our iterative approve on our solution because this is very much the agile way – to jump in and iterate on the solution over time. We’ve done the heavy lifting already of creating the building blocks for our foundational components of knowledge management, automation, metrics, and user experience. Making these strategic shifts was the difficult part, and now we can build on that momentum. In addition to the challenges we talked about, we also see big opportunities for advancing application security in this age of modern engineering. There has never been a better time to push security automation and develop integrated security services for engineering teams as they think about operating in a DevOps environment and push for CI/CD scenarios. Similar to how development, test, and operation roles have merged to shape today’s modern engineer, we feel that a software security assurance program can yield much better results than before if the processes are baked seamlessly into the engineering process. Security teams should leverage this momentum of automation to further expand the scope and coverage in their organization in an effective and efficient manner.

40

9 Appendix A: Resources9.1.1 SDLSecurity Development Lifecyclehttps://www.microsoft.com/sdlApplication security in the changing risk landscapehttps://f5.com/Portals/1/PDF/security/f5-ponemon-report-2016.pdfThe Security Development Lifecycle: SDL: A Process for Developing Demonstrably More Secure Software (Developer Best Practices)https://www.amazon.com/Security-Development-Lifecycle-Developing-Demonstrably/dp/0735622140/ref=sr_1_1?ie=UTF8&qid=1473087879&sr=8-1&keywords=security+development+lifecycle

9.1.2 Modern engineering and DevOpsCreating a culture of modern engineering within Microsoft IThttps://msdn.microsoft.com/en-us/library/mt709101.aspxDevOps—Application Lifecycle Management – Microsofthttps://www.microsoft.com/en-us/cloud-platform/development-operationsThe science of DevOps decodedhttp://www.gartner.com/smarterwithgartner/the-science-of-devops-decoded/

41