Upload
finratech
View
162
Download
3
Tags:
Embed Size (px)
Citation preview
Using Jenkins Enterprise to effectively manage the Jenkins ecosystem across the Enterprise
Sarah Banks, Kostas Gaitanos, Raj Menattamai
Footer
#jenkinsconf About FINRA www.finra.org
2
Financial Industry Regulatory Authority
Ø Largest independent regulator for all securi2es firms doing business in the U.S.
Ø >4,000 brokerage firms
Ø ~161,000 branch offices
Ø ~637,700 registered securi2es representa2ves
Our Mission: Ø Investor Protec2on Ø Market Integrity
Our Technology is powerful enough to look across markets and detect poten2al fraud.
We oversee 50 billion market transac2ons every day
In 2014 , we expelled 18 firms from the industry, suspended 705 brokers and barred 481 brokers from doing business.
We also fined firms more than $134 million, and ordered res2tu2on of $32.3 million to investors who had been harmed.
Computerized cer:fica:on & con:nued educa:on.
Series 7, 63…etc.
Educating & informing investors
Actively supporting firms’ compliance efforts
Providing independent, vigorous regulation
Inviting active industry involvement
& input
Footer
#jenkinsconf Metrics for Jenkins Enterprise based ecosystem activities
# of Apps # of Folders # of Jobs # of Builds/yr (estimate)
# of Deployments/yr
(estimate) ~110 ~610 ~1,400* ~100K
promotable builds
~75K (across all envs)
3
* # of jobs active over the past quarter; valid jobs requiring maintenance ~ 2.5K
>100 plugins currently utilized
Footer
#jenkinsconf Discussion areas
4
• Enable deployments to higher environments via the build pipeline – Development have the ability to deploy to all lower environments
including QC – Enable Prod Deployment group to execute production deployments
using the exact same approach as for the lower environments. • Treat all of Jenkins build-deploy infra as software/source
code – Automatic creation of all infra (i.e folders, jobs, views/pipelines) – Use the same framework to create all that make up an application
Footer
#jenkinsconf Enable deployments to higher environments via the build pipeline
6
Folder Template when instantiated will create:
v Deployment jobs for lower environments • Job template to create standard XL Deploy deployment job *
v Deployment job for QA with a child job “promote_to_prod” • Keep the build forever • Job template to create standard XL Deploy deployment job AND a child job *
v Deployment job for Prod • Job template to create a Prod specific XL Deploy deployment job *
v PROD folder with a Prod Deployment job § RBAC is set § promote_to_prod & prod_deploy jobs are created *
* jobs are created based on corresponding job templates
Footer
#jenkinsconf Folder template groovy code handling RBAC
7
• <com.cloudbees.hudson.plugins.folder.Folder plugin="[email protected]"> • <actions/> • <description></description> • <properties> • <com.cloudbees.jenkins.plugins.foldersplus.SecurityGrantsFolderProperty plugin="[email protected]"> • <securityGrants/> • </com.cloudbees.jenkins.plugins.foldersplus.SecurityGrantsFolderProperty> • <com.cloudbees.hudson.plugins.folder.properties.FolderCredentialsProvider_-FolderCredentialsProperty> • <domainCredentialsMap class="hudson.util.CopyOnWriteMap\$Hash"> • <entry> • <com.cloudbees.plugins.credentials.domains.Domain plugin="[email protected]"> • <specifications/> • </com.cloudbees.plugins.credentials.domains.Domain> • <java.util.concurrent.CopyOnWriteArrayList/> • </entry> • </domainCredentialsMap> • </com.cloudbees.hudson.plugins.folder.properties.FolderCredentialsProvider_-FolderCredentialsProperty> • <com.cloudbees.hudson.plugins.folder.properties.EnvVarsFolderProperty plugin="[email protected]"> • <properties></properties> • </com.cloudbees.hudson.plugins.folder.properties.EnvVarsFolderProperty> • <com.cloudbees.hudson.plugins.folder.properties.SubItemFilterProperty plugin="[email protected]"/> • <com.cloudbees.hudson.plugins.folder.properties.FolderProxyGroupContainer plugin="[email protected]"> • <groups> • <nectar.plugins.rbac.groups.Group> • <name>PPL_ROLE_TEC_ART_LOCAL</name> • <member>PPL_ROLE_TEC_ART</member> • <role>develop_prod</role> • </nectar.plugins.rbac.groups.Group> • </groups> • </com.cloudbees.hudson.plugins.folder.properties.FolderProxyGroupContainer> • </properties>
Footer
#jenkinsconf Treat all of Jenkins build-deploy infra as software/source code
8
The goal is to create all of our build-deploy infrastructure automatically: Ø Folders Ø Jobs Ø Views/Pipelines
v Create a single framework ( groovy job DSL) for all application-components that
make an application
v All of the application related info (i.e folders, environments, pipeline views, components, parameters for plugins used etc…) is captured in a JSON file
v JSON file is the input to the framework, as the framework effectively becomes the “seed” job; when “seed” job runs it will create all of folders, jobs views/pipelines for the application
Footer
#jenkinsconf
9
Properties JSON "AGS": { "scm": "svn", "name": "ENTERPRISEDESKTOP", "rootFolder": "FLDR_ENTERPRISEDESKTOP_DSL_DEMO", "DevQCDeployJobFolder":FLDR_ENTERPRISEDESKTOP_DSL_DEMO/deployment_jobs", "DevQCDeployJobFolderTemplate": "FLDR_DEPLOY_JOBS_use_with_Job_DSL", "ProdDeployJobFolder": "FLDR_ENTERPRISEDESKTOP_DSL_DEMO/deployment_jobs/PROD_JOBS", "ProdDeployJobFolderTemplate": "FLDR_DEPLOY_JOBS_use_with_Job_DSL/PROD_JOBS",
"DevQC": [ { "name": "DEV_deploy" }, : :
JobDSLmethods (class Folders { static void createParentFolders(def asgProperties, def AGS, def AGSFolder,…) { Folders.createAGSFolder(AGSFolder, AGS)
: : }
static void createAGSFolder(def folder, def AGS) { folder.with { :
: : }
JobDSL Framework import jobdslmethods.enterprise.folder.* import groovy.json.JsonSlurper : : Folders.createParentFolders(asgProperties, AGS, AGSFolder, DeploymentJobFolder, DeploymentJobProdFolder) :
Footer
#jenkinsconf Metrics discussion
10
Template Plugins Manual setup of pipeline * 5 mins 30 mins
~4 hrs
Job DSL & Template Plugins
Manual
Jenkins infra as code^
20 mins 4-5 hours
* Times are per job; on an average each app contains 12-13 jobs
~ 1week upfront investment to design/implement
^ metrics captured based on applications with ~20 jobs/pipelines ~3 weeks upfront investment to design/implement