Operating a High Velocity Large Organization with Spring Cloud Microservices

Unless otherwise indicated, these s l ides are © 2013-2016 Pivota l Software , Inc . and l icensed under a Creative Commons Attr ibution-NonCommerc ia l l i cense:

http:/ /c reativecommons .org/ l icenses/by-nc/3.0/

Operating a High Velocity Large Organization with Spring Cloud

MicroservicesNoriaki (Nori) Tatsumi

Capital One



Who we are

2



Data LakePowered by C1 Data Intelligence Team

CC Image by Stanislav Sedov on Flickr


http:/ /c reativecommons .org/ l icenses/by-nc/3.0/4

Continuous Delivery

Build, test, and release fast and frequently to operate high velocity organization

Prerequisite. Not luxury.



Continuous delivery principles• Eliminate non-value added actions• Release process must be repeatable and reliable• Quality is built in• Version everything• Done = “Released”• Small batches of features and experimentations• Everyone is responsible• Kaizen – Improve continuously



Our challengesCategory Examples

Complex systems Large code base and deploymentImplementation sensitivityTest feedback speed

Many teams Collaboration of design & technologyMerge conflicts

Non functional qualities SecurityHigh availability, reliability, maintainabilitySame qualities across components/features

Compliance TraceabilityData lineage

Legacy and 3rd party applications

Implementation of same qualities as the rest

Processes Time consuming reviews and approvals



Customer expectation• Quality• Availability• Velocity - fast and frequent deliveries of features

But they don’t know about the technical challenges



Microservices architecturehttp://martinfowler.com/articles/microservices.html

The twelve factor methodologyhttp://12factor.net/



Intranet/VPN





Spring CloudFor foundation of microservices architecture



Spring Cloud• *Spring Cloud Config• *Spring Cloud Netflix• Spring Cloud Bus• Spring Cloud for Cloud Foundry• Spring Cloud Cloud Foundry

Service Broker• Spring Cloud Cluster• Spring Cloud Consul• *Spring Cloud Security• *Spring Cloud Sleuth• Spring Cloud Data Flow

• Spring Cloud Stream• Spring Cloud Stream Modules• Spring Cloud Task• Spring Cloud Zookeeper• Spring Cloud for Amazon Web

Services• Spring Cloud Connectors• Spring Cloud Starters• Spring Cloud CLI



Technology selection• JVM-based

• Developer productivity• Production support

• Spring Boot• Opinionated view for developer productivity• Production grade qualities

• Netflix OSS• Proven microservices technology



Decoupling• Work in parallel• Quicker and smaller deploys• Domain driven design• Do one scope of things well



Decoupling• Work in parallel• Quicker and smaller deploys• Domain driven design• Do one scope of things well• Technology stack agnostic



Decoupling• Work in parallel• Quicker and smaller deploys• Domain driven design• Do one scope of things well• Technology stack agnostic• Functions are shareable



Decoupling• Work in parallel• Quicker and smaller deploys• Domain driven design• Do one scope of things well• Technology stack agnostic• Functions are shareable• Independent scalability by comp.• Greater resiliency & availability• Continuous delivery friendly



This looks hard to orchestrate



Service discovery (registry)• Netflix Eureka (HashiCorp Consul, Apache Zookeeper)• Locate services• Load balancing• Failover• Resiliency



<application> <name>...</name> <instance> <instanceId>... </instanceId> <hostName>... </hostName> <app>...</app> <ipAddr>...</ipAddr> <status>UP</status> <overriddenstatus>UNKNOWN</overriddenstatus> <port enabled="false">...</port> <securePort enabled="true">...</securePort> <countryId>1</countryId> <dataCenterInfo class="com.netflix.appinfo.AmazonInfo"> <name>Amazon</name> <metadata> <accountId>...</accountId> <local-hostname>... </local-hostname> <instance-id>...</instance-id> <local-ipv4>...</local-ipv4> <instance-type>...</instance-type> <vpc-id>...</vpc-id> <ami-id>...</ami-id> <mac>...</mac> <availability-zone>...</availability-zone> </metadata> </dataCenterInfo> <leaseInfo> <renewalIntervalInSecs>...</renewalIntervalInSecs> <durationInSecs>...</durationInSecs> …..

DEMO TIME!



Service discovery (registry)@SpringBootApplication@EnableEurekaServerpublic class Discovery {

public static void main(String[] args) { SpringApplication.run(Discovery.class, args);}

}



Service discovery (registry)eureka:

instance: appname: discovery app-group-name: bedrock lease-renewal-interval-in-seconds: 30 #sending heartbeats; 90 secs removes from registry eureka.instance.preferIpAddress: false home-page-url: http://${spring.cloud.client.hostname}:${server.port}/discovery health-check-url: http://${spring.cloud.client.hostname}:${management.port}${management.context-path}/health status-page-url: http://${spring.cloud.client.hostname}:${management.port}${management.context-path}/info password: changeme dashboard: path: /discovery client: serviceUrl: defaultZone: http://user:changeme@peer2:8761/eureka/ healthcheck: enabled: true



Making your Spring Boot app discoverable

@SpringBootApplication@EnableDiscoveryClientpublic class Application {

public static void main(String[] args) { SpringApplication.run(Application.class, args);}

}




protected static void registerShutdownHooks() {Runtime.getRuntime().addShutdownHook(new Thread() { @Override public void run() { LOG.info("Shutting down, unregister from discovery service!"); DiscoveryManager.getInstance().shutdownComponent(); }});…..

}




@Bean@Profile("aws")public EurekaInstanceConfigBean eurekaInstanceAwsConfig(InetUtils inetUtils) {

LOG.info("Configuring this instance to be Amazon aware...");EurekaInstanceConfigBean config = new EurekaInstanceConfigBean(inetUtils);AmazonInfo info = AmazonInfo.Builder.newBuilder().autoBuild("eureka");config.setDataCenterInfo(info);return config;

}

Tip: Some Spring Cloud default configurations such as the Eureka instance ID and the Eureka instance port doesn’t take effect when customizing EurekaInstanceConfigBean. Set them explicitly.



Making your non-Spring Boot app discoverableEureka Client (Java)https://github.com/Netflix/eureka/wiki/Understanding-eureka-client-server-communication

Eureka REST API (Polyglot)https://github.com/Netflix/eureka/wiki/Eureka-REST-operationsNote: Omit “/v2” in URI

Sidecar/App Gateway (Polyglot)Sits next to your app. Enables applications to be service discoverable and secure without code modification.

https://github.com/Netflix/eureka/wiki/Understanding-eureka-client-server-communication

https://github.com/Netflix/eureka/wiki/Understanding-eureka-client-server-communication

https://github.com/Netflix/eureka/wiki/Eureka-REST-operations

https://github.com/Netflix/eureka/wiki/Eureka-REST-operations



Service discovery clients• Netflix Eureka Client• Spring Cloud

• Feign• Spring RestTemplate

• Any HTTP client



Spring Boot Admin with Eureka

https://github.com/codecentric/spring-boot-adminDEMO TIME!



Spring Boot Admin with Eureka@SpringBootApplication@EnableAutoConfiguration@EnableDiscoveryClient@EnableAdminServerpublic class Admin extends BaseSpringBootApplication { public static void main(String[] args) { SpringApplication.run(Admin.class, args); }

}



Spring Boot Admin with Eureka@Componentpublic class EurekaApplicationDiscoveryListener extends ApplicationDiscoveryListener { @Autowired public EurekaApplicationDiscoveryListener(DiscoveryClient discoveryClient, ApplicationRegistry registry) { super(discoveryClient, registry); ServiceInstanceConverter converter = new DefaultServiceInstanceConverter() { @Override public Application convert(ServiceInstance instance) { EurekaDiscoveryClient.EurekaServiceInstance eurekaServiceInstance = (EurekaDiscoveryClient.EurekaServiceInstance) instance; final Application temp = super.convert(instance); final Application converted = Application.create(temp) .withHealthUrl(eurekaServiceInstance.getInstanceInfo().getHealthCheckUrl()) .withManagementUrl(eurekaServiceInstance.getInstanceInfo().getStatusPageUrl().replaceFirst("/info", "")) .withServiceUrl(eurekaServiceInstance.getInstanceInfo().getHomePageUrl()) .build(); return converted; } }; setConverter(converter); }}



Spring Boot Admin with Component Auth

@Configuration@AutoConfigureAfter({RevereseZuulProxyConfiguration.class})protected static class ExtendedZuulProxyConfiguration extends ZuulConfiguration { @Bean public ComponentAuthEnrichFilter componentAuthEnrichFilter() { return new ComponentAuthEnrichFilter(); }

}



Centralized monitoring@Componentpublic class MetricsShipper { ….. public JSONObject composeMetrics() { JSONObject metricsJson = new JSONObject(); metricsJson.put("timestamp", System.currentTimeMillis()); metricsJson.put("application", appName); metricsJson.put("instance", eurekaInstanceConfig.getInstanceId()); metricsJson.put("status", healthEndpoint.invoke().getStatus().getCode()); Map<String, Object> metrics = metricsEndpoint.invoke(); for (String metricKey : metrics.keySet()) { metricsJson.put(metricKey.replaceAll("[.]", "-"), metrics.get(metricKey)); } return metricsJson; } @Scheduled(fixedRateString = "${monitoring.shipper.kafka.fixedRate}", initialDelayString = "${monitoring.shipper.kafka.fixedRate}") public void ship() { kafkaTarget.ship(composeMetrics()); }}



Distributed tracing

http://cloud.spring.io/spring-cloud-sleuth/spring-cloud-sleuth.html



Distributed tracing<dependency> <groupId>org.springframework.cloud</groupId> <artifactId>spring-cloud-starter-sleuth</artifactId></dependency>

service1.log:2016-02-26 11:15:47.561 INFO [service1,2485ec27856c56f4,2485ec27856c56f4,true] 68058 --- [nio-8081-exec-1] i.s.c.sleuth.docs.service1.Application : Hello from service1. Calling service2

-----server.tomcat.accesslog.pattern=%h %l %u %t "%r" %s %b %D %I "%{x-b3-

spanid}i" "%{x-b3-traceid}i" "%{x-b3-parentspanid}i”

127.0.0.1 - - [24/May/2016:02:06:44 -0400] "POST /elasticsearch/_msearch?ignore_unavailable=true&preference=1464070003866&timeout=0 HTTP/1.1" 200 148 7 http-nio-8444-exec-5 "27c37d6638ab6c87" "150e347f81964ffd" "150e347f81964ffd"



Distributed tracing



Security*Across all components in various languages• SSO• User authorization• Component to component authentication and authorization• CORS• Appropriate routing• Auditing and logging• Insights and inspection• Rate limiting• A well known entry to platform



Intranet/VPN



Edge gateway@SpringBootApplication@EnableAutoConfiguration@EnableZuulProxypublic class EdgeGateway {

public static void main(String[] args) throws Exception { SpringApplication.run(EdgeGateway.class, args);}

}



Edge gateway - Routingzuul.ignoredServices=*

zuul.routes.root.path=/zuul.routes.root.url=forward:/redirectzuul.routes.app1.path=/app1/**zuul.routes.app1.serviceId=app1zuul.routes.app1.stripPrefix=falsezuul.routes.app1.sensitive-headers=Cookie,Set-Cookie

zuul.routes.app2.path=/app2/**zuul.routes.app2.url=https://app2:8443



Edge gateway - Authentication• SAML 2.0

Sample: https://github.com/vdenotaris/spring-boot-security-saml-sample

• OAuth 2.0Sample: https://github.com/royclarkson/spring-rest-service-oauth

https://github.com/vdenotaris/spring-boot-security-saml-sample



https://github.com/royclarkson/spring-rest-service-oauth





Edge gateway - Filter@Componentpublic class ComponentAuthEnrichFilter extends ZuulFilter { …. @Override public String filterType() { return FilterType.pre.name(); } @Override public int filterOrder() { return FilterOrder.BASIC_AUTH_ENRICH_PRE_FILTER.getOrder(); } @Override public boolean shouldFilter() { return isFilterEnabled; } @Override public Object run() { RequestContext ctx = RequestContext.getCurrentContext(); ctx.addZuulRequestHeader("Authorization", authorizationHeaderValue); return null; }}

• Enrich requests• Inspect requests• Collect stats• Redirect• Etc.



Sidecar / app gateway• Discover service registry• Authentication and authorization• Inspect compliance• Auditing and logging• Distributed tracing• Health check• Monitoring



Sidecar / app gateway

DEMO TIME!



Sidecar / app gateway (@EnableSidecar)

server: port: 5678spring: application: name: sidecarsidecar: port: 8000 health-uri: http://localhost:8000/health.json



Sidecar / app gateway (Custom @EnableZuulProxy)@Componentpublic class SidecarHealthIndicator extends AbstractHealthIndicator { @Override protected void doHealthCheck(Health.Builder builder) throws Exception { if (AppHeartbeatCheck.isHealthy()) { builder.up(); } else { builder.outOfService().withDetail("Failed attempts",AppHeartbeatCheck.getFailedAttempts()); } }}-----------sidecar.healthcheck.fixedRate=5000sidecar.healthcheck.maxAttempts=3sidecar.healthcheck.url=http://localhost:5601sidecar.healthcheck.expectedResponseCode=200



Circuit breaker• Help reduce resources tied up in operations

which are likely to fail with fallback• Avoid waiting on timeouts for the client• Avoid putting loads on a struggling server• Zuul uses Netflix Hystrix

http://martinfowler.com/bliki/CircuitBreaker.html



Circuit breaker@SpringBootApplication@EnableCircuitBreakerpublic class Application { public static void main(String[] args) { new SpringApplicationBuilder(Application.class).web(true).run(args); }}@Componentpublic class StoreIntegration { @HystrixCommand(fallbackMethod = "defaultStores") public Object getStores(Map<String, Object> parameters) { //do stuff that might fail } public Object defaultStores(Map<String, Object> parameters) { return /* something useful */; }}



Zuul gotchas• No sticky session for your legacy and 3rd party apps that might need it• WebSockets, Server-sent Events, HTTP2 not supported



VersioningArtifacts - Semantic versioning• MAJOR version when you make incompatible API changes• MINOR version when you add functionality in a backwards-compatible

manner• PATCH version when you make backwards-compatible bug fixes

Configurations - Git revision hash• All configurations applied via automation



Configuration• There are more configurations to manage than monolith• Spring Boot application properties (application.properties/yml)• Spring Cloud bootstrap context (bootstrap.properties/yml)

• Parent context for main app (loaded before application properties)• Loads configs from external properties (e.g. Spring Cloud Config

Server)• Encryption/decryption of sensitive properties

• Immutable infrastructure with automation



Other keys things…• DevOps culture• Good documentation for the microservices foundation for multiple

teams to use• Automate everything



Take away• It’s not hard to get started with microservices… if you use Spring

Cloud• Start delivering faster and deploy more frequently



Learn More. Stay Connected.

Noriaki Tatsumi

[email protected]

https://www.linkedin.com/company/capital-one

https://twitter.com/capitalonejobs

https://github.com/capitalone