Building block development in managed hosting - Angelo Rossi, Manager, Complex Hosting Services,...

Building Block Development in Managed Hosting

Disclaimer

I am not a developer!

2

Limitations vs. industry best practice

3

Flexibility during

development Testing

Change control during

rollout

QUALITY & STABILITY

Typical MH Client Setup

Silver/Gold/Platinum Diamond

Test Fresh install, single app+db setup, no production data

Test Fresh install, single app+db setup, no production data

Staging Multiple app servers(two), copy of the production data, shared infrastructure with prod environments

Production Multiple app servers Production Multiple app servers

4

Datacenter setup

• Test, Staging and Production environments share the same infrastructure including:– firewalls, switches, routers

– network filers

5

Network setup

• Incoming connections from the internet are blocked with the exception of ports 80 and 443 – no exceptions

• All outgoing connections are allowed

6

Policies

• No shell access to change settings, run scripts

• No cronjobs

• No shell access to restart the services

• No direct database access

7

No shell access

• Existing processes reviewed when moving to Managed Hosting

• More robust change management

• No hacks, they can backfire.

8

No database access

• SLA is the main reason

• Real “Enterprise” integrations, easier to monitor and detect performance issues and disable individual functionalities.

• More robust change control and maintenance: eg. the integrations don’t run if the system is down for maintenance.

9

Development

• Sounds complicated and limiting!

• How am I supposed to develop and test if I have nearly no access to the system?

10

Best Practice

• Limitations? We like to call it “Best Practice”!

11

Solution

• Let’s break down the various phases of the process:

• - development

• - testing

• - deployment

12

Development

• Requirements:

- Full access to the dev system: quick changes, restart services, monitor system resources and troubleshoot. Ideally no network lag and the possibility to take snapshots, revert to a fresh install etc..

- No need to reinvent the wheel: the Developer Virtual Machines are the best solution

- Possible limitations that should be taken into account are different network architecture and speed, and more in general limited app and db performance, however these should help detecting issues at an early stage

13

End of the development cycle

• The B2 or integration is working. All the requirements are fulfilled what’s next?

14

Testing

• All Managed Hosting clients have a Test environment for functional and integration testing.

• network setup is identical to production so firewall/DNS issues can be caught during this phase

• No production data but it is possible to create or import courses and create test user accounts and data.

15

Testing - troubleshooting

• Test is a single application server instance.

• Pros:– all logs can easily be retrieved via system admin > Tools and Utilities >

Logs

– Relatively limited hardware resources. Many performance issues will show up at this stage and be caught before rolling out

• Cons: “session spraying” issues might not be caught until the B2 is deployed to a multi app server system

16

How to install/upgrade a B2 to MH Test

• Administrators can do it from the web interface

• A restart might be required. Managed Hosting Support or your SDM can assist

17

Staging environment - optional

• Only for Diamond clients and clients that have bought an additional service

• copy of the production system data deployed on a multi-app servers setup architecturally similar to Production.

• Diamond clients have single point of contact for maintenance and project management, the Service Delivery Manager.

• Service Delivery Managers keep their clients’ production and staging systems in sync

18

Staging – Change Management

• This step has little to do with the development cycle

• The Service Delivery team works with the client for testing and documenting deployment and config

• This is for testing the change, not the code

• Clients’ business can test the integration with real data

19

Rollout

20

Production

• Deployment to prod can be done by the client or via a support ticket

• Don’t forget a rolling restart, Support will assist

21

Production

• Go-live!

22

Troubleshooting

• Logs!

• What’s different in Dev/Test/Staging?

23

Performance issues

• MH WILL make the B2 inactive in case of performance issues

• Thread dumps will be provided

• Production heap dump is ~12gb, let’s try to avoid that

24

Recommendations

• Connections timeout

• Write smart code

• Handle exceptions and log errors

• Limit database and external resources usage

25

Self hosted clients

• This model could also be used by non hosted clients:– Self contained applications

– No issues when cloning to non-production systems

– Easier maintenance

– Future-proof

26

Questions?

27