DevOpsDays SLC - Getting Along With Your DBOps Team

Endurance International Group

Getting Along with Your DBOps TeamSLC DevOpsDays@nickdemaster

Why do we care?

@nickdemaster #slcdevopsdays

Why do we care?

Your data is super important!

What does the DB Ops team care about?(and how does that apply to dev)


• Atomicity

ACID


ACID

• Atomicity• Atomicity requires that each transaction be "all or nothing": if one part of the transaction fails,

then the entire transaction fails, and the database state is left unchanged. An atomic system must guarantee atomicity in each and every situation, including power failures, errors, and crashes. To the outside world, a committed transaction appears (by its effects on the database) to be indivisible ("atomic"), and an aborted transaction does not happen.


ACID

DevOps Translation:Each code deployment involving a database must be successful and be progressive, not regressive.In the event of a failure in the code deployment, the rollback must be transparent to the user.

• Atomicity• Atomicity requires that each transaction be "all or nothing": if one part of the transaction fails,

then the entire transaction fails, and the database state is left unchanged. An atomic system must guarantee atomicity in each and every situation, including power failures, errors, and crashes. To the outside world, a committed transaction appears (by its effects on the database) to be indivisible ("atomic"), and an aborted transaction does not happen.


ACID

• Atomicity• Consistency


• Consistency• The consistency property ensures that any transaction will bring the database from one valid

state to another. Any data written to the database must be valid according to all defined rules, including constraints, cascades, triggers, and any combination thereof. This does not guarantee correctness of the transaction in all ways the application programmer might have wanted (that is the responsibility of application-level code) but merely that any programming errors cannot result in the violation of any defined rules.

ACID


• Consistency• The consistency property ensures that any transaction will bring the database from one valid

state to another. Any data written to the database must be valid according to all defined rules, including constraints, cascades, triggers, and any combination thereof. This does not guarantee correctness of the transaction in all ways the application programmer might have wanted (that is the responsibility of application-level code) but merely that any programming errors cannot result in the violation of any defined rules.

DevOps Translation:Understand the underlying data you are changing with each code deploy. Make sure that the datastructures you use or change are consistent with the ones currently in place. Keep data types for similardata the same. i.e. If you use database datestamps, don’t switch over to unix timestamps, unless part of a larger initiative.

ACID


• Atomicity• Consistency• Isolation

ACID


ACID

• Isolation• The isolation property ensures that the concurrent execution of transactions results in a system

state that would be obtained if transactions were executed serially, i.e., one after the other. Providing isolation is the main goal of concurrency control. Depending on the concurrency control method (i.e., if it uses strict - as opposed to relaxed - serializability), the effects of an incomplete transaction might not even be visible to another transaction.


DevOps Translation:When doing code deploys, always be on the same page with the way the data is going to bechanged with each deploy. This is most pertinent if you have multiple projects or initiatives going onthat use the same database model. i.e. If you need a column for X and someone else needs the samecolumn for Y, be sure that you don’t crush someone else’s (or your own) changes with an out of order alter/CRUD action.

ACID

• Isolation• The isolation property ensures that the concurrent execution of transactions results in a system

state that would be obtained if transactions were executed serially, i.e., one after the other. Providing isolation is the main goal of concurrency control. Depending on the concurrency control method (i.e., if it uses strict - as opposed to relaxed - serializability), the effects of an incomplete transaction might not even be visible to another transaction.


• Atomicity• Consistency• Isolation• Durability

ACID


• Durability• The durability property ensures that once a transaction has been committed, it will remain

so, even in the event of power loss, crashes, or errors. In a relational database, for instance, once a group of SQL statements execute, the results need to be stored permanently (even if the database crashes immediately thereafter). To defend against power loss, transactions (or their effects) must be recorded in a non-volatile memory.

ACID


DevOps Translation:Understand the mechanics of the durability property of the database. If you are not using a truly ACIDdatabase, you may have to program in checks to assure the data is stored and retrievable correctly –even in the event of a connection or system failure.

ACID

• Durability• The durability property ensures that once a transaction has been committed, it will remain

so, even in the event of power loss, crashes, or errors. In a relational database, for instance, once a group of SQL statements execute, the results need to be stored permanently (even if the database crashes immediately thereafter). To defend against power loss, transactions (or their effects) must be recorded in a non-volatile memory.

In Practice – Schema


In Practice – Database Schema

• Know your data! (love your data)



• Know your data! (love your data)• If your database uses them, Explicit Primary Keys



• Know your data! (love your data)• If your database uses them, Explicit Primary Keys• Correctly sized data fields



• Know your data! (love your data)• If your database uses them, Explicit Primary Keys• Correctly sized data fields• Be able to forecast growth patterns



• Know your data! (love your data)• If your database uses them, Explicit Primary Keys• Correctly sized data fields• Be able to reasonably forecast growth• ConSistentNaming Conventions



• Know your data! (love your data)• If your database uses them, Explicit Primary Keys• Correctly sized data fields• Be able to reasonably forecast growth• ConSistentNaming Conventions• Understand storage conventions of the database engine

(i.e. ACID vs eventual consistency)


• Know your data! (love your data)• If your database uses them, Explicit Primary Keys• Correctly sized data fields• Be able to reasonably forecast growth• ConSistentNaming Conventions• Understand storage conventions of the database engine

(i.e. ACID vs eventual consistency)• Bonus: example queries with explain paths


In Practice – DBA


In Practice – DBA

• Continuous deployment of code, have a system/agreement in place


In Practice – DBA

• Continuous deployment of code, have a system/agreement in place• Audit/Change Management (and if possible roll-forward, roll back)


In Practice – DBA

• Continuous deployment of code, have a system/agreement in place• Audit/Change Management (and if possible roll-forward, roll back)• Data Migrations plans – cause and effect analysis (application/system)


In Practice – DBA

• Continuous deployment of code, have a system/agreement in place• Audit/Change Management (and if possible roll-forward, roll back)• Data Migrations plans – cause and effect analysis (application/system)• Rapid development infrastructure (development > staging > production)


• Continuous deployment of code, have a system/agreement in place• Audit/Change Management (and if possible roll-forward, roll back)• Data Migrations plans – cause and effect analysis (application/system)• Rapid development infrastructure (development > staging > production)• Backups and restores – TEST OFTEN

In Practice – DBA

Getting Along with your DB Ops Team



• Buy into the same goals



• Buy into the same goals• Communicate early and often



• Buy into the same goals• Communicate early and often• Be ready to compromise



• Buy into the same goals• Communicate early and often• Be ready to compromise• Be able to define stakeholders for each data release



• Buy into the same goals• Communicate early and often• Be ready to compromise• Be able to define stakeholders for each data release• Be accountable (bonus: be on pager duty)


• Buy into the same goals• Communicate early and often• Be ready to compromise• Be able to define stakeholders for each data release• Be accountable (bonus: be on pager duty)• TEAM


I am the DB Ops team (and the dev team.. and the ops team… and the printer person...)


• Explain plans

Troubleshooting tips


• Explain plans• Mongo: db.collection.find().explain()



• Explain plans• Mongo: db.collection.find().explain()• MySQL/Postgres: EXPLAIN select ...




• Explain plans• Monitor Monitor Monitor (alert alert alert)



• Explain plans• Monitor Monitor Monitor (alert alert alert)• Practice your deploys on a staging environment


• Explain plans• Monitor Monitor Monitor (alert alert alert)• Practice your deploys on a staging environment• Automation is key!


Endurance International Group

Getting Along with Your DBOps TeamSLC DevOpsDays

@nickdemaster

Technology

DevOpsDays SLC - Getting Along With Your DBOps Team