Database automated build and test - SQL In The City Cambridge

#sqlinthecity

David AtkinsonProduct Manager, Red Gate

David.Atkinson@red-gate.com

Automated Build and Test(or continuous integration)

(for your database)

What will we cover today?

Database build automation:What, Why, How

Practical Demo

Questions encouraged!

#sqlinthecity

Show of hands: How agile is your database development?

1. Database objects/scripts aren’t in version control

2. Database objects/scripts are in version control

3. An automated build system exists to build and test the database

4. An automated release system is used to promote your database to your environments

#sqlinthecity

1. Database objects/scripts aren’t in version control

2. Database objects/scripts are in version control

3. An automated build system exists to build and test the database

4. An automated release system is used to promote your database to your environments

How agile is your database development?

#sqlinthecity

“Continuous Integration is a practice designed to ensure that your software is

always working, and that you get comprehensive feedback in a few minutes as to whether any given change to your system

has broken it.”Jez Humble, ThoughtWorks,

author of “Continuous Delivery”

#sqlinthecity

What is continuous integration?

“Database Continuous Integration is a practice designed to ensure that your

database software is always working, and that you get comprehensive feedback in a

few minutes as to whether any given change to your system has broken it.”

What is continuous integration?

#sqlinthecitydatabase

^

Database Build Automation#sqlinthecity

What is build?

• For application code = compile• For database code = database creation

script – But only for a new installation!

• Upgrade scripts required for existing installations– Need to preserve the state of the data

#sqlinthecity

What is test?

• For .NET code, NUnit.– Runs on a developer’s machine and

build server• What about the database?– tSQLt is an open source framework for

testing SQL Server databases– SQL Test provides SSMS integration

#sqlinthecity

Demo background

Two fictional developers, David and Grant, are working on the website, www.simple-talk.com, a community website for .NET and SQL Server developers

The application comprises• An ASP.Net web application• A SQL Server database

#sqlinthecity

Demo background

Continuous integration has been configured to:

1. Keep a test database up to date with the latest changes

2. Run automated tSQLt tests(on realistic amounts of data)

3. Generate up-to-date documentation4. Generate deployment scripts

#sqlinthecity

Demo background

A few improvements are being made to Simple Talk.

These changes will be deployed to production.

But… only when the dev team has demonstrated that the changes work!

#sqlinthecity

Tools used• TeamCity (CI tool from JetBrains)• SQL Automation Pack (includes TeamCity

plugin)

• Glimpse (open source)

From the SQL Developer Bundle:• SQL Source Control• SQL Test (tSQLt unit testing framework)• SQL Data Generator• SQL Doc

#sqlinthecity

Demo

#sqlinthecity

Custom Migration ScriptsAn introduction

#sqlinthecity

Generating a deployment script

1

12

#sqlinthecity

How changes are made

1 2 3 4 5 6

modify procedure

create table

drop view

create trigger

delete column

#sqlinthecity

1 2 3 4 5 6

modify procedure

create table

drop view

create trigger

delete column

7 8 9 10 11 12

modify trigger

create view

create procedure

create function

drop procedure

modify function

How changes are made#sqlinthecity

Infer deployment script

How SQL Compare sees the world

1

12

#sqlinthecity

But not all changes can be inferred

1 2 3 4 5 6

7 8 9 10 11 12

table rename

column rename add NOT NULL

column without DEFAULT

split table

merge table

data transformation

data motion

merge columnsplit column

#sqlinthecity

SQL Compare and SQL Source Control support custom migration scripts

1 2 3 4 5 6

7 8 9 10 11 12

Custom migration script

#sqlinthecity

Deployment script with migration scripts

1

12

#sqlinthecity

SQL Compare

1

12

1’

Migration scripts

#sqlinthecity

Demo

#sqlinthecity

Recap: database CI

1. Incrementally keeps a test database up to date

2. Runs tSQLt tests on a database with realistic amounts of data

3. Generates always-up-to-date documentation

4. Generates and validates deployment scripts

5. Publishes a Deployment Manager database package

#sqlinthecity

Recap of demo: writing a test

1. A new test is added2. Continuous Integration identifies a failure3. We reproduce the bug locally4. We fix the bug and test it5. We commit the fix to source control

#sqlinthecity

Recap: migration scripts

1. The fix sets the [PublishDate] column from NULL to NOT NULL

2. Grant’s “get latest” fails because existing data that has NULL values

3. The continuous integration build alerts us of the same issue

4. A migration script is added to UPDATE existing data prior to changing the column to NOT NULL

5. Get Latest becomes possible and the continuous integration build once again indicates working software

#sqlinthecity

Latest version at :www.red-gate.com/CI

Questions?

#sqlinthecity

David.Atkinson@red-gate.com