ADO.NET & Data Persistence Frameworks

ADO.NET & Data Persistence Frameworks

Overview

SerializationADO.NETData Tier ApproachesPersistence Frameworks

Serialization

OverviewSerialization ProcessMBV vs MBR

Serialization

What is Serialization?The ability to persist an object’s state data

to a given location (remote server, file, memory, etc.)

The ability to read persisted data from a given location and recreate a new type based on the preserved stateful values

Plays an important role with ADO.NET and distributed architectures

Serialization

Serialization Process

MyObject

Some StorageDevice

NewObject

Formatter

Formatter

File, Memory, Buffer, Socket, etc.

Serialize

Deserialize

XmlSerializer – xmlBinaryFormatter – binarySoapFormatter - soap

Serialization

Serialization is used by .NET Remoting and .NET Web Services to send objects back and forth

The .NET Framework provides 3 built-in class for serialization:XmlSerializerBinaryFormatterSoapFormatter

Serialization

Serialization in Context

Order

Serialize

Deserialize

Order

Serialize

Deserialize

Process

ReceiptReceipt

Machine A Machine B

<order>

<order>

DBSave

Serialization

Differences:XmlSerializer: only serializes public

variables, serializes to standard XMLBinaryFormatter: serializes all variables

(public, private, etc.), serializes to bytesSoapFormatter: serializes all variables

(public, private, etc.), serializes to SOAPDemo

Serialization

Implications: BinaryFormatter is the fastest XmlSerializer is normally second fastest (depends

on the amount of non-public variables) SoapSerializer is normally the slowest XmlSerializer should be used for

multi-platform/language clients (used by Web Services)

BinaryFormatter / SoapFormatter is targeted for .NET environments

Serialization

How objects are passedMBV – Marshal by Value

The caller receives a full copy of the object in its own application domain

Object code is executed in the local applicationMBV objects are declared by using the

[Serializable] attribute, or by inheriting from the ISerializable interface

[Serializable]public class MBVClass{…}ORpublic class MBVClass: ISerializable

Serialization

How objects are passedMBR – Marshal by Reference

The caller receives a proxy to the remote objectObject code is executed in the remote

applicationMBR objects are declared by inheriting from

MarshalByRefObject public class MBRClass : MarshalByRefObject

Serialization

Visualization

Order Order

1. Request Order

2. Receive Order

Order Order

1. Request Order

2. Receive Order Proxy3. Request Calculate Total

3. Request Calculate Total

4. Display Total

5. Calculate Total4. Send Calculate Request

6. Send Total7. Display Total

MBV

MBR

Machine A Machine B

Serialization

MBV vs MBR demo

ADO.NET

OverviewADO.NET ClassesXMLDataDocumentsWhat are the pros and cons of each? Issues with ADO.NET

ADO.NET

What is ADO.NET? Framework that allows you to interact with local and

remote data stores Major overhaul of ADO (few similarities) Optimized libraries for SqlServer (+CE), Oracle Generic libraries for ODBC, OleDb Intrinsic support for Xml Focused on both connected and disconnected

systems

ADO.NET

High-Level View

IDbDataAdapter IDbCommand IDataReader

IDbConnection

DB

Client

A managed provider implements these interfaces to provide access to a specific type of data store

DataSet

In-Memory Disconnected

ADO.NET

Object Model

Connection

Transaction

DataAdapter

Command

Parameter

DataReader

DataSet

DataTable

DataRow

DataColumn

Constraint

DataRelation

DataView

Connected Objects Disconnected Objects

ADO.NET

IDbConnection Represents a network connection to a relational

database Resource intensive, so connections should be kept

open as little as possible (pass through if possible) Connection Pooling is automatically enabled

for .NET IDbConnection implementations a connection pool is created based on an exact matching

algorithm that associates the pool with the connection string

ADO.NET

IDbCommand Represents a SQL statement that is

executed while connected to a data source Provides 3 primary means of submitting a

SQL statement:1. ExecuteNonQuery – nothing returned2. ExecuteScalar – 1 field returned3. ExecuteReader – returns IDataReader

implementation

ADO.NET

IDbCommand cont.Used for standard SQL text and/or stored

proceduresAllows for parameters to be passed in via

an IDataParameter implementation

ADO.NET

IDataReader (Forest Gump)Provides a means of reading one or more

forward-only streams of result sets DEMO

ADO.NET

IDbDataAdapterActs as a bridge between your database

and the disconnected objects in ADO.NETObject’s Fill method provides an efficient

way to fetch the results of a query and place them into a DataSet or DataTable (which can then be used offline)

Reads cached changes from a DataSet or DataTable and submits them to the database

ADO.NET

DataTableOne of the central objects in ADO.NETUsed by DataSet, DataView, etc.Represents one table of data in-memoryAllows you to:

Fetch data from a DB and store it in a DataTableDisconnect from the DB and work with the

DataTable offlineReconnect and synchronize changes

ADO.NET

DataTable cont.DataTable structure is similar to DB Table

structure:DataTable is composed of DataRowsDataRows are composed of DataColumnsConstraints can be set on a DataTable

DataTables can also be created and populated in code (they do note require a corresponding DB Table)

ADO.NET

DataTable cont.Can be remoted (allows for both MBV and

MBR)

ADO.NET

DataViewRepresents a customized view of a

DataTableAllows for:

SortingFilteringEditingSearching

ADO.NET

DataView cont.Allows multiple controls to bind to the same

DataTable, but show different data

ADO.NET

DataSet (La Femme Nikita)Major component of the ADO.NET

architectureCollection of DataTablesAllows for relationships between tables to

be created via DataRelationAllows constraints to be set on data

ADO.NET

DataSet cont.Can read and write data and schema as

XML documentsCan be remoted (allows for both MBV and

MBR)Strongly-typed DataSets can be generated

Can access tables and columns by name, instead of using collection-based methods

Allows for VS.NET Intellisense

ADO.NET

DataSet cont.Ability to merge multiple DataSetsAbility to copy DataSetsUses DiffGram (XML) to load and persist

changesDEMO

ADO.NET

CommandBuilderAutomatically generates single-table

commandsDEMO

ADO.NET

XmlDataDocumentSolves problem of unsychronized access to

relational / XML dataExample:

DataSet (relational) writes out XML fileXmlDocument reads in and manipulates XMLTwo objects dealing with same data in

unsynchronized mannerResult is a disconnect

ADO.NET

XmlDataDocument cont.Synchronizes data between XmlDocument

and DataSetAllows both objects to work on the same set

of dataAllows a single app, using a single set of

data, to harness the power of:DataSets (remoting, databinding, DataViews...)Xml (XPath, XSL, XSLT…)

ADO.NET

Pros and ConsBest Practices

ADO.NET

IDbConnection Best Practices Pass through whenever possible Use constant connectionstrings Use multiple accounts (1 for read access, 1 for

read/write access, etc.) Use Windows Authentication If stored in config file, encrypt Avoid displaying error sensitive error information Avoid using OleDbConnection.State Use the “using” statement in C#

ADO.NET

IDbCommand Best PracticesUse parameters when possible to avoid

SQL injections

ADO.NET

IDataReader Pros and ConsExtremely fast performance (better than

DataSet)Forward-only, read-onlyCan only operate in connected modeMust explicitly close both the IDataReader

and the IDbConnectionNot remotable

ADO.NET

IDataReader Best Practices Use CommandBehavior.CloseConnection and

CommandBehavior.SequentialAccess when possible

Use IDataReader.Get[Type]() whenever possible (performance boost)

Call IDataReader.Cancel() if you’re done w/ a DataSet but still have pending data

Keep connection time to a minimum Use for large volumes of data to avoid memory

footprint

ADO.NET

DataTable / DataSet Pros and Cons Both MBV and MBR behavior Ability to work with data offline Ability to represent the same data in multiple ways

via DataView Data bindable Decreased performance in comparison w/

IDataReader Consumes machine memory Developers must be careful when posting changes

in a distributed environment

ADO.NET

DataSet Best Practices Strongly-type when possible Use for modifiable data and or data that will be

navigated Use for caching of frequently searched items Use DataSet.GetChanges() prior to sending across

the wire Avoid the use of the DataAdapter.Fill overload that

takes startRecord and maxRecords parameters Use MBR behaviour sparingly

ADO.NET

CommandBuilder Pros / Cons Removes the need to manually write code Decreased performance, due to the need to hit a

database twice to retrieve schema information Decreased performance due to very verbose SQL

statements Better to use VS.NET’s built-in code generation MS: Use of the CommandBuilder should be limited

to design time or ad-hoc scenarios

ADO.NET

Performance Matrix1. IDataReader w/ Get[Type]2. IDataReader w/ GetOrdinal3. IDataReader by column name4. Strongly-Typed DataSet w/ custom code5. DataSet w/ custom code6. Strongly-Typed DataSet w/

CommandBuilder7. DataSet w/ CommandBuilder

ADO.NET

Issues w/ ADO.NET No bulk SQL execution (DataSet batch submissions

aren’t done in bulk) No asynchronous calls Can’t write to interfaces…must use providers

directly Inability to get full SQL from

IDbCommand.CommandText when using params No object-relational mapping No SQL API

Data Tier Approaches

OverviewPresentation – DataPresentation – Business – DataPresentation – Business – Service – Data


Presentation - Data

Data Access

Code

Data Access

Code

DBBusLogic

BusLogic

Data Access

Code

Data Access

Code

BusLogic

BusLogic


Presentation – Data Often has best performance Initially the fastest to write (but often requires the

most code over time) Inflexible to change Business logic not available Results in code duplication (business logic…) Error prone (connection strings, closing

connections, etc.) Leaves architecture decisions to implementer (i.e.

DataSet, DataReader, etc.) Ties presentation layer to returned data format


Presentation – Data w/ DAL

Data Access Layer

Data Access Layer

DB

BusLogic

BusLogic

BusLogic

BusLogic


Presentation – Data w/ DALRequires less codeSomewhat flexible to changeBusiness logic not availableResults in code duplication (business

logic…)Ties presentation layer to returned data

format


Presentation – Business – Data (3 Tier)

Data Access Layer

Data Access Layer

DBBus

LogicLayer

BusLogicLayer


Presentation – Business – Data Much more flexible to change Business logic is centralized Does not tie presentation layer to returned data

format More complex to design and build Implicit changes may need to be cascaded through

all layers Does not clearly address remoting issues Business Logic must still be aware of DA classes


Presentation – Business – Service – Data

Data Access Layer

Data Access Layer

DBBus

LogicLayer

BusLogicLayer

ServiceAccessLayer

ServiceAccessLayer

Local Local or remote


Presentation – Business – Service – Data Most flexible to change Most scalable (but decreased performance) Clearly provides remoting capabilities Business Logic does not need to be aware of DA

classes Most complex to design and build Implicit changes may need to be cascaded through

all layers

Persistence Frameworks

Where do Persistence Frameworks fit in?

Data Access Layer

Data Access Layer

DBBus

LogicLayer

BusLogicLayer

ServiceAccessLayer

ServiceAccessLayer

Map Business Entity objects to relational data

Provide CRUD operations

Provide object caching

Provide object versioning

Provide for remote persistence

Map to Business Objects


OverviewPersistence Frameworks in ContextDesired AttributesSample ArchitectureSample FrameworksOther ApproachesQuestions?


What is an Object Persistence Framework?A persistence layer encapsulates the

behavior needed to make objects persistent, in other words to read, write, and delete objects to/from permanent storage

Simplifies life for developers by removing the need for repetitive coding

Focus is on persistence at the Object level vs. the Data level


Persistence Frameworks in Context

Data Access Layer

Data Access Layer

ServiceAccessLayer

ServiceAccessLayer

DBWebsiteWebsite

CachingService

CachingService

BusLogicLayer

BusLogicLayer

BusLogicLayer

BusLogicLayer

Smaller DistributedArchitecture

Client

Server


Persistence Frameworks in Context

DB

WebsiteWebsite

Larger DistributedArchitecture

Caching Server

BusLogicLayer

BusLogicLayer

ServiceAccessLayer

ServiceAccessLayer

DataAccessLayer

DataAccessLayer

CachingService

CachingService

WebsiteWebsiteBus

LogicLayer

BusLogicLayer

BusLogicLayer

BusLogicLayer

Web Services Server

ClientMachine

WebServers


Desired Attributes Support for run-time and design-time object

relational mapping Built-in logging / tracing Object caching Object versioning (optimistic and pessimistic

concurrency) Support for multiple databases Support for multiple persistence types (file,

RDBMS)


Desired Attributes Support for transactions Support for cursors Support for lazy-loading Support for multiple architectures (local, remote…) Built-in security models (object caching

encryption…) Support for batch operations SQL API Configurable performance Still provides access to ADO.NET classes


Sample Architecture


Current .NET Persistence FrameworksOpen Source

SisyphusGentle.NET (DEMO)Wilson ORMapper (DEMO)Bamboo.Prevelance (DEMO)LLBLGen (DEMO)AtomsOJB.NETObjectSpaces*


Current .NET Persistence FrameworksCommercial

EntityBrokerDataObjects.NETLLBLGen Pro .NET N-Tier GeneratorTier DeveloperORM.NETObjectz.NET


Other ApproachesPoly Model (.NET Patterns)Business Object JumpStart

(Developing .NET Enterprise Applications)Serialize object to DB


Caching Considerations Allow for both local and remote caching

Local: Hashtable, MMF, etc. Support both serialized and non-serialized

Remote: .NET Remoting, SQL Server, etc.

Provide Scavengers (LRU, LILO, Expiration…) Provide object encryption Provide size maximums Provide caching stats (Hits/Misses/Duration) Allow for multiple cache instances


Concurrency ApproachesOptimistic/Pessimistic concurrency

In code – error prone upon reboot VersionManager (Timestamp, GUID, etc.)

In DB – resource intensive Timestamp, GUID, etc.

Summary

SerializationADO.NETData Tier ApproachesPersistence Frameworks

Resources

Microsoft Patterns & Practices website MS: Designing Data Tier Components and Passing

Data Through Tiers MS: .NET Data Access Architecture Guide MS: ADO.NET Best Practices Martin Fowler: Patterns of Enterprise Application

Architecture David Sceppa: ADO.NET Clifton Nock: Data Access Patterns Sun: Core J2EE Patterns Andrew Tobias: C# and the .NET Platform