DAT336 Connected vs Disconnected Data Access in ADO.NET
DAT336 Connected vs Disconnected Data Access in ADO.NET
Pablo CastroPablo CastroProgram Manager – ADO.NET TeamProgram Manager – ADO.NET TeamMicrosoft CorporationMicrosoft Corporation
AgendaAgenda
Disconnected is goodDisconnected is goodSome connected bits can helpSome connected bits can help
Balancing connected/disconnectedBalancing connected/disconnectedScrolling and streamingScrolling and streaming
Custom aggregationCustom aggregation
Incremental loadIncremental load
SummarySummary
Disconnected?Disconnected?
In the context of this talk:In the context of this talk:““Disconnected” apps are online Disconnected” apps are online applications that don’t keep database applications that don’t keep database connections open for long periods of time connections open for long periods of time such as the lifetime of a sessionsuch as the lifetime of a session
Usually 3-tier applicationsUsually 3-tier applications
We’re not going to discuss “off-line” We’re not going to discuss “off-line” applications where a user can operate an applications where a user can operate an application even if the server is not application even if the server is not availableavailable
Disconnected is GoodDisconnected is Good
Simpler to designSimpler to design3-tier applications follow the 3-tier applications follow the disconnected model naturallydisconnected model naturally
Simpler to code once the pieces are setSimpler to code once the pieces are seti.e. data binding, marshalling in 3-tier i.e. data binding, marshalling in 3-tier apps, no connection management for apps, no connection management for long-lived business objectslong-lived business objects
Simpler to make apps scaleSimpler to make apps scaleMiddle-tier layer easy to scale-outMiddle-tier layer easy to scale-out
It’s harder to scale-up/out database serversIt’s harder to scale-up/out database servers
Disconnected ADO.NETDisconnected ADO.NET
ADO.NET has first-class disconnected ADO.NET has first-class disconnected apps supportapps support
DataSet as a relational data cacheDataSet as a relational data cachePlays well with remoting & web servicesPlays well with remoting & web services
Single implementation, no database-specific Single implementation, no database-specific behaviorbehavior
DataAdapter/DataSet provide services for DataAdapter/DataSet provide services for getting, updating and merging datagetting, updating and merging data
Custom types for non-relationalCustom types for non-relational
…but too disconnected……but too disconnected…
Handling large volumes of dataHandling large volumes of dataBatch processingBatch processing
ReportingReporting
Custom aggregationCustom aggregation
Performance-sensitive code-pathsPerformance-sensitive code-pathsASP.NET pages in apps without heavy ASP.NET pages in apps without heavy business logicbusiness logic
Gateway applicationsGateway applications
Avoid buffering large Avoid buffering large portions of dataportions of data
Avoid data-shipping costsAvoid data-shipping costs
Discrete Objects ScenariosDiscrete Objects Scenarios
This is the “easy part”This is the “easy part”Stand-alone business objectsStand-alone business objects
Business object state representationBusiness object state representationDataSets – ADO.NET helps with retrieval, DataSets – ADO.NET helps with retrieval, update and change trackingupdate and change tracking
Custom objects – model your business Custom objects – model your business entity state using regular classesentity state using regular classes
Remoting or webservices enable this Remoting or webservices enable this scenario easilyscenario easily
Sessions in the Middle TierScenariosSessions in the Middle TierScenarios
Per-session state can’t be avoided Per-session state can’t be avoided sometimessometimes
If needed, keep only in-memory stateIf needed, keep only in-memory state
No database connections if at all possibleNo database connections if at all possible
Quite common in ASP/ASP.NET appsQuite common in ASP/ASP.NET apps
““Sticky sessions” take care of scale-outSticky sessions” take care of scale-outSo it’s not that bad if you don’t keep So it’s not that bad if you don’t keep connections or other external resourcesconnections or other external resources
Scrolling and StreamingScenariosScrolling and StreamingScenarios
In general, handling large results in In general, handling large results in piecespieces
UI: scrolling, pagingUI: scrolling, paging
Batch processing: chunking, scanning Batch processing: chunking, scanning large resultslarge results
Custom aggregationCustom aggregation
Options change depending on each Options change depending on each casecase
Scrolling, PagingScrolling, Paging
Goal is to fetch rows from a large Goal is to fetch rows from a large result, a few at a timeresult, a few at a time
CursorsCursors
DataAdapter.Fill methodDataAdapter.Fill method
SQL-based solutionsSQL-based solutions
Scrolling, Paging: CursorsScrolling, Paging: Cursors
Not available in all databasesNot available in all databases
Provide scrolling supportProvide scrolling supportSometimes even bi-directionalSometimes even bi-directional
No need for extra logic in the applicationNo need for extra logic in the application
Scalability issuesScalability issuesRequire to maintain stateRequire to maintain state
Database connections, keyset or temporary Database connections, keyset or temporary tables in the databasetables in the database
Cursor escalationCursor escalationServer may need to materialize some/all dataServer may need to materialize some/all data
Scrolling, Paging: CursorsScrolling, Paging: Cursors
Design issuesDesign issuesHard to include in 3-tier applicationsHard to include in 3-tier applications
Need to keep connection/cursor objects Need to keep connection/cursor objects alive in middle tieralive in middle tier
How is data propagated to the How is data propagated to the presentation layer?presentation layer?
Tends to be a chatty interfaceTends to be a chatty interface
Result stabilityResult stabilityNeed to use transactions or static cursors Need to use transactions or static cursors if stability is requiredif stability is required
Scrolling, Paging: Fill()Scrolling, Paging: Fill()
ADO.NET DataAdapter.Fill() methodADO.NET DataAdapter.Fill() methodThere’s an overload that takes first row There’s an overload that takes first row and number of rowsand number of rows
Under the covers, this method:Under the covers, this method:Skips rows as neededSkips rows as needed
Copies as many rows as requested to the Copies as many rows as requested to the target DataTabletarget DataTable
Scans and discards the rest of the rowsScans and discards the rest of the rows
Don’t use it for paging in large Don’t use it for paging in large result-setsresult-sets
Scrolling, Paging: SQLScrolling, Paging: SQL
If at all possible, use SQL constructs If at all possible, use SQL constructs for pagingfor paging
Stored-procedures Stored-procedures if you know the table schema and sort orderif you know the table schema and sort order
SQL can help in other cases SQL can help in other cases where the query is not known but constrainedwhere the query is not known but constrained
Scalability/performance issuesScalability/performance issuesTime taken to execute query is not Time taken to execute query is not amortized across requestsamortized across requests
Scrolling, Paging: SQLScrolling, Paging: SQL
Design issuesDesign issuesFits nicely for 3-tier appsFits nicely for 3-tier apps
Each page is an independent database Each page is an independent database operationoperation
No state held between hitsNo state held between hits
Ad-hoc queries are hard to handleAd-hoc queries are hard to handle
Result stabilityResult stabilityNeed to use transactions if stability is Need to use transactions if stability is requiredrequired
StreamingStreaming
Goal is to handle very large result-setsGoal is to handle very large result-setsNo buffering proportional to size of dataNo buffering proportional to size of data
Concurrency issuesConcurrency issues
InterleavingInterleaving
Common scenarios for streamingCommon scenarios for streamingBatch processingBatch processing
ReportingReporting
Custom aggregationCustom aggregation
Streaming: DataReadersStreaming: DataReaders
ADO.NET providers have an streaming ADO.NET providers have an streaming interfaceinterface
DataReader class exposes a row at a timeDataReader class exposes a row at a timeSome minor buffering might happen internallySome minor buffering might happen internally
Can scan millions of rows without taking Can scan millions of rows without taking much resourcesmuch resources
Scalability issuesScalability issuesUsually large scans happen in batch Usually large scans happen in batch processesprocesses
Not many at the same timeNot many at the same time
Streaming: DataReadersStreaming: DataReaders
Design issuesDesign issuesDataReaders cannot be marshaled across DataReaders cannot be marshaled across tierstiers
Move the batch process code to the middle Move the batch process code to the middle tiertier
Send the data in chunks to the next tier (too Send the data in chunks to the next tier (too much overhead in most cases)much overhead in most cases)
Contention can be highContention can be high
Result stabilityResult stabilityDepends on the isolation levelDepends on the isolation level
Custom Aggregate LogicCustom Aggregate Logic
Similar case: scan lots of rowsSimilar case: scan lots of rowsBut end-result is a small result-setBut end-result is a small result-set
No need to ship the data out of the No need to ship the data out of the serverserver
Cursors can help hereCursors can help here
Scan and aggregate inside the serverScan and aggregate inside the server
Ship only the aggregated information to Ship only the aggregated information to the clientthe client
Avoids moving lots of data across tiersAvoids moving lots of data across tiers
Incremental LoadScenariosIncremental LoadScenarios
Present first bit of data in UI quickPresent first bit of data in UI quick
Incrementally load the rest in the Incrementally load the rest in the backgroundbackground
DataSet merge support is great hereDataSet merge support is great here
Be aware of multi-threading issuesBe aware of multi-threading issues
Incremental Load & MergeIncremental Load & Merge
ADO.NET DataSet can merge resultsADO.NET DataSet can merge results
Incremental load UIIncremental load UIChunking API in the middle tierChunking API in the middle tier
Bring down the first DataSet and display itBring down the first DataSet and display it
As more data comes, merge the DataSets As more data comes, merge the DataSets and update UIand update UI
This even preserves changes in existing dataThis even preserves changes in existing data
Multi-threading issuesMulti-threading issuesDataSet is not thread-safeDataSet is not thread-safe
Same for WinForms UI controlsSame for WinForms UI controls
SummarySummary
Disconnected is goodDisconnected is goodADO.NET has great support for itADO.NET has great support for it
Good to start disconnected by defaultGood to start disconnected by default
You will need some connected piecesYou will need some connected piecesADO.NET also helps thereADO.NET also helps there
You can add connected parts as neededYou can add connected parts as needed
Extremes can hurt your app Extremes can hurt your app performance or scalabilityperformance or scalability
Attend a free chat or web castAttend a free chat or web casthttp://www.microsoft.com/communities/chats/default.mspxhttp://www.microsoft.com/communities/chats/default.mspx http://www.microsoft.com/usa/webcasts/default.asphttp://www.microsoft.com/usa/webcasts/default.asp
List of newsgroupsList of newsgroupshttp://communities2.microsoft.com/http://communities2.microsoft.com/communities/newsgroups/en-us/default.aspxcommunities/newsgroups/en-us/default.aspx
MS Community SitesMS Community Siteshttp://www.microsoft.com/communities/default.mspxhttp://www.microsoft.com/communities/default.mspx
Locate Local User GroupsLocate Local User Groupshttp://www.microsoft.com/communities/usergroups/default.mspxhttp://www.microsoft.com/communities/usergroups/default.mspx
Community sitesCommunity siteshttp://www.microsoft.com/communities/related/default.mspxhttp://www.microsoft.com/communities/related/default.mspx
© 2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.