Upload
smalltalk-solutions
View
216
Download
0
Embed Size (px)
Citation preview
8/7/2019 GemStone Collection Views
1/35
GemStone Collection ViewsGemStone Collection Viewsfor Instant Display of GS Collectionsfor Instant Display of GS Collections
by Paul Baumann
IntercontinentalExchange, Inc.2011.03.15
8/7/2019 GemStone Collection Views
2/35
Just because...
To share ongoing work toward addressingcommon performance problems.
To show techniques that others may findinteresting.
To encourage developers to think outsidethe box.
It was fun to develop.
8/7/2019 GemStone Collection Views
3/35
Installation Sources
Efficient GemStone Enumeration
http://techsupport.gemstone.com/forums
http://www.cincomsmalltalk.com/CincomSmalltalkWiki/PostgreSQL+Access+Page
Public Store Repository
8/7/2019 GemStone Collection Views
4/35
Prerequisites
Efficient GemStone Enumeration base code
http://www.visualworksforums.org/index.php?topic=35.0
- OR -
http://techsupport.gemstone.com/forums/37605-gss-tips-tricks
- OR -
Store Public Repository packageGS_CollectionPerformanceBase
- FROM -
8/7/2019 GemStone Collection Views
5/35
Download files from the GSS Tips & Trickssection of the GemStone forums website:http://techsupport.gemstone.com/forums
Load the parcel named VW_CollectionViews.
Use a topaz session to file-inGS_CollectionViews.gs as SystemUser.
Detailed instructions in the methodBcvCollectionViews class>> #setupInstructions.
Installation Option 1
http://techsupport.gemstone.com/forumshttp://techsupport.gemstone.com/forumshttp://techsupport.gemstone.com/forums8/7/2019 GemStone Collection Views
6/35
Use a VisualWorks image to connect tothe Cincom Public Store Repository
Load GemKit GemStone Source Code Management.
Load PlbCollectionViews.
Detailed instructions in the method
BcvCollectionViews class>> #setupInstructions.
Installation Option 2 Public Store Repository
8/7/2019 GemStone Collection Views
7/35
Session Configuration
Session login parameters must be configuredto connect the Collection View classes.
CollectionViews.BcvCollectionView class
>>setupInstructions_classConnectors
8/7/2019 GemStone Collection Views
8/35
8/7/2019 GemStone Collection Views
9/35
Why Use Collection Views?
An efficient way to display attribute valuesfrom a large GS collection.
An efficient way to #collect: a set of resultobjects from a large GS collection.
An efficient way to replicate results to aGBS client image.
Similar use and benefit as a table View of arelational database.
8/7/2019 GemStone Collection Views
10/35
Relational Database View
In database theory, a view consists of a stored query accessible as a virtualvirtualtabletable composed of the result set of a queryresult set of a query. Unlike ordinary tables (basetables) in a relational database, a view does not form part of the physicalschema: it is a dynamic, virtual table computed or collated from data in theit is a dynamic, virtual table computed or collated from data in thedatabasedatabase. Changing the data in a table alters the data shown in subsequentsubsequentinvocationsinvocations of the view.
Views can provide advantages over tables:
* Views can represent a subset of the datasubset of the data contained in a table* Views can join and simplify multiple tablessimplify multiple tables into a single virtual table* Views can act as aggregated tablesaggregated tables, where the database engine aggregatesdata (sum, average etc) and presents the calculated results as part of thedata
* Views can hide the complexityhide the complexity of data; for example a view could appear as
Sales2000 or Sales2001, transparently partitioning the actual underlyingtable* Views take very little spacevery little space to store; the database contains only thedefinition of a view, not a copy of all the data it presents
* Depending on the SQL engine used, views can provide extra securityextra security* Views can limit exposurelimit exposure of a table or tables to the outer world
8/7/2019 GemStone Collection Views
11/35
SQL Comparison
SQL ViewSELECT
name,
money_received,
money_sent,
(money_received - money_sent)AS balance,
address
FROM table_customers c
JOIN accounts_table a
ON a.customerid = c.customer_id
GS ViewAccount>>balance
^money_received - money_sent
Customer>>account
^Accounts at: customer_id
^account
Customers view: #(
name
(account money_received)
(account money_sent)
(account balance)address
)
8/7/2019 GemStone Collection Views
12/35
SQL View vs. 1.0 GS View
JOIN is achieved through unary messages through theobject graph.
OUTER JOIN is not part of 1.0.
GROUP BY support is not complete in 1.0.
WHERE is achieved with a separate #select:statement.
GS Views can allow updates, but SQL views also allowINSERT and DELETE.
SQL result functions like AVG, COUNT, MAX, MIN, SUM
are not part of 1.0.
1.0 GS Views can only retrieve attribute values from asequence of unary messages (not binary orkeyword).
8/7/2019 GemStone Collection Views
13/35
1.0 Objectives
Quickly replicate some attributes of a large GScollection to a client image for display.
Minimize the time spent gathering attribute values.
Reduce server paging costs.
Avoid creating an object in GS for each result row.
Return data to client as needed (in chunks of rows).
Avoid replicating entire graphs of objects just to returnsome attribute values.
Easily control replication without replication specs.
8/7/2019 GemStone Collection Views
14/35
GS View 1.0 Syntax
aCollection
[by: chunkSize]
view: #([replicationLevels] [attributeAlias @] attributeGetter ...)
chunkSize declares the number of rows
that are replicated in each clienttraversal. 0 disables chunking.
replicationLevels is an integer 0-6 (defaultis 1) that controls the depth that anattribute is replicated to a clientimage. 0 avoids replicating the object.
attributeAlias is a symbol that a row
attribute can be known as; the defaultis the last symbol of theattributeGetter.
attributeGetter is either a symbol or anarray of symbols that will be sent toeach object.
EXAMPLE query from GBS client image:
GBSM evaluate: (Globals select: [:ea | ea isBehavior ])
by: 50 view: #(
name
2 instVarNames
2 instSelectors @ selectors
2 classVarNames2 classSelectors @ (class selectors)
0 yourself
)'.
8/7/2019 GemStone Collection Views
15/35
Traditional Result Sets
Collect to indexed arrays
GBSM evaluate:
(Globals select: [:ea | ea isBehavior])
collect: [:ea |
#[ea name,
ea instVarNames,ea selectors,
ea classVarNames,
ea class selectors,
ea]
].'.
Collect to keyed collection
GBSM evaluate:
(Globals select: [:ea | ea isBehavior ])
collect: [:ea |
Dictionary new
at: #name put: (ea name);
at: #instVarNames
put: (ea instVarNames);
at: #selectors put: (ea selectors);
at: #classVarNames
put: (ea classVarNames);
at: #classSelectors
put: (ea class selectors);
at: #yourself put: ea;
yourself
].'.
Collect to row objects
GBSM evaluate: (Globals select: [:ea | ea isBehavior ])collect: [:ea |
ClassQueryRow newForClass: ea].'.
8/7/2019 GemStone Collection Views
16/35
8/7/2019 GemStone Collection Views
17/35
GS View Row
Has all the advantages of the three TraditionalResult Set styles.
Has distributed workload and chunked replication.
Avoids cost of creating a GS object for each rowattribute, yet still preserves row identity inclient image.
Reduced client memory footprint (only one chunk
at a time plus any individual rows stronglyreferenced).
Rows attributes can be refreshed to reflect thecurrent object state.
8/7/2019 GemStone Collection Views
18/35
Rows are Special
Views are composed of an index foreach attribute--not row instances.
Row instances are only created on-
demand in GS or Client. Identity is preserved for rows created in
the client. Equality is preserved for rows created in
GS. This is the most efficient design for how
views are intended to be used.
8/7/2019 GemStone Collection Views
19/35
Better Replication
You can specify a view-specific faultlevel for each attribute.
Fewer class connectors are needed
You can return attributes through agraph of GS objects that lack aclient representation (like class andconnector).
8/7/2019 GemStone Collection Views
20/35
Attribute Replication Levels
Customers view: #(name
(account money_received)
(account money_sent)
(account balance)
address
)
Customers view: #(1 name
1 (account money_received)
1 (account money_sent)
1 (account balance)
0 address
)
0 replicationLevels leaves the object in GStypically a stub in VW.
Used to keep a row associated with objects
without replicating them.
If an attribute value is already client-replicatedthen it stays replicated.
8/7/2019 GemStone Collection Views
21/35
Attributes Values are a Snapshot
Attribute values do not automaticallyrefresh when the GS object they camefrom does.
Can be useful if you want to see and edit aconsistent view of data regardless ofsubsequent GS changes.
You can refresh attribute values.
You can compare attribute values withcurrent object state.
8/7/2019 GemStone Collection Views
22/35
Views Can Be Refreshed
Requires #yourself as attribute. #refresh only does something if
#yourself is an attribute.
#refreshAttributes warns if #yourselfis not an attribute.
Number of rows does not (normally)
change.
aView refresh
aView refreshAttributes
8/7/2019 GemStone Collection Views
23/35
Rows Can Apply Changes
Requires #yourself as attribute.Row change goes to both row
attributes and to object.If you abort object changes then
attribute changes endure (until you#refresh).
row name: 'Fred'
row asForwarder name: 'Fred'
8/7/2019 GemStone Collection Views
24/35
8/7/2019 GemStone Collection Views
25/35
AdamFred
ChelseaSusanTrish
AdamFred
ChelseaSusanTrish
TroyEdmundTyrelOliverBret
AdamFred
ChelseaSusanTrish
$300$600
$1002$192$4711
$422$1919$6114$1201$592
$receivednameAdamFred
ChelseaSusanTrish
$200$550$902$192$4711
$400$1919$6114$1201$500
$sentAdamFred
ChelseaSusanTrish
$100$50$100
$0$0
$22$0$0
$1201$92
$balanceAdamFred
ChelseaSusanTrish
anAddressanAddressanAddressanAddressanAddress
address
anAddressanAddressanAddressanAddressanAddress
RicoWarrenHelterHaleyMattox
$192$5992$1843$1119$691
$192$4992$1843$1119$691
$0$1000
$0$0$0
anAddressanAddressanAddressanAddressanAddress
View-Attribute-Chunk Structure
8/7/2019 GemStone Collection Views
26/35
Chunking How it Works
1.A view that is created will pause after the first chunk ofrows is processed. A suspended GS process waits toresume chunk gathering.
2.First chunk is replicated to client along with view.3.Client automatically forks a process that asks GS to
resume chunk gathering.
4.Client is able to use the first chunk while GS operates inparallel to gather remaining chunks.
5.When client asks for a row that is not replicated (orcached) then it asks GS for the chunk for that row.
8/7/2019 GemStone Collection Views
27/35
ClientWindowProcess
Server/GemDefaultProcess
Create View
Fork Process
Server/GemGatheringProcess
ClientGatheringProcess
Gather chunkthen wait
Next chunkuntil complete
Fork Process
Show View
Gather chunkthen waitUse active
chunk
Needsdifferent
chunkActivatechunk
ChunkGathered?
Request
Gather specificchunk then wait
Use activechunk
yield
8/7/2019 GemStone Collection Views
28/35
Chunking Replication Efficiency
Client image transports one chunk of rowattributes at a time.
Fewer objects are contained in the GBS
cache and the GS export set. Less memory is used.
User experiences less of a delay.
Faster when GS takes time gatheringattribute values AND client has somedelay in using all the atttribute values
8/7/2019 GemStone Collection Views
29/35
8/7/2019 GemStone Collection Views
30/35
8/7/2019 GemStone Collection Views
31/35
Chunking Not Always Good
Client referenced only one chunk at atime.
Not good when client immediatelyneeds other chunks.
Not good for rapid random rowaccess on clientlike sorting.
Has more traversal costs (if you useall rows).
8/7/2019 GemStone Collection Views
32/35
Views on Non-Sequenceable Collections
Contents is copied to a sequenceablecollection as view is created.
View retains no knowledge of theoriginal collection.
The copying of contents to asequenceable collection is thebiggest performance hit that viewscan experience (entirely due to diskpage reads).
8/7/2019 GemStone Collection Views
33/35
Views on Sequenceable Collections
Much faster than non-sequenceable collections. A view on a sequenceable collection avoids the
page read cost of copying contents. The collection you created the view for is
enumerated to gather attribute values. Do not create a chunked view of a sequenceable
collection that contains objects that changeposition. Changes could affect backgroundchunk gathering.
Workaround options: Avoid chunking for that view.
Create a view for a copy of the collection. Send #gathering_waitForRemaining before
changing the collection.
8/7/2019 GemStone Collection Views
34/35
8/7/2019 GemStone Collection Views
35/35
Future Directions
Attribute change reporting.
View sorting.
Outer Joins.
*companies Attributes from functions.
balance $(money_received - money_sent)
Persistent managed shared views.A new kind multi-indexed RC collection
Use of attribute relation grouping.row group: #(marketType product)