Training Materials

Database and Datamart Basics

January 14, 2005

Table of Contents

Table of Contents.................................................................................................2Why Should I Care About This Database “Stuff”?.................................................3What is a Database?.............................................................................................3Elements of a Relational Database.......................................................................7

Joins...................................................................................................................9Equi-joins........................................................................................................9Left/Right Joins.............................................................................................10Outer Joins....................................................................................................10

Introduction to the SFMA and OSPA Datamart...................................................11General Information........................................................................................11Accessing the Datamart..................................................................................12

Software Needed:.........................................................................................12Access to the Datamart................................................................................12

SFMA Datamart..................................................................................................14Tables on the Datamart...................................................................................15SFMA Financial Tables on the Datamart..........................................................15

Accounting Event Tables..............................................................................15General Ledger Detail Tables.......................................................................16General Ledger Summary Tables.................................................................17Other Special Financial Tables.....................................................................17Appropriation Financial Table.......................................................................17Contract Financial Table...............................................................................17Grant Financial Table...................................................................................18Project Financial Table.................................................................................18Receipt Extr Sum Table................................................................................18Document Financial Tables..........................................................................18Payment Control Table.................................................................................18

Profile Tables...................................................................................................18OSPA Tables on the Datamart.........................................................................19

OSPS DED Tables.........................................................................................19OSPS JS Lab ADJ...........................................................................................19OSPS Labor Cost Tables...............................................................................20OSPS LV Accrual Tables................................................................................20OSPS Net Pay Tables....................................................................................20

General Querying Suggestions...........................................................................20Killing a Query....................................................................................................22

Stopping a Query From Brio.........................................................................22Kill a Query from the Web............................................................................23

Database and Datamart Basics - 2 -

Why Should I Care About This Database “Stuff”?

In some ways, you may not. If you were simply running queries against a database that someone else created, and get the information you want, in the timeline you want, then you would not care at all. However, if you have people asking you questions about information in a database, and the queries are not “canned” for you, or you care only about the three pages in the middle of a 1,387 page long report, then you really need to know about how databases work. If you have need to query against a database on your own, knowing about databases will simplify your time working with them.

What is a Database?

Most simply put, a database is an organized collection of information. Many times, databases are lists of information. A database does not have to be electronic – any collection of organized information is in fact a database. Some examples of this type of database are:

Shopping list (probably the simplest of databases) Your daily to-do list Rolodex file The Yellow Pages The file of agency questions a SARS analyst maintains The card catalog at the library

Examples of electronic databases are: The SFMA Datamart (surprise!) Some types of Excel worksheets A web site An on-line encyclopedia Lexis/Nexis (an on-line paid service combining many text articles) An Access database created by you ORBITS

Databases are usually designed to allow people quick and easy access certain types of information. Look at the Yellow pages – an alphabetized listing of types of services and commodities, actually sorted by commodity and then business name. That is the essence of a database. Look at a card catalog in the library (no fair using the on-line electronic catalog!). You have actually three databases – one by author, one by book title, and one by subject.

The organization of the database comes in when you look at what you are trying to organize. There are three parts to the information in a database:


The collection of records (We’ll refer to this as a table) Records – a collection fields within the table Fields – individual elements of information within the record

Example of a table

The way these elements are organized determines what type of database you have. There is two basic types, plus a hybrid that looks a little like both:

Flat File Relational Hybrid (usually a flat file, with some lookup capability)

Flat file databases are ideal for small amounts of data that needs to be human readable or edited by hand. The record in a flat file database contains all of the fields needed to make up a unique record. An example would be your contacts list in a Rolodex. You have the following fields:

Name (or two fields – First name, Last Name) Agency Number Agency Name Address 1 Address 2 City State Zip Position Title Phone Cell phone FAX


This row is a record

This item is a field

This database now allows you to sort the information on one field, or several. You could simply sort by name. Sorting by agency would be simple. Sorting by agency and name is a little harder, but would still be simple. This type of database is also simple to understand – almost everyone understands how to use a Rolodex file.

In electronic format, one could put their Rolodex on an Excel worksheet similar to below:

This database within Excel is easy to sort and maintain. It is even easy to change if you need a new data element. However, filling in the new data elements can be tedious. This is still a simple list however. Keeping complex data is not impossible with a flat file database, but it becomes much harder to maintain, and people trying to use it would not as easily be able to read or interpret the information presented.

One of the real problems with flat-file databases is updating the information. Say you have 20 contacts at the same agency, at the same address. The agency then moves its offices. In a Rolodex card system, you would copy each one of the cards with the new address, or you would redline the address and written on each card. This is very prone to error. In Excel, you can sort the information, type in the new information on one record and then copy the new information to the new records. However, what happens if you do not notice that you copied the new information to one more (or less) records than you needed? Your database now has an invalid address for one or more of your contacts. In conjunction with the updating issue, there is the issue of maintaining a large amount of repetitive information on each record. In the Excel example, if you had four hundred records, you may have the Agency information including all of the address information repeated multiple times. The larger the flat-file database becomes, the worse the problem is. Maintaining a flat-file database with several thousand, or tens of thousands, of records becomes difficult very quickly. While computing power has reduced in cost dramatically over the years, there are still costs involved in maintaining databases with a large amount of redundant information.


Another problem with the flat-file database is that there is no way to assure that the information for a field gets entered the same way each time – in our example, the agency name for agency 100 may be entered as DHS, Human Services, Department of Human Services or Dep’t of Human Svcs. While we may understand that each of these titles is equivalent, a computer does not. Each one of those titles would be treated as entirely different when accessing, sorting and printing reports for agency 100.

Enter the relational database. This type of database has a more logical way of storing the data. Tables – the groups of records – can be used to represent real objects, with each field representing some attribute unique to that object. In our Rolodex example, the object could be Agency, and attributes of Agency could be Address 1, Address 2, City, State, and Zip. The “relation” in relational comes from the fact that the tables can be linked to each other. For our example, a table containing our contact’s personal information links to an Agency table by an agency code. Relations can be very complex in nature, and would be very difficult to replicate in a flat-file format.

The major advantage of a properly designed relational database is that it should not have any duplication of data. This aids in maintaining the integrity of the data. This also has the advantage of reducing file sizes, important when there are large volumes of data involved, in that it normally speeds up the queries made against such a database. Because relational databases normally store data only once, or with as little repetition as possible, you find that the data is much more consistent. In our example about agency 100, our relational database will have only one record for each agency, with only one agency name. When queried, you will always get the same name for agency 100, rather than what may have been input at the time.

Another advantage is that relational database programs can force integrity of the data to a greater degree than a flat-file database. If you are entering the agency number in a contact record, the relational database program can check to see if there is an entry for that agency number in the agency table. If there is an entry, the program allows the entry to be saved. If there is no entry, the program can be designed to give an error message, and the user may then have the option to enter a new agency record into the system, or correct the agency number. The system will not allow the incorrect information to be entered into the database until the field information matches between tables.

There are some disadvantages to a relational database. One is that adding a field can cause problems with programs that define where data is located by using offset. Rather than identifying where a field ends by some sort of indicator, a database may define where a field is by describing how many characters the field is from the beginning of the record, and then how long the field runs from there. Adding a data field changes the length of the record. All programming has to be redone to allow the system to understand the longer record. In addition, if the record is placed anywhere other than the end of the record, all programming has to be changed for all fields now after the new field


– the beginning point of the field has changed. This is why you find a new field at the end of a record, rather than where it might logically be expected to be located in the record.

Elements of a Relational Database

As discussed earlier, a relational database has several related tables that contain information that no other table contains. The exception to that are those fields common to both tables, which make a unique key. One uses this key to connect tables together to run queries against them all, and assure that you are not duplicating records. The key is the relation in a relational database. The keys between tables are connected through joins. These joins connect the key fields together between the tables you need to use to produce a report. A simple join with a single key field is pictured below:

In the example above, the key field is GL_ACCT. Because there is a join between the SFMFPUSR_GL_SUMMARY and the SFMSPUSR_GL_ACCT, the GL_SUMMARY table does not have to contain anything other than the GL_ACCT that is in SFMSPUSR_GL_ACCT. The GL_SUMMARY does not have fields such as BAL_SHEET_CLASS, COMPT_OBJ_TYPE or GL_ACCT_TITLE as they are already in the GL ACCT table. However, if you create a query with this pair of joined tables, your query will show all fields requested from both tables, based on the join.


GL_SUMMARY Table Fields

GL_ACCT Table Fields

This is the results set of a query using the join described. Even though the GL_SUMMARY table did not have GL_ACCT_TITLE or GL_DR_CR_IND we wished to use, the query returned that information based on what was in the GL_ACCT table.

One record in the GL_ACCT table, but the information is included in all of the records in the results example above. This is a huge savings in storage space for databases, and can make running queries much faster. Queries do not have to search through all records or all fields to find the information requested.

There are some issues with a relational database. The data is not as easy to understand – you have to query the database to see all data elements wanted for one. Reading a flat-file database is easy – all of the data is there in each record. Reading a relational database is more difficult – depending on the level of data “breakdown” into separate tables, on would not be able to read the data at all. Multiple tables also bring more complexity, and complexity requires a higher level of understanding of the data. It also requires knowledge of a database program – this is the only way to access the data needed in a format that the user can read. In addition, relational databases require an electronic format to be useful – imagine trying to use several books to put together an address and phone number as is done in the phone book. Now imagine doing this twenty or thirty times a day!

Another issue with the relational model is that users have to understand the data in the tables. If the user does not understand the presentation of data in the database, that user could create queries that could run for hours, days, weeks or more, depending on the size of the data. After that query ran, the user would have gotten volumes of data, none of it any good. In the Datamart, this error usually happens when users try to join a detail table with a summary table, both with the same information, but at a different level of detail.


As alluded to in the previous paragraph, a relational database can summarize data more simply than a flat-file database. Because a flat-file database is a single table normally, there are no summary tables in which data could be summed, averaged, or manipulated in other ways. A relational database allows this because the data in a particular table is unique to the table, allowing the programming to create another table summarizing information in ways that users need the data. Because the summary tables are much smaller than the detail tables, queries against them are much faster and more efficient.

Joins

There are three basic types of joins used in databases. They are:

Equi-joins – also known as simple joins or inner joins, return data when both joined tables have matching information in the key fields.

Left/Right Joins – these joins determine which table will return all data, even if another table does not have a key field, with nulls in the field missing data.

Outer joins – acts as a combined Left and Right join – returns all data, and places nulls in joined fields where there is no data.

This sounds complicated, but the joins are used in specific cases, and do make sense.

Equi-joinsEqui-joins are the most common type of join for creating queries. The display in one results table all of the records with corresponding records in another table. Correspondence is determined by identical values. In most cases, you have two tables that have a one-to-many relationship – that is, one record in a table relates through the key fields to many records in another table.

If none of the table’s records on the many side of the relationship have a key value (or values) that match a record on the table of the one side, the corresponding records do not appear in the query result.

There is a problem with this type of query, however. If the table with the many side has information that does not match the table in the one side, that data would not show in the results set. For example, we have a data element in the Datamart All Accounting Event table called Operator ID. We also have a profile table called Operator ID, with a key field of the same name. The Operator ID table however, contains only current users, and those users that have had their security on SFMA removed within the last six months. If you were to do a query joining the two tables, and you had someone who input data that had left your agency seven or more months ago, any records with that


employee Operator ID would not be returned in your query. This is a good reason to check your queries against an SFMA hard copy or on-line report.

Left/Right Joins

Left/Right joins are specific types of outer join. The left/right direction refers to which table will return all data, even if there is no correspondence to the joined table. That item returns a null value. In fact, the direction is based on where the tables are placed in the query screen of Brio!

In our example of the Operator ID above, had the join been a left join on the All Accounting Event table to the Operator ID table, all of the records in the All Accounting Event Table would have been returned, even though you had an employee that left seven months ago.

Outer JoinsBrio can use this type of join – it is a combination of a left and right join. Note though that this type of join takes longer to run. One uses this type of join in more complex queries, and is beyond this training.

IndexesAn advantage to the relational database model, especially an electronic database, is the use of indexes. Indexes are actually small databases in their own right. They take particular fields, identify the records that contain them, and sort them. The indexes act like key fields. Use these fields, and your queries can be much quicker. Ignore them, and the query will take much more time.

How is this magic accomplished? Indexes work very similarly to the card catalog at the library. If you have a subject you are interested in, you go to the card catalog and get a list of books, and their location, from the catalog. This makes your search much easier than if you just wandered through the library, looking at books that might have your subject in the title. If you use the index field (or fields), the database has already identified the records that match those fields – your cards in the catalog. The program knows exactly where to get the records requested in the database, and can get them quickly. If not, the database has to plow through every record to make sure it got every record requested.


Introduction to the SFMA and OSPA Datamart

General InformationOur data warehouse is located on an IBM Regatta RISC processor-based minicomputer at the Revenue Building. It contains eight RISC processors, approximately 8 gigabytes of memory and ½ terabyte (500 gigabytes) of disc storage.

The data is loaded from SFMA production after Friday night processing. There is an exception for year-end closing, where two of the financial tables update three times a week in the month of August. SARS normally provides a schedule of which tables update, and when they update. The main reason we do not update all tables on a more frequent basis is the size of the tables – the update takes all evening after processing, and most of the next morning. This tends to make the Datamart much less useful if it is not available for use because it is updating.

The data is in a DB2 database. The data is relational, but combined profiles make up some of the tables – this made design and querying simpler. DB2 is an ODBC (Open Data Base Connection) compliant database. This means that any ODBC-aware software may query the database. We connect to the database through the state WAN backbone.

Even though there is some combination of data, the data is in a “raw” form – you will have to query the database in order to get usable information from it. In addition to knowing how the data is stored on the system, you will need to know how your agency accounting structure in order to get good information out of the system.


Accessing the Datamart

Software Needed:

1. Install TCP/IP software to allow access to the state network. Windows usually install this as part of the operating system.

2. Install the DB2 client ODBC drivers on your computer. Be sure that your

LAN administrator installs the software under your user name rather than an administrator ID. Note: we have a one-click installation file available on the Web, if your administrator does not have a copy of the drivers.

3. Brio Intelligence (if you want to use Brio).

4. Other ODBC compliant software:

a. Microsoft Accessb. Crystal Reportsc. Microsoft Excel (using MS query)d. Paradoxe. Lotus Approachf. Any ODBC compliant software will work with the Datamart.

We support Brio, and can provide some help with Access. Other software you will have to support on your own.

Access to the Datamart

Please be careful with your access to the Datamart. Do not share your password and RACF ID with anyone else. While information on the Datamart is normally considered public information, there is sensitive data such as Social Security Numbers that someone malicious might access using your password. If an inexperienced user were to try to build queries, it could mean rather large bills for those queries – especially if the user builds them inappropriately.

1. Request access to the Datamarta. Have your systems security officer send a request to Systems

Security. The agency security person is the person who sets up access to SFMA.

b. Information to be provided to your security officer:

i. Your Nameii. Your RACF IDiii. Your Telephone number


iv. Agency name / initials (i.e. Veterans’ Affairs / DVA)v. Agency numbervi. Your internet address

c. SARS will process the information through to the security people at

GGDC. GGDC will then e-mail a password to the user. 2. Upon receiving the password from GGDC, you should change the

temporary password. a. To change your password:

i. Open your Web browserii. The following URL goes directly to the login screen:

https://dasdm1.iservices.state.or.us/cgi-bin/login

If you get a dialog box warning about an expired certificate, just click the “OK” button to continue.

iii. Log in using your RACF ID and the password provided by GGDC

iv. You will now be at a screen that allows you to do several

things:

1. Install a new ODBC client and drivers to your desktop – this is where you get the DB2 ODBC drivers.

2. List or kill queries – this allows you to list the queries

you have running, and to stop (kill) all queries running – useful when you have a query that is running much longer than you expected.

3. Change your password – enough said.

4. Log out – this is a good idea after you complete what you planned at this screen – until you do, you are still connected. Shutting the browser down will also disconnect you.

v. Select “Change your password”

vi. Again, type in your RACF ID and the password provided from GGDC. This is extra security for the Datamart and your password.



vii. Type your new password twice as indicated. The password should be structured as follows:

1. Six to eight characters long2. Contain at least one number. 3. At least three characters must be different from the old

password.

viii. Click on the “Change Password” button. You should get a message indicating your password has been changed. If you get an error message, click try again and follow step vii again.

ix. Do not forget to either log out or shut down your browser!

SFMA Datamart

Some general cautions at the beginning here:

The first thing a new user should do is build a report that agrees to a hardcopy report from SFMA. This is to assure yourself that you have created a report, any joins are correct, and that the data you got from the query is in fact what you wanted.

The Datamart is not the “book of record” for the state. The Datamart simply mirrors selected information from SFMA. Original documents and the actual SFMA records are the state’s books of record. The Datamart is available to simplify (we hope) the summarization of the in formation in the books of record.

Reports are only as good as the person producing reports. Review results with a critical eye – if the numbers look wrong, they probably are!

Again – this is raw data we have on the Datamart – the report producer must bring order out of that raw data. There are always risks associated with making assumptions or decisions about the data – this is true for any report. Make sure you know what you are asking for, and at least a general idea of what the amounts should look like.


Tables on the Datamart

There are two types of tables on the Datamart: Financial tables and Profile tables. The main difference between the two, strangely enough, is that the financial tables contain financial information, such as General Ledger balances, and transaction amounts. Use the profile tables to provide titles for various data elements, or to allow a roll-up of information to a higher level than the financial table can do.

Note that if you want detail finer than what is on any of the financial tables, you are not going to have much luck. The Datamart and the query tools can only slice and dice the data as deeply as the data goes. Say you want to get a subset of your comptroller objects. If you do not have any way to tie that subset together using data on the Datamart, you are going to be stuck (likely) having to go to the detailed records and the original documents. An example here is Meals and Lodging. If you use comptroller object only (no agency object), you will not be able to get a report out of SFMA giving you a breakdown of Meals costs separate from Lodging. You might be able to get something based on vendor, but you also might not.

SFMA Financial Tables on the Datamart(See attached listings for details of data elements in the tables)

Accounting Event Tables

1. Active Accounting Event – This table mirrors selected information from the SFMA Active Accounting Event table. Data is roughly three months of active transactions.

2. All Accounting Event - This table mirrors selected information from the

SFMA All Accounting Event table. Data available is for transactions 7/1/2000 to date.

3. Archive Accounting Event - This table mirrors selected information from the SFMA Active Accounting Event table, and archived data from the inception of the SFMA. The period available is dependent on the date of agency implementation onto SFMA. The earliest data for “1A” agencies is March 1995. Note: this is subject to change. Because the Datamart grows at about ½ a gigabyte per week, we are looking at a data purge similar to SFMA.

The accounting event tables are the lowest level of detail in the Datamart. All other financial tables are summaries in some way of this information.

One thing to be aware of is that the date fields in the Datamart are actual date fields. Date fields in SFMA are text fields. While those fields are edited to


assure proper dates are being used, this allows SFMA to contain dates not on the calendar. Period 13 identified through effective date is the best example of this: On SFMA period 13 is identified as 6/31/2004. In the Datamart, date fields are not allowed to have dates not on the calendar – a data validation feature in DB2. In order for us to move data to the Datamart, we have to do some “data washing”. Dates like 6/31/2004 are changed to 6/30/2004, a new data element is introduced called Batch Date Ci. The download program picks up the last digit in the month and records it in this field. This way, any time an incorrect date is entered into SFMA, it is corrected in the Datamart, and an indicator is provided.

Note though, that another data field in the accounting event tables will provide you with the fiscal month. The Fiscal Month field (surprisingly enough!) identifies the appropriate fiscal month, including month 13.

General Ledger Detail Tables

1. General Ledger Detail – This table mirrors selected information on the SFMA General Ledger table. This table summarizes General Ledger balances, and summarizes at the object and agency object level.

2. YE General Ledger Detail - This table mirrors selected information on the

SFMA General Ledger table. However, the data on this table is updated only during year-end close. During the period of mid-July to close in August, this table is updated with the balances in the current fiscal year only. At other times of the year, this table contains the prior fiscal year balances only, and is not updated. This table summarizes General Ledger balances, and summarizes at the object and agency object level.

3. Arch GL Detail - This table mirrors selected information on the SFMA General Ledger table, plus similar data from the archived general ledger data. The period available is dependent on the date of agency implementation onto SFMA. The earliest data for “1A” agencies is March 1995. Note: this is subject to change. Because the Datamart grows at about ½ a gigabyte per week, we are looking at a data purge similar to SFMA.


General Ledger Summary Tables

1. General Ledger Summary – This table mirrors selected information on the SFMA Summary GL table. This table summarizes General Ledger balances, but does not contain any object level detail.

2. YE General Ledger Detail - This table mirrors selected information on the

SFMA Summary GL table. However, the data on this table updates only during close. During the period of mid-July to end of August, this table updates with the balances in the current fiscal year only. At other times of the year, this table contains the prior fiscal year balances only. This table summarizes General Ledger balances, but does not contain any object level detail.

3. Arch GL Detail - This table mirrors selected information on the SFMA Summary GL table, plus similar data from the archived general ledger data. The period available is dependent on the date of agency implementation onto SFMA. The earliest data for “1A” agencies is March 1995. Note: this is subject to change. Because the Datamart grows at about ½ a gigabyte per week, we are looking at a data purge similar to SFMA.

The General Ledger Tables are a higher level of summarization in the Datamart from the accounting event tables. Balances for the tables are by month, and are cumulative by month. This means that a balance in Mo 03 Balance will contain the total of months one, two and three in that field. Monthly information for nominal GL accounts, except for month just closed, must be created using calculated fields.

Other Special Financial Tables

Appropriation Financial Table

The Appropriation Financial table mirrors selected information from the SFMA Appropriation table. This provides information similar to the “62” screen on SFMA.

Contract Financial Table

The Contract Financial table mirrors selected information from the SFMA Cont Fin (contract financial) table. This provides information similar to the “68” screen on SFMA.


Grant Financial Table

The Grant Financial table mirrors selected information from the SFMA Grant table. This provides information similar to the “66” screen on SFMA.

Project Financial Table

The Project Financial table mirrors selected information from the SFMA Project table. This provides information similar to the “80” screen on SFMA.

Receipt Extr Sum Table

The Receipt Extr Sum table mirrors selected information from the SFMA Receipt EX Sum table. This provides information similar to the “XX” screen on SFMA.

Document Financial Tables

1. The Doc Financial table mirrors selected information from the SFMA Document table. This provides information similar to the “64” screen on SFMA. Data available is for transactions 7/1/2000 to date.

2. The Arch Doc Financial table mirrors selected information from the SFMA

Document table, plus all archived information. This provides information similar to the “64” screen on SFMA. Data available is for transactions from inception of SFMA to date.

Payment Control Table

The Payment Control table mirrors selected information from the SFMA PMT Control table. This provides information similar to the “XX” screen on SFMA.

Profile Tables

Agency – D02 ScreenAgency Code 1 – D26 ScreenAgency Code 2 – D27 ScreenAgency Code 3 – D36 ScreenAgy GL – D32 ScreenAgy Obj – D11 ScreenAppn No – 20 ScreenAppr Fund – D22 ScreenBudget Obj – D01 Screen


Cash Fund – D73 ScreenContract – 30 ScreenCompt Obj – D10 ScreenCompt Srce Group – D09 Screen (this is now a tie to ORBITS numbers)Fund D23 – D23 ScreenGAAP Fund – D24 ScreenGL Acct – D31 ScreenGrant No – D47 ScreenGrant Obj – D48 ScreenGrant Phase – 29 ScreenIndex No – 24 ScreenOperator ID – D96A/B ScreenOrg Code – D03 ScreenPCA – 26 ScreenPCA Index Rel – 21 ScreenProgram Code – D04 ScreenProject No – D42 ScreenProject Phase – 27 ScreenState Fund Grp – D20 ScreenSub Grantee – 31 ScreenTDate – Generated – not on SFMA

OSPA Tables on the Datamart

The OSPA tables on the Datamart are not as sophisticated as the SFMA tables. These tables are each a flat-file database made from report output. This does make querying a bit more difficult, as you really have to know exactly what you are looking for, and where to find it.

OSPS DED TablesOSPS Deductions

OSPS Ded - Current BienniumOSPS Ded B - Current and Prior BienniumOSPS Ded PB - Prior BienniumOSPS Ded PP - Prior Prior Biennium (two prior biennia)

OSPS JS Lab ADJOSPS Job Status, Labor and Gross adjustments

OSPS JS Lab ADJ - Current BienniumOSPS JS Lab ADJ B - Current and Prior BienniumOSPS JS Lab ADJ PB - Prior BienniumOSPS JS Lab ADJ PP - Prior Prior Biennium (two prior biennia)


OSPS Labor Cost TablesLabor Cost

OSPS Labor Cost - Current BienniumOSPS Labor Cost B - Current and Prior BienniumOSPS Labor Cost PB - Prior BienniumOSPS Labor Cost PP - Prior Prior Biennium (2 prior biennia)

OSPS LV Accrual TablesLeave Accruals

OSPS LV Accrual - Current BienniumOSPS LV Accrual B - Current and Prior BienniumOSPS LV Accrual PB - Prior BienniumOSPS LV Accrual PP - Prior Prior Biennium (two prior biennia)

OSPS Net Pay TablesNet Pay amounts

OSPS Net Pay - Current BienniumOSPS Net Pay B - Current and Prior BienniumOSPS Net Pay PB - Prior BienniumOSPS Net Pay PP - Prior Prior Biennium (two prior biennia)

Note: When querying the OSPS tables, you must a five-digit agency number in order to query. For example, DAS is agency 107 for SFMS, but 10700 for OSPS purposes. Another example is Parks & Recreation’s agency number on SFMS is 634, but on OSPS, it is 63400.

General Querying Suggestions

1. Make sure to save your queries – especially if you are running data for an auditor. An auditor may want to know what criteria were used to create a report as part of their verification procedures. If you do not have the query, it can be very difficult to reproduce the information. In addition, you may want to use the query that produced a report six months later – if you do not have the query, again, reproducing the query can be difficult.

2. Unless you have a reason, always use the custom joins provided by the Datamart. IRMD created these joins in order to simplify querying, and to


protect new users from runaway queries. If a join does not show up when using custom joins, there is usually a reason, and it is usually because the query will produce inappropriate results.

3. Test your query by limiting the number of rows to 1000. This gives you a reasonable amount of data to look at in the results section before you run a full query.

4. If you think you have results that seem incorrect, check your query first – did you limit your data the way you really wanted it limited, or did you not include a (or include too many) limits? If the limits are still correct, remove one profile table at a time – this will catch a query in which you have inconsistent data (as discussed in the joins section).

5. When building a query, start with your financial table first. When selecting fields for your request, the most efficient way to query is to take as much as you can out of the financial table, and limit the number of joins in your query.

6. Set up your limits based as much on your financial table fields as possible. Limiting a field contained in a profile table tends to slow the query down – the database returns a result set based on the query to the financial table, and then runs through that results set to limit on the profile table. If you can limit only on fields in the financial table, your queries come back more quickly.

7. In Brio, place your financial table to the far left of the query screen – yes, unbelievably, it can affect your query! Place profile tables you wish to join to the right of the financial table. This seems to speed up queries to some degree, especially if you are using an outer join.

8. Try to limit your queries as much as possible – this lowers query time. If you need to look at a broad view of your data, do not hesitate to bring in more data than you think you need. You can always refine your data output (in Brio anyway) via limits in the results set. When you get the results you want, if you expect to rerun the query regularly, transfer the limits in the results set to the query limits.

9. When querying any of the All Accounting Event tables, make sure if possible to include Agency and GL Account in your query. This speeds the query and limits the amount of data returned. Remember, the Accounting Event table by itself is 99+ gigabytes of data in the Datamart.

10. If your OCE does not have the custom joins activated, be very careful of the joins Brio automatically makes. They are made exclusively on whether field names match exactly, not whether they are key fields or not. Try always to make sure your OCE has the custom joins.


11. If using “Show Values” in the results set, be aware that if you have a large dataset, it could take quite some time for the list to come back for selection. We do not recommend that you use “Show Values” in the query section. “Show Values” actually queries against the Datamart, and takes quite some time, depending on the field chosen. Always be sure of the form of the limits you wish to include in the query limits.

12. Always limit your query by AY or FY unless there is an overriding need not to use these limits. Without these limits, you will get data for all years on the database.

13. Never, never, NEVER join two financial tables together. When you run the query, you will get a “Cartesian product” - basically, the number of unique records in one table multiplied by the number of matching records in the other joined table. A runaway query is usually a Cartesian product query.

14. If you have a runaway query, or you decide to end a query, you cannot end a query just by closing down your PC-based program! When you send a query to the Datamart, all of the work goes on at the Datamart processors, not your PC. Closing down your application does not send a message to the Datamart to close the query. If you want to stop a query at the Datamart, you must use the Kill Query process via the Web.

15. Whenever you have Brio open and you have logged in to the Datamart, you have a session running. Open several queries, and you will have several sessions open. Be aware of this if you are going to kill one of the queries. The kill process kills all active sessions, not just the single one you wish to kill.

16. If you do not need detail transaction information, you can use the GL Detail to create income statements. Simply limit the GL Accounts to the nominal revenue, expenditure and transfer control accounts.

Killing a Query

If you wish to terminate a query, the process depends on where you are in the query. These procedures are for Brio – other products have different methods.

Stopping a Query From Brio

If you are already receiving data from the Datamart, indicated by a message in the lower left-hand corer as the example shows, hit ALT+END. This will stop data from coming to the Datamart, and ends the query at the Datamart.


However, the message at the lower left-hand corner says “Waiting for server”, you must follow the kill query process.

Kill a Query from the Web

A warning first: If you kill a query from the web, you will have to close your session of Brio down and restart. The kill query process cuts the communication to Brio, and restarting the session is the only way to reconnect.

Another warning: When you use the kill query process, if you have more than one query running, you kill all queries running, or all session connections open! This is an issue with the software we use to kill queries, and is not something we can change at this time.

To kill a query via the Web:

1. Open your browser and type the following URL:


2. Type in your user name (RACF ID) and Datamart password 3. If you get a message about certificates, just click “OK”. This will put you

at this screen:



4. Click on the “List or Kill Queries” hyperlink.


5. You are now at the screen below. This screen will list all of your open

sessions on the Datamart.

6. Click the “Kill Queries” button. ALL your queries will be killed. This is what the screen will look like after killing queries.

Note that all queries have been removed. Close your browser to log off.


Documents

Training Materials