DR5 Casjobs Overview and Operating Experience

Jared Platson

23 May 2007

          

                                                                                                                                                                

2

The Basics SkyServer Query vs CasJobs Query. • SkyServer SQL Search is a tool that can be used to query data without needing an account.  The query running time is restricted and the results are only available in the web page.• Running a CasJobs query requires an account to log into and the results can be web based or stored in a database for future use.  Once the data subset has been stored in a users MyDB it can be queried much quicker.  

CasJobs Home Page: ( http://casjobs.sdss.org/CasJobs/ ) 

Query page: http://casjobs.sdss.org/CasJobs/SubmitJob.aspx

For DR5 3 queries can be running at the same time.   A user can submit as many jobs as they like only one query will run at a time.

5

CasJobs Admin Queue http://casjobs.sdss.org/CasJobs/admin.aspx

6

MyDB’s via the web interface.http://casjobs.sdss.org/CasJobs/MyDB.aspx

7

CasJobs Service:

Casjobs Service Runs under 2 services:1.  CasJobs Only Service2.  CasJobs Output Service

Output can be several formats like: HTML, CSV, VOTable and Dataset

8

CasJobs Service continued:The logging data server is where MyDB’s and BatchAdmin reside and the data server(s) that contain the actual data being queried.

InternetLogging Server

Data Server

Data Server

Data Server

9

CasJobs Service continued:When a user creates a CasJobs Login they will get their own MyDB database.  named uniquely named like MyDB_123456789. 

The default size for a MyDB is 500MB.  Occasionally we get a request for a larger size (usually from a collab user).  Increasing the size is usually ok.

10

CasJobs Service continued:When a user is created they will each get their own personal database a 'MyDB‘ named like MyDB_WebServiceID.  The default size for a MyDB is 500MB, this can be increased per request.

11

CasJobs Service continued:

There are 2 basic types of query for CasJobs, quick Query and Long query.  A quick query returns the results to the web page while a long query gets added to the jobs queue and the results are then stored in the users MyDB.

12

Monitoring and Troubleshooting

On occasion the CasJobs service might fail. If this happens a system batch script will automatically restart the service and send email notification to system admins.

The time limit for long quries is 10 hours. If a users query exceeds 10 hours we notify the user and ask them to tune their query and resubmit it.

13

Idera Diagnostic Manager (DM): This is the tool we use to monitor all of our SQL Server machines.  DM monitors SQL processes, blocking, memory, I/O, CPU, network packets, Jobs, connections etc.   You can use Windows tools like Performance monitor, Task Manager, and SQL Server tools to get some of this information as well.  Idera packages it all up in one easy to use interface. This tool is helpful but is not required.

The Summary Page:

The Process Page: Very useful information regarding all running process’ on a database server. Here you can find orphaned/suspect jobs and if necessary kill them.