Upload
biovia
View
1.241
Download
0
Tags:
Embed Size (px)
DESCRIPTION
This session will review significant enhancements to Pipeline Pilot in recent releases. Areas covered are: Professional client, administration, security, integration, databases, and collections (chemistry, next gen sequencing, documents and text, statistics, and imaging).
Citation preview
(ATS3-PLAT01) Recent developments in Pipeline Pilot
Ton van DaelenSr. Director, Platform Product Management
The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.
Introduction
• Target audience– Administrators– Developers
• What will you learn?– What is new that affects me or my users (and me indirectly)
• More of What’s New in– (ATS3-PLAT07) Pipeline Pilot Tips, Tricks and Challenges
• Slides and examples will be made available
Topics
• Release history• Administration
– Performance enhancements– Server monitoring– Usage monitoring – Protocol validation– Security
• Integration– .NET components– Python (prototype)
• SharePoint integration
• Pipeline Pilot 8.0 – Dec 2010– Design mode– Local subprotocols– Data flow monitoring
• Pipeline Pilot 8.5 – Aug 2011– Support growing deployments– Isentris and Notebook integration– Support for building ‘apps’
• Pipeline Pilot 8.5 CU1 – Nov 2011– Protocol comparison– Performance
Pipeline Pilot Release History
9.0Enterprise
Workgroup
Personal1-4
6.0
7.0
8.58.0
5.0
Science and Applications Release History
7.5
8.08.0
8.5
Imaging (HCS)Reporting (Canvas elements)Advanced data modeling
Chemistry (MDL formats)Reporting (Stylesheets)Imaging (DICOM)Direct cartridge
Imaging (Big Image Explorer)Chemistry (Content, NEMA, Cheshire)Next generation sequencing
Isentris and Notebook integrationChemRegBioReg
Database Search
Apps
Collections
Mini-Survey
• What server version(s) are you running?– Pre-7.5?– 7.5– 8.0– 8.5
Administration
Performance Testing
• Performance increasingly critical because of ‘apps’– Both Accelrys and custom
• Performance testing– Track our progress– Spot errors
• How do we do this?
Performance Testing
Threads / Windows
Throughput / Windows
Linux / Threads and Throughput
1 5 10 20 50 100
ODBC 7.5 235 202 192 218 1558 18622
ODBC 8.0 224 166 164 156 482 1015
ODBC 8.0 (PoolID)
122 137 117 122 246 367
10003000500070009000
1100013000150001700019000
Latency of PP 7.5, 8.0 & 8.0 PoolID
Avg
Resp
onse
Tim
e (m
s)
1 5 10 20 50 100
ODBC 8.0 224 166 164 156 482 1015
ODBC 8.0 (PoolID) 122 137 117 122 246 367
100
300
500
700
900
1100
Latency of PP 8.0 & 8.0 PoolID
Avg
Resp
onse
Tim
e (m
s)
Pipeline Pilot Configurations Compared (ODBC Test Case)
ODBC latency is 44% lower than 7.5 and 22% lower than 8.0
w/out pooled servers
Improved scalability under load
Server Maintenance and Monitoring
• Administration portal – Run diagnostics– View and cancel running
jobs– View log files– Monitor CPU and memory
• Pro client– Server status indicator
Server Maintenance and Monitoring
• External tools
Windows Performance Counters Ganglia (Linux)
Usage Monitoring
• Why?– Internal accounting (‘Which dept is using imaging?’)– Justify investment (‘Are we using reporting?’)– Administrative (‘What protocols haven’t been used in 1 yr?’)
• Approaches– Usage report in admin portal– Custom analysis of log files– Protocol database analysis using Component Reader
• More in: (ATS3-PLAT11) Advanced Planning
Admin Portal Usage Report
Usage Report Example
Introduced in 8.5. For pre 8.5 servers, install protocols from ftp://download.accelrys.com/protocols/SummaryReport.zip.
Logfile Folder
Remote logfiles. Allows for manual copying if server is
not connected Local logfiles.
Installation logfiles.
Usage Log Analysis Through Protocols
• Usage protocol in C:\Program Files\Accelrys\PPS\apps\scitegic\admin\dataroot\xml
C:\Program Files\Accelrys\PPS\logs\usage\Pipeline Pilot.log
Analysis Example
Runs by User Runs by ClientType Most Frequent Protocols
Analyzing the XMLDB
• Questions to ask– Non-empty Password parameters?– Empty required parameter field?– File references?– Shortcuts as copies?– Protocol for a particular user?– Recent protocols?– Components of a certain collection?– Deprecated components?
• Combined with log file– What components were used recently?– For a migration, what protocols were run recently?
Analyzing the Xmldb
Protocol info:• Author• Date• Component ID
Component info:• Data source references• Passwords
Component ParametersParamName ParamValue ParamTypeSource data\Images\Cards\README.txt URLTypeMaximum LongTypeSourceTag None StringTypeKeep Properties PropertyPreviewTypeHasColumnNames TRUE BoolTypeDelimiter FigureOut StringTypeMultipleValueDelimiter None StringTypeParseNDimensionalArrayStrings TRUE BoolTypeAdditional Options GroupTypeCommentLines StringTypeUndefinedString StringTypeAllLines FALSE BoolTypeTrimWhitespace TRUE BoolTypeCheckSyntax TRUE BoolTypeAddEmptyProperties FALSE BoolTypeOnGeneralError Halt StringTypeOnDataTypeError Halt StringTypeComponentAttributes ComponentReturnsPass StringTypeRegistrant scitegic/generic StringTypeRegistrationDate Sat Feb 26 16:31:03 2011 StringTypeComponentDisplayName Read README.txt StringTypeDerivedFrom Delimited Reader StringTypeCustomErrorText StringTypeComponentIcon DocumentNode StringTypeComponentDisabled 0 LongTypeComponentGUID {7D2AF7C7-A077-4201-B10B-8E8283A5EADD} StringTypeComponentPoint 90 -95 StringTypeComponentHelp 100 StringTypeComponent Sticky Notes StringTypeRegistrationVersion 8.4.5.18 StringTypeComponentLocalID 0 LongTypeComponent Path Components\Data Access and Manipulation\File Readers\Delimited Text ReaderStringType
Accelrys or custom component?
Caption
Help text
Component Reader Example.xml
Date registered
Gotcha!
Component Parameter Interface
Component Parameter Interface
• Inspect protocols before moving to production• Rules checking:
– Hard-coded passwords– Bad file of database references– Broken shortcuts
• Configurable for different deployment modes – Standard protocol, Reports, Web services, RSS
Feeds, SharePoint, etc.
Protocol Validation
• Validation can be set up as hard requirement for publication• Operation is cancelled on validation errors
Protocol Validation
*Not a standard feature. Request from support
(ATS3-PLAT03) Enterprise Protocol & Package Deployment
• Component Review-o-Matic Protocol– Checks Documentation– Checks Parameter Names
• Create your own custom rules
Automating Component Best Practices
See: Components\Database and Application Integration\Utilities\Prototypes\Usability\Component Review-o-Matic
Security
Windows Integrated Authentication
• Run Web Port without providing username /password
• Supported via Kerberos and NTLM • Requirements
– DOMAIN authentication– Kerberos only works on remote servers.
Negotiation on local servers uses NTLM instead of Kerberos, so some restrictions may apply
– Pipeline Pilot Apache service must have valid Service Provider Name on the DOMAIN server. Running as "Local System" usually fulfills this requirement
– Configuration required on server and client’s browser
Windows Integrated Authentication
• Requires Active Directory settings• Enables Secure Services• Authentication provided by NTLM/Kerberos• Limitations
– Delegation not supported– Accessing network resources (e.g., proxy server authentication with
credentials, UNC paths) – Using PP credentials for components that require authentication (e.g.,
database, readers, integration, etc.) – Running protocols from within a protocol (e.g., RunProtocol PilotScript
function, parallel subprotocols)
Server and Client Configuration
• Run Apache service as a user: – To set up Kerberos service principal name for the Pipeline Pilot server, run the following on the
Domain controller: • setspn.exe -a http/SERVERNAME DOMAIN\apache_username
– On the Pipeline Pilot server, use the Services tool and set the Logon User for the "scitegic_apache_8_5_0" service.
• Client Configuration – Internet Explorer:
• Add the server as a trusted site (Tools > Internet Options > Security > Trusted Sites > Custom Level > User Authentication > Logon).
• Select Automatic logon with current user name and password. • If your server is already part of the Local Intranet, select Automatic logon only in Intranet zone.
– Firefox: • Browse to "about:config" and add the server names to the following preferences:
– network.negotiate-auth.trusted-uris – network.automatic-ntlm-auth.trusted-uris
Pass-through (Impersonation)
Scientist
Pipeline Pilot Web Services
Webform Basic
Database
ODBC
Web Services
Basic
SciServer
User Token
Integration
Pipeline Pilot Integration Points
DBs
Pipeline Pilot Enterprise Server
Protocol Runtime Environment (scisvr)
VB Script (On
Client)
Run Program Java Perl Python
VB Script (On
Server)
Admin Portal Help PortalWeb Apps Web Services API
.NET SOAP & HTTP
Telnet / FTP SSH / SCP ODBC /
JDBC
VB Script Cmd Line Java Classes
Perl Scripts
.NET Classes
REST / HTTP
Service
SOAP Service Cmd Line
Professional ClientRun Protocol
Command Line Client
Web Browser
Web Port
JavaScript Client SDK
JavaScript Client
.NET Client SDK
.NET Client
Java Client SDK
Java Client
Web Services API
SOAP Client
.NET
• .NET On Server– Points to a compiled .NET Assembly
• Dynamic .NET– Embed your .NET code directly into
the Server Component
• Leverage .NET supported languages– C#– VB.NET– IronPython
Component Development Update
A look at building a .NET Server Component
.NET Components Demo
• Hello world using dynamic C#
• Hello world using compiled C#
Python (Prototype)
• Requires– ActiveState Python– .NET runtime
• Windows only• Prototype now available
– Looking for testers
properties = component_datarecord.GetRoot().GetProperties()
queryString = properties.GetByName(u"queryString").GetValue().GetString();params = urllib.urlencode({'corpID': queryString})f = urllib.urlopen("http://vm-discoweb-dev.corp.mpi.com/rest/Compound/lookupCorpID/getCompoundProperties?%s" % params)properties.Define("Response", f.read());
SharePoint
Pipeline Pilot & SharePoint
Document Library
SharePointPipeline PilotLists
Pipeline Pilot SharePoint Bridge
Web Parts (3)Web Services
Read Only
Read/Write
Run Job
Protocols
User Created
IT Created
ISV Created
PP Scientifically Enables SharePoint
• Problem/Need: Want to create Pipeline Pilot-based Web Parts and deploy to organization
• Solution: Create packaged (“standalone”) Web Parts (Chapter 4, SharePoint Bridge Admin Guide)
Deploying Custom Web Parts
• Custom Web Parts are available in Web Part Gallery
• Provides a much simpler mechanism for end-users
Using Custom Web Parts
SharePoint Document Library
• Read from / write to document library– Use standard Reader/Writer
components– With Domain Authentication,
R/W will simply work– Alternatives
• Copy between document repository and temporary local version
• Add authentication options to Reader/Writer/Copy components
Summary
• Reasons to upgrade– Administrative– Development– Science and applications
• Plan your upgrade– Engage Accelrys support and services– (ATS3-PLAT11) Advanced Planning
• Resources– Component development guide– Accelrys Community: http://community.accelrys.com/
• Other sessions of interest– (ATS03-PLAT07) Pipeline Pilot Tips, Tricks and Challenges
The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.
For more information on the Accelrys Tech Summits and other IT & Developer information, please visit:https://community.accelrys.com/groups/it-dev