Installation and Development Tools National Center for Supercomputing Applications University of...

Installation and Development Tools

National Center for Supercomputing ApplicationsUniversity of Illinois at Urbana-Champaign

The SEASR project and its Meandre infrastructureare sponsored by The Andrew W. Mellon Foundation

Outline

• Installation

• Meandre servers and clusters

• Development Tools: Eclipse Plugin

Considerations

• Do you want to use SEASR-powered services?

– May not need to install anything (besides a browser)

• Do you want to run analytics on your laptop?

– Quick 3 step process

• Do you want to provide SEASR-powered services?

– Start simple

– Scale as needed

• Deploying all the extra goodies

Using SEASR-Powered Services

• SEASR provides some demo services

• Requires a browser

• You can access them from – Community Hub to execute a flow

• http://seasr.org

– Meandre Server Client to execute a flow; or tune properties and execute a flow• Hosted at http://demo.seasr.org:1714

– Meandre Workbench to execute a flow; or tune properties and execute a flow; or create a flow• Hosted at http://demo.seasr.org:1712

– Zotero to analyze your collections with existing flows

I Need To Run SEASR on my laptop• I want to run on my laptop (server)

– I have copyrighted information

– I have collection for analysis that is too big to be moved

– I just want to test it and have fun with it

• Getting a Meandre server up and running in 3 steps1. Install Java http://www.java.com/en/download/

2. Download the Meandre server jar into a new directory http://seasr.org/meandre/download/

3. Use the “Start-Infrastructure” or type “java –jar meandre-server-1.4.5.jar”

• Access your new installation at– http://localhost:1714/public/services/ping.html

Specialized Downloadable Bundles• On the SEASR/Meandre download site

– http://seasr.org/meandre/download

• Installation bundles available for:

– Mac OS

– Linux

– Windows

• Bundles contain:

– Zip file that includes executable files

– Set of demo components and flows

• Requires Java (1.5 or greater) to be installed

Bundles Include

• The bundle comes with

– Meandre Server

– ZigZag console/compiler/runtime

– Meandre Workbench (also provided as a war file)

• Provides simple scripts to

– Start/stop the Meandre server

– Start/stop the Meandre Workbench

What About Setting Up My Own Server?• You can also deploy the bundles on a

server using the same approach.

Customization of My Server

• This will support– Moderated traffic ??we don’t know what this means??

– Persistent web services can be provided using this server

• Application Server– Workbench can be deployed alongside the Meadre

Server in the embedded Jetty Application Server

– Workbench can be deployed using your favorite application server using the .war file

• Database Options– Meandre uses an embedded Derby as the database

– Meandre can be also be setup to use Mysql for the database

Backend Using Derby Database

• default meandre-config-store.xml <entry key="DB_USER"></entry>

<entry key="DB_DRIVER_CLASS"> org.apache.derby.jdbc.EmbeddedDriver</entry> <entry key="DB">Derby</entry> <entry key="DB_PASSWD"></entry> <entry key="DB_URL”>jdbc:derby:./MeandreStore;create=true;logDevice=./DerbyLog</entry>

Backend Using MySQL

• meandre-config-store.xml to <entry key="DB_USER">USERNAME</entry>

<entry key="DB_DRIVER_CLASS”>com.mysql.jdbc.Driver</entry> <entry key="DB">MySQL</entry> <entry key="DB_PASSWD">PASSWORD</entry> <entry key="DB_URL”><![CDATA[jdbc:mysql://your-server.com/YOURDB?useUnicode=yes&characterEncoding=utf8&autoReconnect=true]]></entry>

• Changing from Derby to MySQL– Stop the server

– Change the meandre-config-store.xml file

– Restart the server

– Now your server is backend on MySQL

Scaling Up ??Bernie will send you this??• Two possible routes– Deploy a farm of self-contained services (via

zigzag)

– Use the Meandre Cluster solution

• Both require your sysadmin/netadmin to provide a highly available load balancer (some virtual appliances available)

• To create a cluster– Use the previous MySQL set up

– Point all the servers to the same database

– The server interface pages will allow you to monitor of the servers

Installing The Workbench

• Use the installation bundles

• Use the war file

– Install your favorite application server

– Deploy the war file against the application server

Installing the Community Hub

• The community hub is a Wordpress plugin• Allows to point to a Meandre server• Makes all the flows available for execution • Pages and posts can add the tag

– [meandre-desc SERVER_REPOSITORY_URL FLOW_URI]– E.g. [meandre-desc

http://demo.seasr.org:1714/public/services/repository.rdf http://test.org/flow/text_processing_demo_1]

• Renders the description of the flow information and provides a simple execute button to allow visitors to run the flow

• Deploy the zip file into Wordpress plugins directory

Eclipse Plugin for Developers

• On the SEASR/Meandre download site

– http://seasr.org/meandre/download/

• Steps for installation

• Exit Eclipse

• Download zip file into Eclipse/dropins directory

• Unzip file

• Restart Eclipse

Developers: Eclipse Plugin

• Uploads components to the Meandre Server

• Lists components installed

• Allows for removal of components

• Shows additional data of interest to a programmer

Meandre: Basic Single Server

The SEASR project and its Meandre infrastructureare sponsored by The Andrew W. Mellon Foundation

Meandre MDX: Cloud Computing• Servers can be

– instantiated on demand

– disposed when done or on demand

• A cluster is formed by at least one server

• The Meandre Distributed Exchange (MDX)

– Orchestrates operational integrity by managing cluster configuration and membership using a shared database resource.

The SEASR project and its Meandre infrastructureare sponsored by The Andrew W. Mellon Foundation

Meandre MDX: The Picture

The SEASR project and its Meandre infrastructureare sponsored by The Andrew W. Mellon Foundation

MDX

Bac

kbon

e

Meandre MDX: The Architecture• Virtualization infrastructure

– Provide a uniform access to the underlying execution environment. It relies on virtualization of machines and the usage of Java for hardware abstraction.

• IO standardization– A unified layer provides access to shared data stores, distributed file-system, specialized metadata stores, and access to other service-oriented architecture gateways.

The SEASR project and its Meandre infrastructureare sponsored by The Andrew W. Mellon Foundation

Meandre MDX: The Architecture• Data-intensive flow infrastructure

– Provide the basic Meandre execution engine for data-intensive flows, component repositories and discovery mechanisms, extensible plugins and web user interfaces (webUIs).

• Interaction layer– Can provide self-contained applications via webUIs, create plugins for third-party services, interact with the embedding application that relies on the Meandre engine, or provide services to the cloud.

The SEASR project and its Meandre infrastructureare sponsored by The Andrew W. Mellon Foundation

Demonstration

• Installation of Meandre

• Meandre Eclipse Plugin

• JIRA, Confluence, Bamboo - what they are and what we use them for

Learning Exercises

• Have participants download and install SEASR on their personal computers

• Have participants sign up for accounts to access the SEASR suite of Atlassian tools

• Use JIRA to log a support request

Discussion Questions

• What challenges (if any) would scholars have installing the SEASR software?

• Do you see your institution's IT department running the SEASR environment or would it be your research group?