Click here to load reader

Co-allocation Using HARC IV. ResourceManagers HARC Workshop University of Manchester

  • View
    215

  • Download
    2

Embed Size (px)

Text of Co-allocation Using HARC IV. ResourceManagers HARC Workshop University of Manchester

  • Co-allocation Using HARCIV. ResourceManagersHARC Workshop University of Manchester

  • PhilosophyNew types of RMs can be written by othersExisting RMs can be customizedInterfaces can be enhanced or changedNone of this means changing the acceptor codeAPI is extensible too

    Good community contribution modelCCT keeps control of the acceptor codeThe acceptor code will become very stable (already less than one commit per month)The community evolves the system

  • Are RMs Easy to InstallHarder than client softwareMuch easier than Acceptors

    Complexity is in the right place:Only a few people install and configure Acceptors (infrastructure), which is hardSome people modify/write RMs, which is not too hardMore people install and configure RMs which is easyMany people install and configure the Client software, which is trivial

  • Pre-installation - PerlRMs are written in perl, to make installation trivialHowever, they need a large number of CPAN modules to be installedSome of these, e.g. Net::SSLeay and Crypt::SSLeay are not trivial

    There is a document which contains things to watch out forLists previously seen problems, with solutionsBasically a list of exceptionsNow 7 pages of text!Theres a lot of AIX content...

  • Pre-installation - CertificateHARC RM needs a certificateWe dont recommend re-using the host certificateGet a service certificate

    UK e-Science CA now supports:harccrm for Compute RMs (CRMs)/C=UK/O=eScience/OU=Manchester/L=MC /CN=harccrm/man2.nw-grid.ac.uk/emailAddress=...harcacceptor for Acceptors/C=UK/O=eScience/OU=Manchester/L=MC /CN=harcacceptor/man4.nw-grid.ac.uk/emailAddress=....

  • Installation ProcedureTheres an installer which installs stuff from the CVS tree - this may changeHARC environment variable points to the root of the repo (negotiation directory)

    You have a subdirectory in$HARC/rm-service/configFor example$HARC/rm-service/config/nw-grid/man2

  • Installation Procedure1. Create Contentsinstall.config - more shortlygrid-mapfile - GT-style mapfile for cert to username mapping (usually a sym-link to /etc/grid-security/grid-mapfile)acceptor_mapfile - a list of the Acceptor DNs, and also their CA cert DNscacerts directory, containing CA Certs for your cert and the Acceptor certs, in PEM format, suffix .crt

    2. Then a trivial Installinstall-rm nw-grid/man2 /usr/local/man2-rm

  • install.configRM_INNER_TYPE=SimpleComputeRM_COMPUTE_NODENAME=man2.nw-grid.ac.ukRM_COMPUTE_BATCH_TYPE=TorqueMauiRM_COMPUTE_MEMORY_MB_PER_CPU=4096RM_COMPUTE_CPUS=8RM_MAUI_COMMAND_DIR=/usr/local/maui/binRM_RESOURCE_DESCRIPTION='The Manchester NW-Grid node, a Dual AMD Opteron Linux cluster

    RM_HOST=130.88.200.242RM_URL=man2-rmRM_PORT=9393

  • install.configRM_INNER_TYPE=SimpleComputeRM_COMPUTE_NODENAME=man2.nw-grid.ac.ukRM_COMPUTE_BATCH_TYPE=TorqueMauiRM_COMPUTE_MEMORY_MB_PER_CPU=4096RM_COMPUTE_CPUS=8RM_MAUI_COMMAND_DIR=/usr/local/maui/binRM_RESOURCE_DESCRIPTION='The Manchester NW-Grid node, a Dual AMD Opteron Linux cluster

    RM_HOST=130.88.200.242RM_URL=man2-rmRM_PORT=9393

    man2.nw-grid.ac.uk https://man2.nw-grid.ac.uk:9393/man2-rm/

  • Installation StepBefore InstallingNeed PERL5LIB and LD_LIBRARY_PATH to be defined in your environment when you installOr can add these to the config fileDont have to set these if you dont need to

    Then a trivial Installinstall-rm nw-grid/man2 /usr/local/man2-rmScript is in $HARC/rm-service/scripts

    What does this do?

  • What happens?rm-service $ scripts/install-rm nw-grid/man2 /Users/jonmaclaren/man2-rmMakefile.crt ... Skippedcct-ca.crt ... 5fb2fc80.0old-uk-escience-ca.crt ... 01621954.0uk-escience-ca.crt ... adcbc9ef.0uk-escience-root.crt ... 8175c1cd.0Notice: Don't forget to place your certificate and key files at: /Users/jonmaclaren/man2-rm/x509/server_cert.pem /Users/jonmaclaren/man2-rm/x509/server_key.pemInstalls Source filesCreates a crontab & scripts for restarting the RMCustomizes some scripts for stopping/starting the RMInstalls and hashes CA certificates

    Output:

  • Whats in /usr/local/man2-rm ?Some Perl ModulesAnd OuterRM.pl which gets runcommands - which configures and runs the RM (based on install.config, etc.)rerun - runs commands in the background from crontabcrontab - crontab line which can be added directly to your crontab (dont cut and paste!)start-rm, stop-rm - control whether rerun will actually start the RM, using a control file (.do_not_restart)./stop-rm./start-rm [ -w ]x509 - subdirectory containing all the CA certs, mapfiles, etc.

  • Perl ModulesJust an overview here...There is a doc online which has some details on these

  • Key ModulesOuterRM - just does the HTTP listening and Acceptor Cert authN/authZMainLoop - handles each requestTransactionManager - remembers what transactions (by TID) are running, and what their states areInnerRM - the main class for different types of RMSimpleComputeRMSimpleNetworkRMBoth inherit from InnerRM

  • SimpleComputeRMHandles batch queue systemsDeals only with processors/memoryTo talk to the scheduler, a subclass of SCBatch is usedSCBatchTorqueMaui.pmSCBatchTorqueMoab.pmSCBatchLoadLeveler.pm - not in CVS yet...Chosen at runtime - RM_COMPUTE_BATCH_TYPESimple modulesLess than 200 linesOverrideinitializemakeReservationcancelReservationgetStatus

  • Customizing InnerRMStartup/shutdowninitialize/removeParsing (validating) the XMLparseResourceElementparseWorkElementmaybe parseScheduleElementCo-allocationtryMakeActiontryCancelActionaddResourceBookingscompleteTransactionBookingsOthers for getTimetable/getStatus

  • Steps for creating a new RMDesign your XMLResource elementWork elementCreate a new subclass of InnerRM.pmUse the utility classes where possibleTo extend the API, create subclasses ofResource.javaWork.java

  • Caveats for RMsNeed to restart to re-read grid-mapfileWhen restarted, they forget the bookingsWant to add persistence so that its trivial for RM developers to utilizeThread handling needs work (soon!)

  • Whats next?Discussion on MPIg...Beer?

  • But first......Any Questions?

Search related