Summer2010 Tutorial

From Ogce

Contents

[edit]

Overview

The Open Grid Computing Environments (OGCE) is a provider of gateway software and includes several components that can be used by themselves or can be integrated to provide more comprehensive solutions. These components include:

  • OGCE Gadget container, a Google gadget-based tool for integrating user interface component
  • XRegistry, a registry service for storing information about other online services and workflows
  • XBaya, a workflow composer and enactment engine
  • GFAC, a factory service that can be used to wrap command line-driven science applications and make them into robust, network-accessible services
  • OGCE Messaging Service, which supports events and notifications across multiple cooperating services.

Additional components include scientific workflow extensions to Apache Orchestration Director Engine (ODE), a WS-BPEL 2.0 Workflow Standard.

[edit]

Goals

The OGCE gateway is a hosted version of the OGCE software stack, visit the [Gateway Portal] to try workflows from Atmospheric science, Molecular Chemistry, Bio Physics, Genome Analysis, EST Pipeline and Astronomy.

This tutorial will walk you through the "Deploy your gateway" process. The goals of this tutorial is to check out, build, and deploy OGCE suite.

  • Outline of the tutorial
  • Setup and Introduction (presentation link)
  • Demonstration of a weather forecasting WRF workflow on TeraGrid - Wink link
  • Deploy & Customize Gadget Container
  • Build & Deploy Workflow Suite
  • Wrap your scientific application into a web service
  • Construct, Execute & Monitor Workflows
[edit]

Tutorial accounts and host machines

TeraGrid accounts train30-train39 are provlded for the tutorial, and we will provide servers on the Quarry testbed for some attendees. You are free to use an existing TeraGrid account instead of the training accounts. Assignments of quarry machines and training accounts are shown below.

  • OLAS (Craig, Cristina): gw42.quarry.iu.teragrid.org; train30, train31
  • UltraScan (Emre): gw33; train32
  • ODI (Frank): gw8; train33
  • LEAD (Yuan): gw13; train34
  • BioVLAB (Hyungro, Heejoon, Youngik, Sun): gw39; train35, train36, train37
  • CGB: gw40; train38
  • Randy Heiland: gw26; train39

For simplicity, we will run a central MySQL server on community.ucs.indiana.edu for XRegistry. Usernames will be the same as your assigned training account (i.e. user train30 will have a database user train30 for XRegistry's MySQL requirements).

[edit]

BigRed UserNames

  • User names on BigRed are tg-traXX
[edit]

Audio/Video Connections and Shared Display

[edit]

Adobe Breeze/Connect Link

We are using Adobe Breeze for shared display with remote attendees. The link is http://breeze.iu.edu/ogcetutorial.

[edit]

PolyCom Connection Information

If you plan to connect via Polycom, please RSVP so that we can test. Remote participants can join by Polycom. See http://kb.iu.edu/data/ause.html for instructions to connect to the IU bridge. Our conference number on Wednesday is 220331. The PIN is 1234#

[edit]

Teleconference Connection

If you prefer to call in, dial 1-800-940-6112, 000704# at the PIN.

[edit]

Hands On Exercise Training Material

Before proceeding, make sure you do not have $M2_HOME, $MAVEN_HOME, or $CATALINA_HOME set. These can cause problems with the build.

[edit]

Checkout and build instructions

  • Make sure you are using Java 1.6 
  java -version
  • Once logged into the train3x account on the gwXX machines to checkout the software needed for the tutorial from sourceforge do 
 svn co https://ogce.svn.sourceforge.net/svnroot/ogce/ogce-all-rc1/
  • Once the checkout is complete change to the ogce-all-rc1 
  cd ogce-all-rc1
  • Untar Apache Maven and add it to your path: 
  tar -zxf  apache-maven-2.2.1-bin.tar.gz
  export PATH=$HOME/ogce-all-rc1/apache-maven-2.2.1/bin:$PATH
  • Some of the components require Maven 2.2.1, so make sure you are using this version. Use mvn -v to verify.
  • Set the MAVEN_OPTS environment variable to increase available heap memory to Maven. 
  export  MAVEN_OPTS="-Xmx1024m -Xms512m"
  • Set JAVA_HOME 
  export JAVA_HOME=/usr/java/jdk1.6.0_19/
  • Edit the pom.xml and make the following changes in the XRegistry Parameters section. [Replace XX in trainXX mentioned below with the number in your training userid.] 
  vi pom.xml 
    
   1. Replace <databaseUrl>jdbc:mysql://localhost:3306/xregistry_ogce</databaseUrl> with the following:
        
         <databaseUrl>jdbc:mysql://community.ucs.indiana.edu:3306/trainXX</databaseUrl>
       
   2. Replace <databaseUser>xreg_ogce</databaseuser> with the following:
        
         <databaseUser>trainXX</databaseUser>
      
   3. Replace <databasePass>xreg_pass</databasePass> with your training password.
  • Edit the Gadget Container's properties file to use your hostname: 
 vi gadget-container/config/ishindig.properties; change site.host from localhost to your machine's name eg. gw40.quarry.iu.teragrid.org
  • Start the build using the mvn goals 
  mvn clean install
  
  Check for permissions on sgfac/target/dist-bin/factory.sh using
  
   ls -l sgfac/target/dist-bin/factory.sh
  
  If execute permissions are not present set them using 
  
  chmod a+x sgfac/target/dist-bin/factory.sh
  • If you just want to rebuild one component, such as the gadget container, you run the command 
  mvn clean install -f gadget-container/pom.xml

Insert non-formatted text hereInsert non-formatted text here

[edit]

Sample Applications

  • To download and untar the file use the commands: 
  wget http://gw40.quarry.iu.teragrid.org/data/sample_applications/ogce_tutorial_apps.tar.gz
  tar zxvf ogce_tutorial_apps.tar.gz


[edit]

XBaya Workflow GUI

http://www.extreme.indiana.edu/xgws/xbaya/xbaya-ogce.jnlp

[edit]

Gadget Container Interface

https://gw33.quarry.iu.teragrid.org:7443/ishindig-webapp

[edit]

Starting Services

  • Start the gadget container with the command ./startup-gadget-container.sh. Verify it is working by pointing your browser to 
    https://your.server.name:7443/ishindig-webapp/

You can shut down the server with ./shutdown-gadget-container.sh. It is also helpful to look at the server's catalina.out file. See OGCE_Gadget_Container for more information.

  • Start the XRegistry server with ./startup-xregistry.sh. Verify it is working by pointing your browser to 
  https://your.server.name:6666/

Shut down xregistry with ./shutdown-xregistry.sh. It is also helpful to check xregistry.log. See XRegistry for more information.

  • Start the GFac server with ./startup-gfac.sh. Verify it is working by pointing your browser to 
  https://your.server.name:23443

See GFAC for more information.

[edit]

Service, Application and Host Descriptions

  • Open the Gadget Container URL and sign up for an account on the main page.
  • Click on the XRegistryInterface tab and click Maximize if the display is windowed.
  • Click on Proxy Configuration and Login to the XRegistry Interface using your assigned user name (trainXX) and password
  • Click on Add Host Description: 
  Enter Host Name - login.bigred.iu.teragrid.org
  Temp Directory  - /N/dc/scratch/$Account_Name  - the Account names are assigned tg-tra30 to tg-tra38. For example the demo url is /N/dc/scratch/tg-tra39
  GateKeeper EndPointR - https://g1.bigred.teragrid.iu.edu:8445/wsrf/services/ManagedJobFactoryService
  Check the box WS Gram Present
  GridFTP End Point - gridftp.bigred.iu.teragrid.org:2812
 GateKeeper JobManager: LoadLeveler
  • Application Description: 
  Login to bigred as tg-traXX@login.bigred.iu.teragrid.org
 Fetch applications by going a wget http://gw40.quarry.iu.teragrid.org/data/sample_applications/ogce_tutorial_apps.tar.gz
  Untar the contentents - tar xzvf ogce_tutorial_apps.tar.gz

Fill the application description with the following details. 
  Executable: /N/u/tg-traXX/BigRed/ogce_tutorial_apps/wrf_wrapper.sh .  
  Project Name - TG-TRA900337N
  Job Type - MPI
  Queue  - NORMAL
  Max Wall Time - 60 
  Processor Count - 16
  Host Count - 8
  • Service Description: 
  Specify a unique name for the Service_Name: ex: WRF_Forecast - Note: This field should not have spaces or special characters as this is passed as XML.
  Method Name: A name of the Method - equivalent to a function in programming language. The workflow will display the service as Service_Name_Method_Name
  • Add Inputs: 
 Namelist_File - URI
 Boundary_File - URI
 Input_File - URI
  • Add Output:

WRF_Output - URI

[edit]

Experiment Builder

  • Click on the ExperimentBuilder tab and click maximize to maximize the gadget window.
  • Login using your trainXX userid and password on the form displayed.
  • Click on New Project and Enter a name for the Project without spaces.
  • Select the Project from the Drop down list and click on New Experiments
  • Type in a name and description for the workflow in the Experiment Wizard Interface presented.
  • Click on New under the Workflows Tab. 
  This will open up the XBaya Composer for workflows. Click Allow if presented with a confirmation dialog.
[edit]

XBaya Workflow Composer

  • Login to the composer using the trainXX account and password.
  • From the list on the left drag the service onto the workspace. This will create a rectangular box with input and output connectors.
  • Click on the arrow next to System Components to open up the System Components tree branch.
  • Drag the Input component onto the workspace.
  • Click on the connector on the Input and drag it to one of the blue input connectors on the Service box.
  • Click on Config displayed inside the Input box and enter a description and a default value of your choice.
  • Drag the Output Component onto teh workspace and connect it to the green output connectors on the service box.
Retrieved from "http://gridfarm018.ucs.indiana.edu/ogce/index.php/Summer2010_Tutorial"
Web site tools