Open Topics

From Ogce

Note: These are open student topics related to science gateways and related infrastructure.

Contents

Gateway Computing Toolkit and Science Gateway

Topic: Science gadgets and gadget based science gateways for the Gadget Container

  • Mentor: Dr. Marlon Pierce
  • Project Background: Science gateways traditionally place a strong emphasis on running high end applications on supercomputers but do not do such a good job enabling collaboration. Social Networking tools can change this.
  • Project Goal: Demonstrate the feasibility of applying social networking tools to problems in scientific collaboration.
  • Project Tasks: Identify candidate science gateways such as genome sequencing and drug discovery; develop usage scenarios; map usage scenarios to social gadget tools; develop gadgets (calendar-based schedulers; annotation feeds); package work for easy deployment.
  • Requirements: Experience with Java and JavaScript programming required. Experience with the Open Social standards and Apache Shindig desired.

Topic: Gadget-building libraries and improvements to JavaScript Cyberaide

  • Mentor: Dr. Marlon Pierce
  • Backup Mentor: Fugang Wang
  • Project Background: Cyberaide JavaScript is a library for building interactive gadgets to Globus-based computational grids. Initial versions of the tools have been developed but much work needs to be done to add numerous improvements.
  • Project Goal: Improve the user interface components and revise the server side architecture to improve stability.
  • Project Tasks: Implement improvements to the file transfer interface; implement improvements to the job submission infrastructure; revise the architecture to improve the robustness; develop interfaces for managing graphs of tasks; use libraries to make innovative mashups.
  • Requirements: Java and JavaScript programming experience required. Experience with specific JavaScript libraries like JQuery highly desired. Experience with Web Services desired.

Topic: Green computing gadgets and services for both home usage and data centers, supercomputing centers

  • Mentor: Dr. Marlon Pierce
  • Backup Mentor: Dr. Gregor von Laszewski, Dr. Lizhe Wang
  • Project Background: Supercomputing centers and data centers are increasingly instrumented to monitor environmental conditions such as temperature and power consumption. This information can be used to manage resources in the most energy-efficient manner. For example, new tasks can be assigned preferentially to more efficient resources.
  • Project Goal: Develop a set of interactive user interfaces as Open Social Gadgets to access machine room monitoring services. Use JavaScript tools such as Raphael.
  • Project Tasks: Develop REST-based monitoring service prototypes; develop plotting gadgets for individual machines and machine room floors.
  • Requirements: JavaScript, REST service development (Java, Python, Perl all ok).

Topic: Web site testing tools for automated testing of nightly builds

  • Mentor: Dr. Marlon Pierce
  • Project Background:
  • Project Goal:
  • Project Tasks:
  • Requirements:

Topic: AJAX based drag and drop composition of Scientific Workflows within XBaya Workflow Composer Framework

  • Mentor: Suresh Marru
  • Project Background:
  • Project Goal: Browser implementation of XBaya similar to flash example Flex XBaya
  • Project Tasks:
  • Requirements: AJAX, JavaScript, and Web Application Enthusiasm

Topic: AJAX based monitoring of Web Service Events to Messaging WS Messenger Framework

  • Mentor: Suresh Marru
  • Project Background:
  • Project Goal:
  • Project Tasks:
  • Requirements: AJAX, JavaScript, and Web Application Enthusiasm

Cloud Computing Tools and Environments

Topic: Stratosphere - a Cloud of Clouds

  • Primary Mentor: Dr. Lizhe Wang
  • Backup Mentor: Dr. Gregor von Laszewski
  • Problem Background: The most challenging scientific problems of today require the assistance of sophisticated modern cyberinfrastructure. However, the current generation of cloud computing infrastructure is based on an intranet of potentially inhomogeneous systems. However, this may not adequately bring enough resources to the scientific research communities. Furthermore, we recognize multiple Cloud middleware infrastructures that prevent potentially the use of large scale supercomputers on a national and even international scale.
  • Project Goal:
    • Provide a mechanism to have multiple clouds be integrated into one science cloud. The result we will call a toolkit named stratosphere.
    • Demonstrate the use of real middleware while using stratosphere, such as TeraGrid and FutureGrid.
    • Demonstrate the usefulness of the tool and conduct performance comparisons between different clouds while using a water threat management application.
Stratosphere will allow researchers to utilize cloud resources at different locations with single sign on. Furthermore, users may be able to choose between clouds such as dryad, Nimbus, Eucalyptus, and frameworks such as hadoop.
  • Project Task: Students will develop middleware and Web interfaces that can be integrated within the Cyberaide Project to address the issue of the creation of on-demand cloud infrastructures while scheduling them through a batch queuing system (Moab). We will dynamically change the number of nodes in a cluster to be assigned to a variety of private scientific clouds. Thus cloud A may use dryad, while cloud B uses Eucalyptus, and the rest of the nodes of the supercomputer are bare bones nodes running MPI applications. The administration and use of such a sophisticated infrastructure is controlled through cyberaide-shell. The final deliverables will be a Web service-based implementation, a GUI interface, monitoring interface and a local virtual machine database configured to demonstrate the capabilities of the stratosphere concept.
  • Requirements:
    • Programming languages: Java, Python
    • Knowledge of distributed computing
    • Senior undergraduate students or postgraduate students are preferred

Topic: Map/Reduce for Scientific Applications

  • Primary Mentor: Dr. Lizhe Wang
  • Backup Mentor: Dr. Gregor von Laszewski
  • Project description

A Water Threat Management (WTM) system is developed to handle contamination incidents in urban Water Distribution Systems. It incorporates Grid middleware, Grid resources, simulation algorithms, sensor networks, a workflow, and a user portal. Any detection of contamination from the sensors located across the water distribution is sent to the optimization engine. The process in the optimization engine includes an Evolutionary Algorithm (EA) to find an optimized solution for the source characterization by the generation process. The arrows between the optimization engine and the simulation engine indicate that they communicate with each other for the generational processing. In the simulation engine, by the use of EPANET, the hydraulic movement of water and change of the water quality is simulated. The processing takes place on Grid resources through middleware with an adaptive workflow for the real-time communication with the sensors or other input data. The workflow can be manipulated through the portal for the simplified management and easier access to the resources. A Map/Reduce implementation will be developed by using MPI for solving the WTM application. The solution will use the Map/Reduce distributed framework and corresponding implementation called Hadoop. Hadoop is an ideal framework for this application because WTM consists of execution of several EPANET simulation runs, and their combined results are fed to the optimization engine. Each such simulation can be made a part of a Map task, and the results of all Map tasks can be combined by a single reduce operation. The optimization engine uses these results to generate new parameters for the next simulation run.

  • Requirements:
    • Programming languages: Java, Python
    • Knowledge of parallel computing
    • Senior undergraduate students or postgraduate students are preferred

Topic: Map/Reduce in a volunteer computing environment

  • Primary Mentor: Dr. Lizhe Wang
  • Backup Mentor: Dr. Gregor von Laszewski
  • Project Background: Map/Reduce is programming model implemented by Google Inc. for processing and generating distributed huge data sets. Hadoop is an open source implementation that supports Map/Reduce programming and Hadoop File System, inspired by Google File System. Volunteer computing is an arrangement where volunteers provide computing resources to projects, which use the resources to do distributed computing and/or storage. BOINC is an open source project for volunteering computing.
  • Project Goal: We would like implement Hadoop framework in the volunteer computing environment, such as BOINC.
  • Project Tasks:
    • Implement BOINC interfaces and functions for Hadoop
    • Implement Map/Reduce software libs in Java with BOINC and Hadoop support
    • Implement a GUI for Hadoop on BOINC.
    • Evaluate performance with a parallel datamining benchmark (Optional)
    • Evaluate performance with real applications, e.g., BLAST or high energy physics application (Optional)
  • Requirements:
    • Programming language: Java or Python
    • Knowledge of distributed computing
    • Senior undergraduate students or postgraduate students are preferred

Topic: Map/Reduce in a production Grid computing environment

  • Mentor: Dr. Lizhe Wang
  • Backup Mentor: Dr. Jie Tao
  • Project Background: Map/Reduce is programming model implemented by Google Inc. for processing and generating distributed huge data sets. Hadoop is an open source implementation that supports Map/Reduce programming and Hadoop File System, inspired by Google File System. Current implementation of Hadoop is limited in a data center scale. Large scale data centric computing, for example high energy physics and bioinformatics applications, require a Map/Reduce-like programming model across multiple compute centers in Grid computing infrastructures.
  • Project Goal: This project aims to extend Hadoop for distributed compute centers in production Grid infrastructures.
  • Project Tasks:
    • Implement CyberFarm, a high performance distributed file system
    • Extend Hadoop libraries to work on CyberFarm file system, including:
      • Developing Hadoop software libraries in Java with CyberFarm.
      • Implementing CyberFarm interfaces and functions for Hadoop.
      • Developing CyberStudio: a portal for Map/Reduce on distributed data centers.
    • Evaluation with data intensive computing applications on distributed centers.
  • Requirements:
    • Programming languages: Java, Python
    • Knowledge of distributed computing
    • Senior undergraduate students or postgraduate students are preferred

Topic: Green Map/Reduce

  • Mentor: Dr. Lizhe Wang
  • Backup Mentor: Dr. Gregor von Laszewski
  • Project Background: Map/Reduce is programming model implemented by Google Inc. for processing and generating distributed huge data sets. Hadoop is an open source implementation that supports Map/Reduce programming and Hadoop File System, inspired by Google File System. Hadoop provides an open source implementation of Map/Reduce on distributed clusters in a data center. It contains a Distributed File System (HDFS) and the map/reduce software framework. HDFS contains one master node (NameNode) and a number of worker nodes (DataNode), which are distributed across multiple racks. Large data files are stored in a number of blocks, which are replicated in part on the various DataNodes. The DataNodes are also distributed across different racks to increase fault tolerance. DataNodes periodically send heartbeat messages to the NameNode to update their status. The block size and replication factor are configurable per file by users. The Map/Reduce application interacts with NameNode to access data blocks in a file. Currently, there is no consideration of energy consumption or environmental impact as part of Hadoop. Increased consumption is obvious when considering replica placement, redistribution, and selection.
  • Project Goal: Plans are to enhance the the existing system with a new service called PowerManager to allow the NameNode to develop energy efficient block operations.
  • Project Tasks:
    • Green Map/Reduce configuration: Users can configure Hadoop by changing the parameter controlling the number of replicas per block via a service level negotiation with the PowerManager service. By decreasing the replica number per rack energy can be saved because the Hadoop system does not need to maintain a large number of block replicas. The power consumption of one block replica can be assessed by our green workload profiling tools. Other parameters such as block size and the map/reduce job number can also be integrated into from the Green SLA negotiation.
    • Map/Reduce thermal and power aware adaptive replica distribution: Hadoop is required to place or re-distribute replicas in the initialization phase or during its execution. Replicas are also selected when applications want to access specific data files. We propose to control replica placement, redistribution, and selection based on the power consumption and temperature of the DataNodes, with the goal of reducing overall energy consumption. It is well known that the temperature distribution and power consumption in a rack or a data center are not uniform and often are a cause for higher energy consumption than necessary. Therefore, the PowerManager service will use this information to select, place and migrate replicas in support of applications on DataNodes that are cooler or that consume less power. The information about the temperature distribution and power consumption can be obtained from the green monitoring and auditing service that we provide.
    • Map/Reduce power aware DataNode management: DataNodes of block replicas can be transitioned into a sleep mode or be powered off to save power or reduce its temperature. The selection of such DataNodes can be controlled through various policies. Examples include the Least Recently Used (LRU) or Least Frequently Used (LFU) policies that we enforce through service choreography. Furthermore, we will provide a better utilization by integrating historical workload characterization and modeling. Policies will be developed to wake up the DataNodes, either periodically or adaptively based on the workloads and the power/thermal conditions of the resources on which the DataNodes are placed. The PowerManager service will also balance the percentage of active block replicas. Placing some DataNodes into “sleep” or “off” state can lead to performance loss for users because additional processing time are required to “power-on” the nodes and to initiate the read or write data. Therefore, it is beneficial to integrate a service level agreement to minimize the impact, while controlling the percentage of the active data nodes in correlation with expected response times.
    • Dynamic scheduling of Map/Reduce jobs: The PowerManager service also provides the ability to dynamically address energy concerns for scheduling Map/Reduce jobs. With dynamic power aware scheduling policies, Map/Reduce jobs are sent dynamically to DataNodes with lower power consumption or lower temperature.
  • Requirements:
    • Programming languages: Java, Python
    • Knowledge of distributed computing
    • Senior undergraduate students or postgraduate students are preferred

Parallel Computing Tool and Environment

Topic: A Software Environment for Green Workload Modeling

  • Mentor: Dr. Lizhe Wang
  • Backup Mentor: Dr. Gregor von Laszewski
  • Project Background: Data center computing now is an active research field of high performance computing. Data center workload has been characterized with some features, like queue time, starting time, end time, arrival rate, job co-relation. However, job's power consumption and thermal impact are unclear.
  • Project Goal: This work leads the creation of an estimation service for power consumption to be integrated into a self-correcting data assimilation system using Kalman filters and other suitable methods.
  • Project Tasks: Initially, this service includes an offline calibration phase and a runtime estimation phase. The offline calibration phase measures the power consumption of a predefined benchmark such as LINPACK and Spec2000 for a compute node. This measurement is than taken for an initial estimate. However, as new calculations are run on this node more realistic application benchmarks are taken and our available information will be enhanced over time. In a later phase of the project we will be able to estimate based on a particular users use pattern by annotating the data in our database and correlating it to familiar patterns in the database. To self correct ourselves we will be integrating the derivation from the predicted value after the calculation s conducted.
  • Requirements:
    • Programming languages: Java, Python
    • Knowledge of parallel computing
    • Senior undergraduate students or postgraduate students are preferred

Topic: Green OpenMP

  • Mentor: Dr. Lizhe Wang
  • Backup Mentor: Dr. Jie Tao
  • Project Background: High performance scientific applications consume significant power consumption. OpenMP [1] is an important parallel programing model on share-memory architectures. Energy saving is not addressed in current OpenMP implementations, such as omniOpenMP and Parallel Java.
  • Project Goal: We will develop power-aware scheduling algorithms in the existing OpenMP compiler
  • Project Tasks:
    • Study OpenMP scheduling techniques and identify opportunities for power saving
    • Implement power aware scheduling of multiple threads in OpenMP compiler
    • Performance evaluation
  • Requirements:
    • Programming languages: Java, Python
    • Knowledge of parallel computing and computer architecture
    • Senior undergraduate students or postgraduate students are preferred

Topic: A Parallel programming environment for multiple virtual machines on multicore architectures

  • Mentor: Dr. Lizhe Wang
  • Backup Mentor: Dr. Jie Tao
  • Project background: Multicore architectures become prevalent now in the context of Moore’s Law shows down and the need to process increasingly large data sets increases. Virtual machines have been widely accepted to build dynamic computing infrastructures, like data centers and computing clouds. It has been widely accepted that the amount of performance gained by the use of a multicore processor is strongly dependent on the software algorithms and implementation. Currently there is little support for deploying and managing virtual machines on multicore architectures for parallel applications. For example, OpenMP applications run across several virtual machines without support from underlying shared memory of multicore architectures.
  • Project goal:This project intends to develop a parallel programming environment, specifically OpenMP, for virtual machines on multicore architectures.
  • Project Tasks: We plan to develop software libraries to support a group of virtual machines with shared memory communication. The OpenMP programming model will be supported across multiple virtual machines in the same group. A library will be implemented to benefit from the shared memory communication, which is expected to be more efficient than the traditional “copy-buffer” based inter-process message exchange. Our work is planned to extend existing virtual machine technologies and popular parallel computing environments, like omniOpenMP.
  • Requirements:
    • Programming languages: C, Java, OpenMP
    • Knowledge of parallel computing and computer architecture
    • Senior undergraduate students or postgraduate students are preferred

Topic: Energy efficient virtual machine management on multi-core achitectures

  • Mentor: Dr. Lizhe Wang
  • Backup Mentor: Dr. Jie Tao
  • Project Background:The virtualization technology has been applied in computer systems for providing on-demand resources. With support of a middleware, called a Virtual Machine Monitor (VMM) or hypervisor, virtual machines can be created with user-specified requirements in, e.g., operating system, memory size, and the number of processors. Performance loss and power consumption caused by virtualization require a comprehensive research effort to understand the system behaviour. Multicore processors [2] [3] will be commonly used in the future to build a data center. Multicore computers and clusters make the study more sophisticated. Existing VMMs are developed in a time when microprocessor design still relied on the uni-processor technology, without much consideration of the specific feature of a multicore architecture.
  • Project Goal: We are interested in developing profiling tools to evaluate how compute performance and power consumption are related while using VM technologies on multicore architectures.
  • Project Tasks:
    • Study the VMMs and running applications on virtual machines of IBM Blades.
    • Design and propose evaluation metrics.
    • Study available tools for performance and energy profiling.
    • Develop a profiling tool for this work.
    • Measure the metrics with NAS, HPC benchmark suites.
    • Analyze the bottlenecks.
  • Requirements:
    • Programming language: C, Python
    • Knowledge of computer architecture and operating system
    • Senior undergraduate students or postgraduate students are preferred

Project Organization

Project Management

  • E-mail list: OGCE at googlegroups.com
  • IRC channel: #OGCE at irc.dal.net
  • Source code management: we have set up several SVNs hosted by Google Code and SourceForge; we will apply strict code management policies.
  • Quality control: We will set a bugzilla server and ticket management system. For each student project, we will set up one mentor and one backup mentor. We will set up frequent meeting and code review processes for quality control.
  • Mentors: The OGCE team has been involved in the open source software project in the field of Grid computing toolkit and service for several years. Dr. Pierce and Dr. von Laszewski have led a number of professional software project development efforts. OGCE mentors have strong expertise in the field of Grid computing, Cloud computing, and parallel computing, and have experience with supervising undergraduate/postgraduate students.

Mentor

Researchers of the OGCE project will work as (backup) mentors:

  • Dr. Marlon Pierce Dr. Pierce's areas of research include Web, Grid, and Cloud computing with a strong emphasis on applying these tools to scientific applications. He has a strong background in both open source software development and distributed computing research. Pierce has supervised and collaborated with many students while leading numerous projects at the Community Grids Laboratory at Indiana University, as is demonstrated by his over 45 publications in computer science. Five of Pierce's students have earned Ph. D.'s in Computer Science. Pierce's research team is funded by the NSF, NIH, NASA, and DOD.
  • Dr. Gregor von Laszewski: Dr. Gregor von Laszewski is conducting state-of-the-art work in Cloud computing and GreenIT at Indiana University as part of the FutureGrid project. He is involved in Grid computing since the term was coined. He is best known for his efforts in making Grids usable and initiating the Java Commodity Grid Kit which provides a basis for many Grid related projects including the Globus toolkit. Dr. von Laszewski has strong experience with supervising undergraduate and postgraduate students. He was an associate professor of Rochester Institute of Technology (2007-2009). He was awarded the Department of Energy Outstanding Mentor Award for Undergraduate Education.
  • Dr. Lizhe Wang: Dr. Wang's expertise fields include energy aware computing and Cloud computing. He has much experience with open source projects development. He has published more than 40 scientific papers and book chapters. He is the coordinator of the IEEE TCSC technical area of Green Computing. He has experience with supervising 12 postgraduate students.
  • Suresh Marru is a senior researcher in the Pervasive Technology Institute at Indiana University and is a scientific workflow and gateway architect. His research interests include distributed, grid, and cloud computing and building complexity hiding interfaces and workflows for various science domains. Marru has supervised 4 masters students and assisted Prof. Dennis Gannon in supervising 12 PhD Students.
  • Fugang Wang: Mr. Wang is a chief software engineer in Pervasive Technology Institute. He has much experience with software development of Web service, SOA, and science gateways. He was awarded a Google Summer Code studentship in the organization of the Globus Toolkit in 2009.
  • Dr. Jie Tao: Dr. Tao is an expert in parallel computing and computer architecture. She is a senior researcher of Karlsruhe Institute of Technology. She has published more than 50 scientific papers and supervised more than 20 postgraduate students.
  • More mentors will be named later

Application Template

  • Name
  • Age
  • Contact information: phone, email, address
  • Background: education, software development experience, knowledge of programming language, ...
  • Interesting topic(s)
  • Your proposed solution(s) for the topic(s)

Contact

Please address your queries to Dr. Lizhe Wang (Lizhe.Wang at gmail.com)

Web site tools