Research Projects

 

Resource Management and Scheduling in Distributed Processing Systems

The KOALA Multicluster Scheduler

KOALA is a scheduler that we have designed and implemented in the PDS group, and that has been deployed on the DAS system. KOALA is our research vehicle for research in scheduling and resource management in multicluster systems, grids, and clouds. Its main original feature was processor co-allocation, but it supports now many more application types, such as Bags-of-Tasks, workflows, and MapReduce applications. KOALA development has been an ongoing effort in several research projects.

The Distributed ASCI Supercomputer (DAS)

The DAS is a six-cluster computer-science infrastructure funded by NWO (the Dutch National Science Foundation) and installed and maintained by the ASCI Research School. One of the clusters is located at TU Delft. The DAS is very important for the research of the PDS group. The KOALA scheduler has been developed for and installed on the DAS.

Infrastructure Virtualization for e-Science (IV-e, part of the national Dutch COMMIT programme, 2011-2017).

This project is a sequel to the VL-e project (see below) on resource management, e-Science applications, workflows and data management in large-scale distributed computing systems such as clouds. The two research topics of the PDS group in this project are further development of the KOALA scheduler and application-specific scheduling. In particular, we currently focus on scheduling data-intensive frameworks such as MapReduce and workflow scheduling.

PhD students: Bogdan Ghit and Alexey Ilyuskin

GUARD-G: Guaranteed Delivery in Grids (2007-2012)

The goal of this project on grid computing is to design and analyze techniques for delivering guaranteed service to applications in grids. The GUARD-G project is part of the GLANCE programme funded by NWO, and is performed jointly with Leiden University.

PhD student: Nezih Yigitbasi
Postdoc: Hashim Mohamed

ALEA: Handling Uncertainties in Large-Scale Distributed Systems (2009-2010)

The goal of ALEAE is to provide models and algorithmic solutions in the field of resource management that cope with uncertainties in large-scale distributed systems. ALEAE is a joint project of Delft University of Technology, INRIA in France, Osaka University in Japan, and the Zuse Institute in Berlin, Germany. One of the main achievements of the ALEAE project is the Failure Trace Archive (FTA), which is a centralized public repository of availability traces of parallel and distributed systems, and tools for their analysis. The purpose of this archive is to facilitate the design, validation, and comparison of fault-tolerant models and algorithms.

Virtual Laboratory for e-Science (2004-2010)

In the Dutch national project Virtual Lab for e-Science (VL-e), we focus on resource management, scheduling, and performance analysis in grids. In particular, we study the management and scheduling of jobs that require co-allocation, that is, the simultaneous allocation of resources (processors, data, etc.) in multiple subsystems making up a grid. For this purpose, we have designed and implemented the KOALA grid scheduler. 

PhD students: Alexandru Iosup and Ozan Sonmez
Postdocs: Alexandru Iosup, Ozan Sonmez and Hashim Mohamed

CoreGRID (2004-2008)

CoreGRID is a Network of Excellence of the European Union in grid computing, with 42 participating universities and public research institutes in Europe. CoreGRID is divided into six work packages or so-called virtual institutes. One of these is the virtual institute on Resource Management and Scheduling, in which the PDS group participates.

Condor (1992-2996)

In this project on grid computing, we focused on resource management across multiple sites. In particular, we designed and implemented the flocking mechanism in Condor for load sharing and job migration across different Condor pools, in cooperation with the main designer of the Condor system, Miron Livny of the University of Wisconsin at Madison.

Peer-to-Peer Systems and Online Social Networks

P2P-Fusion (2006-2009)

P2P-Fusion is an EU project on peer-to-peer systems for creative reuse of multimedia content in virtual communities. The project has seven partners in Finland, Hungary, and the Netherlands.

PhD students: Michel Meulpolder and Rahim Delaviz

I-SHARE (2004-2010)

I-SHARE is a project on sharing technology at different levels in wired and wireless P2P systems. It is part of the BSIK programmme Freeband. As a guiding example, we are defining an architecture for P2P-TV, a P2P system for the dissemination of both live and recorded programs of 10,000+ TV channels. Research issues are how to do recommendations to users on TV programs, how to design the user interface, how to build application-level multicast trees for distributing live video, and in general, how to share the contents of individual video recordings on users' hard disks. 

PhD student: Jan David Mol
Postdoc: Johan Pouwelse

Two-level peer-to-peer systems (TLP2PS, 2003-2008)

The research topic in this NWO-funded project is to exploit the heterogeneity of P2P systems, and in particular, to assess the performance impact of the presence of superpeers, which are peers that have more capabilities than other peers.

PhD student: Pawel Garbacki

Dr.Ir. D.H.J.Epema