20 May 14:00 – 20 May 16:00 in Tutorials / Workshops 1

Dask in HPC

Anderson Banihirwe, Guillaume Eynard-Bontemps

Audience level:
Intermediate

Description

The goal of this workshop is to bring together scientists, software developers and HPC center administrators to share their experiences with interactive supercomputing, using Dask in High Performance Computing (HPC) settings.

Abstract

High Performance Computing (HPC) users perform different kinds of data analysis with a variety of tools and workflows. HPC users tend to be well versed in technologies like MPI (for scaling complex computions), and batch job queuing systems such as PBS, Slurm, etc. With the advent of tools like Jupyter notebooks, there's a growing interest in interactive, "human-in-the-loop" workflows. However, this kind of workflows isn't supported by the traditional MPI with batch processing approach. One remedy to this problem is the Dask toolchain which includes tools like dask-jobqueue that enables interactive supercomputing on HPC systems.

The goal of this workshop is to bring together scientists, software developers and HPC center administrators to share their experiences with interactive/non-interactive supercomputing with Dask in HPC settings. Topics of interest for this workshop include (but are not limited to):

Schedule

Time (UTC) Speaker Title
14:00 - 14:15 Anderson Banihirwe - National Center for Atmospheric Research, USA The Current State of Deploying Dask on HPC systems
14:15 - 14:30 Aamir Shafi - Ohio State University, USA Efficient MPI-based Communication for GPU-Accelerated Dask Applications
14:30 - 14:45 Aurélie Albert - Centre National de la Recherche Scientifique (CNRS), France Benchmarking French High Performance Computing Centers for PANGEO Deployments
14:45 - 15:00 Tina Odaka - Institut Français de Recherche pour l'Exploitation de la Mer (IFREMER), France Real Time Monitoring of HPC Simulation Outputs using Dask and Xarray
15:00 - 15:15 Jens Glaser - Oak Ridge National Laboratory, USA Towards Scalable Data Analytics for Drug Discovery
15:15 - 15:30 Emmanuel Dubois - Centre National d'Etudes Spatiales (CNES), France Using Dask to Produce Digital Surface Models with CARS
15:30 - 16:00 Panel Panel and Q&A