Next generation Radio Telescopes generate vast and ever-increasing quantities of data, but current software is not designed to operate in a parallel, distributed paradigm. This workshop brings together three strands of distributed dask Radio Astronomy development by SARAO, NRAO and SKAO, to provide a forum for the above challenge and to serve as a platform for future developments.
Radio Astronomy Applications with Dask
Principal Investigators
Organising committee
Contact details of Principal Investigators
Aim
The aim of this workshop is to introduce and showcase recent Radio Astronomy Dask development that is now mature enough to be applied by astronomy research scientists for processing datasets from the new generation radio telescopes.
Overview
Next generation radio telescopes must efficiently process large quantities of data during data acquisition and scientific data processing phases. It is already highly time-consuming and challenging to process MeerKAT 32k data with existing Radio Astronomy (RA) software, strongly motivating the need to reimplement the RA software ecosystem in a modern distributed computing paradigm.
Future radio telescopes, such as the Square Kilometer Array (SKA), will add increasing numbers of antennas, increasing the data volumes by a factor O(A2). In fact, the SKA is expected to generate in the order of 256TB of data during the course of an 8 hour observation. As newer instruments support higher sensitivity by adding antennas, increasing the number of channels and reducing integration time, data volumes increase commensurately.
The Dask paradigm offers a compelling solution to the aforementioned challenges. Consequently, multiple RA software development initiatives have invested in Dask, shifting RA software to a new ecosystem capable of reducing large observations in tractable amounts of time on modern HPC and cloud systems. Examples include SARAO’s dask-ms and codex-africanus projects and associated applications, NRAO’s CasaNGI prototype and SKAO’s Rascil Calibration and Imaging library.
Objectives
The primary objective of the workshop is to provide a forum for aforementioned contributors, to strengthen communication between groups and identify possible areas of overlap and collaboration within the Radio Astronomy dask development community.
Additionally, the workshop is a good platform for initial discussions on future development efforts. These may include initial discussion on newer storage formats (zarr, arrow, parquet) suitable for distributed and cloud computing in order to increase standardization and compatibility.
Invited speakers
Invited speakers to present talks on current Dask applications in RA
Proposed program layout
| 40 min slot invited talks | SARAO / Rhodes |
| 30 min slot invited talks | CASA / NRAO |
| 20 min slot invited talks | SKAO |
| 30 min demo | Distributed Streaming Radio Astronomy Reduction with Dask
(Jon Kenyon, SARAO) |
Target audience