An Introduction to Memory Spilling

Memory spilling is an important feature that makes it possible to run Dask applications that would otherwise run out of memory. When low on memory, Dask moves data from GPU memory to main memory and/or data from main memory to disk automatically. In this talk, we will walk through how spilling works in general, its shortcomings, and introduce a new Dask-CUDA approach to overcome these shortcomings.

Categories: Day 1