Using Dask and many GPUs to train a neural network with PyTorch

I took a neural network I had trained on a single CPU to generate pet names and tried retraining it with tons of connected GPUs using Dask, PyTorch, and the package dask-pytorch-ddp. I learned a lot about when is the right time to use multiple GPUs and what the pitfalls can be. In this talk I’ll discuss what these lessons mean for training with GPUs and Dask.

Categories: Day 3