Deep Neural Networks (DNNs) have emerged as a key algorithm for a wide range of difficult applications including image recognition, speech processing, and computer virus detection. Today’s DNNs are often trained on farms of GPUs and then deployed in a wide range of systems from mobile to server. Current trends in DNN architectures are toward deeper and more complex networks, placing more stress on both training and inference. This talk will discuss the challenges associated with emerging DNNs and describe recent work that (1) enables larger and more complex networks to be trained on a single GPU with limited memory capacity; and (2) methods of reducing the memory and computation footprints of DNNs at inference time, enabling them to run with vastly improved energy efficiency. This talk will draw on recent results by researchers at NVIDIA, MIT, and Stanford.