Improving Low-Latency Predictions in Multi-Exit Neural Networks via Block-Dependent Losses.

As the size of a model increases, making predictions using deep neural networks (DNNs) is becoming more computationally expensive. Multi-exit neural network is one promising solution that can flexibly make anytime predictions via early exits, depending on the current test-time budget which may vary over time in practice (e.g.