Improved Information-Theoretic Generalization Bounds for Distributed, Federated, and Iterative Learning.

We consider information-theoretic bounds on the expected generalization error for statistical learning problems in a network setting. In this setting, there are nodes, each with its own independent dataset, and the models from the nodes have to be aggregated into a final centralized model. We consider both simple averaging of the models as well as more complicated multi-round algorithms.