Distributed and Federated Learning
My research in distributed and federated learning focuses on developing efficient and robust optimization algorithms for settings where data and computation are distributed across multiple clients. Two important challenges are reducing communication costs and ensuring reliable learning when some participants provide corrupted or adversarial updates.
I investigate these challenges through two complementary directions: Byzantine-resilient federated learning and communication-efficient optimization using Adaptive Error Feedback.
How can optimization remain reliable when some clients send malicious or corrupted updates?
How can adaptive compression reduce communication while maintaining optimization accuracy?
1. Byzantine-Resilient Federated Learning
Federated learning enables multiple clients to collaboratively train a model without sharing their local datasets. However, heterogeneous data and malicious or corrupted client updates can compromise the reliability of distributed training.
My research focuses on Byzantine-resilient optimization methods for reliable federated learning in the presence of adversarial clients.
- How can federated optimization remain robust to adversarial clients while handling heterogeneous data?
2. Adaptive Error Feedback
Communication is a major bottleneck in distributed optimization. Gradient compression can reduce the amount of information exchanged between workers, but aggressive compression can also affect convergence.
Currently, I study communication-efficient optimization methods that adapt compression to the optimization trajectory.
- How can compression adapt to the optimization trajectory while maintaining convergence guarantees?