Ron Dorfman

Ron Dorfman

Optimization and ML | PhD student at the Technion - Isreal Institute of Technology

Publications

DoCoFL: Downlink Compression for Cross-Device Federated Learning

Published in 40th International Conference on Machine Learning (ICML), 2023

Many compression techniques have been proposed to reduce the communication overhead of Federated Learning training procedures. However, these are typically designed for compressing model updates, which are expected to decay throughout training. As a result, such methods are inapplicable to downlink (i.e., from the parameter server to clients) compression in the cross-device setting, where heterogeneous clients may appear only once during training and thus must download the model parameters. Accordingly, we propose DoCoFL – a new framework for downlink compression in the cross-device setting. Importantly, DoCoFL can be seamlessly combined with many uplink compression schemes, rendering it suitable for bi-directional compression. Through extensive evaluation, we show that DoCoFL offers significant bi-directional bandwidth reduction while achieving competitive accuracy to that of a baseline without any compression.

link


Adapting to Mixing Time in Stochastic Optimizaiton with Markovian Data

Published in 39th International Conference on Machine Learning (ICML), 2022

Spotlight Presentation

We consider stochastic optimization problems where data is drawn from a Markov chain. Existing methods for this setting crucially rely on knowing the mixing time of the chain, which in real-world applications is usually unknown. We propose the first optimization method that does not require the knowledge of the mixing time, yet obtains the optimal asymptotic convergence rate when applied to convex problems. We further show that our approach can be extended to: (i) finding stationary points in non-convex optimization with Markovian data, and (ii) obtaining better dependence on the mixing time in temporal difference (TD) learning; in both cases, our method is completely oblivious to the mixing time. Our method relies on a novel combination of multi-level Monte Carlo (MLMC) gradient estimation together with an adaptive learning method.

link

Offline Meta Reinforcement Learning – Identifiability Challenges and Effective Data Collection Strategies

Published in 35th Conference on Neural Information Processing Systems (NeurIPS), 2021

Consider the following instance of the Offline Meta Reinforcement Learning (OMRL) problem: given the complete training logs of conventional RL agents, trained on different tasks, design a meta-agent that can quickly maximize reward in a new, unseen task from the same task distribution. In particular, while each conventional RL agent explored and exploited its own different task, the meta-agent must identify regularities in the data that lead to effective exploration/exploitation in the unseen task. Here, we take a Bayesian RL (BRL) view, and seek to learn a Bayes-optimal policy from the offline data. Building on the recent VariBAD BRL approach, we develop an off-policy BRL method that learns to plan an exploration strategy based on an adaptive neural belief estimate. However, learning to infer such a belief from offline data brings a new identifiability issue we term MDP ambiguity. We characterize the problem, and suggest resolutions via data collection and modification procedures. Finally, we evaluate our framework on a diverse set of domains, including difficult sparse reward tasks, and demonstrate learning of effective exploration behavior that is qualitatively different from the exploration used by any RL agent in the data.

link      code     

Spatio-Temporal Detection of Cumulonimbus Clouds in Infrared Satellite Images

Published in IEEE International Conference on the Science of Electrical Engineering in Israel (ICSEE), 2018

Best Student Paper Award

In this paper, we address the problem of Cumulonimbus (Cb) cloud detection from Infrared (IR) satellite images. The detection of such storm clouds is of high importance since they pose extreme danger to aviation. We present a joint spatio-temporal detection method that exploits the distinct spatial characteristics of Cb clouds as well as their prototypical evolution over time. The presented method is unsupervised and does not require labeled data or predefined spatial handcrafted features, such as particular shapes, temperatures, textures, and gradients. We demonstrate the performance of the proposed method on several sequences of IR satellite images taken from the middle east region.

link