Ergodic control methods have emerged as a leading approach for synthesizing exploratory behaviors in robotic systems. These approaches target diverse robot applications in which some form of exploration or coverage is required, defined by a distribution, which arise naturally in applications such as surface cleaning, exploration, data collection for learning, localization, surveillance, or manipulation tasks. Ergodic control generates control signals that drive a robot to spend time in areas proportional to the density of an arbitrary spatial distribution (e.g., distributed information). Such a controller produces `natural' exploration behaviors that takes into account that there is a cost in moving from one sample to the next and the impact of physical dynamics of a system. Inherent to the idea of ergodic control, without significant modulation of the theory or implementation, is the possibility of covering/searching/tracking an unsteady target distribution. Ergodic control methods can be applied to diverse robots and has matured in terms of numerical techniques, deployment in hardware, and application domain.
This workshop tutorial will present different techniques to achieve ergodic control including Spectral Multiscale Coverage (SMC) and Heat Equation Driven Area Coverage (HEDAC), from leading researchers in the area from around the world. Demonstrations and discussions on algorithmic implementation, and applications will be discussed during the tutorial. Participants will be encouraged to follow along and run associated python source codes directly from a dedicated website.
Ergodic control is a model-based control approach for coverage and exploration that can be exploited in a wide range of problems requiring the automatic exploration of regions of interest. It is a relatively new research area, which relies on fundamental core principles in signal processing (spectral decomposition with Fourier series for SMC), physics and information theory (diffusion processes for HEDAC), statistics (for methods based on KL divergence), and control synthesis based on optimal control principles. As a topic that spans several research areas, ergodic control has the potential to be of interest to robotics researchers in adjacent areas.
Ergodic control uses spatial distributions to specify the temporal behavior of a dynamical system. This is in contrast to most model-based control methods, where the assumption is that the goal of a system is a particular state or sequence of states. Instead, ergodic control assumes that distributions provide a complete specification of a system's behavior. For example, a conventional tracking problem in robotics is characterized by a target to reach, requiring a controller to be computed to reach this target. In ergodic control, instead of providing a single target point, a probability distribution is given to the robot, which must cover the distribution in an efficient manner (namely, ``tracking a distribution'' instead of ``tracking a point'').
The formulation of ergodic control naturally tackles problems in exploration and exploitation challenge, thus making it relevant to areas in planning and reinforcement learning. Ergodic control is largely overshadowed by reinforcement learning and (naive) stochastic sampling approaches that have readily available open-source toolsets yet result in under-performing exploration. This tutorial aims at reducing this gap and provide awareness to early-stage researchers in robotics that these control techniques exist and, because of the diversity of platforms and applications that they target, could be useful in their own research. For attendees who are familiar with the topic, the tutorial also aims to show some of the most recent advances in this research, to provide an overview of strengths and weaknesses of the various methods used in ergodic control, and to discuss future needs that still need to be investigated. Finally, the fundamental method and properties exploited by different ergodic control approaches make it a fascinating research topic that is relevant both in theoretical developments and in practical applications.
The tutorial targets an audience with a general background in robotics, including young researchers.
The topics of interest for this workshop will include three main components: background and introduction to ergodic control, algorithms that achieve ergodic control, and application areas. Within these main topic components, the workshop will have the organizers present dedicated tutorials on topics for which they are experts.
The full day tutorial will be covered by the 6 organizers, who have diverse expertise in ergodic control. The first part (mostly morning) will cover the fundamental aspects, including the introduction of the basic components (Fourier series, diffusion, KL divergence). Example codes will be run during the tutorial and the attendees will be encouraged to also run the codes on their laptops or tablets, directly through the dedicated website https://ergodiccontrol.github.io/. The second part (mostly afternoon) will concentrate on applications, comparisons and future work.
Below is an outline of the topics that will be presented at the tutorial workshop.
Background and Introduction:| 08:45-09:00 | Welcome and Introduction | (Todd) |
| 09:00-09:15 | Why do we care about ergodic control? | (3-min pitches by organizers) |
| 09:15-09:35 | Technical introduction to ergodic control | (Katie) |
| 09:35-09:55 | Implicit Q&A | (Ian) |
| 09:55-10:00 | Teaser | (Guillaume) |
| 10:00-10:30 | Coffee break with posters presentations | |
| 10:30-11:00 | Sandbox codes | (Muchen, Todd) |
| 11:00-11:20 | Trajectory Optimization for Ergodic Control | (Ian) |
| 11:20-11:40 | Ergodic control using diffusion, HEDAC | (Stefan) |
| 11:40-11:55 | Ergodic control (SMC) in 1D, 2D, 3D and more! | (Sylvain) |
| 11:55-12:00 | Teaser | (Guillaume) |
| 12:00-13:00 | Lunch break (lunch boxes available) | |
| 13:00-13:20 | Sampling-based approaches | (Todd, Guillaume, Muchen) |
| 13:20-13:45 | Applications: HRI and biology | (Katie) |
| 13:45-14:10 | Applications: Search with UAVs | (Stefan) |
| 14:10-14:35 | Applications: Robustness and Optimality in Ergodic Control | (Ian) |
| 14:35-15:00 | Applications: Whole body exploration, insertion tasks, drawing, HEDAC on point clouds | (Sylvain) |
| 15:00-15:25 | Applications: Ergodic control in robot learning: where do target distributions come from? | (Todd) |
| 15:25-15:30 | Teaser | (Guillaume) |
| 15:30-16:00 | Coffee break with posters presentations | |
| 16:00-16:20 | Homogeneous and Heterogeneous Multi-Robot Coverage | (Guillaume) |
| 16:20-16:35 | Learning ergodic heterogeneous distributions | (Ananya, Guillaume, Ian) |
| 16:35-16:50 | Design considerations | (Todd) |
| 16:50-17:00 | Closing Remarks, Grand Challenges | (Todd, Sylvain) |
Run ergodic control sandboxes directly in your browser:
Notebooks:
Run the notebooks directly on Google Colab:
Ergodic control on point clouds (notebook):
Standalone codes:
HEDAC codes:
We invite the submission of abstracts and posters that focus on improving some aspect of existing ergodic control methods or using ergodic control in a particular application.
G. Mathew and I. Mezic. Spectral multiscale coverage: A uniform coverage algorithm for mobile sensor networks. In Proc. IEEE Conf. on Decision and Control, pages 7872–7877, Dec 2009.
G. Mathew and I. Mezic. Metrics for ergodicity and design of ergodic dynamics for multi-agent systems. Physica D: Nonlinear Phenomena, 240(4):432–442, 2011.
L.M. Miller and T.D. Murphey. Trajectory optimization for continuous ergodic exploration. In American Control Conference, pages 4196–4201, June 2013.
L.M. Miller, Y. Silverman, M.A. MacIver, and T.D. Murphey. Ergodic exploration of distributed information. IEEE Trans. on Robotics, 32(1):36–52, Feb 2016.
K. Fitzsimons and T.D. Murphey. Ergodic shared control: Closing the loop on pHRI based on information encoded in motion. ACM Trans. on Human-Robot Interaction, 11(4):1–20, 2022.
I. Abraham, A. Prabhakar, M. J. Z. Hartmann, and T. D. Murphey. Ergodic exploration using binary sensing for nonparametric shape estimation. 2(2):827–834, April 2017.
D. E. Dong, H. P. Berger, and I. Abraham. Time optimal ergodic search. In Proc. Robotics: Science and Systems (RSS), 2023. Best Paper Award.
S. Ivić, B. Crnković, and I. Mezić. Ergodicity-based cooperative multiagent area coverage via a potential field. IEEE Trans. on Cybernetics, 47(8):1983–1993, 2017.
E. Ayvali, H. Salman, and H. Choset. Ergodic coverage in constrained environments using stochastic trajectory optimization. In Proc. IEEE/RSJ Intl Conf. on Intelligent Robots and Systems (IROS), pages 5204–5210, 2017.
G. Sartoretti, A. Rao, and H. Choset. Spectral-based distributed ergodic coverage for heterogeneous multi-agent search. In F. Matsuno, S. Azuma, and M. Yamamoto, editors, Distributed Autonomous Robotic Systems, pages 227–241. Springer, 2022.