ECCOMAS 2024

Conditional Generative Models for Robot Control: new insights and perspectives

  • Califano, Federico (University of Twente)
  • Botteghi, Nicolo (University of Twente)

Please login to view abstract download link

Conditional generation of data through denoising diffusion probabilistic models (DDPMs) has proved to be surprisingly effective in generation of high quality images from language description. We are investigating ways to unravel the potential of DDPMs also in robotics applications, possibly observing and contributing to a new paradigm shifts in the field of robot control and in general control of physical systems, where the generated output is a decision-making sequence. The goal of this talk is to discuss: i) the advantages and the peculiarities (compared to other methods) of controlling a robot through conditional generation of control policies; ii) how to tackle the bottleneck of real-world implementation of trained policies, improving sample-efficiency through dedicated architectures, possibly including physics-informed priors.