Robotics
9 AM-12 PM, February 25, CSL 301
Robotics is a rapidly evolving field with great potential to benefit our society. Robots enhance efficiency, safety, and precision across various industries, contribute to scientific exploration, improve medical practices, support education and research. Consequently, developing adaptive, resilient, and safe systems requires multi-disciplinary synergy among broad areas of research not limited to control, perception, and decision-making. This session will focus on how these areas converge to drive innovation and ensure the future of robotics benefits society.
Time: 11 AM-12 PM
“Building Generalist Robots with Agility via Learning and Control: Humanoids and Beyond”
Abstract: Recent breathtaking advances in AI and robotics have brought us closer to building general-purpose robots in the real world, e.g., humanoids capable of performing a wide range of human tasks in complex environments. Two key challenges in realizing such general-purpose robots are: (1) achieving “breadth” in task/environment diversity, i.e., the generalist aspect, and (2) achieving “depth” in task execution, i.e., the agility aspect.
In this talk, I will present recent works that aim to achieve both generalist-level adaptability and specialist-level agility, demonstrated across various real-world robots, including full-size humanoids, quadrupeds, aerial robots, and ground vehicles. The first part of the talk focuses on learning agile and general-purpose humanoid whole-body control using sim2real reinforcement learning. The second part will discuss the limitations of such end2end sim2real pipelines and how combining learning with control can enhance safety, efficiency, and adaptability.
More details on the presented works are available on the CMU LeCAR Lab website: https://lecar-lab.github.io/
Biography: Guanya Shi is an Assistant Professor at the Robotics Institute at Carnegie Mellon University, leading the Learning and Control for Agile Robotics Lab. He completed his Ph.D. in Control and Dynamical Systems in 2022 from Caltech. Before joining CMU, he was a postdoctoral scholar at the University of Washington. He is broadly interested in the intersection of machine learning and control, spanning the entire spectrum from theory and foundation, algorithm design, to real-world agile robotics. Guanya was the recipient of several awards, including the Simoudis Discovery Prize and the Ben P.C. Chou Doctoral Prize from Caltech, the Rising Star in Data Science, and the Outstanding Student Paper Award Finalist at RSS. Guanya is an Associate Editor of IEEE Robotics and Automation Letters.
Time: 9-9:40 AM
” Learning World Models for Robots”
Abstract: Recent progress in AI can largely be attributed to the emergence of large models trained on large datasets. However, teaching AI agents to reliably interact with our physical world has proven challenging and, consequently, this new paradigm has not materialized as much in robotics as in related areas. In this proposal, I will share my perspective on why it is challenging, what an agent that “understands” our physical world may look like, and our current steps toward this goal. Concretely, I will discuss our work on TD-MPC, a highly data-driven approach to world models (models of the physical world) that can learn from diverse data, improve autonomously through real-world interaction, and scale with data and model size. I will discuss its algorithmic foundations, as well as application to diverse decision-making problems in robotics and beyond.
Biography: Nicklas Hansen is a PhD candidate at University of California San Diego advised by Prof. Xiaolong Wang and Prof. Hao Su. Their research focuses on developing generalist AI agents that learn from physical interaction. Nick has spent time at NVIDIA Research, Meta AI (FAIR), as well as Berkeley AI Research, and received their BS and MS degrees from Technical University of Denmark. They are a recipient of the 2024 NVIDIA Graduate Fellowship, and their work has been featured at top venues in machine learning and robotics.
Student Speakers
Haonan Chen
Time: 9:50-10:05 AM
“Tool-as-Interface: Learning Robot Tool Use from Human Play through Imitation Learning”
Vlas Zyrianov
Time: 10:05-10:20 AM
“LidarDM: Generative LiDAR Simulation in a Generated World”
Shaoxiong Yao
Time: 10:20-10:35 AM
“Safe Leaf Manipulation for Accurate Shape and Pose Estimation of Occluded Fruits”
Runpei Dong
Time: 10:35-10:50 AM
“Learning Getting-Up Policies for Real-World Humanoid Robots”