Toward robust manipulation in complex environments

June 23, 2021 14:00 - 15:00 UTC

Abstract: Recent advances in deep learning and GPU-based computing have enabled significant progress in several areas of robotics, including navigation, visual recognition, and object manipulation. This progress has turned applications such as autonomous driving and delivery tasks in warehouses, hospitals, or hotels into realistic application scenarios. However, robust manipulation in complex settings is still an open research problem. Various efforts at NVIDIA robotics research investigate how deep learning along with physics-based and photo-realistic simulation can be used to train manipulators in virtual environments and then deploy them in the real world. Our work shows promising results on different pieces of the manipulation puzzle, including manipulator control, touch sensing, object pose detection, and object pick and place. In this talk, I will present some of these advances. I will describe a robot manipulator that can open and close cabinet doors and drawers, detect and pickup known and unknown objects, and move these objects to desired locations. Our baseline system is designed to be applicable in a wide variety of environments, only relying on 3D articulated models of the kitchen and the relevant objects. I will discuss lessons learned so far, and various research directions toward enabling more robust and general manipulation systems.

Bio: Dieter Fox is Senior Director of Robotics Research at NVIDIA and Professor in the Paul G. Allen School of Computer Science & Engineering at the University of Washington, where he heads the UW Robotics and State Estimation Lab. Dieter obtained his Ph.D. from the University of Bonn, Germany. His research is in robotics and artificial intelligence, with a focus on state estimation and perception applied to problems such as mapping, object detection and tracking, manipulation, and activity recognition. He has published more than 200 technical papers and is the co-author of the textbook “Probabilistic Robotics”. He is a Fellow of the IEEE, AAAI, and ACM, and recipient of the 2020 IEEE Pioneer in Robotics and Automation Award. Dieter also received several best paper awards at major robotics, AI, and computer vision conferences. He was an editor of the IEEE Transactions on Robotics, program co-chair of the 2008 AAAI Conference on Artificial Intelligence, and program chair of the 2013 Robotics: Science and Systems conference.

Robotics: The Science of Motion

June 25, 2021 13:30 - 14:30 UTC

Abstract: Robots are moving machines submitted to the physical laws of gravity. The talk reports more than 30 years of fundamental research on robot motion planning and control, a research that ranges from computational geometry to probabilistic algorithms via nonholonomic systems. The talk will be illustrated by success stories in Product Lifecycle Management (PLM), mobile robotics, humanoid robotics and biomechanics.

Bio: Jean-Paul Laumond, IEEE Fellow, is a roboticist. He is Directeur de Recherche Emeritus at CNRS. Previously with LAAS-CNRS (team Gepetto) in Toulouse, he joined in 2019 the research unit DIENS affiliated to ENS, CNRS and INRIA (team Willow) in Paris. His research is about robot motion planning and control. In 2001 and 2002 he created and managed Kineo CAM, a spin-off company from LAAS-CNRS devoted to develop and market motion planning technology. Siemens acquired Kineo CAM in 2012. In 2006, he launched the research team Gepetto dedicated to anthropomorphic motion studies along three perspectives: artificial motion for humanoid robots, virtual motion for digital actors, and natural motions of human beings. He has published more than 150 papers in international journals and conferences in Robotics, Computer Science, Automatic Control and in Neurosciences. He teaches Robotics at Ecole Normale Supérieure in Paris. He has been the 2011-2012 recipient of the Chaire Innovation technologique Liliane Bettencourt at Collège de France in Paris. He is the 2016 recipient of the IEEE Inaba Technical Award for Innovation Leading to Production. He is a member of the French Academy of Technologies and of the French Academy of Sciences.

Towards Complex Language in Partially Observed Environments

June 28, 2021 14:00 - 15:00 UTC

Abstract: Robots can act as a force multiplier for people, whether a robot assisting an astronaut with a repair on the International Space station, a UAV taking flight over our cities, or an autonomous vehicle driving through our streets. Existing approaches use action-based representations that do not capture the goal-based meaning of a language expression and do not generalize to partially observed environments. The aim of my research program is to create autonomous robots that can understand complex goal-based commands and execute those commands in partially observed, dynamic environments. I will describe demonstrations of object-search in a POMDP setting with information about object locations provided by language, and mapping between English and Linear Temporal Logic, enabling a robot to understand complex natural language commands in city-scale environments. These advances represent steps towards robots that interpret complex natural language commands in partially observed environments using a decision theoretic framework.

Bio: Stefanie Tellex is an Associate Professor of Computer Science at Brown University. Her group, the Humans To Robots Lab, creates robots that seamlessly collaborate with people to meet their needs using language, gesture, and probabilistic inference, aiming to empower every person with a collaborative robot. She completed her Ph.D. at the MIT Media Lab in 2010, where she developed models for the meanings of spatial prepositions and motion verbs. Her postdoctoral work at MIT CSAIL focused on creating robots that understand natural language. She has published at SIGIR, HRI, RSS, AAAI, IROS, ICAPs and ICMI, winning Best Student Paper at SIGIR and ICMI, Best Paper at RSS, and an award from the CCC Blue Sky Ideas Initiative. Her awards include being named one of IEEE Spectrum's AI's 10 to Watch in 2013, the Richard B. Salomon Faculty Research Award at Brown University, a DARPA Young Faculty Award in 2015, a NASA Early Career Award in 2016, a 2016 Sloan Research Fellowship, and an NSF Career Award in 2017. Her work has been featured in the press on National Public Radio, BBC, MIT Technology Review, Wired and Wired UK, as well as the New Yorker. She was named one of Wired UK's Women Who Changed Science In 2015 and listed as one of MIT Technology Review's Ten Breakthrough Technologies in 2016.

Website: http://h2r.cs.brown.edu/