Caltech Roboticist Says Safety Should Be Top Concern with Humanoid Robots

A quick search of humanoid robot videos will deliver demonstrations of two-legged robots playing soccer, dancing, jumping several feet in the air, even competing in kickboxing matches. It is all dazzling, certainly. But roboticist Aaron Ames, the Booth-Kresa Leadership Chair and director of Caltech's Center for Autonomous Systems and Technologies (CAST) and a leader in the field of robotic safety, has deep concerns. What is crucially missing before humanoids can play a larger role in our lives and society, Ames says, is safety.

Without the adoption of clear safety standards, Ames argues, someone inevitably will get hurt.

"There's this huge push to deliver humanoid robots. And it's truly amazing what we can have robots do now, from a motion perspective," says Ames, Bren Professor of Mechanical and Civil Engineering, Control and Dynamical Systems, and Aerospace at Caltech. Indeed, humanoids have recently demonstrated pulling off parkour-style flips, climbing steep ladders, and performing flashy sword routines. "But if you don't think about safety, none of that is going to matter. And honestly, very few of the humanoid companies are really thinking about safe deployment."

Humanoids, these two-legged robots that look like smooth, often faceless versions of us, have become a center of attention within robotics, both in terms of funding and hype. We are now in a situation where humanoid demonstrations are being driven solely by performance and making the latest "cool" video, Ames says, and companies are rushing to scale and deploy.

However, there is a chasm between pulling off exciting tricks in a controlled environment and being able to navigate in dynamic real-world settings. And humanoids are not yet fully capable of taking in information about and assessing rapidly changing settings. Ames says current work on humanoid performance does not address the kinds of fundamental challenges that will arise when the robots have to sense and respond to nearby, often unfamiliar, obstacles—not step on the family dog, for example. "Little things like that are surprisingly very, very hard for robots," Ames says.

But the biggest safety issues could be far worse.

Ames points to early failures in the autonomous car industry. High-profile safety problems shuttered the Uber Advanced Technologies Group and GM's Cruise program. He doesn't want the same for robotics. "I'm working in my lab at Caltech to deliver solutions," he says. "There is a way to guarantee humanoid robots are safe to operate around people. We just need to be doing it now."

Mathematical Guarantees of Safety

Ames is "Mr. Robot Safety." More than a decade ago, while working on early humanoid designs, he realized there was a need for formal guarantees of safety in robotic systems.

"The reality is that to do safety, you need math," says Ames. "You need formal guarantees, you need physics, you need theorems, and you need rigor."

In 2014, at the IEEE Conference on Decision and Control, Ames presented a new invention: control barrier functions (CBFs), a mathematical tool that guarantees the safety of robotic systems. In the now seminal paper based on that work and published in the journal IEEE Transactions on Automatic Control, Ames and his colleagues showed that CBFs completely characterize safety. Basically, a CBF is a single mathematical equation that defines "safe actions"—the mathematical equivalent of statements like "don't get within a set distance of any moving obstacles," "maintain balance," "respect these limits on your joints," and "stay below a certain speed." By doing this, CBFs decouple concerns about safety from those centered on performance. The CBF safety filter can adjust the input of a learning-based controller concerned with performance to ensure safe actions. That means the AI can focus on performance while its safety is guaranteed by CBFs that act like mathematical guardrails between AI and the physical world.

Over the years, Ames and his students and colleagues have experimentally tested CBFs on drones, quadrupeds, even on an F-16 fighter jet. (There are thousands of papers in the dynamical systems, control, and robotics literature that include the phrase "control barrier function" in their title.) Today, it is something of a universal safety layer in the world of control theory, the branch of engineering that deals with regulating the way dynamic systems behave.

Ames even co-founded a company called 3Laws that commercialized the CBF-based safety layer for robots—not necessarily for humanoids, but for a variety of mobile robotic systems such as ground robots used in warehouses, drones, and autonomous vehicles. It was acquired by Amazon earlier this year. Today, he is an Amazon scholar and, in addition to his lab at Caltech, leads the Safe Autonomy Frontiers (SAF) Lab for Amazon Robotics.

Doing the Hard Work to Make Humanoids Safe

Walk into Ames's workspace at Caltech—what is known as the AMBER Lab (for Advanced Mechanical Bipedal Experimental Robotics Lab)—and you might find students testing control systems on one of the team's humanoids or its quadruped, a sleek, dog-like robot. Other robotic systems might be dangling dormant from gantries, while a modified Unitree G1 humanoid walks autonomously around the lab. On one recent visit, observers noted wooden pallets stacked at the side of the lab, evidence of tests that involved a humanoid jumping atop the pile and safely hopping back down.

Much of the activity in the lab is less dramatic but just as important: coding. Ames's graduate students and postdoctoral scholars spend endless hours at their computers, developing and perfecting the mathematical backend that ensures robotic safety, including the latest incarnations of CBFs.

Ames originally came up with CBFs to address a need he saw in humanoids, but the work was so widely applicable to other robotic systems with a strong immediate need for safety that the focus shifted away from two-legged robots. It has only been in the past year or so that the AMBER Lab has applied CBF to humanoids, but that work is now a major focus for the group. Ames says the math translated over to the legged robots exceptionally well. During the last year, the lab has published or presented dozens of papers in journals and at major robotics conferences showing how CBFs can mathematically guarantee the safety of humanoids.

One of those papers, presented at the IEEE International Conference on Robotics and Automation, demonstrated that CBF combined with newer AI tools such as reinforcement learning—a type of machine learning that rewards robots for behaving in desired ways—allowed a humanoid to successfully avoid collisions with its environment. These ideas were recently extended to help a humanoid dodge a ball thrown at it. This "dodgeball" behavior can also be used to help robots move out of the way of moving obstacles or people. In one set of videos documenting such tests, a humanoid walks up to a person and then stops and reverses course when it comes too close.

In another paper that has been accepted for publication in IEEE Robotics and Automation Letters, the team describes giving a humanoid a LIDAR (light detection and ranging) sensor that essentially shoots out light beams to determine the robot's distance from all of the objects in its environment, even if they are in motion. "This is perceiving the environment, using some nice mathematical tools to map the safety around it, and then using optimization to figure out where to go to be most safe," says Zach Olkin, a graduate student in the AMBER Lab whose work has largely focused on the incorporation of CBF into reinforcement learning, grounding rewards-based AI in the real physical world.

A video of the experiments shows a robot sidestepping to get out of the way of a person walking toward it; in another, a dog interacts with the robot and "herds" it out of the way. "In the real world, robots need to be aware of people and obstacles," Olkin says. "You don't want to just have them come to a stop because then you lose the robot's ability to do anything."

Ames thinks the combination of established control-guided methods, like CBFs, with the AI paradigm represents the immediate future in humanoid safety.

"You cannot use learning to guarantee safety—period," Ames says. While machine learning can optimize performance by learning from data and trying to emulate it, it cannot produce a proof that says the system will never output an unsafe action. It lacks what mathematicians call "describability," a verifiable explanation of why it performs the way it does. "Math is perfect and describable," says Ames. "It lets you make guarantees. You have to be able to write down the equations."

And while that process might make a robotic system slower or less impressive at performing some tasks, it will also make it safe. And Ames notes that being safe is also an important way to improve overall performance. After all, if you're walking and you fall over, you're not performing well. It is through a principled approach to safe robot deployment coupled with the latest innovations in robotics and learning that will pave the way for robots to be deployed, he says.

"I want to see robots in the real world doing useful tasks around people—making our lives better," Ames says. "For that to happen, we need to know we can trust robots to do the right thing."