cryptogon.com

So why haven’t these amazing AI capabilities translated into the kinds of helpful and broadly useful robots we’ve seen in science fiction? Where are the robots that can clean off the table, fold your laundry, and make you breakfast?

Unfortunately, the highly successful generative AI formula—big models trained on lots of Internet-sourced data—doesn’t easily carry over into robotics, because the Internet is not full of robotic-interaction data in the same way that it’s full of text and images. Robots need robot data to learn from, and this data is typically created slowly and tediously by researchers in laboratory environments for very specific tasks. Despite tremendous progress on robot-learning algorithms, without abundant data we still can’t enable robots to perform real-world tasks (like making breakfast) outside the lab. The most impressive results typically only work in a single laboratory, on a single robot, and often involve only a handful of behaviors.

If the abilities of each robot are limited by the time and effort it takes to manually teach it to perform a new task, what if we were to pool together the experiences of many robots, so a new robot could learn from all of them at once? We decided to give it a try. In 2023, our labs at Google and the University of California, Berkeley came together with 32 other robotics laboratories in North America, Europe, and Asia to undertake the RT-X project, with the goal of assembling data, resources, and code to make general-purpose robots a reality.

Here is what we learned from the first phase of this effort.

How to create a generalist robot

Humans are far better at this kind of learning. Our brains can, with a little practice, handle what are essentially changes to our body plan, which happens when we pick up a tool, ride a bicycle, or get in a car. That is, our “embodiment” changes, but our brains adapt. RT-X is aiming for something similar in robots: to enable a single deep neural network to control many different types of robots, a capability called cross-embodiment. The question is whether a deep neural network trained on data from a sufficiently large number of different robots can learn to “drive” all of them—even robots with very different appearances, physical properties, and capabilities. If so, this approach could potentially unlock the power of large datasets for robotic learning.

The scale of this project is very large because it has to be. The RT-X dataset currently contains nearly a million robotic trials for 22 types of robots, including many of the most commonly used robotic arms on the market. The robots in this dataset perform a huge range of behaviors, including picking and placing objects, assembly, and specialized tasks like cable routing. In total, there are about 500 different skills and interactions with thousands of different objects. It’s the largest open-source dataset of real robotic actions in existence.

Surprisingly, we found that our multirobot data could be used with relatively simple machine-learning methods, provided that we follow the recipe of using large neural-network models with large datasets. Leveraging the same kinds of models used in current LLMs like ChatGPT, we were able to train robot-control algorithms that do not require any special features for cross-embodiment. Much like a person can drive a car or ride a bicycle using the same brain, a model trained on the RT-X dataset can simply recognize what kind of robot it’s controlling from what it sees in the robot’s own camera observations. If the robot’s camera sees a UR10 industrial arm, the model sends commands appropriate to a UR10. If the model instead sees a low-cost WidowX hobbyist arm, the model moves it accordingly.

To test the capabilities of our model, five of the laboratories involved in the RT-X collaboration each tested it in a head-to-head comparison against the best control system they had developed independently for their own robot. Each lab’s test involved the tasks it was using for its own research, which included things like picking up and moving objects, opening doors, and routing cables through clips. Remarkably, the single unified model provided improved performance over each laboratory’s own best method, succeeding at the tasks about 50 percent more often on average.

Of the four mice infected with the virus, all began to lose weight five days post-infection. Shortly thereafter, they exhibited symptoms including sluggishness and white eyes.

The four mice died within eight days of inoculation. Researchers described the results as “surprising.”

Researchers then infected eight additional mice, euthanized them, and selected organs from four to analyze. High levels of viral RNA were found in various organs, including the brain, lungs, and eyes. While the viral load in the lungs decreased by the sixth day, it increased in the brain.

“This finding suggested that severe brain infection during the later stages of infection may be the key cause of death in these mice,” the scientists said.

The experiments were on a mutant strain of the pangolin virus, known as GX_P2V(short_3UTR).

“They are denying entry to Border Patrol agents to conduct our duties,” a federal official told CBS News, who wondered “what authority (Texas officials) have over the federal government.” Texans are increasingly wondering about the opposite question.

The U.S. and U.K, with support from Australia, the Netherlands, Bahrain, and Canada, conducted joint strikes tonight against Houthi targets in Yemen, per DOD official. Strikes involved U.S. aircraft, ships and submarines.

The state budgeted nearly $3.5 million to provide 60 migrant families in Brunswick with two years of rent. Maine expects the immigrants to “gain the means” to pay housing costs through state “support” and “guidance.”

Six renegade members of the Chabad-Lubavitch Hasidic movement secretly began digging the 3-foot-high, 20-foot-wide, 50-foot-long tunnel themselves, using crude instruments and their hands. They stuffed the dirt into their pockets so that their work wouldn’t be detected by the sect’s leaders and wider community, a source in the Orthodox community told The Post.

“You’ve seen the movie ‘The Shawshank Redemption’? That’s what these young men did at first: They dug and put the dirt in their pockets,” said Eitan Kalmowitz, a member of the Lubavitcher community in Crown Heights.