IEEE News

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion. RoboCup German Open: 17–21 April 2024, KASSEL, GERMANY AUVSI XPONENTIAL 2024: 22–25 April 2024, SAN DIEGO Eurobot Open 2024: 8–11 May 2024, LA ROCHE-SUR-YON, FRANCE ICRA 2024: 13–17 May 2024, YOKOHAMA, JAPAN RoboCup 2024: 17–22 July 2024, EINDHOVEN, NETHERLANDS Enjoy today’s videos! In the SpaceHopper project, students at ETH Zurich developed a robot capable of moving in low gravity environments through hopping motions. It is intended to be used in future space missions to explore small celestial bodies. The exploration of asteroids and moons could provide insights into the formation of the universe, and they may contain valuable minerals that humanity could use in the future.The project began in 2021 as an ETH focus project for bachelor’s students. Now, it is being continued as a regular research project. A particular challenge in developing exploration robots for asteroids is that, unlike larger celestial bodies like Earth, there is low gravity on asteroids and moons. The students have therefore tested their robot’s functionality in zero gravity during a parabolic flight. The parabolic flight was conducted in collaboration with the European Space Agency as part of the ESA Academy Experiments Programme. [ SpaceHopper ] It’s still kind of wild to me that it’s now possible to just build a robot like Menteebot. Having said that, at present it looks to be a fairly long way from being able to usefully do tasks in a reliable way. [ Menteebot ] Look, it’s the robot we all actually want! [ Github ] I wasn’t quite sure what made this building especially “robot-friendly” until I saw the DEDICATED ROBOT ELEVATOR. [ NAVER ] We are glad to announce the latest updates with our humanoid robot CL-1. In the test, it demonstrates stair climbing in a single stride based on real-time terrain perception. For the very first time, CL-1 accomplishes back and forth running, in a stable and dynamic way! [ LimX Dynamics ] EEWOC [Extended-reach Enhanced Wheeled Orb for Climbing] uses a unique locomotion scheme to climb complex steel structures with its magnetic grippers. Its lightweight and highly extendable tape spring limb can reach over 1.2 meters, allowing it to traverse gaps and obstacles much larger than other existing climbing robots. Its ability to bend allows it to reach around corners and over ledges, and it can transition between surfaces easily thanks to assistance from its wheels. The wheels also let it to drive more quickly and efficiently on the ground. These features make EEWOC well-suited for climbing the complex steel structures seen in real-world environments. [ Paper ] Thanks to its “buttock-contact sensors,” JSK’s musculoskeletal humanoid has mastered(ish) the chair-scoot. [ University of Tokyo ] Thanks, Kento! Physical therapy seems like a great application for a humaonid robot when you don’t really need that humanoid robot to do much of anything. [ Fourier Intelligence ] NASA’s Ingenuity Mars helicopter became the first vehicle to achieve powered, controlled flight on another planet when it took to the Martian skies on 19 April 2021. This video maps the location of the 72 flights that the helicopter took over the course of nearly three years. Ingenuity far surpassed expectations—soaring higher and faster than previously imagined. [ JPL ] No thank you! [ Paper ] MERL introduces a new autonomous robotic assembly technology, offering an initial glimpse into how robots will work in future factories. Unlike conventional approaches where humans set pre-conditions for assembly, our technology empowers robots to adapt to diverse scenarios. We showcase the autonomous assembly of a gear box that was demonstrated live at CES2024. [ Mitsubishi ] Thanks, Devesh! In November, 2023 Digit was deployed in a distribution center unloading totes from an AMR as part of regular facility operations, including a shift during Cyber Monday. [ Agility ] The PR2 just refuses to die. Last time I checked, official support for it ceased in 2016! [ University of Bremen ] DARPA’s Air Combat Evolution (ACE) program has achieved the first-ever in-air tests of AI algorithms autonomously flying a fighter jet against a human-piloted fighter jet in within-visual-range combat scenarios (sometimes referred to as “dogfighting”).In this video, team members discuss what makes the ACE program unlike other aerospace autonomy projects and how it represents a transformational moment in aerospace history, establishing a foundation for ethical, trusted, human-machine teaming for complex military and civilian applications. [ DARPA ] Sometimes robots that exist for one single purpose that they only do moderately successfully while trying really hard are the best of robots. [ CMU ]

Xometry’s essential guide reveals the transformative power of artificial intelligence in supply chain optimisation. It lifts the lid on how machine learning, natural language processing, and big data, can streamline procurement and enhance operations efficiency. The guide showcases applications across various sectors such as healthcare, construction, retail, and more, offering actionable insights and strategies. Readers will explore the workings of AI technologies, their implementation in manufacturing, and future trends in supply chain management, making it a valuable resource for professionals aiming to harness AI’s potential to innovate and optimise their supply chain processes. Download this free whitepaper now!

For more than 50 years, Deep Space Station 43 has been an invaluable tool for space probes as they explore our solar system and push into the beyond. The DSS-43 radio antenna, located at the Canberra Deep Space Communication Complex, near Canberra, Australia, keeps open the line of communication between humans and probes during NASA missions. Today more than 40 percent of all data retrieved by celestial explorers, including Voyagers, New Horizons, and the Mars Curiosity rover, comes through DSS-43. “As Australia’s largest antenna, DSS-43 has provided two-way communication with dozens of robotic spacecraft,” IEEE President-Elect Kathleen Kramer said during a ceremony where the antenna was recognized as an IEEE Milestone. It has supported missions, Kramer noted, “from the Apollo program and NASA’s Mars exploration rovers such as Spirit and Opportunity to the Voyagers’ grand tour of the solar system. “In fact,” she said, “it is the only antenna remaining on Earth capable of communicating with Voyager 2.” Why NASA needed DSS-43 Maintaining two-way contact with spacecraft hurtling billions of kilometers away across the solar system is no mean feat. Researchers at NASA’s Jet Propulsion Laboratory, in Pasadena, Calif., knew that communication with distant space probes would require a dish antenna with unprecedented accuracy. In 1964 they built DSS-42—DSS-43’s predecessor—to support NASA’s Mariner 4 spacecraft as it performed the first-ever successful flyby of Mars in July 1965. The antenna had a 26-meter-diameter dish. Along with two other antennas at JPL and in Spain, DSS-42 obtained the first close-up images of Mars. DSS-42 was retired in 2000. NASA engineers predicted that to carry out missions beyond Mars, the space agency needed more sensitive antennas. So in 1969 they began work on DSS-43, which has a 64-meter-diameter dish. DSS-43 was brought online in December 1972—just in time to receive video and audio transmissions sent by Apollo 17 from the surface of the moon. It had greater reach and sensitivity than DSS-42 even after 42’s dish was upgraded in the early 1980s. The gap between the two antennas’ capabilities widened in 1987, when DSS-43 was equipped with a 70-meter dish in anticipation of Voyager 2’s 1989 encounter with the planet Neptune. DSS-43 has been indispensable in maintaining contact with the deep-space probe ever since. The dish’s size isn’t its only remarkable feature. The dish’s manufacturer took great pains to ensure that its surface had no bumps or rough spots. The smoother the dish surface, the better it is at focusing incident waves onto the signal detector so there’s a higher signal-to-noise ratio. DSS-43 boasts a pointing accuracy of 0.005 degrees (18 arc seconds)—which is important for ensuring that it is pointed directly at the receiver on a distant spacecraft. Voyager 2 broadcasts using a 23-watt radio. But by the time the signals traverse the multibillion-kilometer distance from the heliopause to Earth, their power has faded to a level 20 billion times weaker than what is needed to run a digital watch. Capturing every bit of the incident signals is crucial to gathering useful information from the transmissions. The antenna has a transmitter capable of 400 kilowatts, with a beam width of 0.0038 degrees. Without the 1987 upgrade, signals sent from DSS-43 to a spacecraft venturing outside the solar system likely never would reach their target. NASA’s Deep Space Network The Canberra Deep Space Complex, where DSS-43 resides, is one of three such tracking stations operated by JPL. The other two are DSS-11 at the Goldstone Deep Space Communications Complex near Barstow, Calif., and DSS-63 at the Madrid Deep Space Communications Complex in Robledo de Chavela, Spain. Together, the facilities make up the Deep Space Network, which is the most sensitive scientific telecommunications system on the planet, according to NASA. At any given time, the network is tracking dozens of spacecraft carrying out scientific missions. The three facilities are spaced about 120 degrees longitude apart. The strategic placement ensures that as the Earth rotates, at least one of the antennas has a line of sight to an object being tracked, at least for those close to the plane of the solar system. But DSS-43 is the only member of the trio that can maintain contact with Voyager 2. Ever since its flyby of Neptune’s moon Triton in 1989, Voyager 2 has been on a trajectory below the plane of the planets, so that it no longer has a line of sight with any radio antennas in the Earth’s Northern Hemisphere. To ensure that DSS-43 can still place the longest of long-distance calls, the antenna underwent a round of updates in 2020. A new X-band cone was installed. DSS-43 transmits radio signals in the X (8 to 12 gigahertz) and S (2 to 4 GHz) bands; it can receive signals in the X, S, L (1 to 2 GHz), and K (12 to 40 GHz) bands. The dish’s pointing accuracy also was tested and recertified. Once the updates were completed, test commands were sent to Voyager 2. After about 37 hours, DSS-43 received a response from the space probe confirming it had received the call, and it executed the test commands with no issues. DSS-43 is still relaying signals between Earth and Voyager 2, which passed the heliopause in 2018 and is now some 20 billion km from Earth. [From left] IEEE Region 10 director Lance Fung, Kevin Furguson, IEEE President-Elect Kathleen Kramer, and Ambarish Natu, past chair of the IEEE Australian Capital Territory Section at the IEEE Milestone dedication ceremony held at the Canberra Deep Space Communication Complex in Australia. Furguson is the director of the complex.Ambarish Natu Other important missions DSS-43 has played a vital role in missions closer to Earth as well, including NASA’s Mars Science Laboratory mission. When the space agency sent Curiosity, a golf cart–size rover, to explore the Gale crater and Mount Sharp on Mars in 2011, DSS-43 tracked Curiosity as it made its nail-biting seven-minute descent into Mars’s atmosphere. It took roughly 20 minutes for radio signals to traverse the 320-million km distance between Mars and Earth, and then DSS-43 delivered the good news: The rover had landed safely and was operational. “NASA plans to send future generations of astronauts from the Moon to Mars, and DSS-43 will play an important role as part of NASA’s Deep Space Network,” says Ambarish Natu, an IEEE senior member who is a past chair of the IEEE Australian Capital Territory (ACT) Section. DSS-43 was honored with an IEEE Milestone in March during a ceremony held at the Canberra Deep Space Communication Complex. “This is the second IEEE Milestone recognition given in Australia, and the first for ACT,” Lance Fung, IEEE Region 10 director, said during the ceremony. A plaque recognizing the technology is now displayed at the complex. It reads: First operational in 1972 and later upgraded in 1987, Deep Space Station 43 (DSS-43) is a steerable parabolic antenna that supported the Apollo 17 lunar mission, Viking Mars landers, Pioneer and Mariner planetary probes, and Voyager’s encounters with Jupiter, Saturn, Uranus, and Neptune. Planning for many robotic and human missions to explore the solar system and beyond has included DSS-43 for critical communications and tracking in NASA’s Deep Space Network. Administered by the IEEE History Center and supported by donors, the Milestone program recognizes outstanding technical developments around the world. The IEEE Australian Capital Territory Section sponsored the nomination.

Researchers at Oak Ridge National Laboratory in Tennessee recently announced that they have set a record for wireless EV charging. Their system’s magnetic coils have reached a 100-kilowatt power level. In tests in their lab, the researchers reported their system’s transmitter supplied enough energy to a receiver mounted on the underside of a Hyundai Kona EV to boost the state of charge in the car’s battery by 50 percent (enough for about 150 kilometers of range) in less than 20 minutes. “Impressive,” says Duc Minh Nguyen, a research associate in the Communication Theory Lab at King Abdullah University of Science and Technology (KAUST) in Saudi Arabia. Nguyen is the lead author of several of papers on dynamic wireless charging, including some published when he was working toward his PhD at KAUST. In 15 minutes, “the batteries could take on enough energy to drive for another two-and-a-half or three hours—just in time for another pit stop.”–Omer Onar, Oak Ridge National Laboratory The Oak Ridge announcement marks the latest milestone in work on wireless charging that stretches back more than a decade. As IEEE Spectrum reported in 2018, WiTricity, headquartered in Watertown, Mass., had announced a partnership with an unspecified automaker to install wireless charging receivers on its EVs. Then in 2021, the company revealed that it was working with Hyundai to outfit some of its Genesis GV60 EVs with Wireless charging. (In early 2023, Car Buzz reported that it had sniffed out paperwork pointing to Hyundai’s plans to equip its Ionic 5 EV with wireless charging capability.) The plan, said WiTricity, was to equip EVs with magnetic resonance charging capability so that if such a vehicle were parked over a static charging pad installed in, say, the driver’s garage, the battery would reach full charge overnight. By 2020, we noted, a partnership had been worked out between Jaguar, Momentum Dynamics, Nordic taxi operator Cabonline, and charging company Fortam Recharge. That group set out to outfit 25 Jaguar I-Pace electric SUVs with Momentum Dynamics’ inductive charging receivers. The receivers and transmitters, rated at 50 to 75 kilowatts, were designed so that any of the specially equipped taxis would receive enough energy for 80 kilometers of range by spending 15 minutes above the energized coils embedded in the pavement as the vehicle works its way through a taxi queue. Now, according to Oak Ridge, roughly the same amount of charging time will yield about 1.5 times that range. The Oak Ridge research team admits that installing wireless charging pads is expensive, but they say dynamic and static wireless charging can play an important role in expanding the EV charging infrastructure. This magnetic resonance transmitter pad can wirelessly charge an EV outfitted with a corresponding receiver.Oak Ridge National Laboratory Omad Onar, an R&D staffer in the Power Electronics and Electric Machinery Group at Oak Ridge and a member of the team that developed the newest version of the wireless charging system, envisions the static versions of these wireless charging systems being useful even for extended drives on highways. He imagines them being placed under a section of specially marked parking spaces that allow drivers to pull up and start charging without plugging in. “The usual routine—fueling up, using the restroom, and grabbing coffee or a snack usually takes about 15 minutes or more. In that amount of time, the batteries could take on enough energy to drive for another two-and-a-half or three hours—just in time for another pit stop.” What’s more, says Onar, he and his colleagues are still working to refine the system so it will transfer energy more efficiently than the one-off prototype they built in their lab. Meanwhile, Israeli company Electreon has already installed electrified roads for pilot projects in Sweden, Norway, Italy, and other European countries, and has plans for similar projects in the United States. The company found that by installing a stationary wireless charging spot at one terminal end of a bus route near Tel Aviv University (its first real-world project), electric buses operating on that route were able to ferry passengers back and forth using batteries with one-tenth the storage capacity that was previously deemed necessary. Smaller batteries mean cheaper vehicles. What’s more, says Nguyen, charging a battery in short bursts throughout the day instead of depleting it and filling it with up with, say, an hour-long charge at a supercharging station extends the battery’s life.

Stephen Cass: Hello and welcome to Fixing the Future, an IEEE Spectrum podcast where we look at concrete solutions to tough problems. I’m your host, Stephen Cass, a senior editor at IEEE Spectrum. And before I start, I just want to tell you that you can get the latest coverage of some of Spectrum’s most important beats, including AI, climate change, and robotics, by signing up for one of our free newsletters. Just go to spectrum.ieee.org/newsletters to subscribe. We’ve been covering the drone delivery company Zipline in Spectrum for several years, and I do encourage listeners to check out our great onsite reporting from Rwanda in 2019 when we visited one of Zipline’s dispatch centers for delivering vital medical supplies into rural areas. But now it’s 2024, and Zipline is expanding into commercial drone delivery in the United States, including into urban areas, and hitting some recent milestones. Here to talk about some of those milestones today, we have Keenan Wyrobek, Zipline’s co-founder and CTO. Keenan, welcome to the show. Keenan Wyrobek: Great to be here. Thanks for having me. Cass: So before we get into what’s going on with the United States, can you first catch us up on how things have been going on with Rwanda and the other African countries you’ve been operating in? Wyrobek: Yeah, absolutely. So we’re now operating in eight countries, including here in the US. That includes a handful of countries in Africa, as well as Japan and Europe. So in Africa, it’s really exciting. So the scale is really impressive, basically. As we’ve been operating, started eight years ago with blood, then moved into vaccine delivery and delivering many other things in the healthcare space, as well as outside the healthcare space. We can talk a little bit about in things like animal husbandry and other things. The scale is really what’s exciting. We have a single distribution center there that now regularly flies more than the equivalent of once the equator of the Earth every day. And that’s just from one of a whole bunch of distribution centers. That’s where we are really with that operation today. Cass: So could you talk a little bit about those non-medical systems? Because this was very much how we’d seen blood being parachuted down from these drones and reaching those distant centers. What other things are you delivering there? Wyrobek: Yeah, absolutely. So start with blood, like you said, then vaccines. We’ve now done delivered well over 15 million vaccine doses, lots of other pharmaceutical use cases to hospitals and clinics, and more recently, patient home delivery for chronic care of things like hypertension, HIV-positive patients, and things like that. And then, yeah, moved into some really exciting use cases and things like animal husbandry. One that I’m personally really excited about is supporting these genetic diversity campaigns. It’s one of those things very unglamorous, but really impactful. One of the main sources of protein around the world is cow’s milk. And it turns out the difference between a non-genetically diverse cow and a genetically diverse cow can be 10x difference in milk production. And so one of the things we deliver is bull semen. We’re very good at the cold chain involved in that as we’ve mastered in vaccines and blood. And that’s just one of many things we’re doing in other spaces outside of healthcare directly. Cass: Oh, fascinating. So turning now to the US, it seems like there’s been two big developments recently. One is you’re getting close to deploying Platform 2, which has some really fascinating tech that allows packages to be delivered very precisely by tether. And I do want to talk about that later. But first, I want to talk about a big milestone you had late last year. And this was something that goes by the very unlovely acronym of a BVLOS flight. Can you tell us what a BVLOS stands for and why that flight was such a big deal? Wryobek: Yeah, “beyond visual line of sight.” And so that is basically, before this milestone last year, all drone deliveries, all drone operations in the US were done by people standing on the ground, looking at the sky, that line of sight. And that’s how basically we made sure that the drones were staying clear of aircraft. This is true of everybody. Now, this is important because in places like the United States, many aircraft don’t and aren’t required to carry a transponder, right? So transponders where they have a radio signal that they’re transmitting their location that our drones can listen to and use to maintain separation. And so the holy grail of basically scalable drone operations, of course, it’s physically impossible to have people standing around all the world staring at the sky, and is a sensing solution where you can sense those aircraft and avoid those aircraft. And this is something we’ve been working on for a long time and got the approval for late last year with the FAA, the first-ever use of sensors to detect and avoid for maintaining safety in the US airspace, which is just really, really exciting. That’s now been in operations in two distribution centers here, one in Utah and one in Arkansas ever since. Cass: So could you just tell us a little bit about how that tech works? It just seems to be quite advanced to trust a drone to recognize, “Oh, that is an actual airplane that’s a Cessna that’s going to be here in about two minutes and is a real problem,” or, “No, it’s a hawk, which is just going about his business and I’m not going to ever come close to it at all because it’s so far away. Wryobek: Yeah, this is really fun to talk about. So just to start with what we’re not doing, because most people expect us to use either a radar for this or cameras for this. And basically, those don’t work. And the radar, you would need such a heavy radar system to see 360 degrees all the way around your drone. And this is really important because two things to kind of plan in your mind. One is we’re not talking about autonomous driving where cars are close together. Aircraft never want to be as close together as cars are on a road, right? We’re talking about maintaining hundreds of meters of separation, and so you sense it a long distance. And drones don’t have right of way. So what that means is even if a plane’s coming up behind the drone, you got to sense that plane and get out of the way. And so to have enough radar on your drone that you can actually see far enough to maintain that separation in every direction, you’re talking about something that weighs many times the weight of a drone and it just doesn’t physically close. And so we started there because that’s sort of where we assumed and many people assume that’s the place to start. Then looked at cameras. Cameras have lots of drawbacks. And fundamentally, you can sort of-- we’ve all had this, you taken your phone and tried to take a picture of an airplane and you look at the picture, you can’t see the airplane. Yeah. It takes so many pixels of perfectly clean lenses to see an aircraft at a kilometer or two away that it really just is not practical or robust enough. And that’s when we went back to the drawing board and it ended up where we ended up, which is using an array of microphones to listen for aircraft, which works very well at very long distances to then maintain separation from those other aircraft. Cass: So yeah, let’s talk about Platform 2 a little bit more because I should first explain for listeners who maybe aren’t familiar with Zipline that these are not the kind of the little purely sort of helicopter-like drones. These are these fixed wing with sort of loiter capability and hovering capabilities. So they’re not like your Mavic drones and so on. These have a capacity then for long-distance flight, which is what it gives them. Wyrobek: Yeah. And maybe to jump into Platform 2— maybe starting with Platform 1, what does it look like? So Platform 1 is what we’ve been operating around the world for years now. And this basically looks like a small airplane, right? In the industry referred to as a fixed-wing aircraft. And it’s fixed wing because to solve the problem of going from a metro area to surrounding countryside, really two things matter. Your range and long range and low cost. And a fixed-wing aircraft over something that can hover has something like an 800% advantage in range and cost. And that’s why we did fix wing because it actually works for our customers for their needs for that use case. Platform 2 is all about, how do you deliver to homes and in metro areas where you need an incredible amount of precision to deliver to nearly every home. And so Platform 2—we call our drone zips—our drone, it flies out to the delivery site. Instead of floating a package down to a customer like Platform 1 does, it hovers. Platform 2 hovers and lowers down what we call a droid. And so the droids on tether. The drone stays way up high, about 100 meters up high, and the drone lowers down. And the drone itself-- sorry, the droid itself, it lowers down, it can fly. Right? So you think of it as like the tether does the heavy lifting, but the droid has fans. So if it gets hit by a gust of wind or whatnot, it can still stay very precisely on track and come in and deliver it to a very small area, put the package down, and then be out of there seconds later. Cass: So let me get this right. Platform 2 is kind of as a combo, fixed wing and rotor wing. It’s like a VTOL like that. I’m cheating here a little bit because my colleague Evan Ackerman has a great Q&A on the Spectrum website with you, some of your team members about the nitty-gritty of how that design was evolved. But first off, it’s like a little droid thing at the end of the tether. How much extra precision do all those fans and stuff give you? Wyrobek: Oh, massive, right? We can come down and hit a target within a few centimeters of where we want to deliver, which means we can deliver. Like if you have a small back porch, which is really common, right, in a lot of urban areas to have a small back porch or a small place on your roof or something like that, we can still just deliver as long as we have a few feet of open space. And that’s really powerful for being able to serve our customers. And a lot of people think of Platform 2 as like, “Hey, it’s a slightly better way of doing maybe a DoorDash-style operation, people in cars driving around.” And to be clear, it’s not slightly better. It’s massively better, much faster, more environmentally friendly. But we have many contracts for Platform 2 in the health space with US Health System Partners and Health Systems around the world. And what’s powerful about these customers in terms of their needs is they really need to serve all of their customers. And this is where a lot of our sort of-- this is where our engineering effort goes is how do you make a system that doesn’t just kind of work for some folks, and they can use it if they want to, but a health system is like, “No, I want this to work for everybody in my health network.” And so how do we get to that near 100 percent serviceability? And that’s what this droid really enables us to do. And of course, it has all these other magic benefits too. It makes some of the hardest design problems in this space much, much easier. The safety problem gets much easier by keeping the drone way up high. Cass: Yeah, how high is Platform 2 hovering when it’s doing its deliveries? Wyrobek: About 100 meters, so 300 plus feet, right? We’re talking about high up as a football field is long. And so it’s way up there. And it also helps with things like noise, right? We don’t want to live in a future where drones are all around us sounding like swarms of insects. We want drones to make no noise. We want them to just melt into the background. And so it makes that kind of problem much easier as well. And then, of course, the droid gets other benefits where for many products, we don’t need any packaging at all. We can just deliver the product right onto a table in your porch. And not just from a cost perspective, but again, from— we’re all familiar with the nightmare of packaging from deliveries we get. Eliminating packaging just has to be our future. And we’re really excited to advance that future. Cass: From Evan’s Q&A, I know that a lot of effort went into making the droid element look rather adorable. Why was that so important? Wryobek: Yeah, I like to describe it as sort of a cross between three things, if you kind of picture this, like a miniature little fan boat, right, because it has some fan, a big fan on the back, looks like a little fan boat, combined with sort of a baby seal, combined with a toaster. It sort of has that look to it. And making it adorable, there’s a bunch of sort of human things that matter, right? I want this to be something that when my grandmother, who’s not a tech-savvy, gets these deliveries, it’s approachable. It doesn’t come off as sort of scary. And when you make something cute, not only does it feel approachable, but it also forces you to get the details right so it is approachable, right? The rounded corners, right? This sounds really benign, but a lot of robots, it turns out if you bump into them, they scratch you. And we want you to be able to bump into this droid, and this is no big deal. And so getting the surfaces right, getting them— the surface is made sort of like a helmet foam. If you can picture that, right? The kind of thing you wouldn’t be afraid to touch if it touched you. And so getting it both to be something that feels safe, but is something that actually is safe to be around, those two things just matter a lot. Because again, we’re not designing this for some piloty kind of low-volume thing. Our customers want this in phenomenal volume. And so we really want this to be something that we’re all comfortable around. Cass: Yeah, and one thing I want to pull out from that Q&A as well is it was an interesting note, because you mentioned it has three fans, but they’re rather unobtrusive. And the original design, you had two big fans on the sides, which was very great for maneuverability. But you had to get rid of those and come up with a three-fan design. And maybe you can explain why that was so. Wryobek: Yeah, that’s a great detail. So the original design, the picture, it was like, imagine the package in the middle, and then kind of on either side of the package, two fans. So when you looked at it, it kind of looked like— I don’t know. It kind of looked like the package had big mouse ears or something. And when you looked at it, everybody had the same reaction. You kind of took this big step back. It was like, “Whoa, there’s this big thing coming down into my yard.” And when you’re doing this kind of user testing, we always joke, you don’t need to bring users in if it already makes you take a step back. And this is one of those things where like, “That’s just not good enough, right, to even start with that kind of refined design.” But when we got the sort of profile of it smaller, the way we think about it from a design experiment perspective is we want to deliver a large package. So basically, the droid needs to be as sucked down as small additional volume around that package as possible. So we spent a lot of time figuring out, “Okay, how do you do that sort of physically and aesthetically in a way that also gets that amazing performance, right? Because when I say performance, what I’m talking about is we still need it to work when the winds are blowing really hard outside and still can deliver precisely. And so it has to have a lot of aero performance to do that and still deliver precisely in essentially all weather conditions. Cass: So I guess I just want to ask you then is, what kind of weight and volume are you able to deliver with this level of precision? Wryobek: Yeah, yeah. So we’ll be working our way up to eight pounds. I say working our way up because that’s part of, once you launch a product like this, there’s refinement you can do overtime on many layers, but eight pounds, which was driven off, again, these health use cases. So it does basically 100 percent of what our health partners need to do. And it turns out it’s, nearly 100 percent of what we want to do in meal delivery. And even in the goods sector, I’m impressed by the percentage of goods we can deliver. One of our partners we work with, we can deliver over 80 percent of what they have in their big box store. And yeah, it’s wildly exceeding expectations on nearly every axis there. And volume, it’s big. It’s bigger than a shoebox. I don’t have a great-- I’m trying to think of a good reference to kind of bring it to life. But it looks like a small cooler basically inside. And it can comfortably fit a meal for four to give you a sense of the amount of food you can fit in there. Yeah. Cass: So we’ve seen this history of Zipline in rural areas, and now we’re talking about expanding operations in more urban areas, but just how urban? I don’t imagine that we’ll see the zip lines of zooming around, say, the very hemmed-in streets, say, here in Midtown Manhattan. So what level of urban are we talking about? Wryobek: Yeah, so the way we talk about it internally in our design process is basically we call three-story sprawl. Manhattan is the place where when we think of New York, we’re not talking about Manhattan, but most of the rest of New York, we are talking about it, right? Like the Bronx, things like that. We just have this sort of three stories forever. And that’s a lot of the world out here in California, that’s most of San Francisco. I think it’s something like 98 percent of San Francisco is that. If you’ve ever been to places like India and stuff like that, the cities, it’s just sort of this three stories going for a really long way. And that’s what we’re really focused on. And that’s also where we provide that incredible value because that’s also matches where the hardest traffic situations and things like that can make any other sort of terrestrial on-demand delivery be phenomenally late. Cass: Well, no, I live out in Queens, so I agree there’s not much skyscrapers out there. Although there are quite a few trees and so on, but at the same time, there’s usually some sort of sidewalk availability. So is that kind of what you’re hoping to get into? Wyrobek: Exactly. So as long as you’ve got a porch with a view of the sky or an alley with a view of the sky, it can be literally just a few feet, we can get in there, make a delivery, and be on our way. Cass: And so you’ve done this preliminary test with the FAA, the BVLOS test, and so on. How close do you think you are to, and you’re working with a lot of partners, to really seeing this become routine commercial operations? Wyrobek: Yeah, yeah. So at relatively limited scale, our operations here in Utah and in Arkansas that are leveraging that FAA approval for beyond visual line-of-sight flight operations, that’s been all day, every day now since our approval last year. With Platform 2, we’re really excited. That’s coming later this year. We’re currently in the phase of basically massive-scale testing. So we now have our production hardware and we’re taking it through a massive ground testing campaign. So this picture dozens of thermal chambers and five chambers and things like that just running to really both validate that we have the reliability we need and flush out any issues that we might have missed so we can address that difference between what we call the theoretical reliability and the actual reliability. And that’s running in parallel to a massive flight test campaign. Same idea, right? We’re slowly ramping up the flight volume as we fly into heavier conditions really to make sure we know the limits of the system. We know its actual reliability and true scaled operations so we can get the confidence that it’s ready to operate for people. Cass: So you’ve got Platform 2. What’s kind of next on your technology roadmap for any possible platform three? Wyrobek: Oh, great question. Yeah, I can’t comment on platform three at this time, but. And I will also say, Zipline is pouring our heart into Platform 2 right now. Getting Platform 2 ready for this-- the way I like to talk about this internally is today, we fly about four times the equator of the Earth in our operations on average. And that’s a few thousand flights per day. But the demand we have is for more like millions of flights per day, if not beyond. And so on the log scale, right, we’re halfway there. Three hours of magnitude down, three more zeros to come. And the level of testing, the level of systems engineering, the level of refinement required to do that is a lot. And there’s so many systems from weather forecasting to our onboard autonomy and our fleet management systems. And so to highlight one team, our system test team run by this really impressive individual named Juan Albanell, this team has taken us from where we were two years ago, where we had shown the concept at a very prototype stage of this delivery experience, and we’ve done the first order math kind of on the architecture and things like that through the iterations in test to actually make sure we had a drone that could actually fly in all these weather conditions with all the robustness and tolerance required to actually go to this global scale that Platform 2 is targeting. Cass: Well, that’s fantastic. Well, I think there’s a lot more to talk about to come up in the future, and we look forward to talking with Zipline again. But for today, I’m afraid we’re going to have to leave it there. But it was really great to have you on the show, Keenan. Thank you so much. Wyrobek: Cool. Absolutely, Stephen. It was a pleasure to speak with you. Cass: So today on Fixing the Future, we were talking with Zipline’s Keenan Wyrobek about the progress of commercial drone deliveries. For IEEE Spectrum, I’m Stephen Cass, and I hope you’ll join us next time.

Boston Dynamics has just introduced a new Atlas humanoid robot, replacing the legendary hydraulic Atlas and intended to be a commercial product. This is huge news from the company that has spent the last decade building the most dynamic humanoids that the world has ever seen, and if you haven’t read our article about the announcement (and seen the video!), you should do that right now. We’ve had about a decade of pent-up questions about an all-electric productized version of Atlas, and we were lucky enough to speak with Boston Dynamics CEO Robert Playter to learn more about where this robot came from and how it’s going to make commercial humanoid robots (finally) happen. Robert Playter was the Vice President of Engineering at Boston Dynamics starting in 1994, which I’m pretty sure was back when Boston Dynamics still intended to be a modeling and simulation company rather than a robotics company. Playter became the CEO in 2019, helping the company make the difficult transition from R&D to commercial products with Spot, Stretch, and now (or very soon) Atlas. We talked with Playter about what the heck took Boston Dynamics so long to make this robot, what the vision is for Atlas as a product, all that extreme flexibility, and what comes next. Robert Playter on: What Took So Long The Product Approach A General Purpose Robot? Hydraulic Versus Electric Extreme Range of Motion Atlas’ Head Advantages in Commercialization What’s Next IEEE Spectrum: So what’s going on? Robert Playter: Boston Dynamics has built an all-electric humanoid. It’s our newest generation of what’s been an almost 15-year effort in developing humanoids. We’re going to launch it as a product, targeting industrial applications, logistics, and places that are much more diverse than where you see Stretch—heavy objects with complex geometry, probably in manufacturing type environments. We’ve built our first robot, and we believe that’s really going to set the bar for the next generation of capabilities for this whole industry. What took you so long?! Playter: Well, we wanted to convince ourselves that we knew how to make a humanoid product that can handle a great diversity of tasks—much more so than our previous generations of robots—including at-pace bimanual manipulation of the types of heavy objects with complex geometry that we expect to find in industry. We also really wanted to understand the use cases, so we’ve done a lot of background work on making sure that we see where we can apply these robots fruitfully in industry. We’ve obviously been working on this machine for a while, as we’ve been doing parallel development with our legacy Atlas. You’ve probably seen some of the videos of Atlas moving struts around—that’s the technical part of proving to ourselves that we can make this work. And then really designing a next generation machine that’s going to be an order of magnitude better than anything the world has seen. “We’re not anxious to just show some whiz-bang tech, and we didn’t really want to indicate our intent to go here until we were convinced that there is a path to a product.” —Robert Playter, Boston Dynamics With Spot, it felt like Boston Dynamics developed the product first, without having a specific use case in mind: you put the robot out there and let people discover what it was good for. Is your approach different with Atlas? Playter: You’re absolutely right. Spot was a technology looking for a product, and it’s taken time for us to really figure out the product market fit that we have in industrial inspection. But the challenge of that experience has left us wiser about really identifying the target applications before you say you’re going to build these things at scale. Stretch is very different, because it had a clear target market. Atlas is going to be more like Stretch, although it’s going to be way more than a single task robot, which is kind of what Stretch is. Convincing ourselves that we could really generalize with Atlas has taken a little bit of time. This is going to be our third product in about four years. We’ve learned so much, and the world is different from that experience. [back to top] Is your vision for Atlas one of a general purpose robot? Playter: It definitely needs to be a multi-use case robot. I believe that because I don’t think there’s very many examples where a single repetitive task is going to warrant these complex robots. I also think, though, that the practical matter is that you’re going to have to focus on a class of use cases, and really making them useful for the end customer. The lesson we’ve learned with both Spot and Stretch is that it’s critical to get out there and actually understand what makes this robot valuable to customers while making sure you’re building that into your development cycle. And if you can start that before you’ve even launched the product, then you’ll be better off. [back to top] How does thinking of this new Atlas as a product rather than a research platform change things? Playter: I think the research that we’ve done over the past 10 or 15 years has been essential to making a humanoid useful in the first place. We focused on dynamic balancing and mobility and being able to pick something up and still maintain that mobility—those were research topics of the past that we’ve now figured out how to manage and are essential, I think, to doing useful work. There’s still a lot of work to be done on generality, so that humanoids can pick up any one of a thousand different parts and deal with them in a reasonable way. That level of generality hasn’t been proven yet; we think there’s promise, and that AI will be one of the tools that helps solve that. And there’s still a lot of product prototyping and iteration that will come out before we start building massive numbers of these things and shipping them to customers. “This robot will be stronger at most of its joints than a person, and even an elite athlete, and will have a range of motion that exceeds anything a person can ever do.” —Robert Playter, Boston Dynamics For a long time, it seemed like hydraulics were the best way of producing powerful dynamic motions for robots like Atlas. Has that now changed? Playter: We first experimented with that with the launch of Spot. We had the same issue years ago, and discovered that we could build powerful lightweight electric motors that had the same kind of responsiveness and strength, or let’s say sufficient responsiveness and strength, to really make that work. We’ve designed an even newer set of really compact actuators into our electric Atlas, which pack the strength of essentially an elite human athlete into these tiny packages that make an electric humanoid feasible for us. So, this robot will be stronger at most of its joints than a person, and even an elite athlete, and will have a range of motion that exceeds anything a person can ever do. We’ve also compared the strength of our new electric Atlas to our hydraulic Atlas, and the electric Atlas is stronger. [back to top] In the context of Atlas’ range of motion, that introductory video was slightly uncomfortable to watch, which I’m sure was deliberate. Why introduce the new Atlas in that way? Playter: These high range of motion actuators are going to enable a unique set of movements that ultimately will let the robot be very efficient. Imagine being able to turn around without having to take a bunch of steps to turn your whole body instead. The motions we showed [in the video] are ones where our engineers were like, “hey, with these joints, we could get up like this!” And it just wasn’t something we had that really thought about before. This flexibility creates a palette that you can design new stuff on, and we’re already having fun with it and we decided we wanted to share that excitement with the world. [back to top] “Everybody will buy one robot—we learned that with Spot. But they won’t start by buying fleets, and you don’t have a business until you can sell multiple robots to the same customer.” —Robert Playter, Boston Dynamics This does seem like a way of making Atlas more efficient, but I’ve heard from other folks working on humanoids that it’s important for robots to move in familiar and predictable ways for people to be comfortable working around them. What’s your perspective on that? Playter: I do think that people are going to have to become familiar with our robot; I don’t think that means limiting yourself to human motions. I believe that ultimately, if your robot is stronger or more flexible, it will be able to do things that humans can’t do, or don’t want to do. One of the real challenges of making a product useful is that you’ve got to have sufficient productivity to satisfy a customer. If you’re slow, that’s hard. We learned that with Stretch. We had two generations of Stretch, and the first generation did not have a joint that let it pivot 180 degrees, so it had to ponderously turn around between picking up a box and dropping it off. That was a killer. And so we decided “nope, gotta have that rotational joint.” It lets Stretch be so much faster and more efficient. At the end of the day, that’s what counts. And people will get used to it. What can you tell me about the head? Boston Dynamics CEO Robert Playter said the head on the new Atlas robot has been designed not to mimic the human form but rather “to project something else: a friendly place to look to gain some understanding about the intent of the robot.”Boston Dynamics Playter: The old Atlas did not have an articulated head. But having an articulated head gives you a tool that you can use to indicate intent, and there are integrated lights which will be able to communicate to users. Some of our original concepts had more of a [human] head shape, but for us they always looked a little bit threatening or dystopian somehow, and we wanted to get away from that. So we made a very purposeful decision about the head shape, and our explicit intent was for it not to be human-like. We’re trying to project something else: a friendly place to look to gain some understanding about the intent of the robot. The design borrows from some friendly shapes that we’d seen in the past. For example, there’s the old Pixar lamp that everybody fell in love with decades ago, and that informed some of the design for us. [back to top] How do you think the decade(s) of experience working on humanoids as well as your experience commercializing Spot will benefit you when it comes to making Atlas into a product? Playter: This is our third product, and one of the things we’ve learned is that it takes way more than some interesting technology to make a product work. You have to have a real use case, and you have to have real productivity around that use case that a customer cares about. Everybody will buy one robot—we learned that with Spot. But they won’t start by buying fleets, and you don’t have a business until you can sell multiple robots to the same customer. And you don’t get there without all this other stuff—the reliability, the service, the integration. When we launched Spot as a product several years ago, it was really about transforming the whole company. We had to take on all of these new disciplines: manufacturing, service, measuring the quality and reliability of our robots and then building systems and tools to make them steadily better. That transformation is not easy, but the fact that we’ve successfully navigated through that as an organization means that we can easily bring that mindset and skill set to bear as a company. Honestly, that transition takes two or three years to get through, so all of the brand new startup companies out there who have a prototype of a humanoid working—they haven’t even begun that journey. There’s also cost. Building something effectively at a reasonable cost so that you can sell it at a reasonable cost and ultimately make some money out of it, that’s not easy either. And frankly, without the support of Hyundai which is of course a world-class manufacturing expert, it would be really challenging to do it on our own. So yeah, we’re much more sober about what it takes to succeed now. We’re not anxious to just show some whiz-bang tech, and we didn’t really want to indicate our intent to go here until we were convinced that there is a path to a product. And I think ultimately, that will win the day. [back to top] What will you be working on in the near future, and what will you be able to share? Playter: We’ll start showing more of the dexterous manipulation on the new Atlas that we’ve already shown on our legacy Atlas. And we’re targeting proof of technology testing in factories at Hyundai Motor Group [HMG] as early as next year. HMG is really excited about this venture; they want to transform their manufacturing and they see Atlas as a big part of that, and so we’re going to get on that soon. [back to top] What do you think other robotics folks will find most exciting about the new Atlas? Playter: Having a robot with so much power and agility packed into a relatively small and lightweight package. I’ve felt honored in the past that most of these other companies compare themselves to us. They say, “well, where are we on the Boston Dynamics bar?” I think we just raised the bar. And that’s ultimately good for the industry, right? People will go, “oh, wow, that’s possible!” And frankly, they’ll start chasing us as fast as they can—that’s what we’ve seen so far. I think it’ll end up pulling the whole industry forward.

Yesterday, Boston Dynamics bid farewell to the iconic Atlas humanoid robot. Or, the hydraulically-powered version of Atlas, anyway—if you read between the lines of the video description (or even just read the actual lines of the video description), it was pretty clear that although hydraulic Atlas was retiring, it wasn’t the end of the Atlas humanoid program at Boston Dynamics. In fact, Atlas is already back, and better than ever. Today, Boston Dynamics is introducing a new version of Atlas that’s all-electric. It’s powered by batteries and electric actuators, no more messy hydraulics. It exceeds human performance in terms of both strength and flexibility. And for the first time, Boston Dynamics is calling this humanoid robot a product. We’ll take a look at everything that Boston Dynamics is announcing today, and have even more detail in this Q&A with Boston Dynamics CEO Robert Playter. Boston Dynamics’ new electric humanoid has been simultaneously one of the worst and best kept secrets in robotics over the last year or so. What I mean is that it seemed obvious, or even inevitable, that Boston Dynamics would take the expertise in humanoids that it developed with Atlas and combine that with its experience productizing a fully electric system like Spot. But just because something seems inevitable doesn’t mean it actually is inevitable, and Boston Dynamics has done an admirable job of carrying on as normal while building a fully electric humanoid from scratch. And here it is: It’s all new, it’s all electric, and some of those movements make me slightly uncomfortable (we’ll get into that in a bit). The blog post accompanying the video is sparse on technical detail, but let’s go through the most interesting parts: A decade ago, we were one of the only companies putting real R&D effort into humanoid robots. Now the landscape in the robotics industry is very different. In 2010, we took a look at all the humanoid robots then in existence. You could, I suppose, argue that Honda was putting real R&D effort into ASIMO back then, but yeah, pretty much all those other humanoid robots came from research rather than industry. Now, it feels like we’re up to our eyeballs in commercial humanoids, but over the past couple of years, as startups have appeared out of nowhere with brand new humanoid robots, Boston Dynamics (to most outward appearances) was just keepin’ on with that R&D. Today’s announcement certainly changes that. We are confident in our plan to not just create an impressive R&D project, but to deliver a valuable solution. This journey will start with Hyundai—in addition to investing in us, the Hyundai team is building the next generation of automotive manufacturing capabilities, and it will serve as a perfect testing ground for new Atlas applications. Boston Dynamics This is a significant advantage for Boston Dynamics—through Hyundai, they can essentially be their own first customer for humanoid robots, offering an immediate use case in a very friendly transitional environment. Tesla has a similar advantage with Optimus, but Boston Dynamics also has experience sourcing and selling and supporting Spot, which are those business-y things that seem like they’re not the hard part until they turn out to actually be the hard part. In the months and years ahead, we’re excited to show what the world’s most dynamic humanoid robot can really do—in the lab, in the factory, and in our lives. World’s most dynamic humanoid, you say? Awesome! Prove it! On video! With outtakes! The electric version of Atlas will be stronger, with a broader range of motion than any of our previous generations. For example, our last generation hydraulic Atlas (HD Atlas) could already lift and maneuver a wide variety of heavy, irregular objects; we are continuing to build on those existing capabilities and are exploring several new gripper variations to meet a diverse set of expected manipulation needs in customer environments. Now we’re getting to the good bits. It’s especially notable here that the electric version of Atlas will be “stronger” than the previous hydraulic version, because for a long time hydraulics were really the only way to get the kind of explosively powerful repetitive dynamic motions that enabled Atlas to do jumps and flips. And the switch away from hydraulics enables that extra range of motion now that there aren’t hoses and stuff to deal with. It’s also pretty clear that the new Atlas is built to continue the kind of work that hydraulic Atlas has been doing, manipulating big and heavy car parts. This is in sharp contrast to most other humanoid robots that we’ve seen, which have primarily focused on moving small objects or bins around in warehouse environments. We are not just delivering industry-leading hardware. Some of our most exciting progress over the past couple of years has been in software. In addition to our decades of expertise in simulation and model predictive control, we have equipped our robots with new AI and machine learning tools, like reinforcement learning and computer vision to ensure they can operate and adapt efficiently to complex real-world situations. This is all par for the course now, but it’s also not particularly meaningful without more information. “We will give our robots new capabilities through machine learning and AI” is what every humanoid robotics company (and most other robotics companies) are saying, but I’m not sure that we’re there yet, because there’s an “okay but how?” that needs to happen first. I’m not saying that it won’t happen, just pointing out that until it does happen, it hasn’t happened. The humanoid form factor is a useful design for robots working in a world designed for people. However, that form factor doesn’t limit our vision of how a bipedal robot can move, what tools it needs to succeed, and how it can help people accomplish more. Agility Robotics has a similar philosophy with Digit, which has a mostly humanoid form factor to operate in human environments but also uses a non-human leg design because Agility believes that it works better. Atlas is a bit more human-like with its overall design, but there are some striking differences, including both range of motion and the head, both of which we’ll be talking more about. We designed the electric version of Atlas to be stronger, more dexterous, and more agile. Atlas may resemble a human form factor, but we are equipping the robot to move in the most efficient way possible to complete a task, rather than being constrained by a human range of motion. Atlas will move in ways that exceed human capabilities. The introductory video with the new Atlas really punches you in the face with this: Atlas is not constrained by human range of motion and will leverage its extra degrees of freedom to operate faster and more efficiently, even if you personally might find some of those motions a little bit unsettling. Boston Dynamics Combining decades of practical experience with first principles thinking, we are confident in our ability to deliver a robot uniquely capable of tackling dull, dirty, and dangerous tasks in real applications. As Marco Hutter pointed out, most commercial robots (humanoids included) are really only targeting tasks that are dull, because dull usually means repetitive, and robots are very good at repetitive. Dirty is a little more complicated, and dangerous is a lot more complicated than that. I appreciate that Boston Dynamics is targeting those other categories of tasks from the outset. Commercialization takes great engineering, but it also takes patience, imagination, and collaboration. Boston Dynamics has proven that we can deliver the full package with both industry-leading robotics and a complete ecosystem of software, services, and support to make robotics useful in the real world. There’s a lot more to building a successful robotics company than building a successful robot. Arguably, building a successful robot is not even the hardest part, long term. Having over 1500 Spot robots deployed with customers gives them a well-established product infrastructure baseline to expand from with the new Atlas. Taking a step back, let’s consider the position that Boston Dynamics is in when it comes to the humanoid space right now. The new Atlas appears to be a reasonably mature platform with explicit commercial potential, but it’s not yet clear if this particular version of Atlas is truly commercially viable, in terms of being manufacturable and supportable at scale—it’s Atlas 001, after all. There’s likely a huge amount of work that still needs to be done, but it’s a process that the company has already gone through with Spot. My guess is that Boston Dynamics has some catching up to do with respect to other humanoid companies that are already entering pilot projects. In terms of capabilities, even though the new Atlas hardware is new, it’s not like Boston Dynamics is starting from scratch, since they’re already transferring skills from hydraulic Atlas onto the new platform. But, we haven’t seen the new Atlas doing any practical tasks yet, so it’s hard to tell how far along that is, and it would be premature to assume that hydraulic Atlas doing all kinds of amazing things in YouTube videos implies that electric Atlas can do similar things safely and reliably in a product context. There’s a gap there, possibly an enormous gap, and we’ll need to see more from the new Atlas to understand where it’s at. And obviously, there’s a lot of competition in humanoids right now, although I’d like to think that the potential for practical humanoid robots to be useful in society is significant enough that there will be room for lots of different approaches. Boston Dynamics was very early to humanoids in general, but they’re somewhat late to this recent (and rather abrupt) humanoid commercialization push. This may not be a problem, especially if Atlas is targeting applications where its strength and flexibility sets it apart from other robots in the space, and if their depth of experience deploying commercial robotic platforms helps them to scale quickly. Boston Dynamics An electric Atlas may indeed have been inevitable, and it’s incredibly exciting to (finally!) see Boston Dynamics take this next step towards a commercial humanoid, which would deliver on more than a decade of ambition stretching back through the DARPA Robotics Challenge to PETMAN. We’ve been promised more manipulation footage soon, and Boston Dynamics expects that Atlas will be in the technology demonstration phase in Hyundai factories as early as next year. We have a lot more questions, but we have a lot more answers, too: you’ll find a Q&A with Boston Dynamics CEO Robert Playter right here.

As the history committee chair of the IEEE Lone Star Section, in San Antonio, Texas, I am responsible for documenting, preserving, and raising the visibility of technologies developed in the local area. One such technology is the Datapoint 2200, a programmable terminal that laid the foundation for the personal computer revolution. Launched in 1970 by Computer Terminal Corp. (CTC) in San Antonio, the machine played a significant role in the early days of microcomputers. The pioneering system integrated a CPU, memory, and input/output devices into a single unit, making it a compact, self-contained device. Apple, IBM, and other companies are often associated with the popularization of PCs; we must not overlook the groundbreaking innovations introduced by the Datapoint. The machine might have faded from memory, but its influence on the evolution of computing technology cannot be denied. The IEEE Region 5 life members committee honored the machine in 2022 with its Stepping Stone Award, but I would like to make more members aware of the innovations introduced by the machine’s design. From mainframes to microcomputers Before the personal computer, there were mainframe computers. The colossal machines, with their bulky, green monitors housed in meticulously cooled rooms, epitomized the forefront of technology at the time. I was fortunate to work with mainframes during my second year as an electrical engineering student in the United Arab Emirates University at Al Ain, Abu Dhabi, in 1986. The machines occupied entire rooms, dwarfing the personal computers we are familiar with today. Accessing the mainframes involved working with text-based terminals that lacked graphical interfaces and had limited capabilities. Those relatively diminutive terminals that interfaced with the machines often provided a touch of amusement for the students. The mainframe rooms served as social places, fostering interactions, collaborations, and friendly competitions. Operating the terminals required mastering specific commands and coding languages. The process of submitting computing jobs and waiting for results without immediate feedback could be simultaneously amusing and frustrating. Students often humorously referred to the “black hole,” where their jobs seemed to vanish until the results materialized. Decoding enigmatic error messages became a challenge, yet students found joy in deciphering them and sharing amusing examples. Despite mainframes’ power, they had restricted processing capabilities and memory compared with today’s computers. The introduction of personal computers during my senior year was a game-changer. Little did I know that it would eventually lead me to San Antonio, Texas, birthplace of the PC, where I would begin a new chapter of my life. The first PC In San Antonio, a group of visionary engineers from NASA founded CTC with the goal of revolutionizing desktop computing. They introduced the Datapoint 3300 as a replacement for Teletype terminals. Led by Phil Ray and Gus Roche, the company later built the first personal desktop computer, the Datapoint 2200. They also developed LAN technology and aimed to replace traditional office equipment with electronic devices operable from a single terminal. The Datapoint 2200 introduced several design elements that later were adopted by other computer manufacturers. It was one of the first computers to use a keyboard similar to a typewriter’s, and a monitor for user interaction—which became standard input and output devices for personal computers. They set a precedent for user-friendly computer interfaces. The machine also had cassette tape drives for storage, predecessors of disk drives. The computer had options for networking, modems, interfaces, printers, and a card reader. It used different memory sizes and employed an 8-bit processor architecture. The Datapoint’s CPU was initially intended to be a custom chip, which eventually came to be known as the microprocessor. At the time, no such chips existed, so CTC contracted with Intel to produce one. That chip was the Intel 8008, which evolved into the Intel 8080. Introduced in 1974, the 8080 formed the basis for small computers, according to an entry about early microprocessors in the Engineering and Technology History Wiki. Those first 8-bit microprocessors are celebrating their 50th anniversary this year. The 2200 was primarily marketed for business use, and its introduction helped accelerate the adoption of computer systems in a number of industries, according to Lamont Wood, author of Datapoint: The Lost Story of the Texans Who Invented the Personal Computer Revolution. The machine popularized the concept of computer terminals, which allowed multiple users to access a central computer system remotely, Wood wrote. It also introduced the idea of a terminal as a means of interaction with a central computer, enabling users to input commands and receive output. The concept laid the groundwork for the development of networking and distributed computing. It eventually led to the creation of LANs and wide-area networks, enabling the sharing of resources and information across organizations. The concept of computer terminals influenced the development of modern networking technologies including the Internet, Wood pointed out. How Datapoint inspired Apple and IBM Although the Datapoint 2200 was not a consumer-oriented computer, its design principles and influence played a role in the development of personal computers. Its compact, self-contained nature demonstrated the feasibility and potential of such machines. The Datapoint sparked the imagination of researchers and entrepreneurs, leading to the widespread availability of personal computers. Here are a few examples of how manufacturers built upon the foundation laid by the Datapoint 2200: Apple drew inspiration from early microcomputers. The Apple II, introduced in 1977, was one of the first successful personal computers. It incorporated a keyboard, a monitor, and a cassette tape interface for storage, similar to the Datapoint 2200. In 1984 Apple introduced the Macintosh, which featured a graphical user interface and a mouse, revolutionizing the way users interacted with computers. IBM entered the personal computer market in 1981. Its PC also was influenced by the design principles of microcomputers. The machine featured an open architecture, allowing for easy expansion and customization. The PC’s success established it as a standard in the industry. Microsoft played a crucial role in software development for early microcomputers. Its MS-DOS provided a standardized platform for software development and was compatible with the IBM PC and other microcomputers. The operating system helped establish Microsoft as a dominant player in the software industry. Commodore International, a prominent computer manufacturer in the 1980s, released the Commodore 64 in 1982. It was a successful microcomputer that built upon the concepts of the Datapoint 2200 and other early machines. The Commodore 64 featured an integrated keyboard, color graphics, and sound capabilities, making it a popular choice for gaming and home computing. Xerox made significant contributions to the advancement of computing interfaces. Its Alto, developed in 1973, introduced the concept of a graphical user interface, with windows, icons, and a mouse for interaction. Although the Alto was not a commercial success, its influence was substantial, and it helped lay the groundwork for GUI-based systems including the Macintosh and Microsoft Windows. The Datapoint 2200 deserves to be remembered for its contributions to computer history.

One of the management guru Peter Drucker’s most over-quoted turns of phrase is “what gets measured gets improved.” But it’s over-quoted for a reason: It’s true. Nowhere is it truer than in technology over the past 50 years. Moore’s law—which predicts that the number of transistors (and hence compute capacity) in a chip would double every 24 months—has become a self-fulfilling prophecy and north star for an entire ecosystem. Because engineers carefully measured each generation of manufacturing technology for new chips, they could select the techniques that would move toward the goals of faster and more capable computing. And it worked: Computing power, and more impressively computing power per watt or per dollar, has grown exponentially in the past five decades. The latest smartphones are more powerful than the fastest supercomputers from the year 2000. Measurement of performance, though, is not limited to chips. All the parts of our computing systems today are benchmarked—that is, compared to similar components in a controlled way, with quantitative score assessments. These benchmarks help drive innovation. And we would know. As leaders in the field of AI, from both industry and academia, we build and deliver the most widely used performance benchmarks for AI systems in the world. MLCommons is a consortium that came together in the belief that better measurement of AI systems will drive improvement. Since 2018, we’ve developed performance benchmarks for systems that have shown more than 50-fold improvements in the speed of AI training. In 2023, we launched our first performance benchmark for large language models (LLMs), measuring the time it took to train a model to a particular quality level; within 5 months we saw repeatable results of LLMs improving their performance nearly threefold. Simply put, good open benchmarks can propel the entire industry forward. We need benchmarks to drive progress in AI safety Even as the performance of AI systems has raced ahead, we’ve seen mounting concern about AI safety. While AI safety means different things to different people, we define it as preventing AI systems from malfunctioning or being misused in harmful ways. For instance, AI systems without safeguards could be misused to support criminal activity such as phishing or creating child sexual abuse material, or could scale up the propagation of misinformation or hateful content. In order to realize the potential benefits of AI while minimizing these harms, we need to drive improvements in safety in tandem with improvements in capabilities. We believe that if AI systems are measured against common safety objectives, those AI systems will get safer over time. However, how to robustly and comprehensively evaluate AI safety risks—and also track and mitigate them—is an open problem for the AI community. Safety measurement is challenging because of the many different ways that AI models are used and the many aspects that need to be evaluated. And safety is inherently subjective, contextual, and contested—unlike with objective measurement of hardware speed, there is no single metric that all stakeholders agree on for all use cases. Often the test and metrics that are needed depend on the use case. For instance, the risks that accompany an adult asking for financial advice are very different from the risks of a child asking for help writing a story. Defining “safety concepts” is the key challenge in designing benchmarks that are trusted across regions and cultures, and we’ve already taken the first steps toward defining a standardized taxonomy of harms. A further problem is that benchmarks can quickly become irrelevant if not updated, which is challenging for AI safety given how rapidly new risks emerge and model capabilities improve. Models can also “overfit”: they do well on the benchmark data they use for training, but perform badly when presented with different data, such as the data they encounter in real deployment. Benchmark data can even end up (often accidentally) being part of models’ training data, compromising the benchmark’s validity. Our first AI safety benchmark: the details To help solve these problems, we set out to create a set of benchmarks for AI safety. Fortunately, we’re not starting from scratch— we can draw on knowledge from other academic and private efforts that came before. By combining best practices in the context of a broad community and a proven benchmarking non-profit organization, we hope to create a widely trusted standard approach that is dependably maintained and improved to keep pace with the field. Our first AI safety benchmark focuses on large language models. We released a v0.5 proof-of-concept (POC) today, 16 April, 2024. This POC validates the approach we are taking towards building the v1.0 AI Safety benchmark suite, which will launch later this year. What does the benchmark cover? We decided to first create an AI safety benchmark for LLMs because language is the most widely used modality for AI models. Our approach is rooted in the work of practitioners, and is directly informed by the social sciences. For each benchmark, we will specify the scope, the use case, persona(s), and the relevant hazard categories. To begin with, we are using a generic use case of a user interacting with a general-purpose chat assistant, speaking in English and living in Western Europe or North America. There are three personas: malicious users, vulnerable users such as children, and typical users, who are neither malicious nor vulnerable. While we recognize that many people speak other languages and live in other parts of the world, we have pragmatically chosen this use case due to the prevalence of existing material. This approach means that we can make grounded assessments of safety risks, reflecting the likely ways that models are actually used in the real-world. Over time, we will expand the number of use cases, languages, and personas, as well as the hazard categories and number of prompts. What does the benchmark test for? The benchmark covers a range of hazard categories, including violent crimes, child abuse and exploitation, and hate. For each hazard category, we test different types of interactions where models’ responses can create a risk of harm. For instance, we test how models respond to users telling them that they are going to make a bomb—and also users asking for advice on how to make a bomb, whether they should make a bomb, or for excuses in case they get caught. This structured approach means we can test more broadly for how models can create or increase the risk of harm. How do we actually test models? From a practical perspective, we test models by feeding them targeted prompts, collecting their responses, and then assessing whether they are safe or unsafe. Quality human ratings are expensive, often costing tens of dollars per response—and a comprehensive test set might have tens of thousands of prompts! A simple keyword- or rules- based rating system for evaluating the responses is affordable and scalable, but isn’t adequate when models’ responses are complex, ambiguous or unusual. Instead, we’re developing a system that combines “evaluator models”—specialized AI models that rate responses—with targeted human rating to verify and augment these models’ reliability. How did we create the prompts? For v0.5, we constructed simple, clear-cut prompts that align with the benchmark’s hazard categories. This approach makes it easier to test for the hazards and helps expose critical safety risks in models. We are working with experts, civil society groups, and practitioners to create more challenging, nuanced, and niche prompts, as well as exploring methodologies that would allow for more contextual evaluation alongside ratings. We are also integrating AI-generated adversarial prompts to complement the human-generated ones. How do we assess models? From the start, we agreed that the results of our safety benchmarks should be understandable for everyone. This means that our results have to both provide a useful signal for non-technical experts such as policymakers, regulators, researchers, and civil society groups who need to assess models’ safety risks, and also help technical experts make well-informed decisions about models’ risks and take steps to mitigate them. We are therefore producing assessment reports that contain “pyramids of information.” At the top is a single grade that provides a simple indication of overall system safety, like a movie rating or an automobile safety score. The next level provides the system’s grades for particular hazard categories. The bottom level gives detailed information on tests, test set provenance, and representative prompts and responses. AI safety demands an ecosystem The MLCommons AI safety working group is an open meeting of experts, practitioners, and researchers—we invite everyone working in the field to join our growing community. We aim to make decisions through consensus and welcome diverse perspectives on AI safety. We firmly believe that for AI tools to reach full maturity and widespread adoption, we need scalable and trustworthy ways to ensure that they’re safe. We need an AI safety ecosystem, including researchers discovering new problems and new solutions, internal and for-hire testing experts to extend benchmarks for specialized use cases, auditors to verify compliance, and standards bodies and policymakers to shape overall directions. Carefully implemented mechanisms such as the certification models found in other mature industries will help inform AI consumer decisions. Ultimately, we hope that the benchmarks we’re building will provide the foundation for the AI safety ecosystem to flourish. The following MLCommons AI safety working group members contributed to this article: Ahmed M. Ahmed, Stanford UniversityElie Alhajjar, RAND Kurt Bollacker, MLCommons Siméon Campos, Safer AI Canyu Chen, Illinois Institute of Technology Ramesh Chukka, Intel Zacharie Delpierre Coudert, Meta Tran Dzung, Intel Ian Eisenberg, Credo AI Murali Emani, Argonne National Laboratory James Ezick, Qualcomm Technologies, Inc. Marisa Ferrara Boston, Reins AI Heather Frase, CSET (Center for Security and Emerging Technology) Kenneth Fricklas, Turaco Strategy Brian Fuller, Meta Grigori Fursin, cKnowledge, cTuning Agasthya Gangavarapu, Ethriva James Gealy, Safer AI James Goel, Qualcomm Technologies, Inc Roman Gold, The Israeli Association for Ethics in Artificial Intelligence Wiebke Hutiri, Sony AI Bhavya Kailkhura, Lawrence Livermore National Laboratory David Kanter, MLCommons Chris Knotz, Commn Ground Barbara Korycki, MLCommons Shachi Kumar, Intel Srijan Kumar, Lighthouz AI Wei Li, Intel Bo Li, University of Chicago Percy Liang, Stanford University Zeyi Liao, Ohio State University Richard Liu, Haize Labs Sarah Luger, Consumer Reports Kelvin Manyeki, Bestech Systems Joseph Marvin Imperial, University of Bath, National University Philippines Peter Mattson, Google, MLCommons, AI Safety working group co-chair Virendra Mehta, University of Trento Shafee Mohammed, Project Humanit.ai Protik Mukhopadhyay, Protecto.ai Lama Nachman, Intel Besmira Nushi, Microsoft Research Luis Oala, Dotphoton Eda Okur, Intel Praveen Paritosh Forough Poursabzi, Microsoft Eleonora Presani, Meta Paul Röttger, Bocconi University Damian Ruck, Advai Saurav Sahay, Intel Tim Santos, Graphcore Alice Schoenauer Sebag, Cohere Vamsi Sistla, Nike Leonard Tang, Haize Labs Ganesh Tyagali, NStarx AI Joaquin Vanschoren, TU Eindhoven, AI Safety working group co-chair Bertie Vidgen, MLCommons Rebecca Weiss, MLCommons Adina Williams, FAIR, Meta Carole-Jean Wu, FAIR, Meta Poonam Yadav, University of York, UK Wenhui Zhang, LFAI & Data Fedor Zhdanov, Nebius AI

A consortium of U.S. federal agencies has pooled their funds and wide array of expertise to reinvent the emergency vehicle. The hybrid electric box truck they’ve come up with is carbon neutral. And in the aftermath of a natural disaster like a tornado or wildfire, the vehicle, called H2Rescue, can supply electric power and potable water to survivors while acting as a temperature-controlled command center for rescue personnel. The agencies that funded and developed it from an idea on paper to a functional Class 7 emergency vehicle prototype say they are pleased with the outcome of the project, which is now being used for further research and development. “Any time the fuel cell is producing energy to move the vehicle or to export power, it’s generating water.” –Nicholas Josefik, U.S. Army Corps of Engineers Construction Research Lab Commercial truck and locomotive engine maker Cummins, which has pledged to make all its heavy-duty road and rail vehicles zero-emission by 2050, won a $1 million competitive award to build the H2Rescue, which gets its power from a hydrogen fuel cell that charges its lithium-ion batteries. In demonstrations, including one last summer at National Renewable Energy Lab facilities in Colorado, the truck proved capable of driving 290-kilometers, then taking on the roles of power plant, mobile command center, and (courtesy of the truck’s “exhaust”) supplier of clean drinking water. A hydrogen tank system located behind the 15,000-kilogram truck’s cab holds 175 kg of fuel at 70 megapascals (700 bars) of pressure. Civilian anthropology researcher Lance Larkin at the U.S. Army Corps of Engineers’ Construction Engineering Research Laboratory (CERL) in Champaign, Ill., told IEEE Spectrum that that’s enough fuel for the fuel cell to generate 1,800 kilowatt-hours of energy. Or enough, he says, to keep the lights on in 15 to 20 average U.S. homes for about three days. The fuel cell can provide energy directly to the truck’s powertrain. However, it mainly charges two battery packs with a total capacity of 155-kilowatt-hours because batteries are better than fuel cells at handling the variable power demands that come with vehicle propulsion. When the truck is at a disaster site, the fuel cell can automatically turn itself on and off to keep the batteries charged up while they are exporting electric power to buildings that would otherwise be in the dark. “If it’s called upon to export, say, 3 kilowatts to keep a few computers running, the fuel in its tanks could keep them powered for weeks,” says Nicholas Josefik, an industrial engineer at CERL. As if that weren’t enough, an onboard storage tank captures the water that is the byproduct of the electrochemical reactions in the fuel cell. “Any time the fuel cell is producing energy to move the vehicle or to export power, it’s generating water,” says Josefik. The result: roughly 1,500 liters of clean water available any place where municipal or well water supplies are unavailable or unsafe. “When the H2Rescue drives to a location, you won’t need to pull that generator behind you, because the truck itself is a generator.” —Nicholas Josefik, U.S. Army Corps of Engineers Construction Research Lab Just as important as what it can do, Josefik notes, is what it won’t do: “In a traditional emergency situation, you send in a diesel truck and that diesel truck is pulling a diesel-powered generator, so you can provide power to the site,” he says. “And another diesel truck is pulling in a fuel tank to fuel that diesel generator. A third truck might pull a trailer with a water tank on it. “But when the H2Rescue drives to a location,” he continues, “You won’t need to pull that generator behind you, because the truck itself is a generator. You don’t have to drag a trailer full of water, because you know that while you’re on site, H2Rescue will be your water source.” He adds that H2Rescue will not only allow first responders to eliminate a few pieces of equipment but will also eliminate the air pollution and noise that come standard with diesel-powered vehicles and generators. Larkin recalls that the impetus for developing the zero-emission emergency vehicle came in 2019, when a series of natural disasters across the United States, including wildfires and hurricanes, spurred action. “The organizations that funded this project were observing this and saw a need for an alternative emergency support,” he says. They asked themselves, Larkin notes, “‘What can we do to help our first responders take on these natural disasters?’ The rest, as they say, is history.” Asked when we’ll see the Federal Emergency Management Agency, which is typically in charge of disaster response anywhere in the 50 U.S. states, dispatch the H2Rescue truck to the aftermath of, say, a hurricane, Josefik says, “This is still a research unit. We’re working on trying to build a version 2.0 that could go and support responders to an emergency.” That next version, he says, would be the result of some optimizations suggested by Cummins as it was putting the H2Rescue together. “Because this was a one-off build, [Cummins] identified a number of areas for improvement, like how they would do the wiring and the piping differently, so it’s more compact in the unit.” The aim for the second iteration, Larkin says, is “a turnkey unit, ready to operate without all the extra gauges and monitoring equipment that you wouldn’t want in a vehicle that you would turn over to somebody.” There is no timetable for when the new and improved H2Rescue will go into production. The agencies that allocated the funds for the prototype have not yet put up the money to create its successor.

In a new video posted today, Boston Dynamics is sending off its hydraulic Atlas humanoid robot. “For almost a decade,” the video description reads, “Atlas has sparked our imagination, inspired the next generations of roboticists, and leapt over technical barriers in the field. Now it’s time for our hydraulic Atlas robot to kick back and relax.” Hydraulic Atlas has certainly earned some relaxation; Boston Dynamics has been absolutely merciless with its humanoid research program. This isn’t a criticism—sometimes being merciless to your hardware is necessary to push the envelope of what’s possible. And as spectators, we just just get to enjoy it, and this highlight reel includes unseen footage of Atlas doing things well along with unseen footage of Atlas doing things not so well. Which, let’s be honest, is what we’re all really here for. There’s so much more to the history of Atlas than this video shows. Atlas traces its history back to a DARPA project called PETMAN (Protection Ensemble Test Mannequin), which we first wrote about in 2009, so long ago that we had to dig up our own article on the Wayback Machine. As contributor Mikell Taylor wrote back then: PETMAN is designed to test the suits used by soldiers to protect themselves against chemical warfare agents. It has to be capable of moving just like a soldier—walking, running, bending, reaching, army crawling—to test the suit’s durability in a full range of motion. To really simulate humans as accurately as possible, PETMAN will even be able to “sweat”. Relative to the other humanoid robots out there at the time (the most famous of which, by far, was Honda’s ASIMO), PETMAN’s movement and balance were very, very impressive. Also impressive was the presumably unintentional way in which this PETMAN video synced up with the music video to Stayin’ Alive by the Bee Gees. Anyway, DARPA was suitably impressed by all this impressiveness, and chose Boston Dynamics to build another humanoid robot to be used for the DARPA Robotics Challenge. That robot was unveiled ten years ago. The DRC featured a [still looking for a collective noun for humanoid robots] of Atlases, and it seemed like Boston Dynamics was hooked on the form factor, because less than a year after the DRC Finals the company announced the next generation of Atlas, which could do some useful things like move boxes around. Every six months or so, Boston Dynamics put out a new Atlas video, with the robot running or jumping or dancing or doing parkour, leveraging its powerful hydraulics to impress us every single time. There was really nothing like hydraulic Atlas in terms of dynamic performance, and you could argue that there still isn’t. This is a robot that will be missed. The original rendering of Atlas, followed by four generations of the robot.Boston Dynamics/IEEE Spectrum Now, if you’re wondering why Boston Dynamics is saying “it’s time for our hydraulic Atlas robot to kick back and relax,” rather than just “our Atlas robot,” and if you’re also wondering why the video description ends with “take a look back at everything we’ve accomplished with the Atlas platform “to date,” well, I can’t help you. Some people might attempt to draw some inferences and conclusions from that very specific and deliberate language, but I would certainly not be one of them, because I’m well known for never speculating about anything. I would, however, point out a few things that have been obvious for a while now. Namely, that: Boston Dynamics has been focusing fairly explicitly on commercialization over the past several years Complex hydraulic robots are not product friendly because (among other things) they tend to leave puddles of hydraulic fluid on the carpet Boston Dynamics has been very successful with Spot as a productized electric platform based on earlier hydraulic research platforms Fully electric commercial humanoids really seems to be where robotics is at right now There’s nothing at all new in any of this; the only additional piece of information we have is that the hydraulic Atlas is, as of today, retiring. And I’m just going to leave things there.

AI hiring has been growing at least slightly in most regions around the world, with Hong Kong leading the pack; however, AI careers are losing ground compared with the overall job market, according to the 2024 AI Index Report. This annual effort by Stanford’s Institute for Human-Centered Artificial Intelligence (HAI) draws from a host of data to understand the state of the AI industry today. Stanford’s AI Index looks at the performance of AI models, investment, research, and regulations. But tucked within the 385 pages of the 2024 Index are several insights into AI career trends, based on data from LinkedIn and Lightcast, a labor market analytics firm. Here’s a quick look at that analysis, in four charts. Overall hiring is up—a little But don’t get too excited—as a share of overall labor demand, AI jobs are slipping Python is still the best skill to have Machine learning loses luster

Each year, the AI Index lands on virtual desks with a louder virtual thud—this year, its 393 pages are a testament to the fact that AI is coming off a really big year in 2023. For the past three years, IEEE Spectrum has read the whole damn thing and pulled out a selection of charts that sum up the current state of AI (see our coverage from 2021, 2022, and 2023). This year’s report, published by the Stanford Institute for Human-Centered Artificial Intelligence (HAI), has an expanded chapter on responsible AI and new chapters on AI in science and medicine, as well as its usual roundups of R&D, technical performance, the economy, education, policy and governance, diversity, and public opinion. This year is also the first time that Spectrum has figured into the report, with a citation of an article published here about generative AI’s visual plagiarism problem. 1. Generative AI investment skyrockets While corporate investment was down overall last year, investment in generative AI went through the roof. Nestor Maslej, editor-in-chief of this year’s report, tells Spectrum that the boom is indicative of a broader trend in 2023, as the world grappled with the new capabilities and risks of generative AI systems like ChatGPT and the image-generating DALL-E 2. “The story in the last year has been about people responding [to generative AI],” says Maslej, “whether it’s in policy, whether it’s in public opinion, or whether it’s in industry with a lot more investment.” Another chart in the report shows that most of that private investment in generative AI is happening in the United States. 2. Google is dominating the foundation model race Foundation models are big multipurpose models—for example, OpenAI’s GPT-3 and GPT-4 are the foundation model that enable ChatGPT users to write code or Shakespearean sonnets. Since training these models typically requires vast resources, Industry now makes most of them, with academia only putting out a few. Companies release foundation models both to push the state-of-the-art forward and to give developers a foundation on which to build products and services. Google released the most in 2023. 3. Closed models outperform open ones One of the hot debates in AI right now is whether foundation models should be open or closed, with some arguing passionately that open models are dangerous and others maintaining that open models drive innovation. The AI Index doesn’t wade into that debate, but instead looks at trends such as how many open and closed models have been released (another chart, not included here, shows that of the 149 foundation models released in 2023, 98 were open, 23 gave partial access through an API, and 28 were closed). The chart above reveals another aspect: Closed models outperform open ones on a host of commonly used benchmarks. Maslej says the debate about open versus closed “usually centers around risk concerns, but there’s less discussion about whether there are meaningful performance trade-offs.” 4. Foundation models have gotten super expensive Here’s why industry is dominating the foundation model scene: Training a big one takes very deep pockets. But exactly how deep? AI companies rarely reveal the expenses involved in training their models, but the AI Index went beyond the typical speculation by collaborating with the AI research organization Epoch AI. To come up with their cost estimates, the report explains, the Epoch team “analyzed training duration, as well as the type, quantity, and utilization rate of the training hardware” using information gleaned from publications, press releases, and technical reports. It’s interesting to note that Google’s 2017 transformer model, which introduced the architecture that underpins almost all of today’s large language models, was trained for only US $930. 5. And they have a hefty carbon footprint The AI Index team also estimated the carbon footprint of certain large language models. The report notes that the variance between models is due to factors including model size, data center energy efficiency, and the carbon intensity of energy grids. Another chart in the report (not included here) shows a first guess at emissions related to inference—when a model is doing the work it was trained for—and calls for more disclosures on this topic. As the report notes: “While the per-query emissions of inference may be relatively low, the total impact can surpass that of training when models are queried thousands, if not millions, of times daily.” 6. The United States leads in foundation models While Maslej says the report isn’t trying to “declare a winner to this race,” he does note that the United States is leading in several categories, including number of foundation models released (above) and number of AI systems deemed significant technical advances. However, he notes that China leads in other categories including AI patents granted and installation of industrial robots. 7. Industry calls new PhDs This one is hardly a surprise, given the previously discussed data about industry getting lots of investment for generative AI and releasing lots of exciting models. In 2022 (the most recent year for which the Index has data), 70 precent of new AI PhDs in North America took jobs in industry. It’s a continuation of a trend that’s been playing out over the last few years. 8. Some progress on diversity For years, there’s been little progress on making AI less white and less male. But this year’s report offers a few hopeful signs. For example, the number of non-white and female students taking the AP computer science exam is on the rise. The graph above shows the trends for ethnicity, while another graph, not included here, shows that 30 percent of the students taking the exam are now girls. Another graph in the report shows that at the undergraduate level, there’s also a positive trend in increasing ethnic diversity among North American students earning bachelor degrees in computer science, although the number of women earning CS bachelor degrees has barely budged over the last five years. Says Maslej, “it’s important to know that there’s still a lot of work to be done here.” 9. Chatter in earnings calls Businesses are awake to the possibilities of AI. The Index got data about Fortune 500 companies’ earnings calls from Quid, a market intelligence firm that used natural language processing tools to scan for all mentions of “artificial intelligence,” “AI,” “machine learning,” “ML,” and “deep learning.” Nearly 80 percent of the companies included discussion of AI in their calls. “I think there’s a fear in business leaders that if they don’t use this technology, they’re going to miss out,” Maslej says. And while some of that chatter is likely just CEOs bandying about buzzwords, another graph in the report shows that 55 percent of companies included in a McKinsey survey have implemented AI in at least one business unit. 10. Costs go down, revenues go up And here’s why AI isn’t just a corporate buzzword: The same McKinsey survey showed that the integration of AI has caused companies’ costs to go down and their revenues go up. Overall, 42 percent of respondents said they’d seen reduced costs, and 59 percent claimed increased revenue. Other charts in the report suggest that this impact on the bottom line reflects efficiency gains and better worker productivity. In 2023, a number of studies in different fields showed that AI enabled workers to complete tasks more quickly and produce better quality work. One study looked at coders using Copilot, while others looked at consultants, call center agents, and law students. “These studies also show that although every worker benefits, AI helps lower-skilled workers more than it does high-skilled workers,” says Maslej. 11. Corporations do perceive risks This year, the AI Index team ran a global survey of 1,000 corporations with revenues of at least $500 million to understand how businesses are thinking about responsible AI. The results showed that privacy and data governance is perceived as the greatest risk across the globe, while fairness (often discussed in terms of algorithmic bias) still hasn’t registered with most companies. Another chart in the report shows that companies are taking action on their perceived risks: The majority of organizations across regions have implemented at least one responsible AI measure in response to relevant risks. 12. AI can’t beat humans at everything... yet In recent years, AI systems have outperformed humans on a range of tasks, including reading comprehension and visual reasoning, and Maslej notes that the pace of AI performance improvement has also picked up. “A decade ago, with a benchmark like ImageNet, you could rely on that to challenge AI researchers for for five or six years,” he says. “Now, a new benchmark is introduced for competition-level mathematics and the AI starts at 30 percent, and then in a year it gets to 90 percent.” While there are still complex cognitive tasks where humans outperform AI systems, let’s check in next year to see how that’s going. 13. Developing norms of AI responsibility When an AI company is preparing to release a big model, it’s standard practice to test it against popular benchmarks in the field, thus giving the AI community a sense of how models stack up against each other in terms of technical performance. However, it has been less common to test models against responsible AI benchmarks that assess such things as toxic language output (RealToxicityPrompts and ToxiGen), harmful bias in responses (BOLD and BBQ), and a model’s degree of truthfulness (TruthfulQA). That’s starting to change, as there’s a growing sense that checking one’s model against theses benchmarks is, well, the responsible thing to do. However, another chart in the report shows that consistency is lacking: Developers are testing their models against different benchmarks, making comparisons harder. 14. Laws both boost and constrain AI Between 2016 and 2023, the AI Index found that 33 countries had passed at least one law related to AI, with most of the action occurring in the United States and Europe; in total, 148 AI-related bills have been passed in that timeframe. The Index researchers also classified bills as either expansive laws that aim to enhance a country’s AI capabilities or restrictive laws that place limits on AI applications and usage. While many bills continue to boost AI, the researchers found a global trend toward restrictive legislation. 15. AI makes people nervous The Index’s public opinion data comes from a global survey on attitudes toward AI, with responses from 22,816 adults (ages 16 to 74) in 31 countries. More than half of respondents said that AI makes them nervous, up from 39 percent the year before. And two-thirds of people now expect AI to profoundly change their daily lives in the next few years. Maslej notes that other charts in the index show significant differences in opinion among different demographics, with young people being more inclined toward an optimistic view of how AI will change their lives. Interestingly, “a lot of this kind of AI pessimism comes from Western, well-developed nations,” he says, while respondents in places like Indonesia and Thailand said they expect AI’s benefits to outweigh its harms.

Among the countless challenges of decarbonizing transportation, one of the most compelling involves electric motors. In laboratories all over the world, researchers are now chasing a breakthrough that could kick into high gear the transition to electric transportation: a rugged, compact, powerful electric motor that has high power density and the ability to withstand high temperatures—and that doesn’t have rare-earth permanent magnets. It’s a huge challenge currently preoccupying some of the best machine designers on the planet. More than a few of them are at ZF Friedrichshafen AG, one of the world’s largest suppliers of parts to the automotive industry. In fact, ZF astounded analysts late last year when it announced that it had built a 220-kilowatt traction motor that used no rare-earth elements. Moreover, the company announced, their new motor had characteristics comparable to the rare-earth permanent-magnet synchronous motors that now dominate in electric vehicles. Most EVs have rare-earth-magnet-based motors ranging from 150 to 300 kilowatts, and power densities between 1.1 and 3.0 kilowatts per kilogram. Meanwhile, the company says they’ve developed a rare-earth-free motor right in the middle of that range: 220 kW. (The company has not yet revealed its motor’s specific power—its kW/kg rating.) The ZF machine is a type called a separately-excited (or doubly-excited) synchronous motor. It has electromagnets in both the stator and the rotor, so it does away with the rare-earth permanent magnets used in the rotors of nearly all EV motors on the road today. In a separately-excited synchronous motor, alternating current applied to the stator electromagnets sets up a rotating magnetic field. A separate current applied to the rotor electromagnets energizes them, producing a field that locks on to the rotating stator field, producing torque. “As a matter of fact, 95 percent of the rare earths are mined in China. And this means that if China decides no one else will have rare earths, we can do nothing against it.” —Otmar Scharrer, ZF Friedrichshafen AG So far, these machines have not been used much in EVs, because they require a separate system to transfer power to the spinning rotor magnets, and there’s no ideal way to do that. Many such motors use sliders and brushes to make electrical contact to a spinning surface, but the brushes produce dust and eventually wear out. Alternatively, the power can be transferred via inductance, but in that case the apparatus is typically cumbersome, making the unit complicated and physically large and heavy. Now, though, ZF says it has solved these problems with its experimental motor, which it calls I2SM (for In-Rotor Inductive-Excited Synchronous Motor). Besides not using any rare earth elements, the motor offers a few other advantages in comparison with permanent-magnet synchronous motors. These are linked to the fact that this kind of motor technology offers the ability to precisely control the magnetic field in the rotor—something that’s not possible with permanent magnets. That control, in turn, permits varying the field to get much higher efficiency at high speed, for example. With headquarters in Baden-Württemberg, Germany, ZF Friedrichshafen AG is known for a rich R&D heritage and many commercially successful innovations dating back to 1915, when it began supplying gears and other parts for Zeppelins. Today, the company has some 168,000 employees in 31 countries. Among the customers for its motors and electric drive trains are Mercedes-Benz, BMW, and Jaguar Land Rover. (Late last year, shortly after announcing the I2SM, the company announced the sale of its 3,000,000th motor.) Has ZF just shown the way forward for rare-earth-free EV motors? To learn more about the I 2SM and ZF’s vision of the future of EV traction motors, Spectrum reached out to Otmar Scharrer, ZF’s Senior Vice President, R&D, of Electrified Powertrain Technology. Our interview with him has been edited for concision and clarity. Otmar Scharrer on... The I2SM’s technical bona fides The most promising concepts for future motors The motor’s coils, efficiency, and cooling The prototypes built to date The challenges the team overcame IEEE Spectrum: Why is it important to eliminate or to reduce the use of rare-earth elements in traction motors? ZF Friedrichshafen AG’s Otmar Scharrer is leading a team discovering ways to build motors that don’t depend on permanent magnets—and China’s rare-earth monopolies. ZF Group Otmar Scharrer: Well, there are two reasons for that. One is sustainability. We call them “rare earth” because they really are rare in the earth. You need to move a lot of soil to get to these materials. Therefore, they have a relatively high footprint because, usually, they are dug out of the earth in a mine with excavators and huge trucks. That generates some environmental pollution and, of course, a change of the landscape. That is one thing. The other is that they are relatively expensive. And of course, this is something we always address cautiously as a tier one [automotive industry supplier]. And as a matter of fact, 95 percent of the rare earths are produced in China. And this means that if China decides no one else will have rare earths, we can do nothing against it. The recycling circle [for rare earth elements] will not work because there are just not enough electric motors out there. They still have an active lifetime. When you are ramping up, when you have a steep ramp up in terms of volume, you never can satisfy your demands with recycling. Recycling will only work if you have a constant business and you’re just replacing those units which are failing. I’m sure this will come, but we see this much later when the steep ramp-up has ended. “The power density is the same as for a permanent-magnet machine, because we produce both. And I can tell you that there is no difference.” —Otmar Scharrer, ZF Friedrichshafen AG You had asked a very good question: How much rare-earth metal does a typical traction motor contain? I had to ask my engineers. This is an interesting question. Most of our electric motors are in the range of 150 to 300 kilowatts. This is the main range of power for passenger cars. And those motors typically have 1.5 kilograms of magnet material. And 0.5 percent to 1 percent out of this material is pure [heavy rare-earth elements]. So this is not too much. It’s only 5 to 15 grams. But, yes, it’s a very difficult-to-get material. This is the reason for this [permanent-] magnet-free motor. The concept itself is not new. It has been used for years and years, for decades, because usually, power generation is done with this kind of electric machine. So if you have a huge power plant, for example, a gas power plant, then you would typically find such an externally-excited machine as a generator. We did not use them for passenger cars or for mobile applications because of their weight and size. And some of that weight-and-size problem comes directly from the need to generate a magnetic field in the rotor, to replace the [permanent] magnets. You need to set copper coils under electricity. So you need to carry electric current inside the rotor. This is usually done with sliders. And those sliders generate losses. This is the one thing because you have, typically, carbon brushes touching a metal ring so that you can conduct the electricity. Back to top Those brushes are what make the unit longer, axially, in the direction of the axle? Scharrer: Exactly. That’s the point. And you need an inverter which is able to excite the electric machine. Normal inverters have three phases, and then you need a fourth phase to electrify the rotor. And this is a second obstacle. Many OEMs or e-mobility companies do not have this technology ready. Surprisingly enough, the first ones who brought this into serious production were [Renault]. It was a very small car, a Renault. [Editor's note: the model was the Zoe, which was manufactured from 2013 until March of this year.] It had a relatively weak electric motor, just 75 or 80 kilowatts. They decided to do this because in an electric vehicle, there’s a huge advantage with this kind of externally excited machine. You can switch off and switch on the magnetic field. This is a great safety advantage. Why safety? Think about it. If your bicycle has a generator [for a headlight], it works like an electric motor. If you are moving and the generator is spinning, connected to the wheel, then it is generating electricity. “We have an efficiency of approximately 96 percent. So, very little loss.” —Otmar Scharrer, ZF Friedrichshafen AG The same is happening in an electric machine in the car. If you are driving on the highway at 75 miles an hour, and then suddenly your whole system breaks down, what would happen? In a permanent magnet motor, you would generate enormous voltage because the rotor magnets are still rotating in the stator field. But in a permanent-magnet-free motor, nothing happens. You are just switched off. So it is self-secure. This is a nice feature. And the second feature is even better if you drive at high speed. High speed is something like 75, 80, 90 miles an hour. It’s not too common in most countries. But it’s a German phenomenon, very important here. People like to drive fast. Then you need to address the area of field weakening because [at high speed], the magnetic field would be too strong. You need to weaken the field. And if you don’t have [permanent] magnets, it’s easy: you just adapt the electrically-induced magnetic field to the appropriate value, and you don’t have this field-weakening requirement. And this results in much higher efficiency at high speeds. You called this field weakening at high speed? Scharrer: You need to weaken the magnetic field in order to keep the operation stable. And this weakening happens by additional electricity coming from the battery. And therefore, you have a lower efficiency of the electric motor. Back to top What are the most promising concepts for future EV motors? Scharrer: We believe that our concept is most promising, because as you pointed out a couple of minutes ago, we are growing in actual length when we do an externally excited motor. We thought a lot what we can do to overcome this obstacle. And we came to the conclusion, let’s do it inductively, by electrical inductance. And this has been done by competitors as well, but they simply replaced the slider rings with inductance transmitters. “We are convinced that we can build the same size, the same power level of electric motors as with the permanent magnets.” —Otmar Scharrer, ZF Friedrichshafen AG And this did not change the situation. What we did, we were shrinking the inductive unit to the size of the rotor shaft, and then we put it inside the shaft. And therefore, we reduced this 50-to-90-millimeter growth in axial length. And therefore, as a final result, you know the motor shrinks, the housing gets smaller, you have less weight, and you have the same performance density in comparison with a PSM [permanent-magnet synchronous motor] machine. What is an inductive exciter exactly? Scharrer: Inductive exciter means nothing else than that you transmit electricity without touching anything. You do it with a magnetic field. And we are doing it inside of the rotor shaft. This is where the energy is transmitted from outside to the shaft [and then to the rotor electromagnets]. So the rotor shaft, is that different from the motor shaft, the actual torque shaft? Scharrer: It’s the same. The thing I know with inductance is in a transformer, you have coils next to each other and you can induce a voltage from the energized coil in the other coil. Scharrer: This is exactly what is happening in our rotor shafts. Back to top So you use coils, specially designed, and you induce voltage from one to the other? Scharrer: Yes. And we have a very neat, small package, which has a diameter of less than 30 millimeters. If you can shrink it to that value, then you can put it inside the rotor shaft. So of course, if you have two coils, and they’re spaced next to each other, you have a gap. So that gap enables you to spin, right? Since they’re not touching, they can spin independently. So you had to design something where the field could be transferred. In other words, they could couple even though one of them was spinning. Scharrer: We have a coil in the rotor shaft, which is rotating with the shaft. And then we have another one that is stationary inside the rotor shaft while the shaft rotates around it. And there is an air gap in between. Everything happens inside the rotor shaft. What is the efficiency? How much power do you lose? Scharrer: We have an efficiency of approximately 96 percent. So, very little loss. And for the magnetic field, you don’t need a lot of energy. You need something between 10 and 15 kilowatts for the electric field. Let’s assume a transmitted power of 10 kilowatts, we’ll have losses of about 400 watts. This [relatively low level of loss] is important because we don’t cool the unit actively and therefore it needs this kind of high efficiency. The motor isn’t cooled with liquids? Scharrer: The motor itself is actively cooled, with oil, but the inductive unit is passively cooled, with heat transfer to nearby cooling structures. “A good invention is always easy. If you look as an engineer on good IP, then you say, ‘Okay, that looks nice.’” —Otmar Scharrer, ZF Friedrichshafen AG What are the largest motors you’ve built or what are the largest motors you think you can build, in kilowatts? Scharrer: We don’t think that there is a limitation with this technology. We are convinced that we can build the same size, the same power level of electric motors as with the permanent magnets. You could do 150- or 300-kilowatt motors? Scharrer: Absolutely. Back to top What have you done so far? What prototypes have you built? Scharrer: We have a prototype with 220 kilowatts. And we can easily upgrade it to 300, for example. Or we can shrink it to 150. That is always easy. And what is your specific power of this motor? Scharrer: You mean kilowatts per kilogram? I can’t tell you, to be quite honest. It’s hard to compare, because it always depends on where the borderline is. You never have a motor by itself. You always need a housing as well. What part of the housing are you including in the calculation? But I can tell you one thing: The power density is the same as for a permanent-magnet machine because we produce both. And I can tell you that there is no difference. What automakers do you currently have agreements with? Are you providing electric motors for certain automakers? Who are some of your customers now? Scharrer: We are providing our dedicated hybrid transmissions to BMW, to Jaguar Land Rover, and our electric-axle drives to Mercedes-Benz and Geely Lotus, for example. And we are, of course, in development with a lot of other applications. And I think you understand that I cannot talk about that. So for BMW, Land Rover, Mercedes-Benz, you’re providing electric motors and drivetrain components? Scharrer: BMW and Land Rover. We provide dedicated hybrid transmissions. We provide an eight-speed automatic transmission with a hybrid electric motor up to 160 kilowatts. It’s one of the best hybrid transmissions because you can drive fully electrically with 160 kilowatts, which is quite something. “We achieved the same values, for power density and other characteristics, for as for a [permanent] magnet motor. And this is really a breakthrough because according to our best knowledge, this never happened before.” —Otmar Scharrer, ZF Friedrichshafen AG What were the major challenges you had to overcome, to transmit the power inside the rotor shaft? Back to top Scharrer: The major challenge is, always, it needs to be very small. At the same time, it needs to be super reliable, and it needs to be easy. A good invention is always easy. When you see it, if you look as an engineer on good IP [intellectual property], then you say, “Okay, that looks nice”—it’s quite obvious that it’s a good idea. If the idea is complex and it needs to be explained and you don’t understand it, then usually this is not a good idea to be implemented. And this one is very easy. Straightforward. It’s a good idea: Shrink it, put it into the rotor shaft. So you mean very easy to explain? Scharrer: Yes. Easy to explain because it’s obviously an interesting idea. You just say, “Let’s use part of the rotor shaft for the transmission of the electricity into the rotor shaft, and then we can cut the additional length out of the magnet-free motor.” Okay. That’s a good answer. We have a lot of IP here. This is important because if you have the idea, I mean, the idea is the main thing. What were the specific savings in weight and rotor shaft and so on? Scharrer: Well, again, I would just answer in a very general way. We achieved the same values, for power density and other characteristics, as for a [permanent] magnet motor. And this is really a breakthrough because according to our best knowledge, this never happened before. Do you think the motor will be available before the end of this year or perhaps next year? Scharrer: You mean available for a serious application? Yes. If Volkswagen came to you and said, “Look, we want to use this in our next car,” could you do that before the end of this year, or would it have to be 2025? Scharrer: It would have to be 2025. I mean, technically, the electric motor is very far along. It is already in an A-sample status, which means we are... What kind of status? Scharrer: A-sample. In the automotive industry, you have A, B, or C. For A-sample, you have all the functions, and you have all the features of the product, and those are secured. And then B- is, you are not producing any longer in the prototype shop, but you are producing close to a possibly serious production line. C-sample means you are producing on serious fixtures and tools, but not on a [mass-production] line. And so this is an A-sample, meaning it is about one and a half years away from a conventional SOP ["Start of Production"] with our customer. So we could be very fast. This article was updated on 15 April 2024. An earlier version of this article gave an incorrect figure for the efficiency of the inductive exciter used in the motor. This efficiency is 96 percent, not 98 or 99 percent.

In 2016, the Japanese government announced a plan for the emergence of a new kind of society. Human civilization, the proposal explained, had begun with hunter-gatherers, passed through the agrarian and industrial stages, and was fast approaching the end of the information age. As then Prime Minister Shinzo Abe put it, “We are now witnessing the opening of the fifth chapter.” This chapter, called Society 5.0, would see made-on-demand goods and robot caretakers, taxis, and tractors. Many of the innovations that will enable it, like artificial intelligence, might be obvious. But there is one key technology that is easy to overlook: lasers. The lasers of Society 5.0 will need to meet several criteria. They must be small enough to fit inside everyday devices. They must be low-cost so that the average metalworker or car buyer can afford them—which means they must also be simple to manufacture and use energy efficiently. And because this dawning era will be about mass customization (rather than mass production), they must be highly controllable and adaptive. Semiconductor lasers would seem the perfect candidates, except for one fatal flaw: They are much too dim. Laser brightness—defined as optical power per unit area per unit of solid angle—is a measure of how intensely light can be focused as it exits the laser and how narrowly it diverges as it moves away. The threshold for materials work—cutting, welding, drilling—is on the order of 1 gigawatt per square centimeter per steradian (GW/cm2/sr). However, the brightness of even the brightest commercial semiconductor lasers falls far below that. Brightness is also important for light detection and ranging (lidar) systems in autonomous robots and vehicles. These systems don’t require metal-melting power, but to make precise measurements from long distances or at high speeds, they do require tightly focused beams. Today’s top-line lidar systems employ more than 100 semiconductor lasers whose inherently divergent beams are collimated using a complicated setup of lenses installed by hand. This complexity drives up cost, putting lidar-navigated cars out of reach for most consumers. Multiple 3-millimeter-wide photonic-crystal semiconductor lasers are built on a semiconductor wafer. Susumu Noda Of course, other types of lasers can produce ultrabright beams. Carbon dioxide and fiber lasers, for instance, dominate the market for industrial applications. But compared to speck-size semiconductor lasers, they are enormous. A high-power CO2 laser can be as large as a refrigerator. They are also more expensive, less energy efficient, and harder to control. Over the past couple of decades, our team at Kyoto University has been developing a new type of semiconductor laser that blows through the brightness ceiling of its conventional cousins. We call it the photonic-crystal surface-emitting laser, or PCSEL (pronounced “pick-cell”). Most recently, we fabricated a PCSEL that can be as bright as gas and fiber lasers—bright enough to quickly slice through steel—and proposed a design for one that is 10 to 100 times as bright. Such devices could revolutionize the manufacturing and automotive industries. If we, our collaborating companies, and research groups around the world—such as at National Yang Ming Chiao Tung University, in Hsinchu, Taiwan; the University of Texas at Arlington; and the University of Glasgow—can push PCSEL brightness further still, it would even open the door to exotic applications like inertial-confinement nuclear fusion and light propulsion for spaceflight. Hole-y Grail The magic of PCSELs arises from their unique construction. Like any semiconductor laser, a PCSEL consists of a thin layer of light-generating material, known as the active layer, sandwiched between cladding layers. In fact, for the sake of orientation, it’s helpful to picture the device as a literal sandwich—let’s say a slice of ham between two pieces of bread. Now imagine lifting the sandwich to your mouth, as if you are about to take a bite. If your sandwich were a conventional semiconductor laser, its beam would radiate from the far edge, away from you. This beam is created by passing a current through a stripe in the active “ham” layer. The excited ham atoms spontaneously release photons, which stimulate the release of identical photons, amplifying the light. Mirrors on each end of the stripe then repeatedly reflect these waves; because of interference and loss, only certain frequencies and spatial patterns—or modes—are sustained. When the gain of a mode exceeds losses, the light emerges in a coherent beam, and the laser is said to oscillate in that mode. The problem with this standard stripe approach is that it is very difficult to increase output power without sacrificing beam quality. The power of a semiconductor laser is limited by its emission area because extremely concentrated light can cause catastrophic damage to the semiconductor. You can deliver more power by widening the stripe, which is the strategy used for so-called broad-area lasers. But a wider stripe also gives room for the oscillating light to take zigzag sideways paths, forming what are called higher-order lateral modes. More Modes, More Problems You can visualize the intesity pattern of a lateral mode by imagining that you’ve placed a screen in the cross section of the output beam. Light bouncing back and forth perfectly along the length of the stripe forms the fundamental (zero-order) mode, which has a single peak of intensity in the center of the beam. The first-order mode, from light reflecting at an angle to the edge of the sandwich, has two peaks to the right and left; the second-order mode, from a smaller angle, has a row of three peaks, and so on. For each higher-order mode, the laser effectively operates as a combination of smaller emitters whose narrower apertures cause the beam to diverge rapidly. The resulting mixture of lateral modes therefore makes the laser light spotty and diffuse. Those troublesome modes are why the brightness of conventional semiconductor lasers maxes out around 100 MW/cm2/sr. PCSELs deal with unwanted modes by adding another layer inside the sandwich: the “Swiss cheese” layer. This special extra layer is a semiconductor sheet stamped with a two-dimensional array of nanoscale holes. By tuning the spacing and shape of the holes, we can control the propagation of light inside the laser so that it oscillates in only the fundamental mode, even when the emission area is expanded. The result is a beam that can be both powerful and narrow—that is, bright. Because of their internal physics, PCSELs operate in a completely different way from edge-emitting lasers. Instead of pointing away from you, for instance, the beam from your PCSEL sandwich would now radiate upward, through the top slice of bread. To explain this unusual emission, and why PCSELs can be orders of magnitude brighter than other semiconductor lasers, we must first describe the material properties of the Swiss cheese—in actuality, a fascinating structure called a photonic crystal. How Photonic Crystals Work Photonic crystals control the flow of light in a way that’s similar to how semiconductors control the flow of electrons. Instead of atoms, however, the lattice of a photonic crystal is sculpted out of larger entities—such as holes, cubes, or columns—arranged such that the refractive index changes periodically on the scale of a wavelength of light. Although the quest to artificially construct these marvelous materials began less than 40 years ago, scientists have since learned that they already exist in nature. Opals, peacock feathers, and some butterfly wings, for example, all owe their brilliant iridescence to the intricate play of light within naturally engineered photonic crystals. Understanding how light moves in a photonic crystal is fundamental to PCSEL design. We can predict this behavior by studying the crystal’s photonic band structure, which is analogous to the electronic band structure of a semiconductor. One way to do that is to plot the relationship between frequency and wavenumber—the number of wave cycles that fit within one unit cell of the crystal’s lattice. How Light Moves in a Photonic Crystal Consider, for example, a simple one-dimensional photonic crystal formed by alternating ribbons of glass and air. Light entering the crystal will refract through and partially reflect off each interface, producing overlapping beams that reinforce or weaken one another according to the light’s wavelength and direction. Most waves will travel through the material. But at certain points, called singularity points, the reflections combine perfectly with the incident wave to form a standing wave, which does not propagate. In this case, a singularity occurs when a wave undergoes exactly half a cycle from one air ribbon to the next. There are other singularities wherever a unit cell is an integer multiple of half the wavelength. One of us (Susumu Noda) began experimenting with lasers containing photonic crystal-like structures before these materials even had a name. In the mid 1980s, while at Mitsubishi Electric Corporation, he studied a semiconductor laser called a distributed feedback (DFB) laser. A DFB laser is a basic stripe laser with an extra internal layer containing regularly spaced grooves filled with matter of a slightly different refractive index. This periodic structure behaves somewhat like the 1D photonic crystal described above: It repeatedly reflects light at a single wavelength, as determined by the groove spacing, such that a standing wave emerges. Consequently, the laser oscillates at only that wavelength, which is critical for long-haul fiber-optic transmission and high-sensitivity optical sensing. Steel Slicer As the Mitsubishi team demonstrated, a DFB laser can be enticed to perform other tricks. For instance, when the team set the groove spacing equal to the lasing wavelength in the device, some of the oscillating light diffracted upward, causing the laser to shine not only from the tiny front edge of its active stripe but also from the stripe’s top. However, this surface beam fanned wildly due to the narrow width of the stripe, which also made it difficult to increase the output power. To Noda’s disappointment, his team’s attempts to widen the stripe—and therefore increase brightness—without causing other headaches were unsuccessful. Nevertheless, those early failures planted an intriguing idea: What if laser light could be controlled in two dimensions instead of one? Boosting Brightness Later, at Kyoto University, Noda led research into 2D and 3D photonic crystals just as the field was coming into being. In 1998, his team built the first PCSEL, and we have since honed the design for various functionalities, including high brightness. In a basic PCSEL, the photonic-crystal layer is a 2D square lattice: Each unit cell is a square delineated by four holes. Although the band structure of a 2D photonic crystal is more complicated than that of a 1D crystal, it likewise reveals singularities where we expect standing waves to form. For our devices, we have made use of the singularity that occurs when the distance between neighboring holes is one wavelength. A gallium arsenide laser operating at 940 nanometers, for example, has an internal wavelength of around 280 nm (considering refractive index and temperature). So the holes in a basic gallium arsenide PCSEL would be set about 280 nm apart. The operating principle is this: When waves of that length are generated in the active layer, the holes in the neighboring photonic-crystal layer act like tiny mirrors, bending the light both backward and sideways. The combined effect of multiple such diffractions creates a 2D standing wave, which is then amplified by the active layer. Some of this oscillating light also diffracts upward and downward and leaks out the laser’s top, producing a surface beam of a single wavelength. A key reason this design works is the large refractive index contrast between the semiconductor and the air inside the holes. As Noda discovered while creating the first device, PCSELs with low refractive index contrasts, like those of DFB lasers, do not oscillate coherently. Also unlike a DFB laser, a PCSEL’s surface emission area is broad and usually round. It can therefore produce a higher quality beam with much lower divergence. Bigger and Brighter  As PCSEL size grows to accommodate more optical power, more lateral modes begin to oscillate. Here’s how those modes are eliminated in each device generation. Higher-order lateral modes form when a standing wave has multiple average peaks of intensity. When the emission area of the PCSEL is relatively small, the peaks sit near its edge. Consequently, most of the light leaks out of the sides, and so the higher-order modes do not oscillate. The double lattice causes light diffracting through the crystal to interfere destructively. These cancellations weaken and spread the intensity peaks of the standing waves, causing the higher-order modes to leak heavily again. However, this method alone does not sufficiently suppress those modes in larger devices. Adjustments to the holes and the bottom reflector induce light exiting the laser to lose some of its energy through interference with the standing waves. Because higher-order modes lose more light, they can be selectively cut off. In 2014, our group reported that a PCSEL with a square lattice of triangular holes and an emission area of 200 by 200 μm could operate continuously at around 1 watt while maintaining a spotlike beam that diverged only about 2 degrees. Compared with conventional semiconductor lasers, whose beams typically diverge more than 30 degrees, this performance was remarkable. The next step was to boost optical power, for which we needed a larger device. But here we hit a snag. According to our theoretical models, PCSELs using the single-lattice design could not grow larger than about 200 μm without inviting pesky higher-order lateral modes. In a PCSEL, multiple modes form when the intensity of a standing wave can be distributed in multiple ways due to the interference pattern created by repeated diffractions. In the fundamental (read: desirable) mode, the intensity distribution resembles Mount Fuji, with most of the oscillating light concentrated in the center of the lattice. Each higher-order mode, meanwhile, has two, three, four, or more Mount Fujis. So when the laser’s emission area is relatively small, the intensity peaks of the higher-order modes sit near the lattice’s periphery. Most of their light therefore leaks out of the sides of the device, preventing these modes from oscillating and contributing to the laser beam. But as with conventional lasers, enlarging the emission area makes space for more modes to oscillate. To solve that problem, we added another set of holes to the photonic-crystal layer, creating a double lattice. In our most successful version, a square lattice of circular holes is shifted a quarter wavelength from a second square lattice of elliptical holes. As a result, some of the diffracting light inside the crystal interferes destructively. These cancellations cause the intensity peaks of the lateral modes to weaken and spread. So when we expand the laser’s emission area, light from the higher-order modes still leaks heavily and does not oscillate. Using that approach, we fabricated a PCSEL with a round emission area 1 millimeter in diameter and showed it could produce a 10-W beam under continuous operation. Diverging just one-tenth of a degree, the beam was even slenderer and more collimated than its 200-μm predecessor and more than three times as bright as is possible with a conventional semiconductor laser. Our device also had the advantage of oscillating in a single mode, of course, which conventional lasers of comparable size cannot do. Pushing PCSEL brightness higher required further innovation. At larger diameters, the double-lattice approach alone does not sufficiently suppress higher-order modes, and so they oscillate yet again. We had observed, however, that these modes depart the laser slightly askew, which drew our attention to the backside reflector. (Picture a sheet of tinfoil lining the bottom of your ham and Swiss sandwich.) This 50-watt PCSEL is bright enough to slice through steel. Susumu Noda In previous device generations, this reflector had served simply to bounce downward-diffracted light up and out from the laser’s emitting surface. By adjusting its position (as well as the spacing and shape of the photonic-crystal holes), we found we could control the reflections so that they interfere in a useful way with the 2D standing waves oscillating within the photonic-crystal layer. This interference, or coupling, essentially induces the departing waves to lose some of their energy. The more askew a departing wave, the more light is lost. And poof! No more higher-order modes. That is how, in 2023, we developed a PCSEL whose brightness of 1 GW/cm2/sr rivals that of gas and fiber lasers. With a 3-mm emission diameter, it could lase continuously at up to 50 W while sustaining a beam that diverged a minuscule one-twentieth of a degree. We even used it to cut through steel. As the bright, beautiful beam carved a disc out of a metal plate 100 μm thick, our entire lab huddled around, watching in amazement. More Powerful PCSELs As impressive as the steel-slicing demonstration was, PCSELs must be even more powerful to compete in the industrial marketplace. Manufacturing automobile parts, for instance, requires optical powers on the order of kilowatts. It should be fairly straightforward to build a PCSEL that can handle that kind of power—either by assembling an array of nine 3-mm PCSELs or by expanding the emission area of our current device to 1 cm. At that size, higher-order modes would once again emerge, reducing the beam quality. But because they would still be as bright as high-power gas and fiber lasers, such kilowatt-class PCSELs could begin to usurp their bulkier competitors. To be truly game-changing, 1-cm PCSELs would need to level up by suppressing those higher-order modes. We have already devised a way to do that by fine-tuning the photonic-crystal structure and the position of the reflector. Although we have not yet tested this new recipe in the lab, our theoretical models suggest that it could raise PCSEL brightness as high as 10 to 100 GW/cm2/sr. Just imagine the variety of unique and intricate products that could be made when such concentrated light can be wielded from a tiny package. Especially for those high-power applications, we’ll need to improve the laser’s energy efficiency and thermal management. Even without any optimization, the “wall plug” efficiency of PCSELs is already at 30 to 40 percent, exceeding most carbon-dioxide and fiber lasers. What’s more, we’ve found a path we think could lead to 60 percent efficiency. And as for thermal management, the water-cooling technology we’re using in the lab today should be sufficient for a 1,000-W, 1-cm PCSEL. High-brightness PCSELs could also be used to make smaller and more affordable sensor systems for self-driving cars and robots. Recently, we built a lidar system using a 500-μm PCSEL. Under pulsed operation, we ran it at about 20 W and got a terrifically bright beam. Even at 30 meters, the spot size was only 5 cm. Such high resolution is unheard of for a compact lidar system without external lenses. We then mounted our prototypes—which are roughly the size of a webcam—on robotic carts and programmed them to follow us and one another around the engineering building. In a separate line of work, we have shown that PCSELs can emit multiple beams that can be controlled electronically to point in different directions. This on-chip beam steering is achieved by varying the position and size of the holes in the photonic-crystal layer. Ultimately, it could replace mechanical beam steering in lidar systems. If light detectors were also integrated on the same chip, these all-electronic navigation systems would be seriously miniature and low-cost. Although it will be challenging, we eventually hope to make 3-cm lasers with output powers exceeding 10 kilowatts and beams shining up to 1,000 GW/cm2/sr—brighter than any laser that exists today. At such extreme brightness, PCSELs could replace the huge, electricity-hungry CO2 lasers used to generate plasma pulses for extreme ultraviolet lithography machines, making chip manufacturing much more efficient. They could similarly advance efforts to realize nuclear fusion, a process that involves firing trillions of watts of laser power at a pea-size fuel capsule. Exceptionally bright lasers also raise the possibility of light propulsion for spaceflight. Instead of taking thousands of years to reach faraway stars, a probe boosted by light could make the journey in only a few decades. It may be a cliché, but we cannot think of a more apt prediction for the next chapter of human ingenuity: The future, as they say, is bright.

It’s late in the afternoon of 2 April 2023 on the island of Kauai. The sun is sinking over this beautiful and peaceful place, when, suddenly, at 4:25 pm, there’s a glitch: The largest generator on the island, a 26-megawatt oil-fired turbine, goes offline. This is a more urgent problem than it might sound. The westernmost Hawaiian island of significant size, Kauai is home to around 70,000 residents and 30,000 tourists at any given time. Renewable energy accounts for 70 percent of the energy produced in a typical year—a proportion that’s among the highest in the world and that can be hard to sustain for such a small and isolated grid. During the day, the local system operator, the Kauai Island Utility Cooperative, sometimes reaches levels of 90 percent from solar alone. But on 2 April, the 26-MW generator was running near its peak output, to compensate for the drop in solar output as the sun set. At the moment when it failed, that single generator had been supplying 60 percent of the load for the entire island, with the rest being met by a mix of smaller generators and several utility-scale solar-and-battery systems. Normally, such a sudden loss would spell disaster for a small, islanded grid. But the Kauai grid has a feature that many larger grids lack: a technology called grid-forming inverters. An inverter converts direct-current electricity to grid-compatible alternating current. The island’s grid-forming inverters are connected to those battery systems, and they are a special type—in fact, they had been installed with just such a contingency in mind. They improve the grid’s resilience and allow it to operate largely on resources like batteries, solar photovoltaics, and wind turbines, all of which connect to the grid through inverters. On that April day in 2023, Kauai had over 150 megawatt-hours’ worth of energy stored in batteries—and also the grid-forming inverters necessary to let those batteries respond rapidly and provide stable power to the grid. They worked exactly as intended and kept the grid going without any blackouts. The photovoltaic panels at the Kapaia solar-plus-storage facility, operated by the Kauai Island Utility Cooperative in Hawaii, are capable of generating 13 megawatts under ideal conditions.TESLA A solar-plus-storage facility at the U.S. Navy’s Pacific Missile Range Facility, in the southwestern part of Kauai, is one of two on the island equipped with grid-forming inverters. U.S. NAVY That April event in Kauai offers a preview of the electrical future, especially for places where utilities are now, or soon will be, relying heavily on solar photovoltaic or wind power. Similar inverters have operated for years within smaller off-grid installations. However, using them in a multimegawatt power grid, such as Kauai’s, is a relatively new idea. And it’s catching on fast: At the time of this writing, at least eight major grid-forming projects are either under construction or in operation in Australia, along with others in Asia, Europe, North America, and the Middle East. Reaching net-zero-carbon emissions by 2050, as many international organizations now insist is necessary to stave off dire climate consequences, will require a rapid and massive shift in electricity-generating infrastructures. The International Energy Agency has calculated that to have any hope of achieving this goal would require the addition, every year, of 630 gigawatts of solar photovoltaics and 390 GW of wind starting no later than 2030—figures that are around four times as great as than any annual tally so far. The only economical way to integrate such high levels of renewable energy into our grids is with grid-forming inverters, which can be implemented on any technology that uses an inverter, including wind, solar photovoltaics, batteries, fuel cells, microturbines, and even high-voltage direct-current transmission lines. Grid-forming inverters for utility-scale batteries are available today from Tesla, GPTech, SMA, GE Vernova, EPC Power, Dynapower, Hitachi, Enphase, CE+T, and others. Grid-forming converters for HVDC, which convert high-voltage direct current to alternating current and vice versa, are also commercially available, from companies including Hitachi, Siemens, and GE Vernova. For photovoltaics and wind, grid-forming inverters are not yet commercially available at the size and scale needed for large grids, but they are now being developed by GE Vernova, Enphase, and Solectria. The Grid Depends on Inertia To understand the promise of grid-forming inverters, you must first grasp how our present electrical grid functions, and why it’s inadequate for a future dominated by renewable resources such as solar and wind power. Conventional power plants that run on natural gas, coal, nuclear fuel, or hydropower produce electricity with synchronous generators—large rotating machines that produce AC electricity at a specified frequency and voltage. These generators have some physical characteristics that make them ideal for operating power grids. Among other things, they have a natural tendency to synchronize with one another, which helps make it possible to restart a grid that’s completely blacked out. Most important, a generator has a large rotating mass, namely its rotor. When a synchronous generator is spinning, its rotor, which can weigh well over 100 tonnes, cannot stop quickly. The Kauai electric transmission grid operates at 57.1 kilovolts, an unusual voltage that is a legacy from the island’s sugar-plantation era. The network has grid-forming inverters at the Pacific Missile Range Facility, in the southwest, and at Kapaia, in the southeast. CHRIS PHILPOT This characteristic gives rise to a property called system inertia. It arises naturally from those large generators running in synchrony with one another. Over many years, engineers used the inertia characteristics of the grid to determine how fast a power grid will change its frequency when a failure occurs, and then developed mitigation procedures based on that information. If one or more big generators disconnect from the grid, the sudden imbalance of load to generation creates torque that extracts rotational energy from the remaining synchronous machines, slowing them down and thereby reducing the grid frequency—the frequency is electromechanically linked to the rotational speed of the generators feeding the grid. Fortunately, the kinetic energy stored in all that rotating mass slows this frequency drop and typically allows the remaining generators enough time to ramp up their power output to meet the additional load. Electricity grids are designed so that even if the network loses its largest generator, running at full output, the other generators can pick up the additional load and the frequency nadir never falls below a specific threshold. In the United States, where nominal grid frequency is 60 hertz, the threshold is generally between 59.3 and 59.5 Hz. As long as the frequency remains above this point, local blackouts are unlikely to occur. Why We Need Grid-Forming Inverters Wind turbines, photovoltaics, and battery-storage systems differ from conventional generators because they all produce direct current (DC) electricity—they don’t have a heartbeat like alternating current does. With the exception of wind turbines, these are not rotating machines. And most modern wind turbines aren’t synchronously rotating machines from a grid standpoint—the frequency of their AC output depends on the wind speed. So that variable-frequency AC is rectified to DC before being converted to an AC waveform that matches the grid’s. As mentioned, inverters convert the DC electricity to grid-compatible AC. A conventional, or grid-following, inverter uses power transistors that repeatedly and rapidly switch the polarity applied to a load. By switching at high speed, under software control, the inverter produces a high-frequency AC signal that is filtered by capacitors and other components to produce a smooth AC current output. So in this scheme, the software shapes the output waveform. In contrast, with synchronous generators the output waveform is determined by the physical and electrical characteristics of the generator. Grid-following inverters operate only if they can “see” an existing voltage and frequency on the grid that they can synchronize to. They rely on controls that sense the frequency of the voltage waveform and lock onto that signal, usually by means of a technology called a phase-locked loop. So if the grid goes down, these inverters will stop injecting power because there is no voltage to follow. A key point here is that grid-following inverters do not deliver any inertia. Przemyslaw Koralewicz, David Corbus, Shahil Shah, and Robb Wallen, researchers at the National Renewable Energy Laboratory, evaluate a grid-forming inverter used on Kauai at the NREL Flatirons Campus. DENNIS SCHROEDER/NREL Grid-following inverters work fine when inverter-based power sources are relatively scarce. But as the levels of inverter-based resources rise above 60 to 70 percent, things start to get challenging. That’s why system operators around the world are beginning to put the brakes on renewable deployment and curtailing the operation of existing renewable plants. For example, the Electric Reliability Council of Texas (ERCOT) regularly curtails the use of renewables in that state because of stability issues arising from too many grid-following inverters. It doesn’t have to be this way. When the level of inverter-based power sources on a grid is high, the inverters themselves could support grid-frequency stability. And when the level is very high, they could form the voltage and frequency of the grid. In other words, they could collectively set the pulse, rather than follow it. That’s what grid-forming inverters do. The Difference Between Grid Forming and Grid Following Grid-forming (GFM) and grid-following (GFL) inverters share several key characteristics. Both can inject current into the grid during a disturbance. Also, both types of inverters can support the voltage on a grid by controlling their reactive power, which is the product of the voltage and the current that are out of phase with each other. Both kinds of inverters can also help prop up the frequency on the grid, by controlling their active power, which is the product of the voltage and current that are in phase with each other. What makes grid-forming inverters different from grid-following inverters is mainly software. GFM inverters are controlled by code designed to maintain a stable output voltage waveform, but they also allow the magnitude and phase of that waveform to change over time. What does that mean in practice? The unifying characteristic of all GFM inverters is that they hold a constant voltage magnitude and frequency on short timescales—for example, a few dozen milliseconds—while allowing that waveform’s magnitude and frequency to change over several seconds to synchronize with other nearby sources, such as traditional generators and other GFM inverters. Some GFM inverters, called virtual synchronous machines, achieve this response by mimicking the physical and electrical characteristics of a synchronous generator, using control equations that describe how it operates. Other GFM inverters are programmed to simply hold a constant target voltage and frequency, allowing that target voltage and frequency to change slowly over time to synchronize with the rest of the power grid following what is called a droop curve. A droop curve is a formula used by grid operators to indicate how a generator should respond to a deviation from nominal voltage or frequency on its grid. There are many variations of these two basic GFM control methods, and other methods have been proposed as well. At least eight major grid-forming projects are either under construction or in operation in Australia, along with others in Asia, Europe, North America, and the Middle East. To better understand this concept, imagine that a transmission line shorts to ground or a generator trips due to a lightning strike. (Such problems typically occur multiple times a week, even on the best-run grids.) The key advantage of a GFM inverter in such a situation is that it does not need to quickly sense frequency and voltage decline on the grid to respond. Instead, a GFM inverter just holds its own voltage and frequency relatively constant by injecting whatever current is needed to achieve that, subject to its physical limits. In other words, a GFM inverter is programmed to act like an AC voltage source behind some small impedance (impedance is the opposition to AC current arising from resistance, capacitance, and inductance). In response to an abrupt drop in grid voltage, its digital controller increases current output by allowing more current to pass through its power transistors, without even needing to measure the change it’s responding to. In response to falling grid frequency, the controller increases power. GFL controls, on the other hand, need to first measure the change in voltage or frequency, and then take an appropriate control action before adjusting their output current to mitigate the change. This GFL strategy works if the response does not need to be superfast (as in microseconds). But as the grid becomes weaker (meaning there are fewer voltage sources nearby), GFL controls tend to become unstable. That’s because by the time they measure the voltage and adjust their output, the voltage has already changed significantly, and fast injection of current at that point can potentially lead to a dangerous positive feedback loop. Adding more GFL inverters also tends to reduce stability because it becomes more difficult for the remaining voltage sources to stabilize them all. When a GFM inverter responds with a surge in current, it must do so within tightly prescribed limits. It must inject enough current to provide some stability but not enough to damage the power transistors that control the current flow. Increasing the maximum current flow is possible, but it requires increasing the capacity of the power transistors and other components, which can significantly increase cost. So most inverters (both GFM and GFL) don’t provide current surges larger than about 10 to 30 percent above their rated steady-state current. For comparison, a synchronous generator can inject around 500 to 700 percent more than its rated current for several AC line cycles (around a tenth of a second, say) without sustaining any damage. For a large generator, this can amount to thousands of amperes. Because of this difference between inverters and synchronous generators, the protection technologies used in power grids will need to be adjusted to account for lower levels of fault current. What the Kauai Episode Reveals The 2 April event on Kauai offered an unusual opportunity to study the performance of GFM inverters during a disturbance. After the event, one of us (Andy Hoke) along with Jin Tan and Shuan Dong and some coworkers at the National Renewable Energy Laboratory, collaborated with the Kauai Island Utility Cooperative (KIUC) to get a clear understanding of how the remaining system generators and inverter-based resources interacted with each other during the disturbance. What we determined will help power grids of the future operate at levels of inverter-based resources up to 100 percent. NREL researchers started by creating a model of the Kauai grid. We then used a technique called electromagnetic transient (EMT) simulation, which yields information on the AC waveforms on a sub-millisecond basis. In addition, we conducted hardware tests at NREL’s Flatirons Campus on a scaled-down replica of one of Kauai’s solar-battery plants, to evaluate the grid-forming control algorithms for inverters deployed on the island.The leap from power systems like Kauai’s, with a peak demand of roughly 80 MW, to ones like South Australia’s, at 3,000 MW, is a big one. But it’s nothing compared to what will come next: grids with peak demands of 85,000 MW (in Texas) and 742,000 MW (the rest of the continental United States). Several challenges need to be solved before we can attempt such leaps. They include creating standard GFM specifications so that inverter vendors can create products. We also need accurate models that can be used to simulate the performance of GFM inverters, so we can understand their impact on the grid. Some progress in standardization is already happening. In the United States, for example, the North American Electric Reliability Corporation (NERC) recently published a recommendation that all future large-scale battery-storage systems have grid-forming capability. Standards for GFM performance and validation are also starting to emerge in some countries, including Australia, Finland, and Great Britain. In the United States, the Department of Energy recently backed a consortium to tackle building and integrating inverter-based resources into power grids. Led by the National Renewable Energy Laboratory, the University of Texas at Austin, and the Electric Power Research Institute, the Universal Interoperability for Grid-Forming Inverters (UNIFI) Consortium aims to address the fundamental challenges in integrating very high levels of inverter-based resources with synchronous generators in power grids. The consortium now has over 30 members from industry, academia, and research laboratories. A recording of the frequency responses to two different grid disruptions on Kauai shows the advantages of grid-forming inverters. The red trace shows the relatively contained response with two grid-forming inverter systems in operation. The blue trace shows the more extreme response to an earlier, comparable disruption, at a time when there was only one grid-forming plant online.NATIONAL RENEWABLE ENERGY LABORATORY At 4:25 pm on 2 April, there were two large GFM solar-battery plants, one large GFL solar-battery plant, one large oil-fired turbine, one small diesel plant, two small hydro plants, one small biomass plant, and a handful of other solar generators online. Immediately after the oil-fired turbine failed, the AC frequency dropped quickly from 60 Hz to just above 59 Hz during the first 3 seconds [red trace in the figure above]. As the frequency dropped, the two GFM-equipped plants quickly ramped up power, with one plant quadrupling its output and the other doubling its output in less than 1/20 of a second. In contrast, the remaining synchronous machines contributed some rapid but unsustained active power via their inertial responses, but took several seconds to produce sustained increases in their output. It is safe to say, and it has been confirmed through EMT simulation, that without the two GFM plants, the entire grid would have experienced a blackout. Coincidentally, an almost identical generator failure had occurred a couple of years earlier, on 21 November 2021. In this case, only one solar-battery plant had grid-forming inverters. As in the 2023 event, the three large solar-battery plants quickly ramped up power and prevented a blackout. However, the frequency and voltage throughout the grid began to oscillate around 20 times per second [the blue trace in the figure above], indicating a major grid stability problem and causing some customers to be automatically disconnected. NREL’s EMT simulations, hardware tests, and controls analysis all confirmed that the severe oscillation was due to a combination of grid-following inverters tuned for extremely fast response and a lack of sufficient grid strength to support those GFL inverters. In other words, the 2021 event illustrates how too many conventional GFL inverters can erode stability. Comparing the two events demonstrates the value of GFM inverter controls—not just to provide fast yet stable responses to grid events but also to stabilize nearby GFL inverters and allow the entire grid to maintain operations without a blackout. Australia Commissions Big GFM Projects In sunny South Australia, solar power now routinely supplies all or nearly all of the power needed during the middle of the day. Shown here is the chart for 31 December 2023, in which solar supplied slightly more power than the state needed at around 1:30 p.m. AUSTRALIAN ENERGY MARKET OPERATOR (AEMO) The next step for inverter-dominated power grids is to go big. Some of the most important deployments are in South Australia. As in Kauai, the South Australian grid now has such high levels of solar generation that it regularly experiences days in which the solar generation can exceed the peak demand during the middle of the day [see figure at left]. The most well-known of the GFM resources in Australia is the Hornsdale Power Reserve in South Australia. This 150-MW/194-MWh system, which uses Tesla’s Powerpack 2 lithium-ion batteries, was originally installed in 2017 and was upgraded to grid-forming capability in 2020. Australia’s largest battery (500 MW/1,000 MWh) with grid-forming inverters is expected to start operating in Liddell, New South Wales, later this year. This battery, from AGL Energy, will be located at the site of a decommissioned coal plant. This and several other larger GFM systems are expected to start working on the South Australia grid over the next year. The leap from power systems like Kauai’s, with a peak demand of roughly 80 MW, to ones like South Australia’s, at 3,000 MW, is a big one. But it’s nothing compared to what will come next: grids with peak demands of 85,000 MW (in Texas) and 742,000 MW (the rest of the continental United States). Several challenges need to be solved before we can attempt such leaps. They include creating standard GFM specifications so that inverter vendors can create products. We also need accurate models that can be used to simulate the performance of GFM inverters, so we can understand their impact on the grid. Some progress in standardization is already happening. In the United States, for example, the North American Electric Reliability Corporation (NERC) recently published a recommendation that all future large-scale battery-storage systems have grid-forming capability. Standards for GFM performance and validation are also starting to emerge in some countries, including Australia, Finland, and Great Britain. In the United States, the Department of Energy recently backed a consortium to tackle building and integrating inverter-based resources into power grids. Led by the National Renewable Energy Laboratory, the University of Texas at Austin, and the Electric Power Research Institute, the Universal Interoperability for Grid-Forming Inverters (UNIFI) Consortium aims to address the fundamental challenges in integrating very high levels of inverter-based resources with synchronous generators in power grids. The consortium now has over 30 members from industry, academia, and research laboratories. One of Australia’s major energy-storage facilities is the Hornsdale Power Reserve, at 150 megawatts and 194 megawatt-hours. Hornsdale, along with another facility called the Riverina Battery, are the country’s two largest grid-forming installations. NEOEN In addition to specifications, we need computer models of GFM inverters to verify their performance in large-scale systems. Without such verification, grid operators won’t trust the performance of new GFM technologies. Using GFM models built by the UNIFI Consortium, system operators and utilities such as the Western Electricity Coordinating Council, American Electric Power, and ERCOT (the Texas’s grid-reliability organization) are conducting studies to understand how GFM technology can help their grids. Getting to a Greener Grid As we progress toward a future grid dominated by inverter-based generation, a question naturally arises: Will all inverters need to be grid-forming? No. Several studies and simulations have indicated that we’ll need just enough GFM inverters to strengthen each area of the grid so that nearby GFL inverters remain stable. How many GFMs is that? The answer depends on the characteristics of the grid and other generators. Some initial studies have shown that a power system can operate with 100 percent inverter-based resources if around 30 percent are grid-forming. More research is needed to understand how that number depends on details such as the grid topology and the control details of both the GFLs and the GFMs. Ultimately, though, electricity generation that is completely carbon free in its operation is within our grasp. Our challenge now is to make the leap from small to large to very large systems. We know what we have to do, and it will not require technologies that are far more advanced than what we already have. It will take testing, validation in real-world scenarios, and standardization so that synchronous generators and inverters can unify their operations to create a reliable and robust power grid. Manufacturers, utilities, and regulators will have to work together to make this happen rapidly and smoothly. Only then can we begin the next stage of the grid’s evolution, to large-scale systems that are truly carbon neutral.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion. RoboCup German Open: 17–21 April 2024, KASSEL, GERMANY AUVSI XPONENTIAL 2024: 22–25 April 2024, SAN DIEGO Eurobot Open 2024: 8–11 May 2024, LA ROCHE-SUR-YON, FRANCE ICRA 2024: 13–17 May 2024, YOKOHAMA, JAPAN RoboCup 2024: 17–22 July 2024, EINDHOVEN, NETHERLANDS Cybathlon 2024: 25–27 October 2024, ZURICH Enjoy today’s videos! I think suggesting that robots can’t fall is much less useful than instead suggesting that robots can fall and get quickly and easily get back up again. [ Deep Robotics ] Sanctuary AI says that this video shows Phoenix operating at “human-equivalent speed,” but they don’t specify which human or under which conditions. Though it’s faster than I would be, that’s for sure. [ Sanctuary AI ] “Suzume” is an animated film by Makoto Shinkai, in which one of the characters gets turned into a three-legged chair: Shintaro Inoue from JSK Lab at the University of Tokyo has managed to build a robotic version of that same chair, which is pretty impressive: [ Github ] Thanks, Shintaro! Humanoid robot EVE training for home assistance like putting groceries into the kitchen cabinets. [ 1X ] This is the RAM—robotic autonomous mower. It can be dropped anywhere in the world and will wake up with a mission to make tall grass around it shorter. Here is a quick clip of it working on the Presidio in SF. [ Electric Sheep ] This year, our robots braved a Finnish winter for the first time. As the snow clears and the days get longer, we’re looking back on how our robots made thousands of deliveries to S Group customers during the colder months. [ Starship ] Agility Robotics is doing its best to answer the (very common) question of “Okay, but what can humanoid robots actually do?” [ Agility Robotics ] Digit is great and everything, but Cassie will always be one of my favorite robots. [ CoRIS ] Adopting omnidirectional Field of View (FoV) cameras in aerial robots vastly improves perception ability, significantly advancing aerial robotics’s capabilities in inspection, reconstruction, and rescue tasks. We propose OmniNxt, a fully open-source aerial robotics platform with omnidirectional perception. [ OmniNxt ] The MAkEable framework enhances mobile manipulation in settings designed around humans by streamlining the process of sharing learned skills and experiences among different robots and contexts. Practical tests confirm its efficiency in a range of scenarios, involving different robots, in tasks such as object grasping, coordinated use of both hands in tasks, and the exchange of skills among humanoid robots. [ Paper ] We conducted trials of Ringbot outdoors on a 400 meter track. With a power source of 2300 milliamp-hours and 11.1 Volts, Ringbot managed to cover approximately 3 kilometers in 37 minutes. We commanded its target speed and direction using a remote joystick controller (Steam Deck), and Ringbot experienced five falls during this trial. [ Paper ] There is a notable lack of consistency about where exactly Boston Dynamics wants you to think Spot’s eyes are. [ Boston Dynamics ] As with every single cooking video, there’s a lot of background prep that’s required for this robot to cook an entire meal, but I would utterly demolish those fries. [ Dino Robotics ] Here’s everything you need to know about Wing delivery drones, except for how much human time they actually require and the true cost of making deliveries by drone, because those things aren’t fun to talk about. [ Wing ] This CMU Teruko Yata Memorial Lecture is by Agility Robotics’ Jonathan Hurst, on “Human-Centric Robots and How Learning Enables Generality.” Humans have dreamt of robot helpers forever. What’s new is that this dream is becoming real. New developments in AI, building on foundations of hardware and passive dynamics, enable vastly improved generality. Robots can step out of highly structured environments and become more human-centric: operating in human spaces, interacting with people, and doing some basic human workflows. By connecting a Large Language Model, Digit can convert natural language high-level requests into complex robot instructions, composing the library of skills together, using human context to achieve real work in the human world. All of this is new—and it is never going back: AI will drive a fast-following robot revolution that is going to change the way we live. [ CMU ]

We tend to think about hopping robots from the ground up. That is, they start on the ground, and then, by hopping, incorporate a aerial phase into their locomotion. But there’s no reason why aerial robots can’t approach hopping from the other direction, by adding a hopping ground phase to flight. Hopcopter is the first robot that I’ve ever seen give this a try, and it’s remarkably effective, combining a tiny quadrotor with a springy leg to hop hop hop all over the place. Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu So why in the air is it worth adding a pogo stick to an otherwise perfectly functional quadrotor? Well, flying is certainly a valuable ability to have, but does take a lot of energy. If you pay close attention to birds (acknowledged experts in the space), they tend to spend a substantial amount of time doing their level best not to fly, often by walking on the ground or jumping around in trees. Not flying most of the time is arguably one of the things that makes birds so successful—it’s that multimodal locomotion capability that has helped them to adapt to so many different environments and situations. Hopcopter is multimodal as well, although in a slightly more restrictive sense: Its two modes are flying and intermittent flying. But the intermittent flying is very important, because cutting down on that flight phase gives Hopcopter some of the same efficiency benefits that birds experience. By itself, a quadrotor of hopcopter’s size can stay airborne for about 400 seconds, while Hopcopter can hop continuously for more than 20 minutes. If your objective is to cover as much distance as possible, Hopcopter might not be as effective as a legless quadrotor. But if your objective is instead something like inspection or search and rescue, where you need to spend a fair amount of time not moving very much, hopping could be significantly more effective. Hopcopter is a small quadcopter (specifically a Crazyflie) attached to a springy pogo-stick leg.Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu Hopcopter can reposition itself on the fly to hop off of different surfaces.Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu The actual hopping is mostly passive. Hopcopter’s leg is two rigid pieces connected by rubber bands, with a Crazyflie microcopter stapled to the top. During a hop, the Crazyflie can add directional thrust to keep the hops hopping and alter its direction as well as its height, from 0.6 meters to 1.6 meters. There isn’t a lot of room for extra sensors on Hopcopter, but the addition of some stabilizing fins allow for continuous hopping without any positional feedback. Besides vertical hopping, Hopcopter can also position itself in midair to hop off of surfaces at other orientations, allowing it to almost instantaneously change direction, which is a neat trick. And it can even do mid air somersaults, because why not? Hopcopter’s repertoire of tricks includes somersaults.Songnan Bai, Runze Ding, Song Li, and Bingxuan Pu The researchers, based at the City University of Hong Kong, say that the Hopcopter technology (namely, the elastic leg) could be easily applied to most other quadcopter platforms, turning them into Hopcopters as well. And if you’re more interested in extra payload rather than extra endurance, it’s possible to use hopping in situations where a payload would be too heavy for continuous flight. The researchers published their work 10 April in Science Robotics.

The idea of powering civilization from gigantic solar plants in orbit is older than any space program, but despite seven decades of rocket science, the concept—to gather near-constant sunlight tens of thousands of kilometers above the equator, beam it to Earth as microwaves, and convert it to electricity—still remains tantalizingly over the horizon. Several recently published deep-dive analyses commissioned by NASA and the European Space Agency have thrown cold water on the hope that space solar power could affordably generate many gigawatts of clean energy in the near future. And yet the dream lives on. The dream achieved a kind of lift-off in January 2023. That’s when SSPD-1, a solar space-power demonstrator satellite carrying a bevy of new technologies designed at the California Institute of Technology, blasted into low Earth orbit for a year-long mission. Mindful of concerns about the technical feasibility of robotic in-space assembly of satellites, each an order of magnitude larger than the International Space Station, the Caltech team has been looking at very different approaches to space solar power. For an update on what the SSPD-1 mission achieved and how it will shape future concepts for space solar-power satellites, IEEE Spectrum spoke with Ali Hajimiri, an IEEE Fellow, professor of electrical engineering at Caltech, and codirector of the school’s space-based solar power project. The interview has been condensed and edited for length and clarity. SSPD-1 flew with several different testbeds. Let’s start with the MAPLE (Microwave Array for Power-transfer Low-orbit Experiment) testbed for wireless power transmission: When you and your team went up on the roof of your building on campus in May 2023 and aimed your antennas to where the satellite was passing over, did your equipment pick up actual power being beamed down or just a diagnostic signal? Ali Hajimiri is the codirector of Caltech’s space-based solar power project.Caltech Ali Hajimiri: I would call it a detection. The primary purpose of the MAPLE experiment was to demonstrate wireless energy transfer in space using flexible, lightweight structures and also standard CMOS integrated circuits. On one side are the antennas that transmit the power, and on the flip side are our custom CMOS chips that are part of the power-transfer electronics. The point of these things is to be very lightweight, to reduce the cost of launch into space, and to be very flexible for storage and deployment, because we want to wrap it and unwrap it like a sail. I see—wrap them up to fit inside a rocket and then unwrap and stretch them flat once they are released into orbit. Hajimiri: MAPLE’s primary objective was to demonstrate that these flimsy-looking arrays and CMOS integrated circuits can operate in space. And not only that, but that they can steer wireless energy transfer to different targets in space, different receivers. And by energy transfer I mean net power out at the receiver side. We did demonstrate power transfer in space, and we made a lot of measurements. We are writing up the details now and will publish those results. The second part of this experiment—really a stretch goal—was to demonstrate that ability to point the beam to the right place on Earth and see whether we picked up the expected power levels. Now, the larger the transmission array is in space, the greater the ability to focus the energy to a smaller spot on the ground. Right, because diffraction of the beam limits the size of the spot, as a function of the transmitter size and the frequency of the microwaves. Hajimiri: Yes. The array we had in space for MAPLE was very small. As a result, the transmitter spread the power over a very large area. So we captured a very small fraction of the energy—that’s why I call it a detection; it was not net positive power. But we measured it. We wanted to see: Do we get what we predict from our calculations? And we found it was in the right range of power levels we expected from an experiment like that. So, comparable in power to the signals that come down in standard communication satellite operations. Hajimiri: But done using this flexible, lightweight system—that’s what makes it better. You can imagine developing the next generation of communication satellites or space-based sensors being built with these to make the system significantly cheaper and lighter and easier to deploy. The satellites used now for Starlink and Kuiper—they work great, but they are bulky and heavy. With this technology for the next generation, you could deploy hundreds of them with a very small and much cheaper launch. It could lead to a much more effective Internet in the sky. Tell me about ALBA, the experiment on the mission that tested 32 different and novel kinds of photovoltaic solar cells to see how they perform in space. What were the key takeaways? Hajimiri: My Caltech colleague Harry Atwater led that experiment. What works best on Earth is not necessarily what works best in space. In space there is a lot of radiation damage, and they were able to measure degradation rates over months. On the other hand, there is no water vapor in space, no air oxidation, which is good for materials like perovskites that have problems with those things. So Harry and his team are exploring the trade-offs and developing a lot of new cells that are much cheaper and lighter: Cells made with thin films of perovskites or semiconductors like gallium arsenide, cells that use quantum dots, or use waveguides or other optics to concentrate the light. Many of these cells show very large promise. Very thin layers of gallium arsenide, in particular, seem very conducive to making cells that are lightweight but very high performance and much lower in cost because they need very little semiconductor material. Many of the design concepts for solar-power satellites, including one your group published in a 2022 preprint, incorporate concentrators to reduce the amount of photovoltaic area and mass needed. Hajimiri: A challenge with that design is the rather narrow acceptance angle: Things have to be aligned just right so that the focused sunlight hits the cell properly. That’s one of the reasons we’ve pulled away from that approach and moved toward a flat design. A view from inside MAPLE: On the right is the array of flexible microwave power transmitters, and on the left are receivers they transmit that power to.Caltech There are some other major differences between the Caltech power satellite design and the other concepts out there. For example, the other designs I’ve seen would use microwaves in the Wi-Fi range, between 2 and 6 gigahertz, because cheap components are available for those frequencies. But yours is at 10 GHz? Hajimiri: Exactly—and it’s a major advantage because when you double the frequency, the size of the systems in space and on the ground go down by a factor of four. We can do that basically because we build our own microchips and have a lot of capabilities in millimeter-wave circuit design. We’ve actually demonstrated some of these flexible panels that work at 28 GHz. And your design avoids the need for robots to do major assembly of components in space? Hajimiri: Our idea is to deploy a fleet of these sail-like structures that then all fly in close formation. They are not attached to each other. That translates to a major cost reduction. Each one of them has little thrusters on the edges, and it contains internal sensors that let it measure its own shape as it flies and then correct the phase of its transmission accordingly. Each would also track its own position relative to the neighbors and its angle to the sun. From your perspective as an electrical engineer, what are the really hard problems still to be solved? Hajimiri: Time synchronization between all parts of the transmitter array is incredibly crucial and one of the most interesting challenges for the future. Because the transmitter is a phased array, each of the million little antennas in the array has to synchronize precisely with the phase of its neighbors in order to steer the beam onto the receiver station on the ground. Hajimiri: Right. To give you a sense of the level of timing precision that we need across an array like this: We have to reduce phase noise and timing jitter to just a few picoseconds across the entire kilometer-wide transmitter. In the lab, we do that with wires of precise length or optical fibers that feed into CMOS chips with photodiodes built into them. We have some ideas about how to do that wirelessly, but we have no delusions: This is a long journey. What other challenges loom large? Hajimiri: The enormous scale of the system and the new manufacturing infrastructure needed to make it is very different from anything humanity has ever built. If I were to rank the challenges, I would put getting the will, resources, and mindshare behind a project of this magnitude as number one.

Last December, the AI Institute announced that it was opening an office in Zurich as a European counterpart to its Boston headquarters and recruited Marco Hutter to helm the office. Hutter also runs the Robotic Systems Lab at ETH Zurich, arguably best known as the origin of the ANYmal quadruped robot (but it also does tons of other cool stuff). We’re doing our best to keep close tabs on the institute, because it’s one of a vanishingly small number of places that currently exist where roboticists have the kind of long-term resources and vision necessary to make substantial progress on really hard problems that aren’t quite right for either industry or academia. The institute is still scaling up (and the branch in Zurich has only just kicked things off), but we did spot some projects that the Boston folks have been working on, and as you can see from the clips at the top of this page, they’re looking pretty cool. Meanwhile, we had a chance to check in with Marco Hutter to get a sense of what the Zurich office will be working on and how he’s going to be solving all of the hard problems in robotics. All of them! How much can you tell us about what you’ll be working on at the AI Institute? Marco Hutter: If you know the research that I’ve been doing in the past at ETH and with our startups, there’s an overlap on making systems more mobile, making systems more able to interact with the world, making systems in general more capable on the hardware and software side. And that’s what the institute strives for. The institute describes itself as a research organization that aims to solve the most important and fundamental problems in robotics and AI. What do you think those problems are? Marco Hutter is the head of the AI Institute’s new Zurich branch.Swiss Robotics Day Hutter: There are lots of problems. If you’re looking at robots today, we have to admit that they’re still pretty stupid. The way they move, their capability of understanding their environment, the way they’re able to interact with unstructured environments—I think we’re still lacking a lot of skills on the robotic side to make robots useful in all of the tasks we wish them to do. So we have the ambition of having these robots taking over all these dull, dirty, and dangerous jobs. But if we’re honest, today the biggest impact is really only for the dull part. And I think these dirty and dangerous jobs, where we really need support from robots, that’s still going to take a lot of fundamental work on the robotics and AI side to make enough progress for robots to become useful tools. What is it about the institute that you think will help robotics make more progress in these areas? Hutter: I think the institute is one of these unique places where we are trying to bring the benefits of the academic world and the benefits from this corporate world together. In academia, we have all kinds of crazy ideas and we try to develop them in all different directions, but at the same time, we have limited engineering support, and we can only go so far. Making robust and reliable hardware systems is a massive effort, and that kind of engineering is much better done in a corporate lab. You’ve seen this a little bit with the type of work my lab has been doing in the past. We built simple quadrupeds with a little bit of mobility, but in order to make them robust, we eventually had to spin it out. We had to bring it to the corporate world, because for a research group, a pure academic group, it would have been impossible. But at the same time, you’re losing something, right? Once you go into your corporate world and you’re running a business, you have to be very focused; you can’t be that explorative and free anymore. So if you bring these two things together through the institute, with long-term planning, enough financial support, and brilliant people both in the U.S. and Europe working together, I think that’s what will hopefully help us make significant progress in the next couple of years. “We’re very different from a traditional company, where at some point you need to have a product that makes money. Here, it’s really about solving problems and taking the next step.” —Marco Hutter, AI Institute And what will that actually mean in the context of dynamically mobile robots? Hutter: If you look at Boston Dynamics’ Atlas doing parkour, or ANYmal doing parkour, these are still demonstrations. You don’t see robots running around in the forests or robots working in mines and doing all kinds of crazy maintenance operations, or in industrial facilities, or construction sites, you name it. We need to not only be able to do this once as a prototype demonstration, but to have all the capabilities that bring that together with environmental perception and understanding to make this athletic intelligence more capable and more adaptable to all kinds of different environments. This is not something that from today to tomorrow we’re going to see it being revolutionized—it will be gradual, steady progress because I think there’s still a lot of fundamental work that needs to be done. I feel like the mobility of legged robots has improved a lot over the last five years or so, and a lot of that progress has come from Boston Dynamics and also from your lab. Do you feel the same? Hutter: There has always been progress; the question is how much you can zoom in or zoom out. I think one thing has changed quite a bit, and that’s the availability of robotic systems to all kinds of different research groups. If you look back a decade, people had to build their own robots, they had to do the control for the robots, they had to work on the perception for the robots, and putting everything together like that makes it extremely fragile and very challenging to make something that works more than once. That has changed, which allows us to make faster progress. Marc Raibert (founder of the AI Institute) likes to show videos of mountain goats to illustrate what robots should be (or will be?) capable of. Does that kind of thing inspire you as well? Hutter: If you look at the animal kingdom, there’s so many things you can draw inspiration from. And a lot of this stuff is not only the cognitive side; it’s really about pairing the cognitive side with the mechanical intelligence of things like the simple-seeming hooves of mountain goats. But they’re really not that simple, they’re pretty complex in how they interact with the environment. Having one of these things and not the other won’t allow the animal to move across its challenging environment. It’s the same thing with the robots. It’s always been like this in robotics, where you push on the hardware side, and your controls become better, so you hit a hardware limitation. So both things have to evolve hand in hand. Otherwise, you have an over-dimensioned hardware system that you can’t use because you don’t have the right controls, or you have very sophisticated controls and your hardware system can’t keep up. How do you feel about all of the investment into humanoids right now, when quadrupedal robots with arms have been around for quite a while? Hutter: There’s a lot of ongoing research on quadrupeds with arms, and the nice thing is that these technologies that are developed for mobile systems with arms are the same technologies that are used in humanoids. It’s not different from a research point of view, it’s just a different form factor for the system. I think from an application point of view, the story from all of these companies making humanoids is that our environment has been adapted to humans quite a bit. A lot of tasks are at the height of a human standing, right? A quadruped doesn’t have the height to see things or to manipulate things on a table. It’s really application dependent, and I wouldn’t say that one system is better than the other.

Rapid and resourceful technological improvisation has long been a mainstay of warfare, but the war in Ukraine is taking it to a new level. This improvisation is most conspicuous in the ceaselessly evolving struggle between weaponized drones and electronic warfare, a cornerstone of this war. Weaponized civilian first-person-view (FPV) drones began dramatically reshaping the landscape of the war in the summer of 2023. Prior to this revolution, various commercial drones played critical roles, primarily for intelligence, surveillance, and reconnaissance. Since 2014, the main means of defending against these drones has been electronic warfare (EW), in its many forms. The iterative, lethal dance between drones and EW has unfolded a rich technological tapestry, revealing insights into a likely future of warfare where EW and drones intertwine. After the invasion of Crimea, in 2014, Ukrainian forces depended heavily on commercial off-the-shelf drones, such as models from DJI, for reconnaissance and surveillance. These were not FPV drones, for the most part. Russia’s response involved deploying military-grade EW systems alongside law-enforcement tools like Aeroscope, a product from DJI that allows instant identification and tracking of drones from their radio emissions. Aeroscope, while originally a standard tool used by law enforcement to detect and track illegal drone flights, soon revealed its military potential by pinpointing both the drone and its operator. On both sides of the line you’ll find much the same kind of people doing much the same thing: hacking. This application turned a security feature into a significant tactical asset, providing Russian artillery units with precise coordinates for their targets—namely, Ukrainian drone operators. To circumvent this vulnerability, groups of Ukrainian volunteers innovated. By updating the firmware of the DJI drones, they closed the backdoors that allowed the drones to be tracked by Aeroscope. Nevertheless, after the start of the conflict in Crimea, commercial, off-the-shelf drones were considered a last-resort asset used by volunteers to compensate for the lack of proper military systems. To be sure, the impact of civilian drones during this period was not comparable to what occurred after the February 2022 invasion. As Russia’s “thunder-run” strategy became bogged down shortly after the invasion, Russian forces found themselves unexpectedly vulnerable to civilian drones, in part because most of their full-scale military EW systems were not very mobile. During a training exercise in southern Ukraine in May 2023, a drone pilot maneuvered a flier to a height of 100 meters before dropping a dummy anti-tank grenade on to a pile of tires. The test, pictured here, worked—that night the pilot’s team repeated the exercise over occupied territory, blowing up a Russian armored vehicle. Emre Caylak/Guardian/eyevine/Redux The Russians could have compensated by deploying many Aeroscope terminals then, but they didn’t, because most Russian officers at the time had a dismissive view of the capabilities of civilian drones in a high-intensity conflict. That failure opened a window of opportunity that Ukrainian armed-forces units exploited aggressively. Military personnel, assisted by many volunteer technical specialists, gained a decisive intelligence advantage for their forces by quickly fielding fleets of hundreds of camera drones connected to simple yet effective battlefield-management systems. They soon began modifying commercial drones to attack, with grenade tosses and, ultimately, “kamikaze” operations. Besides the DJI models, one of the key drones was the R18, an octocopter developed by the Ukrainian company Aerorozvidka, capable of carrying three grenades or small bombs. As casualties mounted, Russian officers soon realized the extent of the threat posed by these drones. How Russian electronic warfare evolved to counter the drone threat By spring 2023, as the front lines stabilized following strategic withdrawals and counteroffensives, it was clear that the nature of drone warfare had evolved. Russian defenses had adapted, deploying more sophisticated counter-drone systems. Russian forces were also beginning to use drones, setting the stage for the nuanced cat-and-mouse game that has been going on ever since. The modular construction of first-person-view drones allowed for rapid evolution to enhance their resilience against electronic warfare. For example, early on, most Russian EW efforts primarily focused on jamming the drones’ radio links for control and video. This wasn’t too hard, given that DJI’s OcuSync protocol was not designed to withstand dense jamming environments. So by April 2023, Ukrainian drone units had begun pivoting toward first-person-view (FPV) drones with modular construction, enabling rapid adaptation to, and evasion of, EW countermeasures. The Russian awakening to the importance of drones coincided with the stabilization of the front lines, around August 2022. Sluggish Russian offensives came at a high cost, with an increasing proportion of casualties caused directly or indirectly by drone operators. By this time, the Ukrainians were hacking commercial drones, such as DJI Mavics, to “anonymize” them, rendering Aeroscope useless. It was also at this time that the Russians began to adopt commercial drones and develop their own tactics, techniques, and procedures, leveraging their EW and artillery advantages while attempting to compensate for their delay in combat-drone usage. On 4 March, a Ukrainian soldier flew a drone at a testing site near the town of Kreminna in eastern Ukraine. The drone was powered by a blue battery pack and carried a dummy bomb.David Guttenfelder/The New York Times/Redux Throughout 2023, when the primary EW tactic employed was jamming, the DJI drones began to fall out of favor for attack roles. When the density of Russian jammer usage surpassed a certain threshold, DJI’s OcuSync radio protocol, which controls a drone’s flight direction and video, could not cope with it. Being proprietary, OcuSync’s frequency band and power are not modifiable. A jammer can attack both the control and video signals, and the drone becomes unrecoverable most of the time. As a result, DJI drones have lately been used farther from the front lines and relegated mainly to roles in intelligence, surveillance, and reconnaissance. Meanwhile, the modular construction of FPVs allowed for rapid evolution to enhance their resilience against EW. The Ukraine war greatly boosted the world’s production of FPV drones; at this point there are thousands of FPV models and modifications. A “kamikaze” first-person-view drone with an attached PG-7L round, intended for use with an RPG-7 grenade launcher, is readied for a mission near the town of Horlivka, in the Donetsk region, on 17 January 2024. The drone was prepared by a Ukrainian serviceman of the Rarog UAV squadron of the 24th Separate Mechanized Brigade.Inna Varenytsia/Reuters/Redux As of early 2024, analog video signals are the most popular option by far. This technology offers drone operators a brief window of several seconds to correct the drone’s path upon detecting interference, for example as a result of jamming, before signal loss. Additionally, drone manufacturers have access to more powerful video transmitters, up to 5 watts, which are more resistant to jamming. Furthermore, the 1.2-gigahertz frequency band is gaining popularity over the previously dominant 5.8-GHz band due to its superior obstacle penetration and because fewer jammers are targeting that band. However, the lack of encryption in analog video transmitter systems means that a drone’s visual feed can be intercepted by any receiver. So various mitigation strategies have been explored. These include adding encryption layers and using digital-control and video protocols such as HDZero, Walksnail, or, especially, any of several new open-source alternatives. In the war zone, the most popular of these open-source control radio protocols is ExpressLRS, or ELRS. Being open-source, ELRS not only offers more affordable hardware than its main rival, TBS Crossfire, it is also modifiable via its software. It has been hacked in order to use frequency bands other than its original 868 to 915 megahertz. This adaptation produces serious headaches for EW operators, because they have to cover a much wider band. As of March 2024, Ukrainian drone operators are performing final tests on 433-MHz ELRS transmitter-receiver pairs, further challenging prevailing EW methods. Distributed mass in the transparent battlefield Nevertheless, the most important recent disruption of all in the drone-versus-EW struggle is distributed mass. Instead of an envisioned blitzkrieg-style swarm with big clouds of drones hitting many closely spaced targets during very short bursts, an ever-growing number of drones are covering more widely dispersed targets over a much longer time period, whenever the weather is conducive. Distributed mass is a cornerstone of the emerging transparent battlefield, in which many different sensors and platforms transmit huge amounts of data that is integrated in real time to provide a comprehensive view of the battlefield. One offshoot of this strategy is that more and more kamikaze drones are directed toward a constantly expanding range of targets. Electronic warfare is adapting to this new reality, confronting mass with mass: massive numbers of drones against massive numbers of RF sensors and jammers. Ukraine is the first true war of the hackers. Attacks now often consist of far more commercial drones than a suite of RF detectors or jammers could handle even six months ago. With brute-force jamming, even if defenders are willing to accept high rates of damage inflicted on their own offensive drones, these previous EW systems are just not up to the task. So for now, at least, the drone hackers are in the lead in this deadly game of “hacksymmetrical” warfare. Their development cycle is far too rapid for conventional electronic warfare to keep pace. But the EW forces are not standing still. Both sides are either developing or acquiring civilian RF-detecting equipment, while military-tech startups and even small volunteer groups are developing new, simple, and good-enough jammers in essentially the same improvised ways that hackers would. Ukrainian soldiers familiarized themselves with a portable drone jammer during a training session in Kharkiv, Ukraine, on 11 March 2024.Diego Herrera Carcedo/Anadolu/Getty Images Two examples illustrate this trend. Increasingly affordable, short-range jammers are being installed on tanks, armored personnel carriers, trucks, pickups, and even 4x4s. Although limited and unsophisticated, these systems contribute to drone-threat mitigation. In addition, a growing number of soldiers on the front line carry simple, commercial radio-frequency (RF) scanners with them. Configured to detect drones across various frequency bands, these devices, though far from perfect, have begun to save lives by providing precious additional seconds of warning before an imminent drone attack. The electronic battlefield has now become a massive game of cat and mouse. Because commercial drones have proven so lethal and disruptive, drone operators have become high-priority targets. As a result, operators have had to reinvent camouflage techniques, while the hackers who drive the evolution of their drones are working on every modification of RF equipment that offers an advantage. Besides the frequency-band modification described above, hackers have developed and refined two-way, two-signal repeaters for drones. Such systems are attached to another drone that hovers close to the operator and well above the ground, relaying signals to and from the attacking drone. Such repeaters more than double the practical range of drone communications, and thus the EW “cats” in this game have to search a much wider area than before. Hackers and an emerging cottage industry of war startups are raising the stakes. Their primary goal is to erode the effectiveness of jammers by attacking them autonomously. In this countermeasure, offensive drones are equipped with home-on-jam systems. Over the next several months, increasingly sophisticated versions of these systems will be fielded. These home-on-jam capabilities will autonomously target any jamming emission within range; this range, which is classified, depends on emission power at a rate that is believed to be 0.3 kilometers per watt. In other words, if a jammer has 100 W of signal power, it can be detected up to 30 km away, and then attacked. After these advances allow the drone “mice” to hunt the EW cat, what will happen to the cat? The challenge is unprecedented and the outcome uncertain. But on both sides of the line you’ll find much the same kind of people doing much the same thing: hacking. Civilian hackers have for years lent their skills to such shady enterprises as narco-trafficking and organized crime. Now hacking is a major, indispensable component of a full-fledged war, and its practitioners have emerged from a gray zone of plausible deniability into the limelight of military prominence. Ukraine is the first true war of the hackers. The implications for Western militaries are ominous. We have neither masses of drones nor masses of EW tech. What is worse, the world’s best hackers are completely disconnected from the development of defense systems. The Ukrainian experience, where a vibrant war startup scene is emerging, suggests a model for integrating maverick hackers into our defense strategies. As the first hacker war continues to unfold, it serves as a reminder that in the era of electronic and drone warfare, the most critical assets are not just the technologies we deploy but also the scale and the depth of the human ingenuity behind them.

No matter where professionals are in their tech career—whether just starting out or well established—it’s never a bad time for them to reassess their skills to ensure they are aligned with market needs. As the professional home for engineers and technical professionals, IEEE offers a wealth of career-development resources. To showcase them, from 14 to 20 April the organization is holding its annual Education Week. The event highlights the array of educational opportunities, webinars, online courses, activities, and scholarships provided by IEEE’s organizational units, societies, and councils around the globe. Individuals can participate in IEEE Education Week by exploring dozens of live and virtual events. Here are a few highlights: IEEE: Educating for the Future. Tom Coughlin, IEEE’s president and CEO, kicks off the week on 15 April with a keynote presentation at noon EDT. Coughlin’s priorities include retaining younger members, engaging industry, developing workforce programs, and focusing on the future of education. Investing in Your Future: The Importance of Continuing Education for Engineers. At 11 a.m. on 18 April, learn about the IEEE Professional Development Suite of specialized business and leadership training programs. Essential Business Skills for Engineers: Bridging the Gap Between Business and Engineering. Join IEEE and representatives from the Rutgers Business School to learn how engineers and technical professionals can grow their career through management training. This event—to be held at 10 a.m. on 16 April Singapore Standard Time and 10 p.m EDT on 17 April—is primarily for engineering professionals in the Asia Pacific region. Attendees will be introduced to the IEEE | Rutgers Online Mini-MBA Program for Engineers program. Add Value and Attendees to Your Events With IEEE Credentialing. Learn about the benefits of IEEE digital certificates and badges at noon EDT on 17 April. The session covers how to find events that offer professional development hours and continuing education units. IEEE–Eta Kappa Nu 2024 TechX. The honor society’s three-day virtual event, 17 to 19 April, addresses opportunities and challenges presented by new technology, along with Q&A sessions with experts. TechX includes a virtual job fair and networking events. What You Should Know About the IEEE Learning Network. At noon EDT on 16 April, learn how the platform can help you advance your career with eLearning courses on that cover emerging technologies. Best Practices for Service Learning From Past EPICS in IEEE Project Leaders. Leah Jamieson, the 2007 IEEE president, is set to lead a panel discussion on the IEEE Engineering Projects in Community Service program at 9:30 a.m. on 16 April. Jamieson, who helped found EPICS at Purdue University, and other project leaders will share their experiences. TryEngineering and Keysight: Inspiring the Engineers of Tomorrow. IEEE and Keysight Technologies, a manufacturer of electronics test and measurement equipment and software, recently partnered to develop lesson plans on electronics and the power of simulations. Learn more about the program at 10:30 a.m. on 17 April. Global Semiconductors: IEEE Resources and Communities for Those Working in the Semiconductor Industry. This session, at 1 p.m. on 18 April, can explain which IEEE groups offer educational materials for semiconductor engineers. Offers and discounts The Education Week website lists special offers and discounts. The IEEE Learning Network, for example, is offering some of its most popular courses for US $10 each. They cover artificial intelligence standards, configuration management, the Internet of Things, smart cities, and more. You can use the code ILNIEW24 until 30 April. Be sure to complete the IEEE Education Week quiz by noon EDT on 20 April for a chance to earn an IEEE Education Week 2024 digital badge, which can be displayed on social media. To learn more about IEEE Education Week, watch this video or follow the event on Facebook or X.

Although the race to power the massive ambitions of AI companies might seem like it’s all about Nvidia, there is a real competition going in AI accelerator chips. The latest example: At Intel’s Vision 2024 event this week in Phoenix, Ariz., the company gave the first architectural details of its third-generation AI accelerator, Gaudi 3. With the predecessor chip, the company had touted how close to parity its performance was to Nvidia’s top chip of the time, H100, and claimed a superior ratio of price versus performance. With Gaudi 3, it’s pointing to large-language-model (LLM) performance where it can claim outright superiority. But, looming in the background is Nvidia’s next GPU, the Blackwell B200, expected to arrive later this year. Gaudi Architecture Evolution Gaudi 3 doubles down on its predecessor Gaudi 2’s architecture, literally in some cases. Instead of Gaudi 2’s single chip, Gaudi 3 is made up of two identical silicon dies joined by a high-bandwidth connection. Each has a central region of 48 megabytes of cache memory. Surrounding that are the chip’s AI workforce—four engines for matrix multiplication and 32 programmable units called tensor processor cores. All that is surrounded by connections to memory and capped with media processing and network infrastructure at one end. Intel says that all that combines to produce double the AI compute of Gaudi 2 using 8-bit floating-point infrastructure that has emerged as key to training transformer models. It also provides a fourfold boost for computations using the BFloat 16 number format. Gaudi 3 LLM Performance Intel projects a 40 percent faster training time for the GPT-3 175B large language model versus the H100 and even better results for the 7-billion and 8-billion parameter versions of Llama2. For inferencing, the contest was much closer, according to Intel, where the new chip delivered 95 to 170 percent of the performance of H100 for two versions of Llama. Though for the Falcon 180B model, Gaudi 3 achieved as much as a fourfold advantage. Unsurprisingly, the advantage was smaller against the Nvidia H200—80 to 110 percent for Llama and 3.8x for Falcon. Intel claims more dramatic results when measuring power efficiency, where it projects as much as 220 percent H100’s value on Llama and 230 percent on Falcon. “Our customers are telling us that what they find limiting is getting enough power to the data center,” says Intel’s Habana Labs chief operating officer Eitan Medina. The energy-efficiency results were best when the LLMs were tasked with delivering a longer output. Medina puts that advantage down to the Gaudi architecture’s large-matrix math engines. These are 512 bits across. Other architectures use many smaller engines to perform the same calculation, but Gaudi’s supersize version “needs almost an order of magnitude less memory bandwidth to feed it,” he says. Gaudi 3 Versus Blackwell It’s speculation to compare accelerators before they’re in hand, but there are a couple of data points to compare, particular in memory and memory bandwidth. Memory has always been important in AI, and as generative AI has taken hold and popular models reach the tens of billions of parameters in size it’s become even more critical. Both make use of high-bandwidth memory (HBM), which is a stack of DRAM memory dies atop a control chip. In high-end accelerators, it sits inside the same package as the logic silicon, surrounding it on at least two sides. Chipmakers use advanced packaging, such as Intel’s EMIB silicon bridges or TSMC’s chip-on-wafer-on-silicon (CoWoS), to provide a high-bandwidth path between the logic and memory. As the chart shows, Gaudi 3 has more HBM than H100, but less than H200, B200, or AMD’s MI300. It’s memory bandwidth is also superior to H100’s. Possibly of importance to Gaudi’s price competitiveness, it uses the less expensive HBM2e versus the others’ HBM3 or HBM3e, which are thought to be a significant fraction of the tens of thousands of dollars the accelerators reportedly sell for. One more point of comparison is that Gaudi 3 is made using TSMC’s N5 (sometimes called 5-nanometer) process technology. Intel has basically been a process node behind Nvidia for generations of Gaudi, so it’s been stuck comparing its latest chip to one that was at least one rung higher on the Moore’s Law ladder. With Gaudi 3, that part of the race is narrowing slightly. The new chip uses the same process as H100 and H200. What’s more, instead of moving to 3-nm technology, the coming competitor Blackwell is done on a process called N4P. TSMC describes N4P as being in the same 5-nm family as N5 but delivering an 11 percent performance boost, 22 percent better efficiency, and 6 percent higher density. In terms of Moore’s Law, the big question is what technology the next generation of Gaudi, currently code-named Falcon Shores, will use. So far the product has relied on TSMC technology while Intel gets its foundry business up and running. But next year Intel will begin offering its 18A technology to foundry customers and will already be using 20A internally. These two nodes bring the next generation of transistor technology, nanosheets, with backside power delivery, a combination TSMC is not planning until 2026.

I’ve enjoyed reading magazine articles about Ethernet’s 50th anniversary, including one in the The Institute. Invented by computer scientists Robert Metcalfe and David Boggs, Ethernet has been extraordinarily impactful. Metcalfe, an IEEE Fellow, received the 1996 IEEE Medal of Honor as well as the 2022 Turing Award from the Association for Computing Machinery for his work. But there is more to the story that is not widely known. During the 1980s and early 1990s, I led Digital Equipment Corp.’s networking advanced development group in Massachusetts. I was a firsthand witness in what was a period of great opportunity for LAN technologies and intense competition between standardization efforts. DEC, Intel, and Xerox poised themselves to profit from Ethernet’s launch in the 1970s. But during the 1980s other LAN technologies emerged as competitors. Prime contenders included the token ring, promoted by IBM, and the token bus. (Today Ethernet and both token-based technologies are part of the IEEE 802 family of standards.) All those LANs have some basic parts in common. One is the 48-bit media access control (MAC) address, a unique number assigned during a computer’s network port manufacturing process. The MAC addresses are used inside the LAN only, but they are critical to its operation. And usually, along with the general-purpose computers on the network, they have at least one special-purpose computer: a router, whose main job is to send data to—and receive it from—the Internet on behalf of all the other computers on the LAN. In a decades-old conceptual model of networking, the LAN itself (the wires and low-level hardware) is referred to as Layer 2, or the data link layer. Routers mostly deal with another kind of address: a network address that is used both within the LAN and outside it. Many readers likely have heard the terms Internet Protocol and IP address. With some exceptions, the IP address (a network address) in a data packet is sufficient to ensure that packet can be delivered anywhere on the Internet by a sequence of other routers operated by service providers and carriers. Routers and the operations they perform are referred to as Layer 3, or the network layer. In a token ring LAN, shielded twisted-pair copper wires connect each computer to its upstream and downstream neighbors in an endless ring structure. Each computer forwards data from its upstream neighbor to its downstream one but can send its own data to the network only after it receives a short data packet—a token—from the upstream neighbor. If it has no data to transmit, it just passes the token to its downstream neighbor, and so on. In a token bus LAN, a coaxial cable connects all the network’s computers, but the wiring doesn’t control the order in which the computers pass the token. The computers agree on the sequence in which they pass the token, forming an endless virtual ring around which data and tokens circulate. Ethernet, meanwhile, had become synonymous with coaxial cable connections that used a method called carrier sense multiple access with collision detection for managing transmissions. In the CSMA/CD method, computers that want to transmit a data packet first listen to see if another computer is transmitting. If not, the computer sends its packet while listening to determine whether that packet collides with one from another computer. Collisions can happen because signal propagation between computers is not instantaneous. In the case of a collision, the sending computer resends its packet with a delay that has both a random component and an exponentially increasing component that depends on the number of collisions. The need to detect collisions involves tradeoffs among data rate, physical length, and minimum packet size. Increasing the data rate by an order of magnitude means either reducing the physical length or increasing the minimum packet size by roughly the same factor. The designers of Ethernet had wisely chosen a sweet spot among the tradeoffs: 10 megabits per second and a length of 1,500 meters. A threat from fiber Meanwhile, a coalition of companies—including my employer, DEC—was developing a new ANSI LAN standard: the Fiber Distributed Data Interface. The FDDI approach used a variation of the token bus protocol to transmit data over optical fiber, promising speeds of 100 Mb/s, far faster than Ethernet’s 10 Mb/s. A barrage of technical publications released analyses of the throughputs and latencies of competing LAN technologies under various workloads. Given the results and the much greater network performance demands expected from speedier processors, RAM, and nonvolatile storage, Ethernet’s limited performance was a serious problem. FDDI seemed a better bet for creating higher speed LANs than Ethernet, although FDDI used expensive components and complex technology, especially for fault recovery. But all shared media access protocols had one or more unattractive features or performance limitations, thanks to the complexity involved in sharing a wire or optical fiber. A solution emerges I thought that a better approach than either FDDI or a faster version of Ethernet would be to develop a LAN technology that performed store-and-forward switching. One evening in 1983, just before leaving work to go home, I visited the office of Mark Kempf, a principal engineer and a member of my team. Mark, one of the best engineers I have ever worked with, had designed the popular and profitable DECServer 100 terminal server, which used the local-area transport (LAT) protocol created by Bruce Mann from DEC’s corporate architecture group. Terminal servers connect groups of dumb terminals, with only RS-232 serial ports, to computer systems with Ethernet ports. I told Mark about my idea of using store-and-forward switching to increase LAN performance. The next morning he came in with an idea for a learning bridge (also known as a Layer 2 switch or simply a switch). The bridge would connect to two Ethernet LANs. By listening to all traffic on each LAN, the device would learn the MAC addresses of the computers on both Ethernets (remembering which computer was on which Ethernet) and then selectively forward the appropriate packets between the LANs based upon the destination MAC address. The computers on the two networks didn’t need to know which path their data would take on the extended LAN; to them, the bridge was invisible. The bridge would need to receive and process some 30,000 packets per second (15,000 pp/s per Ethernet) and decide whether to forward each one. Although the 30,000 pp/s requirement was near the limit of what could be done using the best microprocessor technology of the time, the Motorola 68000, Mark was confident he could build a two-Ethernet bridge using only off-the-shelf components including a specialized hardware engine he would design using programmable array logic (PAL) devices and dedicated static RAM to look up the 48-bit MAC addresses. Mark’s contributions have not been widely recognized. One exception is the textbook Network Algorithmics by George Varghese. In a misconfigured network—one with bridges connecting Ethernets in a loop—packets could circulate forever. We felt confident that we could figure out a way to prevent that. In a pinch, a product could ship without the safety feature. And clearly a two-port device was only the starting point. Multiple-port devices could follow, though they would require custom components. I took our idea to three levels of management, looking for approval to build a prototype of the learning bridge that Mark envisioned. Before the end of the day, we had a green light with the understanding that a product would follow if the prototype was successful. Developing the bridge My immediate manager at DEC, Tony Lauck, challenged several engineers and architects to solve the problem of packet looping in misconfigured networks. Within a few days, we had several potential solutions. Radia Perlman, an architect in Tony’s group, provided the clear winner: the spanning tree protocol. In Perlman’s approach, the bridges detect each other, select a root bridge according to specified criteria, and then compute a minimum spanning tree. An MST is a mathematical structure that, in this case, describes how to efficiently connect LANs and bridges without loops. The MST was then used to place any bridge whose presence would create a loop into backup mode. As a side benefit, it provided automated recovery in the case of a bridge failure. The logic module of a disassembled LANBridge 100, which was released by Digital Equipment Corp. in 1986. Alan Kirby Mark designed the hardware and timing-sensitive low-level code, while software engineer Bob Shelly wrote the remaining programs. And in 1986, DEC introduced the technology as the LANBridge 100, product code DEBET-AA. Soon after, DEC developed DEBET-RC, a version that supported a 3-kilometer optical fiber span between bridges. Manuals for some of the DEBET-RCs can be found on the Bitsavers website. Mark’s idea didn’t replace Ethernet—and that was its brilliance. By allowing store-and-forward switching between existing CSMA/CD coax-based Ethernets, bridges allowed easy upgrades of existing LANs. Since any collision would not propagate beyond the bridge, connecting two Ethernets with a bridge would immediately double the length limit of a single Ethernet cable alone. More importantly, placing computers that communicated heavily with each other on the same Ethernet cable would isolate that traffic to that cable, while the bridge would still allow communication with computers on other Ethernet cables. That reduced the traffic on both cables, increasing capacity while reducing the frequency of collisions. Taken to its limit, it eventually meant giving each computer its own Ethernet cable, with a multiport bridge connecting them all. That is what led to a gradual migration away from CSMA/CD over coax to the now ubiquitous copper and fiber links between individual computers and a dedicated switch port. The speed of the links is no longer limited by the constraints of collision detection. Over time, the change completely transformed how people think of Ethernet. A bridge could even have ports for different LAN types if the associated packet headers were sufficiently similar. Our team later developed GIGAswitch, a multiport device supporting both Ethernet and FDDI. The existence of bridges with increasingly higher performance took the wind out of the sails of those developing new shared media LAN access protocols. FDDI later faded from the marketplace in the face of faster Ethernet versions. Bridge technology was not without controversy, of course. Some engineers continue to believe that Layer 2 switching is a bad idea and that all you need are faster Layer 3 routers to transfer packets between LANs. At the time, however, IP had not won at the network level, and DECNet, IBM’s SNA, and other network protocols were fighting for dominance. Switching at Layer 2 would work with any network protocol. Mark received a U.S. patent for the device in 1986. DEC offered to license it on a no-cost basis, allowing any company to use the technology. That led to an IEEE standardization effort. Established networking companies and startups adopted and began working to improve the switching technology. Other enhancements—including switch-specific ASICs, virtual LANs, and the development of faster and less expensive physical media and associated electronics—steadily contributed to Ethernet’s longevity and popularity. The lasting value of Ethernet lies not in CSMA/CD or its original coaxial media but in the easily understood and functional service that it provided for protocol designers. The switches in many home networks today are directly descended from the innovation. And modern data centers have numerous switches with individual ports running between 40 and 800 gigabits per second. The data center switch market alone accounts for more than US $10 billion in annual revenue. Lauck, my DEC manager, once said that the value of an architecture can be measured by the number of technology generations over which it is useful. By that measure, Ethernet has been enormously successful. The same can be said of Layer 2 switching. No one knows what would have happened to Ethernet had Mark not invented the learning bridge. Perhaps someone else would have come up with the idea. But it’s also possible that Ethernet would have slowly withered away. To me, Mark saved Ethernet.

You can’t see, hear, taste, feel, or smell it, but software is everywhere around us. It underpins modern civilization even while consuming more energy, wealth, and time than it needs to and burping out a significant amount of carbon dioxide into the atmosphere. The software industry and the code it ships need to be much more efficient in order to minimize the emissions attributable to programs running in data centers and over transmission networks. Two approaches to software development featured in Spectrum‘s April 2024 issue can help us get there. In “Why Bloat Is Still Software’s Biggest Vulnerability,” Bert Hubert pays homage to the famed computer scientist and inventor of Pascal, Niklaus Wirth, whose influential essay “A Plea for Lean Software” appeared in IEEE Computer in 1995. Wirth’s essay built on a methodology first conceived by Spectrum contributing editor Robert N. Charette, who in the early 1990s adapted the Toyota Production System for software development. Hubert points out that bloated code offers giant attack surfaces for bad actors. Malicious hacks and ransomware attacks, not to mention run-of-the-mill software failures, are like the weather now: partly cloudy with a 50 percent chance of your app crashing or your personal information being circulated on the Dark Web. Back in the day, limited compute resources forced programmers to write lean code. Now, with much more robust resources at hand, coders are writing millions of lines of code for relatively simple apps that call on hundreds of libraries of, as Hubert says, “unknown provenance.” “There’s an already existing large segment of the software-development ecosystem that cares about this space—they just haven’t known what to do.” —Asim Hussain, Green Web Foundation Among other things, he argues for legislation along the lines of what the European Union is trying to enforce: “NIS2 for important services; the Cyber Resilience Act for almost all commercial software and electronic devices; and a revamped Product Liability Directive that also extends to software.” Hubert, a software developer himself, walks the lean walk: His 3-megabyte image-sharing program Trifecta does the same job as other programs that use hundreds of megabytes of code. Lean software should, in theory, be green software. In other words, it should run so efficiently that it reduces the amount of energy used in data centers and transmission networks. Overall, the IT and communications sectors are estimated to account for 2 to 4 percent of global greenhouse gas emissions and, according to one 2018 study, could by 2040 reach 14 percent. And that study came out prior to the explosion in AI applications, whose insatiable hunger for computing resources and the power required to feed the algorithms exacerbates an already complicated problem. Thankfully, several groups are working on solutions, including the Green Web Foundation. The GWF was spun up almost 20 years ago to figure out how the Internet is powered, and now has a goal of a fossil-free Internet by 2030. There are three main ways to achieve that objective, according to the foundation’s chair and executive director Asim Hussain: Use less energy, use fewer physical resources, and use energy more prudently—by, for instance, having your apps do more when there’s power from wind and solar available and less when there’s not. “There’s an already existing large segment of the software-development ecosystem that cares about this space—they just haven’t known what to do,” Hussain told Spectrum contributing editor Rina Diane Caballar. They do now, thanks to Caballar’s extensive reporting and the handy how-to guide she includes in “We Need to Decarbonize Software.” Programmers have the tools to make software leaner and greener. Now it’s up to them, and as we’ve seen in the EU, their legislators, to make sustainable and secure code their top priority. Software doesn’t have to suck.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion. RoboCup German Open: 17–21 April 2024, KASSEL, GERMANY AUVSI XPONENTIAL 2024: 22–25 April 2024, SAN DIEGO Eurobot Open 2024: 8–11 May 2024, LA ROCHE-SUR-YON, FRANCE ICRA 2024: 13–17 May 2024, YOKOHAMA, JAPAN RoboCup 2024: 17–22 July 2024, EINDHOVEN, NETHERLANDS Enjoy today’s videos! USC, UPenn, Texas A&M, Oregon State, Georgia Tech, Temple University, and NASA Johnson Space Center are teaching dog-like robots to navigate craters of the moon and other challenging planetary surfaces in research funded by NASA. [ USC ] AMBIDEX is a revolutionary robot that is fast, lightweight, and capable of human-like manipulation. We have added a sensor head and the torso and the waist to greatly expand the range of movement. Compared to the previous arm-centered version, the overall impression and balance has completely changed. [ Naver Labs ] It still needs a lot of work, but the six-armed pollinator, Stickbug, can autonomously navigate and pollinate flowers in a greenhouse now. I think “needs a lot of work” really means “needs a couple more arms.” [ Paper ] Experience the future of robotics as UBTECH’s humanoid robot integrates with Baidu’s ERNIE through AppBuilder! Witness robots [that] understand language and autonomously perform tasks like folding clothes and object sorting. [ UBTECH ] I know the fins on this robot are for walking underwater rather than on land, but watching it move, I feel like it’s destined to evolve into something a little more terrestrial. [ Paper ] via [ HERO Lab ] iRobot has a new Roomba that vacuums and mops—and at $275, it’s a pretty good deal. Also, if you are a robot vacuum owner, please, please remember to clean the poor thing out from time to time. Here’s how to do it with a Roomba: [ iRobot ] The video demonstrates the wave-basin testing of a 43 kg (95 lb) amphibious cycloidal propeller unmanned underwater vehicle (Cyclo-UUV) developed at the Advanced Vertical Flight Laboratory, Texas A&M University. The use of cyclo-propellers allows for 360 degree thrust vectoring for more robust dynamic controllability compared to UUVs with conventional screw propellers. [ AVFL ] Sony is still upgrading Aibo with new features, like the ability to listen to your terrible music and dance along. [ Aibo ] Operating robots precisely and at high speeds has been a long-standing goal of robotics research. To enable precise and safe dynamic motions, we introduce a four degree-of-freedom (DoF) tendon-driven robot arm. Tendons allow placing the actuation at the base to reduce the robot’s inertia, which we show significantly reduces peak collision forces compared to conventional motor-driven systems. Pairing our robot with pneumatic muscles allows generating high forces and highly accelerated motions, while benefiting from impact resilience through passive compliance. [ Max Planck Institute ] Rovers on Mars have previously been caught in loose soils, and turning the wheels dug them deeper, just like a car stuck in sand. To avoid this, Rosalind Franklin has a unique wheel-walking locomotion mode to overcome difficult terrain, as well as autonomous navigation software. [ ESA ] Cassie is able to walk on sand, gravel, and rocks inside the Robot Playground at the University of Michigan. Aww, they stopped before they got to the fun rocks. [ Paper ] via [ Michigan Robotics ] Not bad for 2016, right? [ Namiki Lab ] MOMO has learned the Bam Yang Gang dance moves with its hand dexterity. 🙂 By analyzing 2D dance videos, we extract detailed hand skeleton data, allowing us to recreate the moves in 3D using a hand model. With this information, MOMO replicates the dance motions with its arm and hand joints. [ RILAB ] via [ KIMLAB ] This UPenn GRASP SFI Seminar is from Eric Jang at 1X Technologies, on “Data Engines for Humanoid Robots.” 1X’s mission is to create an abundant supply of physical labor through androids that work alongside humans. I will share some of the progress 1X has been making towards general-purpose mobile manipulation. We have scaled up the number of tasks our androids can do by combining an end-to-end learning strategy with a no-code system to add new robotic capabilities. Our Android Operations team trains their own models on the data they gather themselves, producing an extremely high-quality “farm-to-table” dataset that can be used to learn extremely capable behaviors. I’ll also share an early preview of the progress we’ve been making towards a generalist “World Model” for humanoid robots. [ UPenn ] This Microsoft Future Leaders in Robotics and AI Seminar is from Chahat Deep Singh at the University of Maryland, on “Minimal Perception: Enabling Autonomy in Palm-Sized Robots.” The solution to robot autonomy lies at the intersection of AI, computer vision, computational imaging, and robotics—resulting in minimal robots. This talk explores the challenge of developing a minimal perception framework for tiny robots (less than 6 inches) used in field operations such as space inspections in confined spaces and robot pollination. Furthermore, we will delve into the realm of selective perception, embodied AI, and the future of robot autonomy in the palm of your hands. [ UMD ]

The stretchable battery is gaining momentum in the electronics industry, where it might one day serve as an energy storage medium in fitness trackers, wearable electronics, and even smart clothing. Researchers believe the concept will become more valuable in the next decade, as electronic devices migrate closer and closer to human skin. “For many applications, such as wearables, stretchability is necessary since our skin stretches as we move,” said James Pikul, a professor of mechanical engineering at the University of Wisconsin–Madison. “A battery that only flexes would feel uncomfortable to wear.” A stretchable battery behaves like a rubber band, whereas flexible batteries are more like a piece of paper, which can bend but not stretch. Pikul and others around the world are now working on batteries that stretch. The new batteries differ from commonly known “flexible batteries” in that they withstand axial tension forces—longitudinal forces on a body that include tension and compression—and will stretch elastically when such force is applied. In essence, a stretchable battery behaves like a rubber band, whereas flexible batteries are more like a piece of paper, which can bend but not stretch. Patches Argue for Stretches Recent interest in stretchable batteries stems from growing use of unpowered wearable patches that monitor blood and even sweat. Gatorade, for example, now markets a skin patch called the Gx Sweat Patch, which helps track personal hydration. Numerous other companies offer wearable medical patches, many of which would benefit from the integration of a power source. “We’re seeing microelectronics being used everywhere,” said Thierry Djenizian, a professor in the flexible electronics department at the School of Mines Saint-Etienne in France. “And those electronics need power. One solution is the development of microbatteries that can be completely invisible.” “We stretched the battery, twisted it, hit it with a hammer, and still we showed that it could consistently power a servo motor under all of those deformations.” —James Pikul, University of Wisconsin–Madison Djenizian is part of a group that published a paper in February in the journal Advanced Materials Technologies on a stretchable lithium-ion wire battery. The wire battery, which measures 1.4 millimeters in diameter and more than 20 centimeters long, uses a twisted copper fabric as a current collector. It has been fabricated using conventional battery chemistries such as lithium cobalt oxide (LCO) and lithium nickel cobalt aluminum (NCA, popular for a time in Tesla cars). The researchers report that their battery can be stretched up to 22 percent and can be used in applications including biomedical patches, health trackers, smart textiles, and wristwatches. Djenizian said that a big part of the battery’s appeal is simple comfort. “If you’re doing a yoga stretch, your shirt can pull back on you. And if you have batteries that are not stretchable, you’ll feel it.” Similarly, Pikul is part of a group that published a paper in March in the journal Advanced Functional Materials on stretchable metal-air batteries. The new metal-air battery addresses a simple fact of battery life—that hard metals may make good anodes and cathodes, but they don’t stretch. The solution to that dilemma is an architecture in which the battery’s metal electrodes are allowed to slide freely between its enclosure and its electrolyte, both of which do stretch. The result is that the active parts of the battery–the anode and cathode–don’t need to stretch. In essence, they slide across the face of the electrolyte. The electrolyte is made from a hydrogel, a substance roughly the consistency of a soft contact lens. “The anode and the cathode are blocks, and they just slide across the other components that are doing the stretching,” Pikul explained. The wire battery uses a metal anode, typically zinc, and a carbon cloth loaded with platinum, as the “air” cathode. The battery is not rechargeable, and its applications include medical patches and hearing aids. Adding Zinc to the Mix Researchers are also developing stretchable batteries engineered for safety, which can even be used in applications where the wire battery comes in contact with, for example, the wet skin of a perspiring user. In a paper published in February in the journal Small, authors Zhao Wang and Jian Zhu say the key to such batteries is a stretchable zinc-ion chemistry that uses an aqueous electrolyte. Such batteries are safer than lithium-ion, which uses an “inherently flammable” organic electrolyte, they say. “Stretchable batteries with aqueous electrolytes can give us absolute safety and reliable power during deformation,” Zhu wrote in an email. The authors describe numerous zinc-ion chemistries, mostly involving a zinc anode and a manganese oxide cathode or a silver cathode. Energy capacity of stretchable zinc-ion ranges from a few milliampere-hours per gram to as much as 300 mAh per gram. “In comparison with conventional lithium-ion batteries, stretchable zinc-ion batteries have a lower energy density, but they can drive most power-consumption modules,” including sensors, transistors, and displays, Zhu said. With careful engineering, he said, the batteries can be stretched more than 900 percent. Unlike cellphone batteries, which consume much of the volume and weight of the overall product, the new breed of stretchable batteries is expected to be virtually invisible. Most are less than 2 millimeters in diameter, and weigh just a few grams. Moreover, durability does not seem to be an issue with any of the thin, stretchable batteries. Researchers said they subjected their stretchable batteries to substantial abuse without incident. “We stretched the battery, twisted it, hit it with a hammer, and still we showed that it could consistently power a servo motor under all of those deformations,” Pikul said. Battery experts believe the stretchable concept is viable, and will likely find a market. “Yes, in principle a stretchable battery could be made, provided there is a suitable anode,” said Donald Sadoway, a retired materials science professor from MIT and founder of Sadoway Labs Foundation, a nonprofit research institution aimed at new battery discoveries. “But maybe flexible is what is needed, not necessarily stretchable.” Sadoway added that he built a stretchable wristwatch battery in the 1990s, but found it was too early for the market. None of today’s researchers know when the new breed of batteries will reach the market, but they expect demand for them to grow. “In the past 10 years, there’s been all these advances in stretchable electronics, and now there are a lot of new applications,” Pikul said. “So there’s a need to power these stretchable devices, and the logical solution is to have stretchable batteries.”

Andrew Ng has serious street cred in artificial intelligence. He pioneered the use of graphics processing units (GPUs) to train deep learning models in the late 2000s with his students at Stanford University, cofounded Google Brain in 2011, and then served for three years as chief scientist for Baidu, where he helped build the Chinese tech giant’s AI group. So when he says he has identified the next big shift in artificial intelligence, people listen. And that’s what he told IEEE Spectrum in an exclusive Q&A. Ng’s current efforts are focused on his company Landing AI, which built a platform called LandingLens to help manufacturers improve visual inspection with computer vision. He has also become something of an evangelist for what he calls the data-centric AI movement, which he says can yield “small data” solutions to big issues in AI, including model efficiency, accuracy, and bias. Andrew Ng on... What’s next for really big models The career advice he didn’t listen to Defining the data-centric AI movement Synthetic data Why Landing AI asks its customers to do the work The great advances in deep learning over the past decade or so have been powered by ever-bigger models crunching ever-bigger amounts of data. Some people argue that that’s an unsustainable trajectory. Do you agree that it can’t go on that way? Andrew Ng: This is a big question. We’ve seen foundation models in NLP [natural language processing]. I’m excited about NLP models getting even bigger, and also about the potential of building foundation models in computer vision. I think there’s lots of signal to still be exploited in video: We have not been able to build foundation models yet for video because of compute bandwidth and the cost of processing video, as opposed to tokenized text. So I think that this engine of scaling up deep learning algorithms, which has been running for something like 15 years now, still has steam in it. Having said that, it only applies to certain problems, and there’s a set of other problems that need small data solutions. When you say you want a foundation model for computer vision, what do you mean by that? Ng: This is a term coined by Percy Liang and some of my friends at Stanford to refer to very large models, trained on very large data sets, that can be tuned for specific applications. For example, GPT-3 is an example of a foundation model [for NLP]. Foundation models offer a lot of promise as a new paradigm in developing machine learning applications, but also challenges in terms of making sure that they’re reasonably fair and free from bias, especially if many of us will be building on top of them. What needs to happen for someone to build a foundation model for video? Ng: I think there is a scalability problem. The compute power needed to process the large volume of images for video is significant, and I think that’s why foundation models have arisen first in NLP. Many researchers are working on this, and I think we’re seeing early signs of such models being developed in computer vision. But I’m confident that if a semiconductor maker gave us 10 times more processor power, we could easily find 10 times more video to build such models for vision. Having said that, a lot of what’s happened over the past decade is that deep learning has happened in consumer-facing companies that have large user bases, sometimes billions of users, and therefore very large data sets. While that paradigm of machine learning has driven a lot of economic value in consumer software, I find that that recipe of scale doesn’t work for other industries. Back to top It’s funny to hear you say that, because your early work was at a consumer-facing company with millions of users. Ng: Over a decade ago, when I proposed starting the Google Brain project to use Google’s compute infrastructure to build very large neural networks, it was a controversial step. One very senior person pulled me aside and warned me that starting Google Brain would be bad for my career. I think he felt that the action couldn’t just be in scaling up, and that I should instead focus on architecture innovation. “In many industries where giant data sets simply don’t exist, I think the focus has to shift from big data to good data. Having 50 thoughtfully engineered examples can be sufficient to explain to the neural network what you want it to learn.” —Andrew Ng, CEO & Founder, Landing AI I remember when my students and I published the first NeurIPS workshop paper advocating using CUDA, a platform for processing on GPUs, for deep learning—a different senior person in AI sat me down and said, “CUDA is really complicated to program. As a programming paradigm, this seems like too much work.” I did manage to convince him; the other person I did not convince. I expect they’re both convinced now. Ng: I think so, yes. Over the past year as I’ve been speaking to people about the data-centric AI movement, I’ve been getting flashbacks to when I was speaking to people about deep learning and scalability 10 or 15 years ago. In the past year, I’ve been getting the same mix of “there’s nothing new here” and “this seems like the wrong direction.” Back to top How do you define data-centric AI, and why do you consider it a movement? Ng: Data-centric AI is the discipline of systematically engineering the data needed to successfully build an AI system. For an AI system, you have to implement some algorithm, say a neural network, in code and then train it on your data set. The dominant paradigm over the last decade was to download the data set while you focus on improving the code. Thanks to that paradigm, over the last decade deep learning networks have improved significantly, to the point where for a lot of applications the code—the neural network architecture—is basically a solved problem. So for many practical applications, it’s now more productive to hold the neural network architecture fixed, and instead find ways to improve the data. When I started speaking about this, there were many practitioners who, completely appropriately, raised their hands and said, “Yes, we’ve been doing this for 20 years.” This is the time to take the things that some individuals have been doing intuitively and make it a systematic engineering discipline. The data-centric AI movement is much bigger than one company or group of researchers. My collaborators and I organized a data-centric AI workshop at NeurIPS, and I was really delighted at the number of authors and presenters that showed up. You often talk about companies or institutions that have only a small amount of data to work with. How can data-centric AI help them? Ng: You hear a lot about vision systems built with millions of images—I once built a face recognition system using 350 million images. Architectures built for hundreds of millions of images don’t work with only 50 images. But it turns out, if you have 50 really good examples, you can build something valuable, like a defect-inspection system. In many industries where giant data sets simply don’t exist, I think the focus has to shift from big data to good data. Having 50 thoughtfully engineered examples can be sufficient to explain to the neural network what you want it to learn. When you talk about training a model with just 50 images, does that really mean you’re taking an existing model that was trained on a very large data set and fine-tuning it? Or do you mean a brand new model that’s designed to learn only from that small data set? Ng: Let me describe what Landing AI does. When doing visual inspection for manufacturers, we often use our own flavor of RetinaNet. It is a pretrained model. Having said that, the pretraining is a small piece of the puzzle. What’s a bigger piece of the puzzle is providing tools that enable the manufacturer to pick the right set of images [to use for fine-tuning] and label them in a consistent way. There’s a very practical problem we’ve seen spanning vision, NLP, and speech, where even human annotators don’t agree on the appropriate label. For big data applications, the common response has been: If the data is noisy, let’s just get a lot of data and the algorithm will average over it. But if you can develop tools that flag where the data’s inconsistent and give you a very targeted way to improve the consistency of the data, that turns out to be a more efficient way to get a high-performing system. “Collecting more data often helps, but if you try to collect more data for everything, that can be a very expensive activity.” —Andrew Ng For example, if you have 10,000 images where 30 images are of one class, and those 30 images are labeled inconsistently, one of the things we do is build tools to draw your attention to the subset of data that’s inconsistent. So you can very quickly relabel those images to be more consistent, and this leads to improvement in performance. Could this focus on high-quality data help with bias in data sets? If you’re able to curate the data more before training? Ng: Very much so. Many researchers have pointed out that biased data is one factor among many leading to biased systems. There have been many thoughtful efforts to engineer the data. At the NeurIPS workshop, Olga Russakovsky gave a really nice talk on this. At the main NeurIPS conference, I also really enjoyed Mary Gray’s presentation, which touched on how data-centric AI is one piece of the solution, but not the entire solution. New tools like Datasheets for Datasets also seem like an important piece of the puzzle. One of the powerful tools that data-centric AI gives us is the ability to engineer a subset of the data. Imagine training a machine-learning system and finding that its performance is okay for most of the data set, but its performance is biased for just a subset of the data. If you try to change the whole neural network architecture to improve the performance on just that subset, it’s quite difficult. But if you can engineer a subset of the data you can address the problem in a much more targeted way. When you talk about engineering the data, what do you mean exactly? Ng: In AI, data cleaning is important, but the way the data has been cleaned has often been in very manual ways. In computer vision, someone may visualize images through a Jupyter notebook and maybe spot the problem, and maybe fix it. But I’m excited about tools that allow you to have a very large data set, tools that draw your attention quickly and efficiently to the subset of data where, say, the labels are noisy. Or to quickly bring your attention to the one class among 100 classes where it would benefit you to collect more data. Collecting more data often helps, but if you try to collect more data for everything, that can be a very expensive activity. For example, I once figured out that a speech-recognition system was performing poorly when there was car noise in the background. Knowing that allowed me to collect more data with car noise in the background, rather than trying to collect more data for everything, which would have been expensive and slow. Back to top What about using synthetic data, is that often a good solution? Ng: I think synthetic data is an important tool in the tool chest of data-centric AI. At the NeurIPS workshop, Anima Anandkumar gave a great talk that touched on synthetic data. I think there are important uses of synthetic data that go beyond just being a preprocessing step for increasing the data set for a learning algorithm. I’d love to see more tools to let developers use synthetic data generation as part of the closed loop of iterative machine learning development. Do you mean that synthetic data would allow you to try the model on more data sets? Ng: Not really. Here’s an example. Let’s say you’re trying to detect defects in a smartphone casing. There are many different types of defects on smartphones. It could be a scratch, a dent, pit marks, discoloration of the material, other types of blemishes. If you train the model and then find through error analysis that it’s doing well overall but it’s performing poorly on pit marks, then synthetic data generation allows you to address the problem in a more targeted way. You could generate more data just for the pit-mark category. “In the consumer software Internet, we could train a handful of machine-learning models to serve a billion users. In manufacturing, you might have 10,000 manufacturers building 10,000 custom AI models.” —Andrew Ng Synthetic data generation is a very powerful tool, but there are many simpler tools that I will often try first. Such as data augmentation, improving labeling consistency, or just asking a factory to collect more data. Back to top To make these issues more concrete, can you walk me through an example? When a company approaches Landing AI and says it has a problem with visual inspection, how do you onboard them and work toward deployment? Ng: When a customer approaches us we usually have a conversation about their inspection problem and look at a few images to verify that the problem is feasible with computer vision. Assuming it is, we ask them to upload the data to the LandingLens platform. We often advise them on the methodology of data-centric AI and help them label the data. One of the foci of Landing AI is to empower manufacturing companies to do the machine learning work themselves. A lot of our work is making sure the software is fast and easy to use. Through the iterative process of machine learning development, we advise customers on things like how to train models on the platform, when and how to improve the labeling of data so the performance of the model improves. Our training and software supports them all the way through deploying the trained model to an edge device in the factory. How do you deal with changing needs? If products change or lighting conditions change in the factory, can the model keep up? Ng: It varies by manufacturer. There is data drift in many contexts. But there are some manufacturers that have been running the same manufacturing line for 20 years now with few changes, so they don’t expect changes in the next five years. Those stable environments make things easier. For other manufacturers, we provide tools to flag when there’s a significant data-drift issue. I find it really important to empower manufacturing customers to correct data, retrain, and update the model. Because if something changes and it’s 3 a.m. in the United States, I want them to be able to adapt their learning algorithm right away to maintain operations. In the consumer software Internet, we could train a handful of machine-learning models to serve a billion users. In manufacturing, you might have 10,000 manufacturers building 10,000 custom AI models. The challenge is, how do you do that without Landing AI having to hire 10,000 machine learning specialists? So you’re saying that to make it scale, you have to empower customers to do a lot of the training and other work. Ng: Yes, exactly! This is an industry-wide problem in AI, not just in manufacturing. Look at health care. Every hospital has its own slightly different format for electronic health records. How can every hospital train its own custom AI model? Expecting every hospital’s IT personnel to invent new neural-network architectures is unrealistic. The only way out of this dilemma is to build tools that empower the customers to build their own models by giving them tools to engineer the data and express their domain knowledge. That’s what Landing AI is executing in computer vision, and the field of AI needs other teams to execute this in other domains. Is there anything else you think it’s important for people to understand about the work you’re doing or the data-centric AI movement? Ng: In the last decade, the biggest shift in AI was a shift to deep learning. I think it’s quite possible that in this decade the biggest shift will be to data-centric AI. With the maturity of today’s neural network architectures, I think for a lot of the practical applications the bottleneck will be whether we can efficiently get the data we need to develop systems that work well. The data-centric AI movement has tremendous energy and momentum across the whole community. I hope more researchers and developers will jump in and work on it. Back to top This article appears in the April 2022 print issue as “Andrew Ng, AI Minimalist.”

The end of Moore’s Law is looming. Engineers and designers can do only so much to miniaturize transistors and pack as many of them as possible into chips. So they’re turning to other approaches to chip design, incorporating technologies like AI into the process. Samsung, for instance, is adding AI to its memory chips to enable processing in memory, thereby saving energy and speeding up machine learning. Speaking of speed, Google’s TPU V4 AI chip has doubled its processing power compared with that of its previous version. But AI holds still more promise and potential for the semiconductor industry. To better understand how AI is set to revolutionize chip design, we spoke with Heather Gorr, senior product manager for MathWorks’ MATLAB platform. How is AI currently being used to design the next generation of chips? Heather Gorr: AI is such an important technology because it’s involved in most parts of the cycle, including the design and manufacturing process. There’s a lot of important applications here, even in the general process engineering where we want to optimize things. I think defect detection is a big one at all phases of the process, especially in manufacturing. But even thinking ahead in the design process, [AI now plays a significant role] when you’re designing the light and the sensors and all the different components. There’s a lot of anomaly detection and fault mitigation that you really want to consider. Heather GorrMathWorks Then, thinking about the logistical modeling that you see in any industry, there is always planned downtime that you want to mitigate; but you also end up having unplanned downtime. So, looking back at that historical data of when you’ve had those moments where maybe it took a bit longer than expected to manufacture something, you can take a look at all of that data and use AI to try to identify the proximate cause or to see something that might jump out even in the processing and design phases. We think of AI oftentimes as a predictive tool, or as a robot doing something, but a lot of times you get a lot of insight from the data through AI. What are the benefits of using AI for chip design? Gorr: Historically, we’ve seen a lot of physics-based modeling, which is a very intensive process. We want to do a reduced order model, where instead of solving such a computationally expensive and extensive model, we can do something a little cheaper. You could create a surrogate model, so to speak, of that physics-based model, use the data, and then do your parameter sweeps, your optimizations, your Monte Carlo simulations using the surrogate model. That takes a lot less time computationally than solving the physics-based equations directly. So, we’re seeing that benefit in many ways, including the efficiency and economy that are the results of iterating quickly on the experiments and the simulations that will really help in the design. So it’s like having a digital twin in a sense? Gorr: Exactly. That’s pretty much what people are doing, where you have the physical system model and the experimental data. Then, in conjunction, you have this other model that you could tweak and tune and try different parameters and experiments that let sweep through all of those different situations and come up with a better design in the end. So, it’s going to be more efficient and, as you said, cheaper? Gorr: Yeah, definitely. Especially in the experimentation and design phases, where you’re trying different things. That’s obviously going to yield dramatic cost savings if you’re actually manufacturing and producing [the chips]. You want to simulate, test, experiment as much as possible without making something using the actual process engineering. We’ve talked about the benefits. How about the drawbacks? Gorr: The [AI-based experimental models] tend to not be as accurate as physics-based models. Of course, that’s why you do many simulations and parameter sweeps. But that’s also the benefit of having that digital twin, where you can keep that in mind—it’s not going to be as accurate as that precise model that we’ve developed over the years. Both chip design and manufacturing are system intensive; you have to consider every little part. And that can be really challenging. It’s a case where you might have models to predict something and different parts of it, but you still need to bring it all together. One of the other things to think about too is that you need the data to build the models. You have to incorporate data from all sorts of different sensors and different sorts of teams, and so that heightens the challenge. How can engineers use AI to better prepare and extract insights from hardware or sensor data? Gorr: We always think about using AI to predict something or do some robot task, but you can use AI to come up with patterns and pick out things you might not have noticed before on your own. People will use AI when they have high-frequency data coming from many different sensors, and a lot of times it’s useful to explore the frequency domain and things like data synchronization or resampling. Those can be really challenging if you’re not sure where to start. One of the things I would say is, use the tools that are available. There’s a vast community of people working on these things, and you can find lots of examples [of applications and techniques] on GitHub or MATLAB Central, where people have shared nice examples, even little apps they’ve created. I think many of us are buried in data and just not sure what to do with it, so definitely take advantage of what’s already out there in the community. You can explore and see what makes sense to you, and bring in that balance of domain knowledge and the insight you get from the tools and AI. What should engineers and designers consider when using AI for chip design? Gorr: Think through what problems you’re trying to solve or what insights you might hope to find, and try to be clear about that. Consider all of the different components, and document and test each of those different parts. Consider all of the people involved, and explain and hand off in a way that is sensible for the whole team. How do you think AI will affect chip designers’ jobs? Gorr: It’s going to free up a lot of human capital for more advanced tasks. We can use AI to reduce waste, to optimize the materials, to optimize the design, but then you still have that human involved whenever it comes to decision-making. I think it’s a great example of people and technology working hand in hand. It’s also an industry where all people involved—even on the manufacturing floor—need to have some level of understanding of what’s happening, so this is a great industry for advancing AI because of how we test things and how we think about them before we put them on the chip. How do you envision the future of AI and chip design? Gorr: It’s very much dependent on that human element—involving people in the process and having that interpretable model. We can do many things with the mathematical minutiae of modeling, but it comes down to how people are using it, how everybody in the process is understanding and applying it. Communication and involvement of people of all skill levels in the process are going to be really important. We’re going to see less of those superprecise predictions and more transparency of information, sharing, and that digital twin—not only using AI but also using our human knowledge and all of the work that many people have done over the years.

Quantum computing is a devilishly complex technology, with many technical hurdles impacting its development. Of these challenges two critical issues stand out: miniaturization and qubit quality. IBM has adopted the superconducting qubit road map of reaching a 1,121-qubit processor by 2023, leading to the expectation that 1,000 qubits with today’s qubit form factor is feasible. However, current approaches will require very large chips (50 millimeters on a side, or larger) at the scale of small wafers, or the use of chiplets on multichip modules. While this approach will work, the aim is to attain a better path toward scalability. Now researchers at MIT have been able to both reduce the size of the qubits and done so in a way that reduces the interference that occurs between neighboring qubits. The MIT researchers have increased the number of superconducting qubits that can be added onto a device by a factor of 100. “We are addressing both qubit miniaturization and quality,” said William Oliver, the director for the Center for Quantum Engineering at MIT. “Unlike conventional transistor scaling, where only the number really matters, for qubits, large numbers are not sufficient, they must also be high-performance. Sacrificing performance for qubit number is not a useful trade in quantum computing. They must go hand in hand.” The key to this big increase in qubit density and reduction of interference comes down to the use of two-dimensional materials, in particular the 2D insulator hexagonal boron nitride (hBN). The MIT researchers demonstrated that a few atomic monolayers of hBN can be stacked to form the insulator in the capacitors of a superconducting qubit. Just like other capacitors, the capacitors in these superconducting circuits take the form of a sandwich in which an insulator material is sandwiched between two metal plates. The big difference for these capacitors is that the superconducting circuits can operate only at extremely low temperatures—less than 0.02 degrees above absolute zero (-273.15 °C). Superconducting qubits are measured at temperatures as low as 20 millikelvin in a dilution refrigerator.Nathan Fiske/MIT In that environment, insulating materials that are available for the job, such as PE-CVD silicon oxide or silicon nitride, have quite a few defects that are too lossy for quantum computing applications. To get around these material shortcomings, most superconducting circuits use what are called coplanar capacitors. In these capacitors, the plates are positioned laterally to one another, rather than on top of one another. As a result, the intrinsic silicon substrate below the plates and to a smaller degree the vacuum above the plates serve as the capacitor dielectric. Intrinsic silicon is chemically pure and therefore has few defects, and the large size dilutes the electric field at the plate interfaces, all of which leads to a low-loss capacitor. The lateral size of each plate in this open-face design ends up being quite large (typically 100 by 100 micrometers) in order to achieve the required capacitance. In an effort to move away from the large lateral configuration, the MIT researchers embarked on a search for an insulator that has very few defects and is compatible with superconducting capacitor plates. “We chose to study hBN because it is the most widely used insulator in 2D material research due to its cleanliness and chemical inertness,” said colead author Joel Wang, a research scientist in the Engineering Quantum Systems group of the MIT Research Laboratory for Electronics. On either side of the hBN, the MIT researchers used the 2D superconducting material, niobium diselenide. One of the trickiest aspects of fabricating the capacitors was working with the niobium diselenide, which oxidizes in seconds when exposed to air, according to Wang. This necessitates that the assembly of the capacitor occur in a glove box filled with argon gas. While this would seemingly complicate the scaling up of the production of these capacitors, Wang doesn’t regard this as a limiting factor. “What determines the quality factor of the capacitor are the two interfaces between the two materials,” said Wang. “Once the sandwich is made, the two interfaces are “sealed” and we don’t see any noticeable degradation over time when exposed to the atmosphere.” This lack of degradation is because around 90 percent of the electric field is contained within the sandwich structure, so the oxidation of the outer surface of the niobium diselenide does not play a significant role anymore. This ultimately makes the capacitor footprint much smaller, and it accounts for the reduction in cross talk between the neighboring qubits. “The main challenge for scaling up the fabrication will be the wafer-scale growth of hBN and 2D superconductors like [niobium diselenide], and how one can do wafer-scale stacking of these films,” added Wang. Wang believes that this research has shown 2D hBN to be a good insulator candidate for superconducting qubits. He says that the groundwork the MIT team has done will serve as a road map for using other hybrid 2D materials to build superconducting circuits.