IEEE Novosti

IEEE Spectrum IEEE Spectrum

  • Boston Dynamics Unleashes New Spot Variant for Research
    by Evan Ackerman on 28. Marta 2024. at 19:03

    At NVIDIA GTC last week, Boston Dynamics CTO Aaron Saunders gave a talk about deploying AI in real world robots—namely, how Spot is leveraging reinforcement learning to get better at locomotion (We spoke with Saunders last year about robots falling over). And Spot has gotten a lot better—a Spot robot takes a tumble on average once every 50 kilometers, even as the Spot fleet collectively walks enough to circle the Earth every three months. That fleet consists of a lot of commercial deployments, which is impressive for any mobile robot, but part of the reason for that is because the current version of Spot is really not intended for robotics research, even though over 100 universities are home to at least one Spot. Boston Dynamics has not provided developer access to Spot’s joints, meaning that anyone who has wanted to explore quadrupedal mobility has had to find some other platform that’s a bit more open and allows for some experimentation. Boston Dynamics is now announcing a new variant of Spot that includes a low-level application programming interface (API) that gives joint-level control of the robot. This will give (nearly) full control over how Spot moves its legs, which is a huge opportunity for the robotics community, since we’ll now be able to find out exactly what Spot is capable of. For example, we’ve already heard from a credible source that Spot is capable of running much, much faster than Boston Dynamics has publicly shown, and it’s safe to assume that a speedier Spot is just the start. An example of a new Spot capability when a custom locomotion controller can be used on the robot.Boston Dynamics When you buy a Spot robot from Boston Dynamics, it arrives already knowing how to walk. It’s very, very good at walking. Boston Dynamics is so confident in Spot’s walking ability that you’re only allowed high-level control of the robot: You tell it where to go, it decides how to get there. If you want to do robotics research using Spot as a mobility platform, that’s totally fine, but if you want to do research on quadrupedal locomotion, it hasn’t been possible with Spot. But that’s changing. The Spot RL Researcher Kit is a collaboration between Boston Dynamics, Nvidia, and the AI Institute. It includes a joint-level control API, an Nvidia Jetson AGX Orin payload, and a simulation environment for Spot based on Nvidia Isaac Lab. The kit will be officially released later this year, but Boston Dynamics is starting a slow rollout through an early adopter beta program. From a certain perspective, Boston Dynamics did this whole thing with Spot backwards by first creating a commercial product and only then making it into a research platform. “At the beginning, we felt like it would be great to include that research capability, but that it wasn’t going to drive the adoption of this technology,” Saunders told us after his GTC session. Instead, Boston Dynamics first focused on getting lots of Spots out into the world in a useful way, and only now, when the company feels like they’ve gotten there, is the time right to unleash a fully-featured research version of Spot. “It was really just getting comfortable with our current product that enabled us to go back and say, ‘how can we now provide people with the kind of access that they’re itching for?’” Getting to this point has taken a huge amount of work for Boston Dynamics. Predictably, Spot started out as a novelty for most early adopters, becoming a project for different flavors of innovation groups within businesses rather than an industrial asset. “I think there’s been a change there,” Saunders says. “We’re working with operational customers a lot more, and the composure of our sales is shifting away from being dominated by early adopters and we’re starting to see repeat sales and interest in larger fleets of robots.” Deploying and supporting a large fleet of Spots is one of the things that allowed Boston Dynamics to feel comfortable offering a research version. Researchers are not particularly friendly to their robots, because the goal of research is often to push the envelope of what’s possible. And part of that process includes getting very well acquainted with what turns out to be not possible, resulting in robots that end up on the floor, sometimes in pieces. The research version of Spot will include a mandatory Spot Care Service Plan, which exists to serve commercial customers but will almost certainly provide more value to the research community who want to see what kinds of crazy things they can get Spot to do. Exactly how crazy those crazy things will be remains to be seen. Boston Dynamics is starting out with a beta program for the research Spots partially because they’re not quite sure yet how many safeguards to put in place within the API. “We need to see where the problems are,” Saunders says. “We still have a little work to do to really hone in how our customers are going to use it.” Deciding how much Spot should be able to put itself at risk in the name of research may be a difficult question to answer, but I’m pretty sure that the beta program participants are going to do their best to find out how much tolerance Boston Dynamics has for Spot shenanigans. I just hope that whatever happens, they share as much video of it as possible. The Spot Early Adopter Program for the new RL Researcher Kit is open for applications here.

  • A New Alliance Is Advancing Augmented Reality
    by Kathy Pretz on 28. Marta 2024. at 18:00

    Apple’s Vision Pro headset might just be the breakthrough product that the augmented-reality industry has been waiting for to catalyze the widespread adoption of AR technology, according to the IEEE-ISTO AR Alliance. The new alliance’s goal is to foster and encourage the development of AR technologies, components, devices, solutions, and platforms. It is a program of the IEEE Industry Standards and Technology Organization (ISTO), a global, not-for-profit corporation that provides its member organizations with legal and financial infrastructure and administrative support for standards development and market adoption of emerging technologies. In addition to other headsets on the market, such as Meta’s Oculus Quest 3, the US $3,499 Vision Pro could invigorate the market, members of the AR Alliance say. “Unlike many other devices, Vision Pro goes beyond gaming and entertainment,” says Bharath Rajagopalan, the alliance’s chair. “Users can engage in productivity activities such as checking email and reading documents. Even though the headset isn’t technically pure AR, it is a way of letting people simultaneously see the real world and the digital world and interact with content in an intuitive and compelling way.” “The launch of the Vision Pro could be a watershed moment for the AR industry,” adds Rajagopalan, who is director of strategic marketing for STMicroelectronics, a global semiconductor company. STM is one of the alliance’s founding members. “When a technology leader like Apple comes into this domain in such a big way, it helps to validate the category,” Rajagopalan says. “To advance AR, we need to enable everyday use of the technology for the average user, not just the techies of the world.” “To advance AR, we need to enable everyday use of the technology for the average user, not just the techies of the world.” After trying out the Vision Pro, Rajagopalan says he believes the headset, which integrates an array of technologies, is the kind of device that is needed to bring together companies in the hardware ecosystem to advance AR technology. “Apple is unique in the sense that it develops, or facilitates the development of, the many critical technologies and components for such a device,” he says. But companies that don’t or can’t take that approach “will have to pick and choose different classes of technologies and components to integrate into their products. To facilitate faster time to market, enable interoperability, ensure a robust supply chain, and drive for lower costs, having standards and protocols will be a key element.” That’s where the AR Alliance comes in. Leveraging existing technical standards The alliance grew out of the LaSAR Alliance, a collaboration of companies that worked to advance laser-scanning technology for use as displays on AR wearables such as smart glasses. The organization decided to expand the scope and rebrand the alliance last year after fulfilling its mission of establishing laser scanning as a core AR display technology. While the alliance was reorganizing, Apple unveiled the Vision Pro. Several LaSAR Alliance members said the timing was right to advance AR beyond display technologies—which led to the AR Alliance. Corning, Dispelix, Essilor Luxottica, Microoled, and OptoFidelity are some of the other member organizations and founding members, in addition to STM. “We decided to take what we did for laser-scanning technology and think about how we could address the overall needs of the AR marketplace,” Rajagopalan says. “We asked ourselves what were the things that were really going to help this industry and the market advance. Just like for other industries, the way to do it is to have standards and protocols and an ecosystem that companies can coalesce around. “A benefit of the alliance being part of IEEE-ISTO is that it can leverage IEEE’s work on existing AR, VR, and mixed-reality standards.” Tackling image quality, connectivity, and certification The first step the alliance took was to meet with companies building AR hardware and software to find out their needs and to solicit ideas. The businesses “asked for a neutral environment that allows companies to come together—not competitively but cooperatively,” Rajagopalan says. Their other requests were to work together on addressing key challenges such as improving image quality, tackling connectivity challenges, and developing technical standards necessary for product certifications. With more headsets expected to come on the market, Rajagopalan says, the alliance wants to ensure there are uniform ways to measure their image quality. Currently none exist for AR devices, so the alliance’s image quality methods, metrics, and metrology working group is researching and developing approaches and techniques. Another group is working to establish connectivity protocols for smart glasses, so-called light AR. Such glasses let users access digital content such as maps, calendars, and other apps that deliver relevant information, and they can display instructions while the user interacts with the physical world. A compelling user experience requires smooth, reliable wireless connectivity. Working with existing standards but adapting them for the needs of smart glasses is the approach the alliance is taking, such as by adapting latency and bandwidth specifications, Rajagopalan says. Another group of the AR Alliance is assisting the Underwriters Laboratory Technical Committee 8400 with establishing standards for testing and certifying augmented, virtual, and mixed-reality products. UL is in the process of defining standards and certification requirements. UL wanted a single voice for the industry, Rajagopalan says, and it didn’t want one that might be biased in favor of one company. “The reason for forming this alliance is so that we can be that voice of the industry,” he says. “Our tagline is Building AR Together. That’s the whole point.” To participate in the alliance, you can apply here. This article has been updated from an earlier version.

  • Salt-Sized Sensors Mimic the Brain
    by Gwendolyn Rak on 28. Marta 2024. at 14:54

    To gain a better understanding of the brain, why not draw inspiration from it? At least, that’s what researchers at Brown University did, by building a wireless communications system that mimics the brain using an array of tiny silicon sensors, each the size of a grain of sand. The researchers hope that the technology could one day be used in implantable brain-machine interfaces to read brain activity. Each sensor, measuring 300 by 300 micrometers, acts as a wireless node in a large array, analogous to neurons in the brain. When a node senses an event, such as a change in temperature or neural activity, the device sends the data as a “spike” signal, consisting of a series of short radiofrequency pulses, to a central receiver. That receiver then decodes the information. “The brain is exquisitely efficient in handling large amounts of data,” says Arto Nurmikko, a professor of engineering and physics at Brown University. That’s why his lab chose to develop a network of unobtrusive microsensors that are “neuromorphic,” meaning they are inspired by how the brain works. And the similarities don’t end there—Nurmikko says that the wireless signals and computing methods are also inspired by the brain. The team published their results on 19 March in Nature Electronics. Thinking Like a Brain Like neurons, these sensors are event-driven and only send signals to the receiver when a change occurs. While digital communication encodes information in a sequence of ones and zeros, this system cuts down the amount of data transmitted by using periods of inactivity to infer where zeros would be sent. Importantly, this leads to significant energy savings, which in turn allows for a larger collection of microsensors. But with so many sensors sending information to a common receiver, it can be difficult to keep the data streams straight. The researchers deployed a neuromorphic computing technique to decode the signals in real time. “The brain is exquisitely efficient in handling large amounts of data.” —Arto Nurmikko, Brown University The researchers also conducted simulations to test the system’s error rate, which increases with more sensors. In addition to 78 fabricated sensors, they ran simulations of networks consisting of 200, 500, and 1,000 nodes using a real data set from primate brain recordings. In each, the system predicted the hand movement of a non-human primate with an error rate below 0.1 percent, which is acceptable for brain-computer applications. Nurmikko says the team will next test the wireless implanted sensor network in rodents. While the technology could be applied to any part of the body where biomedical researchers aim to monitor physiological activity, the primary goal is use in a brain-machine interface that can probe a large region of the brain, says Nurmikko. The sensors could also be modified for use in wearable technology or environmental sensors. There are key advantages of the system for biomedical uses, such as the small, unobtrusive design. But these applications also impose a key limitation: The sensors are externally powered by a wireless beam to avoid the need for batteries, and the body can only safely absorb so much radiofrequency energy. In other words, the system is not limited by bandwidth, but instead by power delivery. “From a practical point of view, it always comes back to the question of, where do you get your energy?” says Nurmikko. Brain-Machine Interface Possibilities The research provides “an important contribution, which demonstrates the feasibility and potential of neuromorphic communications for future use cases of low-power wireless sensing, communication, and decision making,” says Osvaldo Simeone, a professor at King’s College London and one of the researchers who first designed and simulated a neuromorphic communication system in 2020. The idea of a wireless network probing the brain is not new, says Federico Corradi, a researcher and assistant professor of electrical engineering at Eindhoven University of Technology. In 2011, for example, a researcher at UC Berkeley gave a presentation on “neural dust” in which he proposed a hypothetical class of nanometer-sized wireless sensors. “But now, it’s materializing slowly,” Corradi says. One important element of the Brown researcher’s design is its simplicity, says Corradi. The sensor’s architecture does not include a battery or clock embedded within the chips, making it ideal for scalable, low-power systems. “It opens a lot of possibilities.” Additionally, Corradi points to the sensor’s asynchronous nature as a key advantage—and limitation. This aspect of the sensor preserves time information, which is essential for studying the brain. But this feature could also introduce problems if the relative timing of events gets out of whack. Corradi believes this work is part of a larger trend toward neuromorphic systems, a “new wave of brain-machine interfaces that I hope we will see in the coming future.”

  • How We’ll Reach a 1 Trillion Transistor GPU
    by Mark Liu on 28. Marta 2024. at 14:30

    In 1997 the IBM Deep Blue supercomputer defeated world chess champion Garry Kasparov. It was a groundbreaking demonstration of supercomputer technology and a first glimpse into how high-performance computing might one day overtake human-level intelligence. In the 10 years that followed, we began to use artificial intelligence for many practical tasks, such as facial recognition, language translation, and recommending movies and merchandise. Fast-forward another decade and a half and artificial intelligence has advanced to the point where it can “synthesize knowledge.” Generative AI, such as ChatGPT and Stable Diffusion, can compose poems, create artwork, diagnose disease, write summary reports and computer code, and even design integrated circuits that rival those made by humans. Tremendous opportunities lie ahead for artificial intelligence to become a digital assistant to all human endeavors. ChatGPT is a good example of how AI has democratized the use of high-performance computing, providing benefits to every individual in society. All those marvelous AI applications have been due to three factors: innovations in efficient machine-learning algorithms, the availability of massive amounts of data on which to train neural networks, and progress in energy-efficient computing through the advancement of semiconductor technology. This last contribution to the generative AI revolution has received less than its fair share of credit, despite its ubiquity. Over the last three decades, the major milestones in AI were all enabled by the leading-edge semiconductor technology of the time and would have been impossible without it. Deep Blue was implemented with a mix of 0.6- and 0.35-micrometer-node chip-manufacturing technology. The deep neural network that won the ImageNet competition, kicking off the current era of machine learning, was implemented with 40-nanometer technology. AlphaGo conquered the game of Go using 28-nm technology, and the initial version of ChatGPT was trained on computers built with 5-nm technology. The most recent incarnation of ChatGPT is powered by servers using even more advanced 4-nm technology. Each layer of the computer systems involved, from software and algorithms down to the architecture, circuit design, and device technology, acts as a multiplier for the performance of AI. But it’s fair to say that the foundational transistor-device technology is what has enabled the advancement of the layers above. If the AI revolution is to continue at its current pace, it’s going to need even more from the semiconductor industry. Within a decade, it will need a 1-trillion-transistor GPU—that is, a GPU with 10 times as many devices as is typical today. Advances in semiconductor technology [top line]—including new materials, advances in lithography, new types of transistors, and advanced packaging—have driven the development of more capable AI systems [bottom line] Relentless Growth in AI Model Sizes The computation and memory access required for AI training have increased by orders of magnitude in the past five years. Training GPT-3, for example, requires the equivalent of more than 5 billion billion operations per second of computation for an entire day (that’s 5,000 petaflops-days), and 3 trillion bytes (3 terabytes) of memory capacity. Both the computing power and the memory access needed for new generative AI applications continue to grow rapidly. We now need to answer a pressing question: How can semiconductor technology keep pace? From Integrated Devices to Integrated Chiplets Since the invention of the integrated circuit, semiconductor technology has been about scaling down in feature size so that we can cram more transistors into a thumbnail-size chip. Today, integration has risen one level higher; we are going beyond 2D scaling into 3D system integration. We are now putting together many chips into a tightly integrated, massively interconnected system. This is a paradigm shift in semiconductor-technology integration. In the era of AI, the capability of a system is directly proportional to the number of transistors integrated into that system. One of the main limitations is that lithographic chipmaking tools have been designed to make ICs of no more than about 800 square millimeters, what’s called the reticle limit. But we can now extend the size of the integrated system beyond lithography’s reticle limit. By attaching several chips onto a larger interposer—a piece of silicon into which interconnects are built—we can integrate a system that contains a much larger number of devices than what is possible on a single chip. For example, TSMC’s chip-on-wafer-on-substrate (CoWoS) technology can accommodate up to six reticle fields’ worth of compute chips, along with a dozen high-bandwidth-memory (HBM) chips. How Nvidia Uses CoWoS Advanced Packaging CoWoS, TSMC’s chip-on-wafer-on-silicon advanced packaging technology, has already been deployed in products. Examples include the Nvidia Ampere and Hopper GPUs. Each consists of one GPU die with six high-bandwidth memory cubes all on a silicon interposer. The compute GPU die is about as large as chipmaking tools will currently allow. Ampere has 54 billion transistors, and Hopper has 80 billion. The transition from 7-nm technology to the denser 4-nm technology made it possible to pack 50 percent more transistors on essentially the same area. Ampere and Hopper are the workhorses for today’s large language model (LLM) training. It takes tens of thousands of these processors to train ChatGPT. HBMs are an example of the other key semiconductor technology that is increasingly important for AI: the ability to integrate systems by stacking chips atop one another, what we at TSMC call system-on-integrated-chips (SoIC). An HBM consists of a stack of vertically interconnected chips of DRAM atop a control logic IC. It uses vertical interconnects called through-silicon-vias (TSVs) to get signals through each chip and solder bumps to form the connections between the memory chips. Today, high-performance GPUs use HBM extensively. Going forward, 3D SoIC technology can provide a “bumpless alternative” to the conventional HBM technology of today, delivering far denser vertical interconnection between the stacked chips. Recent advances have shown HBM test structures with 12 layers of chips stacked using hybrid bonding, a copper-to-copper connection with a higher density than solder bumps can provide. Bonded at low temperature on top of a larger base logic chip, this memory system has a total thickness of just 600 µm. With a high-performance computing system composed of a large number of dies running large AI models, high-speed wired communication may quickly limit the computation speed. Today, optical interconnects are already being used to connect server racks in data centers. We will soon need optical interfaces based on silicon photonics that are packaged together with GPUs and CPUs. This will allow the scaling up of energy- and area-efficient bandwidths for direct, optical GPU-to-GPU communication, such that hundreds of servers can behave as a single giant GPU with a unified memory. Because of the demand from AI applications, silicon photonics will become one of the semiconductor industry’s most important enabling technologies. Toward a Trillion Transistor GPU How AMD Uses 3D Technology The AMD MI300A Accelerated Processor Unit leverages not just CoWoS but also TSMC’s 3D technology, silicon-on-integrated-circuits (SoIC). The MI300A combines GPU and CPU cores designed to handle the largest AI workloads. The GPU performs the intensive matrix multiplication operations for AI, while the CPU controls the operations of the entire system, and the high-bandwidth memories (HBM) are unified to serve both. The 9 compute dies built with 5-nm technology are stacked on top of 4 base dies of 6-nm technology, which are dedicated to cache and I/O traffic. The base dies and HBM sit atop silicon interposers. The compute part of the processor is composed of 150 billion transistors. As noted already, typical GPU chips used for AI training have already reached the reticle field limit. And their transistor count is about 100 billion devices. The continuation of the trend of increasing transistor count will require multiple chips, interconnected with 2.5D or 3D integration, to perform the computation. The integration of multiple chips, either by CoWoS or SoIC and related advanced packaging technologies, allows for a much larger total transistor count per system than can be squeezed into a single chip. We forecast that within a decade a multichiplet GPU will have more than 1 trillion transistors. We’ll need to link all these chiplets together in a 3D stack, but fortunately, industry has been able to rapidly scale down the pitch of vertical interconnects, increasing the density of connections. And there is plenty of room for more. We see no reason why the interconnect density can’t grow by an order of magnitude, and even beyond. Toward a Trillion Transistors Vertical connection density in 3D chips has increased at roughly the same rate as the number of transistors in a GPU. Energy-Efficient Performance Trend for GPUs So, how do all these innovative hardware technologies contribute to the performance of a system? We can see the trend already in server GPUs if we look at the steady improvement in a metric called energy-efficient performance. EEP is a combined measure of the energy efficiency and speed of a system. Over the past 15 years, the semiconductor industry has increased energy-efficient performance about threefold every two years. We believe this trend will continue at historical rates. It will be driven by innovations from many sources, including new materials, device and integration technology, extreme ultraviolet (EUV) lithography, circuit design, system architecture design, and the co-optimization of all these technology elements, among other things. Largely thanks to advances in semiconductor technology, a measure called energy-efficient performance is on track to triple every two years (EEP units are 1/femtojoule-picoseconds). In particular, the EEP increase will be enabled by the advanced packaging technologies we’ve been discussing here. Additionally, concepts such as system-technology co-optimization (STCO), where the different functional parts of a GPU are separated onto their own chiplets and built using the best performing and most economical technologies for each, will become increasingly critical. A Mead-Conway Moment for 3D Integrated Circuits In 1978, Carver Mead, a professor at the California Institute of Technology, and Lynn Conway at Xerox PARC invented a computer-aided design method for integrated circuits. They used a set of design rules to describe chip scaling so that engineers could easily design very-large-scale integration (VLSI) circuits without much knowledge of process technology. That same sort of capability is needed for 3D chip design. Today, designers need to know chip design, system-architecture design, and hardware and software optimization. Manufacturers need to know chip technology, 3D IC technology, and advanced packaging technology. As we did in 1978, we again need a common language to describe these technologies in a way that electronic design tools understand. Such a hardware description language gives designers a free hand to work on a 3D IC system design, regardless of the underlying technology. It’s on the way: An open-source standard, called 3Dblox, has already been embraced by most of today’s technology companies and electronic design automation (EDA) companies. The Future Beyond the Tunnel In the era of artificial intelligence, semiconductor technology is a key enabler for new AI capabilities and applications. A new GPU is no longer restricted by the standard sizes and form factors of the past. New semiconductor technology is no longer limited to scaling down the next-generation transistors on a two-dimensional plane. An integrated AI system can be composed of as many energy-efficient transistors as is practical, an efficient system architecture for specialized compute workloads, and an optimized relationship between software and hardware. For the past 50 years, semiconductor-technology development has felt like walking inside a tunnel. The road ahead was clear, as there was a well-defined path. And everyone knew what needed to be done: shrink the transistor. Now, we have reached the end of the tunnel. From here, semiconductor technology will get harder to develop. Yet, beyond the tunnel, many more possibilities lie ahead. We are no longer bound by the confines of the past.

  • The Most Hackable Handheld Ham Radio Yet
    by Stephen Cass on 27. Marta 2024. at 19:00

    All right, confession time. I don’t use my handheld ham radio for much more than eavesdropping on the subway dispatcher when my train rumbles to a mysterious halt in a dark tunnel. But even I couldn’t help but hear the buzz surrounding a new handheld, Quansheng’s UV-K5. It caught my attention in part because for over a decade, Baofeng has been the name in Chinese handhelds. In 2012 Baofeng made waves with its UV-5R radio, upending the sleepy handheld-transceiver market. Prior to the 5R, the price tag of the cheapest VHF/UHF handheld was a little north of US $100. The 5R sold for a quarter to a third of that. Hams groused about the 5R’s so-so technical performance—and then bought a couple anyway, so they’d always have a radio in their car or workplace. Now it’s Quansheng that’s making a splash. The UV-K5, released last year, might be the most hackable handheld ever, with a small army of dedicated hams adding a raft of software-based improvements and new features. I had to have one, and $30 later, I did. Like Baofeng’s 5R, Quansheng’s K5 as a radio transceiver is fine. (I’m using K5 here to refer to both the original K5 and the new K5(8) model.) The key technical distinction between the 5R and K5 is a seemingly minor design choice. With Baofeng’s 5R, the firmware resides in read-only memory. But Quansheng stores the K5’s firmware in flash memory and made it possible to rewrite that memory with the same USB programming cable used to assign frequencies to preset channels. This feature has opened the door for improvements to the K5 that are well beyond what Quansheng offers out of the box. Hopefully, this design will inspire other radio makers to offer more support for modders, in turn bringing more innovation to the VHF and UHF radio bands. Quansheng probably thought of its design purely in terms of fixing software bugs or adjusting for regulatory changes—it offers a free install tool for uploading official firmware releases to the radio. But the prospect of an updatable radio dangled an irresistible temptation for folks to start reverse engineering the firmware and hardware so they could try writing their own code. Modifications to date have generally taken the form of patches to the official firmware, rather than wholesale rewrites. With the official firmware taking up most of the radio’s 64 kilobytes of flash memory, such mods have to fit into less than 3 KB. And the CPU is not brimming with compute power—it’s a 48-megahertz, 32-bit ARM-based processor with 8 KB of RAM. Nonetheless, I found the results impressive. For example, one mod installs a fairly sophisticated graphical spectrum analyzer: You can adjust the bandwidth, set a threshold for tuning into detected peaks automatically, and specify frequencies to ignore, among other things. Another mod allows you to exchange text messages between K5s. Other mods improve the K5’s ability to receive AM signals, meaning you can, say, listen in on aviation bands more clearly. And there are plenty of fun little mods that do things like change up the system fonts or replace the start-up message with a line-art image of your choice. Updatable firmware dangled an irresistible temptation for folks to start reverse engineering… Installing many of these mods is ridiculously easy. Normally at this point in a Hands On article that involves hacking some consumer electronics, things get pretty heroic as I futz with the hardware or unravel a software-installation enigma. But not this time. A modder known as whosmatt has created a Web-based patcher/flasher for the K5 that lets you pick a selection of mods from a menu. It then combines them with the official firmware to create a custom image for uploading (as long as you don’t exceed the total amount of memory). In fact, if you’re using Chrome, Edge, or Opera, you don’t even need to use Quansheng’s installer to upload the firmware: You can update the radio’s flash memory directly from the browser via the built-in Web Serial API and the USB programming cable. (The instructions say this will work only on Linux and Windows, but I was able to do it using a Mac as well.) Web Serial could do with some improved error handling, though. The first USB programming cable I used was a bit flaky, but where Quansheng’s installer would halt and flag a communications error with a failed upload, Web Serial would silently crash and take the whole Windows operating system with it. There are even more K5 mods available than are in whosmatt’s online patcher. If you want to play with those or start writing your own mods, Python-based toolchains exist to assist you. This block diagram of the UV-K5 is based on the work of Phil McAllen. Hams have reverse engineered many details of the radio’s hardware and software.James Provost Of course, allowing unfettered modding of the K5’s transceiver does raise the possibility of abuse. For example, the Quansheng firmware blocks transmitting on the aviation band, to prevent illegal and hazardous interference. But this block can be removed by a patch (although to be a significant threat, you’d likely need an amplifier to boost the K5’s 5-watt signal). However, hams have always had the ability to behave badly, with or without firmware blocks. Such blocks are convenient for guarding against accidental abuse, but the truth is that unless problematic signals are persistent enough to allow a transmitter’s location to be triangulated, amateur radio must continue to rely on an honor system, whether that means not jamming a neighbor’s TV or transmitting on forbidden frequencies. Many of the most exciting uses of ham radio today involve digital processing, and that processing is normally done using a computer connected to a transceiver. With embedded controllers becoming ever more powerful, the K5 modding scene points toward a future where more processing happens in-radio and where you can add new functions the way apps are added to smartphones. Here’s hoping manufacturers embrace that future!

  • Nvidia Tops Llama 2, Stable Diffusion Speed Trials
    by Samuel K. Moore on 27. Marta 2024. at 18:45

    Times change, and so must benchmarks. Now that we’re firmly in the age of massive generative AI, it’s time to add two such behemoths, Llama 2 70B and Stable Diffusion XL, to MLPerf’s inferencing tests. Version 4.0 of the benchmark tests more than 8,500 results from 23 submitting organizations. As has been the case from the beginning, computers with Nvidia GPUs came out on top, particularly those with its H200 processor. But AI accelerators from Intel and Qualcomm were in the mix as well. MLPerf started pushing into the LLM world last year when it added a text summarization benchmark GPT-J (a 6 billion parameter open-source model). With 70 billion parameters, Llama 2 is an order of magnitude larger. Therefore it requires what the organizer MLCommons, a San Francisco-based AI consortium, calls “a different class of hardware.” “In terms of model parameters, Llama-2 is a dramatic increase to the models in the inference suite,” Mitchelle Rasquinha, a software engineer at Google and co-chair of the MLPerf Inference working group, said in a press release. Stable Diffusion XL, the new text-to-image generation benchmark, comes in at 2.6 billion parameters, less than half the size of GPT-J. The recommender system test, revised last year, is larger than both. MLPerf benchmarks run the range of sizes, with the latest, such as Llama 2 70B in the many tens of billions of parameters.MLCommons The tests are divided between systems meant for use in data centers and those intended for use by devices out in the world, or the “edge” as its called. For each benchmark, a computer can be tested in what’s called an offline mode or in a more realistic manner. In offline mode, it runs through the test data as fast as possible to determine its maximum throughput. The more realistic tests are meant to simulate things like a stream of data coming from a camera in a smartphone, multiple streams of data from all the cameras and sensors in a car, or as queries in a data center setup, for example. Additionally, the power consumption of some systems was tracked during tasks. Data center inference results The top performers in the new generative AI categories was an Nvidia H200 system that combined eight of the GPUs with two Intel Xeon CPUs. It managed just under 14 queries per second for Stable Diffusion and about 27,000 tokens per second for Llama 2 70B. Its nearest competition were 8-GPU H100 systems. And the performance difference wasn’t huge for Stable Diffusion, about 1 query per second, but the difference was larger for Llama 2 70B. H200s are the same Hopper architecture as the H100, but with about 75 percent more high-bandwidth memory and 43 percent more memory bandwidth. According to Nvidia’s Dave Salvator, memory is particularly important in LLMs, which perform better if they can fit entirely on the chip with other key data. The memory difference showed in the Llama 2 results, where H200 sped ahead of H100 by about 45 percent. According to the company, systems with H100 GPUs were 2.4-2.9 times faster than H100 systems from the results of last September, thanks to software improvements. Although H200 was the star of Nvidia’s benchmark show, its newest GPU architecture, Blackwell, officially unveiled last week, looms in the background. Salvator wouldn’t say when computers with that GPU might debut in the benchmark tables. For its part, Intel continued to offer its Gaudi 2 accelerator as the only option to Nvidia, at least among the companies participating in MLPerf’s inferencing benchmarks. On raw performance, Intel’s 7-nanometer chip delivered a little less than half the performance of 5-nm H100 in an 8-GPU configuration for Stable Diffusion XL. Its Gaudi 2 delivered results closer to one-third the Nvidia performance for Llama 2 70B. However, Intel argues that if you’re measuring performance per dollar (something they did themselves, not with MLPerf), the Gaudi 2 is about equal to the H100. For Stable Diffusion, Intel calculates it beats H100 by about 25 percent on performance per dollar. For Llama 2 70B it’s either an even contest or 21 percent worse, depending on whether you’re measuring in server or offline mode. Gaudi 2’s successor, Gaudi 3 is expected to arrive later this year. Intel also touted several CPU-only entries that showed a reasonable level of inferencing performance is possible in the absence of a GPU, though not on Llama 2 70B or Stable Diffusion. This was the first appearance of Intel’s 5th generation Xeon CPUs in the MLPerf inferencing competition, and the company claims a performance boost ranging from 18 percent to 91 percent over 4th generation Xeon systems from September 2023 results. Edge inferencing results As large as it is, Llama 2 70B wasn’t tested in the edge category, but Stable Diffusion XL was. Here the top performer was a system using two Nvidia L40S GPUs and an Intel Xeon CPU. Performance here is measured in latency and in samples per second. The system, submitted by Taipei-based cloud infrastructure company Wiwynn, produced answers in less than 2 seconds in single-stream mode. When driven in offline mode, it generates 1.26 results per second. Power consumption In the data center category, the contest around energy efficiency was between Nvidia and Qualcomm. The latter has focused on energy efficient inference since introducing the Cloud AI 100 processor more than a year ago. Qualcomm introduced a new generation of the accelerator chip the Cloud AI 100 Ultra late last year, and its first results showed up in the edge and data center performance benchmarks above. Compared to the Cloud AI 100 Pro results, Ultra produced a 2.5 to 3 times performance boost while consuming less than 150 Watts per chip. Among the edge inference entrance, Qualcomm was the only company to attempt Stable Diffusion XL, managing 0.6 samples per second using 578 watts.

  • This Startup’s AI Tool Makes Moving Day Easier
    by Edd Gent on 27. Marta 2024. at 16:00

    Engineers are used to being experts in their field, but when Zach Rattner cofounded his artificial-intelligence startup, Yembo, he quickly realized he needed to get comfortable with being out of his depth. He found the transition from employee to business owner to be a steep learning curve. Taking on a host of unfamiliar responsibilities like finance and sales required a significant shift in mind-set. Rattner cofounded Yembo in 2016 to develop an AI-based tool for moving companies that creates an inventory of objects in a home by analyzing video taken with a smartphone. Today, the startup employs 70 people worldwide and operates in 36 countries, and Rattner says he’s excited to get out of bed every morning because he’s building a product that simply wouldn’t exist otherwise. Zach Rattner Employer: Yembo Occupation: Chief technology officer and cofounder Education: Bachelor’s degree in computer engineering, Virginia Tech “I’m making a dent in the universe,” he says. “We are bringing about change. We are going into an industry and improving it.” How Yembo grew out of a family business Rattner has his wife to thank for his startup idea. From 2011 to 2015, she worked for a moving company, and she sometimes told him about the challenges facing the industry. A major headache for these companies, he says, is the time-consuming task of taking a manual inventory of everything to be moved. At the time, he was a software engineer in Qualcomm’s internal incubator in San Diego, where employees’ innovative ideas are turned into new products. In that role, he got a lot of hands-on experience with AI and computer vision, and he realized that object-detection algorithms could be used to automatically catalog items in a house. Rattner reports that his clients are able to complete three times more inspections in a day than traditional methods. Also his customers have increased their chances of getting jobs by 27 because they’re able to get quotes out faster than the competition, often in the same day. “Comparing Yembo’s survey to a virtual option like Zoom or FaceTime, our clients have reported being able to perform three to five times as many surveys per day with the same headcount,” he says. “If you compare us to an in-house visit, the savings are even more since Yembo doesn’t have drive time.” Getting used to not being an expert In 2016, he quit his job to become a consultant and work on his startup idea in his spare time. A few months later, he decided the idea had potential, and he convinced a former Qualcomm colleague, Siddharth Mohan, to join him in cofounding Yembo. Rattner admits that the responsibilities that come with starting a new business took some getting used to. In the early days, you’re not only building the technology, he says, you also have to get involved in marketing, finance, sales, and a host of other areas you have little experience in. “If you try to become that rigorous expert at everything, it can be crippling, because you don’t have enough time in the day,” Rattner says. “You just need to get comfortable being horrible at some things.” As the company has grown, Rattner has become less hands-on, but he still gets involved in all aspects of the business and is prepared to tackle the most challenging problems on any front. In 2020, the company branched out, developing a tool for property insurers by adapting the original AI algorithms to provide the information needed for an accurate insurance quote. Along with cataloging the contents of a home, this version of the AI tool extracts information about the house itself, including a high-fidelity 3D model that can be used to take measurements virtually. The software can also be used to assess damage when a homeowner makes a claim. “It feels like it’s a brand-new startup again,” Rattner says. A teenage Web developer From a young age, Rattner had an entrepreneurial streak. As a 7-year-old, he created a website to display his stamp collection. By his teens, he was freelancing as a Web developer. “I had this strange moment where I had to confess to my parents that I had a side job online,” he says. “I told them I had a couple of hundred dollars I needed to deposit into their bank account. They weren’t annoyed; they were impressed.” When he entered Virginia Tech in 2007 to study computer engineering, he discovered his roommate had also been doing freelance Web development. Together they came up with an idea for a tool that would allow people to build websites without writing code. They were accepted into a startup incubator to further develop their idea. But acceptance came with an offer of only US $15,000 for funding and the stipulation that they had to drop out of college. As he was writing the startup’s business plan, Rattner realized that his idea wasn’t financially sustainable long term and turned the offer down. “That is where I learned there’s more to running a startup than just the technology,” he says. This experience reinforced his conviction that betting everything on one great business idea wasn’t a smart move. He decided to finish school and get some experience at a major tech company before striking out on his own. Managing Qualcomm’s internal incubator In 2010, the summer before his senior year, he interned at Qualcomm. As 4G technology was just rolling out, the company was growing rapidly, and it offered Rattner a full-time job. He joined in 2011 after earning his bachelor’s degree in computer engineering. Rattner started out at Qualcomm as a modem software engineer, working on technology that measured cellphone signal strength and searched for the best cell connections. He took algorithms designed by others and used his coding skills to squeeze them onto the meager hardware available on cellphones of the era. Rattner says the scale of Qualcomm’s operations forced him to develop a rigorous approach to engineering quality. “You just need to get comfortable being horrible at some things.” “If you ship code on something that has a billion installs a year and there’s a bug, it will be found,” he says. Eventually, Rattner decided there was more to life than signal bars, and he began looking for new career opportunities. That’s when he discovered Qualcomm’s internal incubator. After having one of his ideas accepted and following the project through to completion, Rattner accepted a job to help to manage the program. “I got as close as I could to running a startup inside a big company,” he says. A book about running a startup Rattner wrote a book about his journey as a startup founder called Grow Up Fast, which he self-published last year. In it, he offers a few tips for those looking to follow in his footsteps. Rattner suggests developing concrete skills and obtaining experience before trying to make it on your own. One way to do this is to get a job at a big tech company, he says, since they tend to have a wealth of experienced employees you can learn from. It’s crucial to lean on others, he writes. Joining startup communities can be a good way to meet people in a similar situation whom you can turn to for advice when you hit roadblocks. And the best way to master the parts of the job that don’t come naturally to you is to seek out those who excel at them, he points out. “There’s a lot you can learn from just observing, studying, and asking questions of others,” he says. Most important, Rattner advises, is to simply learn by doing. “You can’t think of running a business as if you’re at school, where you study, practice, and eventually get good at it, because you’re going to be thrown into situations that are completely unforeseen,” he says. “It’s about being willing to put yourself out there and take that first step.”

  • 5 Ways to Strengthen the AI Acquisition Process
    by Cari Miller on 26. Marta 2024. at 18:00

    In our last article, A How-To Guide on Acquiring AI Systems, we explained why the IEEE P3119 Standard for the Procurement of Artificial Intelligence (AI) and Automated Decision Systems (ADS) is needed. In this article, we give further details about the draft standard and the use of regulatory “sandboxes” to test the developing standard against real-world AI procurement use cases. Strengthening AI procurement practices The IEEE P3119 draft standard is designed to help strengthen AI procurement approaches, using due diligence to ensure that agencies are critically evaluating the AI services and tools they acquire. The standard can give government agencies a method to ensure transparency from AI vendors about associated risks. The standard is not meant to replace traditional procurement processes, but rather to optimize established practices. IEEE P3119’s risk-based-approach to AI procurement follows the general principles in IEEE’s Ethically Aligned Design treatise, which prioritizes human well-being. The draft guidance is written in accessible language and includes practical tools and rubrics. For example, it includes a scoring guide to help analyze the claims vendors make about their AI solutions. The IEEE P3119 standard is composed of five processes that will help users identify, mitigate, and monitor harms commonly associated with high-risk AI systems such as the automated decision systems found in education, health, employment, and many public sector areas. An overview of the standard’s five processes is depicted below. Gisele Waters Steps for defining problems and business needs The five processes are 1) defining the problem and solution requirements, 2) evaluating vendors, 3) evaluating solutions, 4) negotiating contracts, and 5) monitoring contracts. These occur across four stages: pre-procurement, procurement, contracting, and post-procurement. The processes will be integrated into what already happens in conventional global procurement cycles. While the working group was developing the standard, it discovered that traditional procurement approaches often skip a pre-procurement stage of defining the problem or business need. Today, AI vendors offer solutions in search of problems instead of addressing problems that need solutions. That’s why the working group created tools to assist agencies with defining a problem and to assess the organization’s appetite for risk. These tools help agencies proactively plan procurements and outline appropriate solution requirements. During the stage in which bids are solicited from vendors (often called the “request for proposals” or “invitation to tender” stage), the vendor evaluation and solution evaluation processes work in tandem to provide a deeper analysis. The vendor’s organizational AI governance practices and policies are assessed and scored, as are their solutions. With the standard, buyers will be required to get robust disclosure about the target AI systems to better understand what’s being sold. These AI transparency requirements are missing in existing procurement practices. The contracting stage addresses gaps in existing software and information technology contract templates, which are not adequately evaluating the nuances and risks of AI systems. The standard offers reference contract language inspired by Amsterdam’s Contractual Terms for Algorithms, the European model contractual clauses, and clauses issued by the Society for Computers and Law AI Group. “The working group created tools to assist agencies with defining a problem and to assess the organization’s appetite for risk. These tools help agencies proactively plan procurements and outline appropriate solution requirements.” Providers will be able to help control for the risks they identified in the earlier processes by aligning them with curated clauses in their contracts. This reference contract language can be indispensable to agencies negotiating with AI vendors. When technical knowledge of the product being procured is extremely limited, having curated clauses can help agencies negotiate with AI vendors and advocate to protect the public interest. The post-procurement stage involves monitoring for the identified risks, as well as terms and conditions embedded into the contract. Key performance indicators and metrics are also continuously assessed. The five processes offer a risk-based approach that most agencies can apply across a variety of AI procurement use cases. Sandboxes explore innovation and existing processes In advance of the market deployment of AI systems, sandboxes are opportunities to explore and evaluate existing processes for the procurement of AI solutions. Sandboxes are sometimes used in software development. They are isolated environments where new concepts and simulations can be tested. Harvard’s AI Sandbox, for example, enables university researchers to study security and privacy risks in generative AI. Regulatory sandboxes are real-life testing environments for technologies and procedures that are not yet fully compliant with existing laws and regulations. They are typically enabled over a limited time period in a “safe space” where legal constraints are often “reduced” and agile exploration of innovation can occur. Regulatory sandboxes can contribute to evidence-based lawmaking and can provide feedback that allows agencies to identify possible challenges to new laws, standards and technologies. We sought a regulatory sandbox to test our assumptions and the components of the developing standard, aiming to explore how the standard would fare on real-world AI use cases. In search of sandbox partners last year, we engaged with 12 government agencies representing local, regional, and transnational jurisdictions. The agencies all expressed interest in responsible AI procurement. Together, we advocated for a sandbox “proof of concept” collaboration in which the IEEE Standards Association, IEEE P3119 working group members, and our partners could test the standard’s guidance and tools against a retrospective or future AI procurement use case. During several months of meetings we have learned which agencies have personnel with both the authority and the bandwidth needed to partner with us. Two entities in particular have shown promise as potential sandbox partners: an agency representing the European Union and a consortium of local government councils in the United Kingdom. Our aspiration is to use a sandbox to assess the differences between current AI procurement procedures and what could be if the draft standard adapts the status quo. For mutual gain, the sandbox would test for strengths and weaknesses in both existing procurement practices and our IEEE P3119 drafted components. After conversations with government agencies, we faced the reality that a sandbox collaboration requires lengthy authorizations and considerations for IEEE and the government entity. The European agency for instance navigates compliance with the EU AI Act, General Data Protection Regulation, and its own acquisition regimes while managing procurement processes. Likewise, the U.K. councils bring requirements from their multi-layered regulatory environment. Those requirements, while not surprising, should be recognized as substantial technical and political challenges to getting sandboxes approved. The role of regulatory sandboxes, especially for AI-enabled public services in high-risk domains, is critical to informing innovation in procurement practices. A regulatory sandbox can help us learn whether a voluntary consensus-based standard can make a difference in the procurement of AI solutions. Testing the standard in collaboration with sandbox partners would give it a better chance of successful adoption. We look forward to continuing our discussions and engagements with our potential partners. The approved IEEE 3119 standard is expected to be published early next year and possibly before the end of this year.

  • Unico's Battery Testing Enters a Competitive Industry
    by Willie Jones on 25. Marta 2024. at 19:40

    As EV production and, with it, battery production accelerate over the next decade, so too will the demand for rigorous EV battery testing. But producing a battery that will stand up to the task of powering an electric vehicle for years under different weather conditions and unpredictable usage patterns is no mean feat. Battery manufacturers will have to turn out many more units suitable for EV demands while still performing the array of tests they must carry out to assure automakers and consumers that the batteries they’re turning out meet basic standards for performance, safety, and durability. At the International Battery Seminar & Exhibit held 12-15 March in Orlando, many of the world’s leading battery makers, test equipment manufacturers, and theorists gathered to talk about advances in the field and to present new products claiming to make the battery production process quicker and cheaper. One such product family was the BAT300 series electric power testing devices made by Unico. Unico says the compact units are well suited for use at three critical battery-production stages: cell formation; fault testing as cells are gathered into packs and packs are gathered into full batteries; and for end-of-the-line charge cycling to help predict how many charge-discharge cycles a battery will go through before it is no longer capable of holding enough charge for vehicle propulsion. Unico vice president of engineering Don Wright told IEEE Spectrum that the BAT300 series is also good for conducting capacity tests when a company wants to check cells it purchased from another manufacturer. “Battery companies have expended enormous effort on predictive modeling.” —Paul Kohl, Georgia Institute of Technology At the battery seminar, Wright showed off a BAT300 test channel designed to convert a 400 or 800 volt direct current down to a 10 volt current for cell formation or cell testing. The potential benefits of Unico’s product are significant enough that it was named a finalist for the battery seminar’s “Best in Show” award. But important questions remain. “It may be a wonderful product. But it’s such a complicated set of things [Unico] is claiming,” says Paul Kohl, a professor at the Georgia Institute of Technology’s School of Chemical and Biomolecular Engineering in Atlanta. “Does it really do all those things?” says Kohl, whose primary research interests include advanced interconnects for integrated circuits and electrochemical devices for energy conversion and storage including lithium-ion batteries and alkaline fuel cells. “If it did any one of those things really well,” says Kohl, “that would be amazing.” “Battery companies have expended enormous effort on predictive modeling, trying to be able to forecast when a battery will reach the end of its service life without having to cycle test it to that point,” says Kohl. “They’re all very private about exactly what they do, but everyone does pretty much the same thing. They cycle batteries and look for signatures in the current and voltage which follow a trend. And they’ll take a few batteries all the way to the end of life, then use that empirical data for extrapolating to the end of life for the thousands of other batteries that they put through only a few cycles.” Big manufacturers already have battery testing tools The open question is whether Unico’s product does it better than the tools major battery manufacturers such as LG or Samsung already use. There are thousands of publications about this, including plenty sponsored by the U.S. Department of Energy. “Does Unico have something that is somehow better than what everyone else has been doing?” Kohl says. “Only the market will tell.” Kohl says the BAT300 series might be wonderful for battery manufacturers which are not on the scale of LG or Samsung: ”Companies that are, by far, the world’s experts in how batteries age,” he says. Smaller companies that don’t have the in-house expertise the giants have acquired would suddenly have an advanced tool that can raise the quality of their batteries. “It would take a lot of faith for you to do a simple electrical test and guarantee that nothing bad will happen if you smash this battery with a hammer.” —Paul Kohl, Georgia Institute of Technology Though electric vehicle batteries rarely catch fire—at least compared with the propensity of fires in vehicles with internal combustion engines—the conflagrations caused by cheap, knockoff batteries at the low end of the EV market have engendered a somewhat unwarranted wariness about the relative safety of EVs. Kohl and Wright agree in their hope that equipment like Unico’s will help bring an end to companies buying or selling batteries that have not undergone testing protocols that make EV battery fires even more of a rarity. Still, Kohl points out, “In this unit they presented, they can only do so much. It can do an electrical test, looking for certain signatures that indicate issues that could cause electrical faults or premature loss of capacity at some later time, but [it doesn’t carry out] the array of Underwriters Laboratory–style physical tests where they pound batteries, heat them up, and drive nails through them to see what it takes to get them to catch fire or otherwise fail. It would take a lot of faith for you to do a simple electrical test and guarantee that nothing bad will happen if you smash this battery with a hammer, or if it gets into a car accident, or the temperature goes up to 150 degrees Fahrenheit.” Better battery testing tech could jumpstart the used EV market To be fair, Unico says that the BAT300 series is intended for applications where expensive, research-level tools deliver more than what’s needed, but cheap, entry-level tools don’t do quite enough. One possible application is the used vehicle market. Once cars have been driven off of car lots, tests that call for physical abuse are out of the question. That is the point at which detailed but straightforward and reasonably-priced electrical analysis would be indispensable. According to a study done by the United Kingdom’s Society of Motor Manufacturers and Traders (SMMT), the country’s used EV market, though tiny today at 1.6 percent of sales, will eventually be as large as today’s used car market for vehicles with internal combustion engines, which accounts for 82 percent of sales. “ Different makes and models test in different ways, so an individual manufacturer would struggle to test any other vehicles which their dealerships might sell after a trade-in,” says Alexander Johns, partnership lead at Altelium, a consultancy headquartered in London that provides warranties and insurance to companies that deal with batteries at any stage in their life cycle. Altelium also conducts battery health tests and issues battery health certificates that present an honest broker’s perception of what an electric vehicle should sell for on the secondary market. Because of the wide variance in how batteries are set up in electric vehicles, says Johns, “the [secondary] market is adopting third-party solutions which could offer nearly universally applicable test services.” That’s a sweet spot which seems like a natural fit for tools like the ones Unico produces. Perhaps insurers like Altelium will use tools like Unico’s to certify the capacity of EV batteries and tell a would-be purchaser the estimated number of remaining charge cycles before a car’s battery will need to be replaced. That all-important data will be critical to setting a purchaser’s expectations and go a long way toward leaving a good impression.

  • How to Boot Up a New Engineering Program
    by Michael Koziol on 25. Marta 2024. at 19:00

    Starting a new engineering program at a university is no simple task. But that’s just what Brandeis University in Waltham, Mass., is doing. By 2026, the university will offer an undergraduate engineering degree—but without creating an engineering department. Instead, Brandeis aims to lean on its strong liberal arts tradition, in hope of offering something different from the more than 3,500 other engineering programs in the United States accredited by the Accreditation Board for Engineering and Technology (ABET). IEEE Spectrum spoke with Seth Fraden, one of the new program’s interim cochairs, about getting a new engineering program up and running. What prompted offering an engineering degree? Seth Fraden: We saw that we had 90 percent of all the elements that are necessary for a vibrant engineering program—the basic sciences, math, physics, computer science, life science, all put in a social context through the liberal arts. We see our new program as a way of bridging science and society through technology, and it seems like a natural fit for us without having to build everything from scratch. Seth Fraden Seth Fraden is a professor of physics at Brandeis University. He is serving as one of the two interim cochairs for the university’s new engineering degree. Brandeis’s engineering degree will be accredited by ABET. Why is that important? Fraden: Being the new kids on the block in engineering, it’s natural to want to reassure the community at large that we’re committed to outstanding quality. Beyond that, ABET has very well-thought-out criteria for what defines excellence and leaves each individual program the freedom to define the learning objectives, the tools, the assessment, and how to continuously improve. It’s a set of very-well-founded principles that we would support, even in the absence of this certification. What is the first course you’re offering? Fraden: We’re doing an introduction to design. It’s a course in which the students develop prosthetics for both animal and human use. It’s open to all students at Brandeis, but it’s still quite substantive: They’re working in Python, they’re working with CAD programs, and they’re working on substantive projects using open-source designs. The idea is to get students excited about engineering, but also to have them learn the fundamentals of ideation—going from planning to design to fabrication, and then this will help them decide whether or not engineering is the major for them. How do you see liberal arts such as history and ethics being part of engineering? Fraden: Many of our students want to intervene in the world and transform it into a better place. If you solely focus on the production of the technology, you’re incapable of achieving that objective. You need to know the impact of that technology on society. How is this thing going to be produced? Who says what labor is going to go into manufacturing? What’s its life cycle? How’s it going to be disposed of? You need to have a full-throttled liberal arts education to understand the environmental, ecological, economical, and historical consequences of your intervention as a technologist. How will you develop an engineering culture? Fraden: We’re not going to have a department. It will be the only R1 [top-tier research institution] engineering major without a department. We see that as a strength, not a weakness. We’re going to embed new engineering faculty throughout all our sciences, in order to have a positive influence on the departments and to promote technology development. That said, we want there to be a strong engineering culture, and we want the students to have a distinctive engineering identity, something that a scientist like myself—though I am enthusiastic about engineering—doesn’t have in my bones. In order to do that, our instructors will each come from an engineering background, and will work together to build a culture of engineering. This article appears in the April 2024 print issue as “5 Questions for Seth Fraden.”

  • Inkjets Are for More Than Just Printing
    by Phillip W. Barth on 25. Marta 2024. at 18:00

    In the early 1980s, offices were noisy places, filled with the sound of metal striking inked ribbons to mark characters on paper. IBM Selectric typewriters clacked, daisy wheel printers clattered, and dot-matrix printers made loud ripping sounds. Today, those noises are gone. And though we do spend more time reading on screens, we haven’t stopped printing on paper. The main reason for the quiet? The inkjet printer. While laser printers do the big printing jobs in commercial settings, the inkjet printer has become the printer most of us use at home and at the office. The printhead of an inkjet printer performs a remarkable task. Even at the coarse resolution of 96 dots per inch (dpi), as was typical for the first models in the 1980s, the distance from dot center to dot center is a mere 260 micrometers. To fill a standard letter page that has 2.5-centimeter margins would require more than half a million individual ink droplets. Delivery of those tiny droplets involves moving them with very precise control, repeated a vast number of times as rapidly as possible. This process is ideally suited for microelectromechanical systems (MEMS), which are electronic devices with microscopic components that employ movement. If there is a way to package something in microscopic droplets with the appropriate fluid properties, chances are someone is looking to adapt inkjet technology to work with it. As with all microtechnology, the specs of inkjet systems have evolved considerably over time. A typical inkjet printhead in the mid-1980s had 12 nozzles working in parallel, each one emitting up to 1,350 droplets per second, to print 150 alphanumeric characters per second. Today, a high-end inkjet printhead used in a commercial printing press may contain 21,000 nozzles, each nozzle printing 20,000 to 150,000 dots per second. Each drop of ink may be just 1.5 picoliters—a picoliter is one-trillionth of a liter—and measure roughly 14 micrometers in diameter. Surpassing the visions of its creators, the inkjet technology used in these printers has found a host of applications beyond putting dots on paper. These include making DNA microarrays for genomics, creating electrical traces for printed circuit boards, and building 3D-printed structures. Future uses could include personalized medicine and development of advanced batteries. Indeed, a search for patents containing the word “inkjet” today returns more than 92,000 results. If there is a way to package something in microscopic droplets with the appropriate fluid properties, chances are someone is looking to adapt inkjet technology to work with it. How MEMS Transformed Inkjet Printing Inkjet technology dates back to 1948, when Swedish inventor Rune Elmqvist patented a chart recorder wherein a very thin glass tube emitting a continuous jet of ink was steered to make a trace on a moving strip of paper. A couple of years later, he demonstrated his invention in the form of a device for recording electrocardiograms. In 1965, Richard G. Sweet of Stanford University developed a chart recorder in which the jet of ink was broken into a uniform stream of electrically charged droplets. Diverter electrodes on either side of the stream could permit the drops to proceed straight to the paper, or else deflect them onto an absorbent pad or into a gutter to be collected and reused. In April 1984, the HP ThinkJet [top] ushered in the era of desktop inkjet printing. The Thinkjet’s ink cartridge [bottom] delivered thousands of microscopic droplets a second from 12 nozzles. The MEMS technology to perform that feat was entirely within the printhead.HP This technology is called continuous inkjet printing, and by 1976 IBM had incorporated it in a commercial printer, the IBM 6640. But continuous inkjets lose ink to evaporation even when recycling is used, limiting their appeal. To get around the wastefulness of continuous inkjets, others worked on developing drop-on-demand inkjet printers, where each orifice on the printhead emits one drop of ink at a time, avoiding the waste of a continual flow of drops. Surface tension holds the ink in place in a tiny open nozzle until a mechanism pushes the ink to eject a drop. Each drop hitting the paper creates a dot, and moving the printhead back and forth builds up an image. A printhead with multiple orifices can emit many drops of ink simultaneously, so each pass of the printhead across the page adds a strip of the image, not just a single drop-thin line. In the late 1970s, Siemens was the first to sell a drop-on-demand inkjet printer. It came not as a stand-alone device like a modern desktop printer, but as an integral part of a computer terminal, the Siemens PT80i (Printer Terminal 80 Inkjet). The printer used piezoelectric actuators surrounding 12 ink tubes, which fed 12 nozzles to shoot ink droplets, printing 270 characters per second. Piezoelectric devices rely on how some materials, such as ceramic lead-zirconate-titanate (PZT), change shape when subjected to a voltage. This effect has proved extremely useful in MEMS in general, for generating precise forces and motion on command. If a layer of PZT is bonded to a nonpiezoelectric material, forming what’s called a bimorph, it will bend when exposed to a voltage. In the piezoelectric inkjet nozzle, the bending of the bimorph pushes ink out of the orifice. [For another application of piezoelectric MEMS technology, see “How Ultrasound Became Ultra Small.”] The HP Jet Fusion 5200 industrial 3D printer uses an inkjet process to build parts out of nylon, polypropylene, or polyurethane.HP This novel printing technology, however, was not yet as dependable as proven impact printers in the 1970s, and the whole Siemens terminal became unusable if the printer failed, so it didn’t catch on. Meanwhile, researchers at both Hewlett-Packard and Canon noticed that ink would boil and splatter when exposed to a hot element like a soldering iron, and they decided to turn that splattering into a useful inkjet printing mechanism. They knew that a resistor could be used as a heating element and could be miniaturized with the same technology as that used for integrated circuits. In the printers they built, each ink nozzle contains a resistor instead of a piezoelectric actuator. An electrical pulse heats the resistor, which flash-boils a thin layer of the ink, forming a rapidly expanding vapor bubble that pushes a droplet of ink out through the orifice. This work led to two competing versions of thermal inkjet technology coming to market at nearly the same time 40 years ago. (The same year, 1984, Epson introduced a stand-alone piezoelectric inkjet printer.) Hewlett-Packard’s HP ThinkJet was its first desktop inkjet printer based on its thermal technology, and it was designed to connect to a personal computer for everyday printing. It had an immediate advantage over the recently developed laser printers: It was much cheaper. A desktop laser printer from HP cost US $3,500 (about $10,500 today); HP’s 2225A ThinkJet cost only $495 ($1,500 today). Inkjet printers also used far less power than laser printers did and were quieter. Admittedly, inkjets didn’t have great resolution—96 dpi compared with 300 for laser printers in those early days—and they were slow. But the advantages outweighed the disadvantages (more so as the technology improved), and inkjet printers came to dominate the desktop and home printer markets. Today, more than 20 companies make inkjet printers, generating a market of more than $100 billion annually and continuing to grow at more than 8 percent per year. Printing DNA Microarrays With Inkjets While the business of making inkjet printers matured and grew, some companies began exploring what other kinds of “ink” might be delivered with an inkjet. One of these was Agilent Technologies, a spin-off of Hewlett Packard with a focus on life-science and chemical-analysis technologies. Agilent developed a way to print strands of DNA from the four nucleic acid bases—cytosine (C), guanine (G), adenine (A), and thymine (T). Specifically, the company adapted existing DNA chemistries plus inkjet printing techniques to build microarrays of DNA on glass slides for genomics work, such as measuring which genes are being expressed in an organism under various conditions. Academic researchers have shared open-source methods for converting existing inkjet printers to build their own microarrays, albeit with specs that are much more modest than the commercial systems. A DNA microarray consists of a substrate, usually glass, with an array of small regions called spots where DNA strands are attached. Agilent produces arrays with as many as a million spots on a single 2.5-by-7.6-cm slide. An open-source system puts up to 10,000 in a somewhat smaller area. Each DNA strand is made of sequences of the bases C, G, A, and T. In double-stranded DNA, the strands have complementary sequences, which join up like rungs of a ladder, C joining with G, and A with T. In the resting state [top] of an inkjet nozzle, the ink is held in place by surface tension. In a thermal inkjet nozzle [left], a voltage pulse to the heating resistor flash-vaporizes a thin layer of ink, producing an expanding vapor bubble [left, middle] that pushes a droplet of ink out through the orifice [left, bottom]. In less than a millisecond the vapor recondenses and the chamber cools and refills with ink, returning the nozzle to the resting state. A piezoelectric inkjet nozzle [right] is driven by a piezoelectric bimorph, which bends when a voltage is applied [right, middle] to push out an ink droplet [right, bottom]. A DNA microarray uses single-stranded DNA, and each spot has millions of strands with a common sequence. When a sample with copies of the complementary strand washes over the spot, those strands bind together with the strands anchored in the spot. The sample strands are tagged with fluorescent molecules, and the user learns which DNA sequences were present in the sample by examining which spots light up. In Agilent’s method for fabricating a microarray, the printer makes multiple passes over the substrate, each pass adding one base to each strand in the spots, with intermediate steps to prepare for the next pass. Adding a base is actually a three-step process. Each of the growing strands in the microarray spots has a molecular “cap” at the end that prevents the indiscriminate addition of more bases. So the first step is to remove or deactivate those caps by washing a solution over the nascent microarray. The second step is analogous to printing a page: At each spot on the microarray, the inkjet adds a dot of liquid containing the next monomer molecule (modified versions of C, G, A, or T) to be added to the end of the strand. These monomers each include a new cap so that only one molecule gets added to each strand. Although the newly added monomers are now attached to the strands, the connection is not fully stable, and so the third step applies an oxidizer solution that modifies the bonds, fully integrating the new monomers into the DNA structure. Rinse and repeat. The versatility of the open-source inkjet construction allows researchers to rapidly build prototype arrays with whatever sequences they want to try out. A new array can be designed, synthesized, and used to analyze DNA in a single day. One group reported a cycle time of 10 to 20 minutes to attach each base with their system, or about 13 hours to produce a batch of arrays, each with about 10,000 spots containing 40-base strands. For comparison’s sake, Agilent’s commercial microarrays typically have strands up to 60 bases long. Agilent also uses its inkjet system to synthesize another genomic workhorse known as an oligonucleotide library. The process is the same as for making a microarray, but at the end all the strands are cleaved from the substrate, dried, and packaged together in a single tube for the customer. Agilent’s inkjet-printed libraries have strands up to about 230 bases long. 3D Printing Using Two Inkjet Inks Invent Medical—a Czech Republic startup partnered with HP—custom printed this helmet for correcting head-shape deformities [top]. An HP Jet Fusion 5200 printed this lipstick holder [bottom].Top: Invent Medical/HP; bottom: YOO Makeup/HP In addition to printing two-dimensional pages and building one-dimensional molecular strands, inkjet technology has for many years been used to produce three-dimensional objects. One approach is a variant of powder-bed 3D printing, in which objects are built up by fusing or binding layers of powder in the desired pattern. The inkjet printhead applies droplets of a liquid binding agent to each layer of powder in the regions that will form the finished 3D items. The HP Multi Jet Fusion (MJF) line of 3D printers extends this approach by depositing two types of ink: One is a binding promoter and the other a detailing agent, which is applied at the edges of the pattern to prevent the promoter from bleeding into the surrounding powder. A printhead carrying a wide array of inkjet nozzles dispenses these inks, and the array is quickly followed by a lighting bar to heat the powder, fusing it in the regions where the binding promoter is present. A fresh layer of powder is then spread over the entire printing area in readiness for the next cycle of the process. At the end, compressed air and a vacuum hose remove the unfused powder to reveal the completed 3D objects. The HP MJF printers perform this in a volume of up to 38 by 28 by 38 cm. This model of a handheld vacuum cleaner with colored, translucent, and transparent parts was printed in one piece on a Mimaki inkjet 3D printer.Mimaki Engineering Co. A quite different approach has been taken by Mimaki Engineering Co. of Japan, which has introduced 3D printers with piezoelectric inkjet heads that dispense droplets of resin. The resins are photopolymers that are cured by ultraviolet light-emitting diodes after each layer is printed. Instead of using a powder bed that fills the entire build area, the printer deposits the resins on top of the growing structure. To deal with steep overhangs—such as an outstretched arm of a figurine—one of the resins produces a water-soluble material, which is used to build supports where needed. After the build is finished, these supports can be dissolved away. Seven other resins provide colors that can include CMYK—the familiar cyan, magenta, yellow, and black inks of consumer inkjet printers—as well as white and clear, for a total of 10 million color combinations, comparable to the color depth that the human eye can discriminate. The resulting parts can combine solid color, colored transparency and translucency, and colorless transparency. The printer provides a volume for building that measures 51 by 51 by 30 cm. Unlike with a powder-bed machine, small test parts can be made without filling the entire volume. In general, however, the Mimaki approach is slower than that of the HP MJF because it uses smaller printheads instead of a wide one that can cross the entire area in one sweep. Inkjet’s Future Inkjet printing’s strength is the ability to pattern various inks over large areas in short, rapid production runs at a reasonable cost. It cannot generally compete with standard high-volume production approaches, because those will usually be cheaper. Thus, a car enthusiast, for instance, may embrace 3D inkjet printing to make bespoke parts for repairs or other tinkering, but a high-volume car-parts manufacturer is not going to introduce such printers to its factory lines. Similarly, a company may build individual figurines from a customer’s design, printed by 3D inkjet, but the same technique won’t be economical for mass-producing models of the latest superhero. With many potential applications, it isn’t clear if there is a niche where the inkjet approach will win. An example is the use of 3D inkjet printing for personalized medicine. The idea is to produce tablets of a medication customized for a specific patient. Such personalized pills can include simple fine-tuning of the dose for an individual, as well as adjustments to the drug’s release rate—from very rapid to slow and sustained—through modifications to the binding agents and structure of the tablet. Rather than juggling multiple medications on a complicated schedule each day, a patient could take a single daily polypill—a 3D-printed tablet containing multiple medications, each with a different rate of release. Researchers are exploring how to adapt existing 3D printing techniques, including inkjet, to make these personalized medications. Inkjet systems are particularly suited for printing drugs in the form of thin films, such as transdermal patches to be applied to the skin and buccal films to be held in the cheek, where drugs can pass directly to the bloodstream without first going through the digestive system. A DNA microarray can be fabricated using an inkjet system to build custom-designed strands of DNA at each spot of the array. The printhead delivers a droplet of “ink” [left] containing modified monomers of one nucleotide [G, C, A, or T] to each spot. On the first print cycle, these monomers attach to the chemically treated glass surface. On subsequent print runs [right], a single monomer joins on the end of each growing DNA strand. Each monomer includes a protective cap to prevent other monomers from joining. Additional processes [not depicted here] wash away the nucleotide ink, apply a catalyst to complete the monomer bonding, and strip away the protective caps in preparation for the next printing step.Chris Philpot These printed personalized medicines, however, would be expensive compared to fixed doses rolling off standard high-volume production lines. Thus the technique is likely to be reserved for relatively rare conditions. Another potential application of 3D inkjet printing is in the fabrication of advanced lithium-ion batteries. The charging and discharging of these batteries relies on lithium ions moving from the battery’s electrodes to its electrolyte and back again, in the process releasing or absorbing electrons that produce the current flow. The energy-storage density of the standard electrode design can be increased by using thicker electrodes, but this compromises the power density—the rate of energy release—because a smaller proportion of the electrode is in close contact with the electrolyte. A 3D inkjet could build electrodes with a detailed microstructure that allows the electrolyte to penetrate throughout the electrode volume. This could boost the ability of the active lithium ions and electrons to reach the entire electrode efficiently even when the electrode is larger, thereby increasing the energy storage and power density in tandem. For this vision to become a reality, however, researchers will need to learn more about how to formulate the “inks” for printing these electrodes: What are the best particle sizes and solvents to make an ink with fluid properties suitable for use in an inkjet system and that will produce stable printed structures with good electrochemical properties? We think it is unlikely, however, that inkjet printing could compete with high-volume manufacturing on price. Inkjet printing of prototypes, on the other hand, may uncover an optimal battery design that can then be adapted for production by conventional techniques. Inkjet systems have been demonstrated for a wide variety of applications beyond what we have discussed above: Living cells can be printed, for instance, to form tissue structures for in vitro experiments. MEMS such as microscopic motors have been printed using inks containing nanoparticles of gold and silver as conductors and resin-based inks to act as insulators. Flexible sensors for health care monitoring have been printed using an electrically conducting polymer that responds to temperature differentials. And then there are all the ways inkjets are used to create images on media other than office printouts, such as printing of textiles, inkjet robots to apply custom automotive paint jobs, and the “Giclée” printing of fine art using archival-quality inks and substrates. Each of these applications is like a colored dot on the vast canvas of human technology and activity. And while the dots from inkjets, powered by MEMS, may be only a single color among many others on that metaphorical page, the picture would be very different without them.

  • Reach Your Space Goals With High-Bandwidth Devices
    by Avnet on 25. Marta 2024. at 15:29

    This sponsored article is brought to you by Avnet and AMD. The use of artificial intelligence (AI) in image processing has brought about significant advancements in the exploration of space and our own Earth. In space applications, images are captured using various instruments, such as telescopes, satellites, and rovers. However, processing large amounts of data can be time-consuming and require significant computational power. This is where AI and powerful processors come in. AI algorithms, with the support of industry-leading processors, can handle large amounts of data quickly and accurately. They can also learn from previous data to improve their analysis of new data. AI algorithms can be trained to recognize patterns in images, classify them, and identify anomalies. One of the main applications of AI in image processing for space is in the detection and classification of objects. For example, AI can be used to detect and classify asteroids or other celestial bodies. This information can be used to determine the trajectory of these objects and their potential impact on Earth. AI can also identify and classify space debris, which is a growing concern for space missions. Combining AI and Cutting-Edge Processors This also applies to objects closer to home. Earth-observing instruments rely on AI algorithms and processors to collect large amounts of information each day. However, transforming that raw data into information that can be acted upon is the real challenge, especially when instruments need to determine which data points are the most important. There are wildfires, harmful algal blooms, volcanic eruptions, and heavy snowfalls. If we can observe these events better, we will have an opportunity to react by making the world safer and cleaner for humans. AI and cutting-edge processors can also be used to improve the resolution of images captured by telescopes and satellites. By analyzing the data captured by these instruments, AI algorithms can remove noise and improve the clarity of the images. This can help scientists better understand the structure and composition of celestial bodies as well as detect water on other planets. By analyzing the spectral signatures of the surface of these planets, AI algorithms can detect the presence of water. This information can be used to determine if these planets are habitable. AI can also identify other important minerals and resources on these planets. AMD processors and advancements in AI are enabling limitless possibilities for space exploration. AMD processors are transforming image processing for space applications with high throughput, low latency, and unlimited in-orbit reconfigurability. This enables scientists to analyze large amounts of data quickly and accurately. When paired with AI algorithms, the AMD KintexTM UltraScale, AMD VersalTM SoCs, and AMD VirtexTM devices can detect and classify objects, improve the resolution of images, detect the presence of water and other imperative resources on other planets. AMD processors and advancements in AI are enabling limitless possibilities for space exploration. Learn how Avnet and AMD are enabling the next generation of space applications that bring AI, imaging and more together to help humans explore beyond our limits. We’ve compiled several case studies that illustrate these cutting-edge technologies in real-world applications. Click here to explore how.

  • We Need to Decarbonize Software
    by Rina Diane Caballar on 23. Marta 2024. at 14:00

    Software may be eating the world, but it is also heating it. In December 2023, representatives from nearly 200 countries gathered in Dubai for COP28, the U.N.’s climate-change conference, to discuss the urgent need to lower emissions. Meanwhile, COP28’s website produced 3.69 grams of carbon dioxide (CO2) per page load, according to the website sustainability scoring tool Ecograder. That appears to be a tiny amount, but if the site gets 10,000 views each month for a year, its emissions would be a little over that of a one-way flight from San Francisco to Toronto. This was not inevitable. Based on Ecograder’s analysis, unused code, improperly sized images, and third-party scripts, among other things, affect the COP28 website’s emissions. These all factor into the energy used for data transfer, loading, and processing, consuming a lot of power on users’ devices. Fixing and optimizing these things could chop a whopping 93 percent from the website’s per-page-load emissions, Ecograder notes. While software on its own doesn’t release any emissions, it runs on hardware in data centers and steers data through transmission networks, which account for about 1 percent of energy-related greenhouse gas emissions each. The information and communications technology sector as a whole is responsible for an estimated 2 to 4 percent of global greenhouse gas emissions. By 2040, that number could reach 14 percent—almost as much carbon as that emitted by air, land, and sea transport combined. Within the sphere of software, artificial intelligence has its own sustainability issues. AI company Hugging Face estimated the carbon footprint of its BLOOM large language model across its entire life cycle, from equipment manufacturing to deployment. The company found that BLOOM’s final training emitted 50 tonnes of CO2—equivalent to about a dozen flights from New York City to Sydney. Green software engineering is an emerging discipline consisting of best practices to build applications that reduce carbon emissions. The green software movement is fast gaining momentum. Companies like Salesforce have launched their own software sustainability initiatives, while the Green Software Foundation now comprises 64 member organizations, including tech giants Google, Intel, and Microsoft. But the sector will have to embrace these practices even more broadly if they are to prevent worsening emissions from developing and using software. What Is Green Software Engineering? The path to green software began more than 10 years ago. The Sustainable Web Design Community Group of the World Wide Web Consortium (W3C) was established in 2013, while the Green Web Foundation began in 2006 as a way to understand the kinds of energy that power the Internet. Now, the Green Web Foundation is working toward the ambitious goal of a fossil-free Internet by 2030. Green Software Resources The Green Software Foundation offers a catalog of green software patterns for AI, the cloud, and the Web. The W3C’s Sustainable Web Design Community Group released a draft of its Web sustainability guidelines, with both tactical and technical recommendations for business and product strategy, user-experience design, Web development, and hosting and infrastructure. The draft guidelines also include impact and effort ratings to give software engineers an idea of the level of difficulty in terms of implementation and the level of impact in terms of sustainability. “There’s an already existing large segment of the software-development ecosystem that cares about this space—they just haven’t known what to do,” says Asim Hussain, chairperson and executive director of the Green Software Foundation and former director of green software and ecosystems at Intel. What to do, according to Hussain, falls under three main pillars: energy efficiency, or using less energy; hardware efficiency, or using fewer physical resources; and carbon-aware computing, or using energy more intelligently. Carbon-aware computing, Hussain adds, is about doing more with your applications during the periods when the electricity comes from clean or low-carbon sources—such as when wind and solar power are available—and doing less when it doesn’t. The Case for Sustainable Software So why should programmers care about making their software sustainable? For one, green software is efficient software, allowing coders to cultivate faster, higher-quality systems, says Kaspar Kinsiveer, a team lead and sustainable-software strategist at the software-development firm Helmes. These efficient systems could also mean lower costs for companies. “One of the main misconceptions about green software is that you have to do something extra, and it will cost extra,” Kinsiveer says. “It doesn’t cost extra—you just have to do things right.” Green software is efficient software, allowing coders to cultivate faster, higher-quality systems. Other motivating factors, especially on the business side of software, are the upcoming legislation and regulations related to sustainability. In the European Union, for instance, the Corporate Sustainability Reporting Directive requires companies to report more on their environmental footprint, energy usage, and emissions, including the emissions related to the use of their products. Yet other developers may be motivated by the climate crisis itself, wanting to play their part in fostering a habitable planet for the coming generations. And software engineers have tremendous influence on the actual purpose and emissions of what they build. “It’s not just lines of code. Those lines have an impact on human beings,” says June Sallou, a postdoctoral researcher specializing in sustainable-software engineering at the Delft University of Technology, in the Netherlands. Because of AI’s societal impact in particular, she adds, developers have a responsibility to ensure that what they’re creating isn’t damaging the environment. Building Greener Websites and Apps The makers of COP28’s website could have taken a page from directories like Lowwwcarbon, which highlights examples of existing low-carbon websites. The company website of the Netherlands-based Web design and branding firm Tijgerbrood, for instance, emits less than 0.1 grams of carbon per page view. Creating sustainable websites like Tijgerbrood’s is a team effort that involves different roles—from business analysts who define software requirements to designers, architects, and those in charge of operations—and includes green practices that can be applied at each stage of the software-development process. Tips for Greener Websites and Apps First, analysts will have to consider if the feature, app, or software they’re designing should even be developed in the first place. Tech is often about creating the next new thing, but making software sustainable also entails decisions on what not to build, and that may require a shift in mind-set. The design stage is all about choosing efficient algorithms and architectures. “Think about sustainability before going into the solution—and not after,” says Chiara Lanza, a researcher at the Sustainable AI unit of the Centre Tecnològic de Telecomunicacions de Catalunya, in Barcelona. During the development stage, programmers need to focus on optimizing code. “We need the overall amount of energy we’re using to run software to go down. Some of that will come from writing [code] efficiently,” says Hannah Smith, a sustainable digital tech consultant and director of operations at the Green Web Foundation. Tijgerbrood’s website optimized the company’s code by using low-resolution images and modern image formats, loading animations only when a user scrolls them into view, and removing unnecessary code. These techniques help speed up data transfer, loading, and processing on a user’s device. The website also uses minimal JavaScript. “When a user loads a website [with] a lot of JavaScript, it causes them to use a lot more energy on their own device because their device is having to do all the work of reading the JavaScript and running [it],” explains Smith. When it comes to operations, one of the most impactful actions you can take is to select a sustainable Web hosting or cloud-computing provider. The Green Web Foundation has a tool to check if your website runs on green energy, as well as a directory of hosting providers powered by renewable energy. You can also ask your hosting provider if you can scale how your software runs in the cloud so that peak usage is powered by green energy or pause or switch off certain services during nonpeak hours. AI the Green Way Programmers can apply green software strategies when developing AI as well. Trimming training data is one of the major ways to make AI systems greener. Starting with data collection and preprocessing, it’s worth thinking about how much data is really needed to do the job. It may pay to clean the dataset to remove unnecessary data, or select only a subset of the dataset for training. “The larger your dataset is, the more time and computation it will take for the algorithm to go through all the data,” hence using up more energy, says Sallou. For instance, in a study of six different AI algorithms that detect SMS spam messages, Sallou and her colleagues found that the random forest algorithm, which combines the output of a collection of decision trees to make a prediction, was the most energy-greedy algorithm. But reducing the size of the training dataset to 20 percent—only 1,000 data points out of 5,000—dropped the energy consumption of training by nearly 75 percent, with only a 0.06 percent loss in accuracy. Choosing a greener algorithm could also save carbon. Tools like CodeCarbon and ML CO2 Impact can help make the choice by estimating the energy usage and carbon footprint of training different AI models. Tips for Greener AI Tools for Measuring Software’s Carbon Footprint To write green code, developers need a way of measuring the actual carbon emissions across a system’s entire life cycle. It’s a complex feat, given the myriad processes involved. If we take AI as an example, its life cycle encompasses raw material extraction, materials manufacturing, hardware manufacturing, model training, model deployment, and disposal—and not all of these stages have available data. “We don’t understand huge parts of the ecosystem at the moment, and access to reliable data is tough,” Smith says. The biggest need, she adds, is “open data that we can rely on and trust” from big tech data-center operators and cloud providers like Amazon, Google, and Microsoft. Until that data surfaces, a more practical approach would be to measure how much power software consumes. “Just knowing the energy consumption of running a piece of software can impact how software engineers can improve the code,” Sallou says. Developers themselves are heeding the call for more measurement, and they’re building tools to meet this demand. The W3C’s Sustainable Web Design Community Group, for instance, plans to provide a test suite to measure the impacts of implementing its Web sustainability guidelines. Similarly, the Green Software Foundation wrote a specification to calculate the carbon intensity of software systems. For accurate measurements, Lanza suggests isolating the hardware in which a system runs from any other operations and to avoid running any other programs that could influence measurements. Other tools developers can use to measure the impact of green software engineering practices include dashboards that give an overview of the estimated carbon emissions associated with cloud workloads, such as the AWS Customer Carbon Footprint Tool and Microsoft’s Azure Emissions Impact Dashboard; energy profilers or power monitors like Intel’s Performance Counter Monitor; and tools that help calculate the carbon footprint of websites, such as Ecograder, Firefox Profiler, and Website Carbon Calculator. Green Software Measurement Tools  Developers can use these tools to measure the impact of green software engineering practices. AI Estimate the energy usage and carbon footprint of training AI models with these tools. carbontracker experiment-impact-tracker ML CO2 Impact Cloud These dashboards give an overview of the estimated carbon emissions associated with cloud workloads. AWS Customer Carbon Footprint Tool Google Cloud Carbon Footprint Microsoft Azure Emissions Impact Dashboard Cloud Carbon Footprint (free, open source, provider agnostic) Code Integrate emissions estimation at the code level using these tools. Carbon-Aware SDK CodeCarbon Impact Framework Middleware These energy profilers or power monitors provide APIs (application programming interfaces) to measure power consumption of apps or track energy metrics of processors. Intel’s Performance Counter Monitor PowerAPI Web These tools help calculate the carbon footprint of websites. Are my third parties green? CO2.js Ecograder Firefox Profiler Website Carbon Calculator The Future Is Green Green software engineering is growing and evolving, but we need more awareness to help the discipline to become more widespread. This is why, in addition to its Green Software for Practitioners course, the Green Software Foundation aims to create more training courses, some of which may even lead to certifications. Likewise, Sallou coteaches a graduate course in sustainable software engineering, whose syllabus is open and can be used as a foundation for anyone looking to build a similar course. Providing this knowledge to students early on, she says, could ensure they bring it to their workplaces as future software engineers. In the realm of artificial intelligence, Navveen Balani, an AI expert and Google Cloud Certified Fellow who also serves on the Green Software Foundation’s steering committee, notes that AI could inherently include green AI principles in the coming years, much like how security considerations are now an integral part of software development. “This shift will align AI innovation with environmental sustainability, making green AI not just a specialty but an implied standard in the field,” he says. As for the Web, Smith hopes the Green Web Foundation will cease to exist by 2030. “Our dream as an organization is that we’re not needed, we meet our goal, and the Internet is green by default,” she says. Kinsiveer has observed that in the past, software had to be optimized and built well because hardware then was lacking. As hardware performance and innovation leveled up, “the quality of programming itself went down,” he says. But now, the industry is coming full circle, going back to its efficiency roots and adding sustainability to the mix. “The future is green software,” Kinsiveer says. “I cannot imagine it any other way.”

  • Video Friday: Project GR00T
    by Evan Ackerman on 22. Marta 2024. at 18:07

    Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion. Eurobot Open 2024: 8–11 May 2024, LA ROCHE-SUR-YON, FRANCE ICRA 2024: 13–17 May 2024, YOKOHAMA, JAPAN RoboCup 2024: 17–22 July 2024, EINDHOVEN, NETHERLANDS Enjoy today’s videos! See NVIDIA’s journey from pioneering advanced autonomous vehicle hardware and simulation tools to accelerated perception and manipulation for autonomous mobile robots and industrial arms, culminating in the next wave of cutting-edge AI for humanoid robots. [ NVIDIA ] In release 4.0, we advanced Spot’s locomotion abilities thanks to the power of reinforcement learning. Paul Domanico, Robotics Engineer at Boston Dynamics talks through how Spot’s hybrid approach of combining reinforcement learning with model predictive control creates an even more stable robot in the most antagonistic environments. [ Boston Dynamics ] We’re excited to share our latest progress on teaching EVEs general-purpose skills. Everything in the video is all autonomous, all 1X speed, all controlled with a single set of neural network weights. [ 1X ] What I find interesting about the Unitree H1 doing a standing flip is where it decides to put its legs. [ Unitree ] At the MODEX Exposition in March of 2024, Pickle Robot demonstrated picking freight from a random pile similar to what you see in a messy truck trailer after it has bounced across many miles of highway. The piles of boxes were never the same and the demonstration was run live in front of crowds of onlookers 25 times over 4 days. No other robotic trailer/container unloading system has yet to demonstrate this ability to pick from unstructured piles. [ Pickle ] RunRu is a car-like robot, a robot-like car, with autonomy, sociability, and operability. This is a new type of personal vehicle that aims to create a “Jinba-Ittai” relationship with its passengers, who are not only always assertive, but also sometimes whine. [ ICD-LAB ] Verdie went to GTC this year and won the hearts of people but maybe not the other robots. [ Electric Sheep ] The “DEEPRobotics AI+” merges AI capabilities with robotic software systems to continuously boost embodied intelligence. The showcased achievement is a result of training a new AI and software system. [ DEEP Robotics ] If you want to collect data for robot grasping, using Stretch and a pair of tongs is about as affordable as it gets. [ Hello Robot ] The real reason why Digit’s legs look backwards is so that it doesn’t bang its shins taking GPUs out of the oven. Meanwhile, some of us can bake our GPUs without even needing an oven. [ Agility ] P1 is LimX Dynamics’ innovative point-foot biped robot, serving as an important platform for the systematic development and modular testing of reinforcement learning. It is utilized to advance the research and iteration of basic biped locomotion abilities. The success of P1 in conquering forest terrain is a testament to LimX Dynamics’ systematic R&D in reinforcement learning. [ LimX ] And now, this. [ Suzumori Endo Lab ] Cooking in kitchens is fun. BUT doing it collaboratively with two robots is even more satisfying! We introduce MOSAIC, a modular framework that coordinates multiple robots to closely collaborate and cook with humans via natural language interaction and a repository of skills. [ Cornell ] neoDavid is a Robust Humanoid with Dexterous Manipulation Skills, developed at DLR. The main focus in the development of neoDavid is to get as close to human capabilities as possible—especially in terms of dynamics, dexterity and robustness. [ DLR ] Welcome to our customer spotlight video series where we showcase some of the remarkable robots that our customers have been working on. In this episode we showcase three Clearpath Robotics UGVs that our customers are using to create robotic assistants for three different applications. [ Clearpath ] This video presents KIMLAB’s new three-fingered robotic hand, featuring soft tactile sensors for enhanced grasping capabilities. Leveraging cost-effective 3D printing materials, it ensures robustness and operational efficiency. [ KIMLAB ] Various perception-aware planning approaches have attempted to enhance the state estimation accuracy during maneuvers, while the feature matchability among frames, a crucial factor influencing estimation accuracy, has often been overlooked. In this paper, we present APACE, an Agile and Perception-Aware trajeCtory gEneration framework for quadrotors aggressive flight, that takes into account feature matchability during trajectory planning. [ Paper ] via [ HKUST ] In this video, we see Samuel Kunz, the pilot of the RSL Assistance Robot Race team from ETH Zurich, as he participates in the CYBATHLON Challenges 2024. Samuel completed all four designated tasks—retrieving a parcel from a mailbox, using a toothbrush, hanging a scarf on a clothesline, and emptying a dishwasher—with the help of an assistance robot. He achieved a perfect score of 40 out of 40 points and secured first place in the race, completing the tasks in 6.34 minutes. [ CYBATHLON ] Florian Ledoux is a wildlife photographer with a deep love for the Arctic and its wildlife. Using the Mavic 3 Pro, he steps onto the ice ready to capture the raw beauty and the stories of this chilly, remote place. [ DJI ]

  • 6G Terahertz Devices Demand 3D Electronics
    by Michael Koziol on 22. Marta 2024. at 14:50

    Smartphones have a scaling problem. Specifically, the radio-frequency (RF) filters that every phone—and every wireless device in general—uses to extract information from isolated wireless signals are too big, too flat, and too numerous. And without these filters, wireless communications simply wouldn’t work at all. “They are literally the entire backbone of wireless systems,” says Roozbeh Tabrizian, a researcher at the University of Florida in Gainesville. So Tabrizian and other researchers at the University of Florida have now developed an alternative three-dimensional RF filter that can save space in smartphones and IoT devices. If these 3D filters one day replace bulky stacks of 2D filters, it would leave more room for other components, such as batteries. They could also make it easier to push wireless communications into terahertz frequencies, an important spectrum range being researched for 6G cellular technologies. “Very soon, we’ll have trillions of devices connected to wireless networks, and you need new bands: You just need a whole range of frequencies and a whole range of filters.” —Roozbeh Tabrizian, University of Florida The filters currently used by wireless devices are called planar piezoelectric resonators. Each resonator is a different thickness—a resonator’s specific thickness is directly tied to the band of wireless frequencies that the resonator responds to. Any wireless device that relies on multiple bands of spectrum—increasingly commonplace today—requires more and more of these flat resonators. But planar resonator technology has revealed a number of weaknesses as wireless signals proliferate and as the spectrum those signals relies on broadens. One is that it’s getting more difficult to make the filters thin enough for the new swaths of spectrum that wireless researchers are interested in harnessing for next-gen communications. Another involves space. It’s proving increasingly challenging to cram all of the signal filters needed into devices. The vertical fins for ferroelectric-gate fin resonators can be constructed in the same manner as FinFET semiconductors.Faysal Hakim/Roozbeh Tabrizian/University of Florida “Very soon, we’ll have trillions of devices connected to wireless networks, and you need new bands: You just need a whole range of frequencies and a whole range of filters,” says Tabrizian. “If you open up a cellphone, there are five or six specific frequencies, and that’s it. Five or six frequencies cannot handle that. It’s as if you have five or six streets, and now you want to accommodate the traffic of a city of 10 million people.” To make the switch to a 3D filter, Tabrizian and his fellow researchers took a page from another industry that made the jump to the third dimension: semiconductors. When, in the continuous quest to shrink down chip sizes, it seemed like the industry might finally be hitting the end of the road, a new approach that raised electron channels above the semiconductor substrate breathed new life into Moore’s Law. The chip design is called FinFET (for “fin field-effect transistor,” where “fin” refers to the shark-fin-like vertical electron channel). “The fact that we can change the width of the fin plays a huge role in making the technology much more capable.” —Roozbeh Tabrizian, University of Florida “We definitely got inspired [by FinFETS],” says Tabrizian. “The fact that planar transistors were converted to fins was just to make sure the effective size of the transistor was smaller while having the same active area.” Despite taking inspiration from FinFETs, Tabrizian says there are some fundamental differences in the way the vertical fins need to be implemented for RF filters, compared to chips. “If you think of FinFETs, all the fins are nearly the same width. People are not changing the dimension of the fin.” Not so for filters, which must have fins of different widths. That way, each fin on the filter can be tuned to different frequencies, allowing one 3D filter to process multiple spectrum bands. “The fact that we can change the width of the fin plays a huge role in making the technology much more capable,” says Tabrizian. Tabrizian’s group have already manufactured multiple three-dimensional filters, called ferroelectric-gate fin (FGF) resonators, that spanned frequencies between 3 and 28 gigahertz. They also constructed a spectral processor comprised of six integrated FGF resonators that covered frequencies between 9 and 12 GHz (By way of comparision, 5G’s coveted midband spectrum falls between 1 and 6 GHz). The researchers published their work in January in Nature Electronics. It’s still early days for 3D filter development, and Tabrizian acknowledges that the road ahead is long. But again taking inspiration from FinFETs, he sees a clear path of development for FGF resonators. “The good news is we can already guess what a lot of these challenges are by looking at FinFET technology,” he says. Incorporating FGF resonators into commercial devices someday will require solving several manufacturing problems, such as figuring out how to increase the density of fins on the filter and improving the electrical contacts. “Fortunately, since we already have FinFETs going through a lot of these answers, the manufacturing part is already being addressed,” Tabrizian says. One thing the research group is already working on is the process design kit, or PDK, for FGF resonators. PDKs are commonplace in the semiconductor industry, and they function as a kind of guidebook for designers to fabricate chips based on components detailed by a chip foundry. Tabrizian also sees a lot of potential for future manufacturing to integrate FGF resonators and semiconductors into one component, given their similarities in design and fabrication. “It’s human innovation and creativity to come up with new types of architectures, which may revolutionize the way that we think about having resonators and filters and transistors.”

  • Here Are 6 Actual Uses for Near-Term Quantum Computers
    by Dina Genkina on 21. Marta 2024. at 19:08

    Although recent findings have poured cold water on quantum computing hype, don’t count the technology out yet. On 4 March, Google and XPrize announced a US $5 million prize to anyone who comes up with use cases for quantum computers. If that sounds like an admission that use cases don’t already exist, it isn’t, says Ryan Babbush, head of quantum algorithms at Google. “We do know of some applications that these devices would be quite impactful for,” he says. “A quantum computer is a special purpose accelerator,” says Matthias Troyer, corporate vice president of Microsoft Quantum and member of the Xprize competition’s advisory board. “It can have a huge impact for special problems where quantum mechanics can help you solve them.” The kinds of problems for which quantum computers could be useful hark back to their historical roots. In 1981, physicist Richard Feynman proposed the idea of a quantum computer as a means of simulating the full complexity of the quantum world. “The commercial impact of solving quantum systems is in chemistry, material science, and pharma.” —Matthias Troyer, Microsoft Quantum Since then, scientists have come up with ingenious algorithms to make quantum computers useful for non-quantum things, such as searching databases or breaking cryptography. However, the database search algorithms don’t promise viable speedups in the foreseeable future, and destroying Internet security seems like a dubious reason to build a new machine. But a recent study suggests that quantum computers will be able to simulate quantum phenomena of interest to several industries well before they can make headway in those other applications. “The commercial impact of solving quantum systems is in chemistry, material science, and pharma,” Troyer says. And these are industries of significance, Troyer adds. “From the Stone Age to the Bronze Age, the Iron Age, the Steel Age, the Silicon Age, we define progress through materials progress.” On that path to the possible new Quantum Age, here are a few examples with proven quantum advantage on machines that quantum computing researchers expect within the coming decade. And with any luck, Troyer hopes that the $5 million prize will incentivize the scientific community to find even more ways to put quantum algorithms to use in the real world. “The goal [of the prize] is that we want to have more quantum scientists get interested in not just developing quantum algorithms and the theory of them but also asking: Where can they be applied? How can we use quantum computers to tackle the world’s big problems?” Drug Metabolism In a 2022 paper published in PNAS, a collaboration between pharmaceutical company Boehringer Ingelheim, Columbia University, Google Quantum AI, and quantum simulation company QSimulate examined an enzyme called cytochrome P450. This particular enzyme is responsible for metabolizing roughly 70 percent of the drugs that enter the human body. The oxidation process by which the enzyme metabolizes drugs is inherently quantum, in a way that is difficult to simulate classically (classical methods work well when there are not a lot of quantum correlations at different scales). They found that a quantum computer of a few million qubits would be able to simulate the process faster and more precisely than state-of-the-art classical techniques. “We find that the higher accuracy offered by a quantum computer is needed to correctly resolve the chemistry in this system, so not only will a quantum computer be better, it will be necessary,” the researchers (including Babbush) wrote in a blog post. CO2 Sequestration One strategy to lower the amount of carbon dioxide in the atmosphere is sequestration—using a catalyst to react with the carbon dioxide and form a compound that can be stored for a long time. Sequestration strategies exist, but are not cost or energy efficient enough to make a significant dent in the current carbon emissions. Several recent studies have suggested that quantum computers of the near future should be able to model carbon dioxide reactions with various catalysts more accurately than classical computers. If true, this would allow scientists to more effectively estimate the efficiency of various sequestration candidates. Agricultural Fertilization Most farmland today is fertilized with ammonia produced under high temperature and pressure in large plants via the Haber-Bosch process. In 2017, a team at Microsoft Research and ETH Zurich considered an alternative ammonia production method—nitrogen fixation by way of the enzyme nitrogenase—that would work at ambient temperature and pressure. This reaction cannot be accurately simulated by classical methods, the researchers showed, but it is within the reach of a classical and quantum computer working in tandem. “If, for example, you could find a chemical process for nitrogen fixation is a small scale in a village on a farm, that would have a huge impact on the food security,” says Troyer, who was involved in the research. Alternate Battery Cathodes Many lithium-ion batteries use cobalt in their cathodes. Cobalt mining has some practical drawbacks, including environmental concerns and unsafe labor practices. One alternative to cobalt is nickel. In a study published in 2023, a collaboration between chemical producer BASF, Google Quantum AI, Macquarie University in Sydney, and QSimulate considered what it would take to simulate a nickel-based cathode, lithium nickel oxide, on a quantum computer. Pure lithium nickel oxide, the researchers said, is unstable in production, and its basic structure is poorly understood. Having a better simulation of the material’s ground state may suggest methods for making a stable version. The quantum computing requirements to adequately simulate this problem are “out of reach of the first error-corrected quantum computers,” the authors wrote in a blog post, “but we expect this number to come down with future algorithmic improvements.” Fusion Reactions In 2022, the National Ignition Facility made headlines with the first inertial fusion reaction to produce more energy than was put directly into it. In an inertial fusion reaction, a tritium-deuterium mixture is heated with lasers until it forms a plasma that collapses into itself, initiating the fusion reaction. This plasma is extremely difficult to simulate, says Babbush, who was involved with the study. “The Department of Energy is already spending hundreds of millions of CPU hours if not billions of CPU hours, just computing one quantity,” he says. In a preprint, Babbush and his collaborators outlined an algorithm that a quantum computer could use to model the reaction in its full complexity. This, like the battery cathode research, would require more qubits than are currently available, but the authors believe future hardware and algorithmic improvements may close this gap. Improving Quantum Sensors Unlike quantum computers, quantum sensors are already having an impact in the real world. These sensors can measure magnetic fields more precisely than any other technology, and are being used for brain scans, gravity measurements for mapping geological activity, and more. The output of the quantum sensor is quantum data, but it’s currently read out as classical data, traditional ones and zeros that miss some of the full quantum complexity. A 2022 study from a collaboration between Google, Caltech, Harvard, UC Berkeley, and Microsoft has shown that if the output of a quantum sensor is instead funneled into a quantum computer, they can use a clever algorithm to learn relevant properties with exponentially fewer copies of the data from the sensor, speeding up readout. They demonstrated their quantum algorithm on a simulated sensor, showing that this algorithms is within reach for even currently existing quantum computers. And More There are also advantageous quantum algorithms still in search of definitive use cases, and prize money is being offered to also motivate that search. Among those algorithms are solving certain types of linear differential equations, and finding patterns in data that are not accessible classically. In addition, classically, many algorithms can’t be proven to work efficiently with pencil and paper, says Jay Gambetta, vice president at IBM Quantum. Instead, people try heuristic algorithms out on real hardware, and some of them perform surprisingly well. With quantum computers, Gambetta argues, the hardware state of the art is on the cusp of being good enough to test out many more heuristic algorithms, so many more use cases should be forthcoming. “I think we can finally start to do algorithm discovery using hardware,” Gambetta says. “And to me that’s opening a different avenue for accelerated scientific discovery. And I think that’s what’s most exciting.”

  • The Story Behind Pixar’s RenderMan CGI Software
    by Joanna Goodrich on 21. Marta 2024. at 18:00

    Watching movies and TV series that use digital visual effects to create fantastical worlds lets people escape reality for a few hours. Thanks to advancements in computer-generated technology used to produce films and shows, those worlds are highly realistic. In many cases, it can be difficult to tell what’s real and what isn’t. The groundbreaking tools that make it easier for computers to produce realistic images, introduced as RenderMan by Pixar in 1988, came after years of development by computer scientists Robert L. Cook, Loren Carpenter, Tom Porter, and Patrick M. Hanrahan. RenderMan, a project launched by computer graphics pioneer Edwin Catmull, is behind much of today’s computer-generated imagery and animation, including in the recent fan favorites Avatar: The Way of Water, The Mandalorian, and Nimona. The technology was honored with an IEEE Milestone in December during a ceremony held at Pixar’s Emeryville, Calif., headquarters. The ceremony is available to watch on demand. The pioneers of RenderMan attended the IEEE Milestone ceremony at Pixar’s Emeryville, Calif., headquarters. From left, Patrick M. Hanrahan, Robert L. Cook, Loren Carpenter, Alvy Ray Smith, Brian Berg (Milestone proposer), Jim Morris (Pixar President), Ed Catmull, Tom Porter, Tony Apodaca, Darwyn Peachey, and Steve May.IEEE/Pixar Animation Studios “I feel deeply honored that IEEE recognizes this achievement with a Milestone award,” Catmull, a Pixar founder, said at the ceremony. “Everyone’s dedication and hard work [while developing the technology] brought us to this moment.” Cook, Carpenter, and Porter collaborated as part of Lucasfilm’s computer graphics research group, an entity that later became Pixar. Hanrahan joined Pixar after its launch. The development of the software that would eventually become RenderMan started long before. From Utah and NYIT to Lucasfilm As a doctoral student studying computer science at the University of Utah, in Salt Lake City, Catmull developed the principle of texture mapping, a method for adding complexity to a computer-generated surface. It later was incorporated into RenderMan. After graduation, Catmull joined the New York Institute of Technology on Long Island as director of its recently launched computer graphics research program. NYIT’s founder, Alexander Schure, started the program with the goal of using computers to create animated movies. Malcolm Blanchard, Alvy Ray Smith, and David DiFrancesco soon joined the lab. While at the University of Utah, Blanchard designed and built hardware that clipped 3D shapes to only what was potentially visible. Before joining NYIT, Smith, an IEEE life member, helped develop SuperPaint at the Xerox Palo Alto Research Center in California. It was one of the first computer raster graphics editor programs. DiFrancesco, an artist and scientist, worked with Smith on the project. “I feel deeply honored that IEEE recognizes this achievement with a Milestone award.” —Edwin Catmull During the next five years the team created so many pioneering rendering technologies that when Catmull tried to list all its achievements years later, he opted to stop at four pages, according to an article on the history of Pixar. The team’s technologies include Tween, software that enables a computer to automatically generate the intermediate frames between key frames of action; and the Alpha Channel, which combines separate elements into one image. “We didn’t keep [our work] secret,” Catmull said in an interview with The Institute. The team created a 22-minute short using its technology. It soon caught the attention of Hollywood producer George Lucas, founder of Lucasfilm and originator of the Star Wars and Indiana Jones franchises. Lucas, aiming to digitize the film production process, recruited Catmull in 1979 to head the company’s newly created computer division. He tasked the group with developing a digital, nonlinear film editing system, a digital sound editing system, and a laser film printer. From left, Ed Catmull, Alvy Ray Smith, and Loren Carpenter at Lucasfilm in 1982.Pixar Animation Studios During the next year, Catmull brought Smith, DiFrancesco, and Blanchard to join him. But Smith, who became director of the division’s graphics group, soon realized that Lucas didn’t fully understand what computer graphics could do for the film industry, he told The Institute in 2018. To show Lucas what was possible, the team decided to develop a rendering program that could create complex, photorealistic imagery virtually indistinguishable from filmed live action images. They started a rendering team to develop the program. In 1981 Carpenter and Cook came on board. Carpenter had been working in the computer-aided design group at Boeing, in Seattle, where he developed procedural modeling. The algorithm creates 3D models and textures from sets of rules. Cook, then a recent graduate of Cornell, had published a paper on why nearly every computer-generated object had a plasticlike appearance. His article found its way to Catmull and Smith, who were impressed with his research and offered Cook a job as a software engineer. And then the project that was to become RenderMan was born. From REYES to RenderMan The program was not always known as RenderMan. It originally was named REYES (render everything you ever saw). Carpenter and Cook wanted REYES to create scenery that mimicked real life, add motion blur, and eliminate “jaggies,” the saw-toothed curved or diagonal lines displayed on low-resolution monitors. Computer scientist Loren Carpenter, along with Robert L. Cook, took on the initial challenge of developing the computer graphics software that became RenderMan.Pixar Animation Studios No desktop computer at the time was fast enough to process the algorithms being developed. Carpenter and Cook realized they eventually would have to build their own computer. But they first would have to overcome two obstacles, Cook explained at the Milestone ceremony. “Computers like having a single type of object, but scenes require many different types of objects,” he said. Also, he added, computers “like operating on groups of objects together, but rendering has two different natural groupings that conflict.” Those are shading (which you do on every point on the same surface) and hiding (the things you do at every individual pixel). RenderMan was used to create Toy Story, the first fully computer-generated animated movie. It was released in 1995.Pixar Animation Studios/Walt Disney Pictures/Alamy Carpenter created the REYES algorithm to resolve the two issues. He defined a standard object and called it a micropolygon, a tiny, single-color quadrilateral less than half a pixel wide. He figured about 80 million micropolygons were needed per the 1,000 polygons that typically made up an object. Then he split the rendering into two steps: one to calculate the colors of the micropolygons and the other to use them to determine the pixel colors. To eliminate jaggies, Cook devised the so-called Monte Carlo technique. It randomly picks points—which eliminates jaggies and the interference of light. Noise patterns appeared, however. “But there is another way you can pick points,” Cook explains. “You pick a point randomly, and the next point randomly, but you throw the next one out if it is too close to the first point.” Problems RenderMan Had to Solve Computer scientists Robert L. Cook and Loren Carpenter took on the initial challenge of developing the computer graphics software that became Renderman. They created this list of tasks for the tool: Create complex scenes, with several different computer-generated objects in a scene to mimic real life. Give texture to wooden, metallic, or liquid objects. (At the time, computer-generated objects had a plasticlike appearance.) Eliminate jaggies, the saw-toothed appearance of curved or diagonal lines when viewed on a low-resolution monitor. They appear at an object’s edges, mirror reflection, and the bending of light rays when they pass through transparent substances such as glass or water. Create motion blur or the apparent streaking of moving objects caused by rapid movement. Simulate the depth-of-field effect, in which objects within some range of distances in a scene appear in focus, and objects nearer or farther than this range appear out of focus. Generate shadows only in the direction of the light source. Create reflections and refractions on shiny surfaces. Control how deep light penetrates the surface of an object. That is known as a Poisson disk distribution, modeled on the distribution of cells in the human retina—which also have a seemingly random pattern but with a consistent minimum spacing. The Monte Carlo change eliminated the annoying visual effects. The distribution also simplified several other tasks handled by the rendering software, including the creation of motion blur and a simulated depth-of-field effect. “Creating motion blur was probably the single hardest problem we had,” Catmull told IEEE Spectrum in a 2001 interview. Porter, who was part of the graphics group, came up with a way to use the random sampling notion to solve motion blur. In addition to determining what colors appear at each point, the computer considers how that changes over time. Using point sampling, Cook explained, the group found a simple solution: “To every one of your random samples, you assign a random time between when a traditional camera’s shutter would open and when that shutter would close.” Averaging the times creates a blurred image. The same process worked to simulate the depth-of-field effect: an image in which some areas are in focus and some are not. To create the lack of focus using point sampling, the software assigns a random spot on an imaginary lens to each randomly selected point. Tom Porter, who was part of the graphics group, came up with a way to use the so-called Monte Carlo technique to solve motion blur. Pixar Animation Studios They initially tried the software on a VAX 11/780 computer, running at 1 million instructions per second. But it was too slow, so the team developed the Pixar image computer. It executed instructions at a rate of 40 MIPS, making it 200 times faster than the VAX 11/780, according to the Rhode Island Computer Museum. Pixar was funded thanks to Steve Jobs In 1984 the graphics group invited animator John Lasseter to direct the first short film using REYES. The Adventures of André & Wally B. featured a boy who wakes in a forest after being annoyed by a pesky bumblebee. The movie premiered at the Association for Computing Machinery’s 1984 Special Interest Group on Computer Graphics and Interactive Techniques conference. After the premiere, visual effects studio Industrial Light and Magic asked the team to create the first CGI-animated character to be used in a live-action feature-length film. The group developed the stained-glass knight character for the 1985 Young Sherlock Holmes movie. In 1986 the Lucasfilm graphics group, now with 40 employees, was spun out into a separate company that went on to become Pixar. “At first it was a hardware company,” Smith says. “We turned a prototype of the Pixar image computer into a product. Then we met with some 35 venture capitalists and 10 companies to see if they would fund Pixar. Finally, Steve Jobs signed on as our majority shareholder.” REYES contained a custom-built interface that had been written to work with software used by Lucasfilm, so the teams needed to replace it with a more open interface that would work with any program sending it a scene description, according to the Milestone entry on the Engineering and Technology History Wiki. In 1988, with the help of computer graphics pioneer Hanrahan, Pixar scientists designed a new interface. Hanrahan helped refine shading language concepts as well. Thanks to a conversation that Hanrahan had about futuristic rendering software being so small it could fit inside a pocket, like a Sony Walkman, REYES became RenderMan. Pixar released the tool in 1988. Three years later, Pixar and Walt Disney Studios partnered to “make and distribute at least one computer-generated animated movie,” according to Pixar’s website. The first film was Toy Story, released in 1995. The first fully computer-generated animated movie, it became a blockbuster. RenderMan is still the standard rendering tool used in the film industry. As of 2022, the program had been used in more than 500 movies. The films it helped create have received 15 Academy Awards for Best Animated Feature and 26 for Best Visual Effects. In 2019 Catmull and Hanrahan received the Turing Award from the Association for Computing Machinery for “fundamental contributions to 3-D computer graphics and the revolutionary impact of these techniques on computer-generated imagery in filmmaking and other applications.” A plaque recognizing the technology is displayed next to the entrance gates at Pixar’s campus. It reads: RenderMan software revolutionized photorealistic rendering, significantly advancing the creation of 3D computer animation and visual effects. Starting in 1981, key inventions during its development at Lucasfilm and Pixar included shading languages, stochastic antialiasing, and simulation of motion blur, depth of field, and penumbras. RenderMan’s broad film industry adoption following its 1988 introduction led to numerous Oscars for Best Visual Effects and an Academy Award of Merit for its developers. Administered by the IEEE History Center and supported by donors, the Milestone program recognizes outstanding technical developments around the world. The IEEE Oakland–East Bay Section in California sponsored the nomination, which was initiated by IEEE Senior Member Brian Berg, vice chair of IEEE’s history committee. “It’s important for this history to be documented,” Smith told The Institute, “and I see that as one of the roles of IEEE: recording the history of its own field. “Engineers and IEEE members often don’t look at how technological innovations affect the people who were part of the development. I think the [Milestone] dedication ceremonies are special because they help highlight not just the tech but the people surrounding the tech.”

  • Supercomputing’s Future Is Green and Interconnected
    by Dina Genkina on 20. Marta 2024. at 16:58

    While the Top500 list ranks the 500 biggest high-performance computers (HPCs) in the world, its cousin the Green500 reranks the same 500 supercomputers according to their energy efficiency. For the last three iterations of the list, Henri—a small supercomputer operated by the Flatiron Institute, in New York City—has been named the world’s most energy-efficient high-performance computer. Built in the fall of 2022, Henri was the first system to use Nvidia’s H100 GPUs, a.k.a Hopper. To learn the secrets of building and maintaining the most energy-efficient supercomputer, we caught up with Henri’s architect, Ian Fisk, who is codirector of the Scientific Computing Core at the Flatiron Institute. Flatiron is an internal research division of the Simons Foundation that brings together researchers using modern computational tools to advance our understanding of science. The Flatiron Insitute’s Ian Fisk on... Building Henri Getting and keeping the top Green500 spot The future of Henri and its successors Powering science IEEE Spectrum: Where did the name Henri come from? Ian Fisk: The name came about for a silly reason. Our previous machine was called Rusty. So, when asked by the vendor what the machine name was going to be, we said, “Well, by our naming convention, it’ll be Rusty, and it’s using [Nvidia’s] H100 chip, so it’d be Rusty Hopper.’ But Rusty Hopper sounds like a country singer from the 1980s, so they didn’t want to call it that. And one of the Nvidia engineers who decided that you might be able to actually build a machine that would make the Top500 and be the top of the Green500 had just had a son named Henri. So, we were asked by the vendor if we might consider naming it after that person, which we thought was sweet. Since the Green500 measures performance per watt, it doesn’t matter how fast you are, it matters how fast you are for how many watts you used. —Ian Fisk, Flatiron Institute Did you set out to build the world’s greenest supercomputer? Fisk: Nvidia sold us that gear at an educational-discount price in part because we were aiming for this benchmark. It was good for us because it gave us some exposure, but we really wanted the hardware for the scientists, and it was a way for us to get access to H100s very early. But to do that, we had to do the test in November 2022. So the equipment came to the loading dock in October, and it was assembled into a computer and then tested in record time. If there was an award for the fast 500, we would also be the winner. My Trip to See Henri A few weeks ago, I hopped on a train from the bustling Penn Station of New York City to the land of warehouses and outlet malls known as Secaucus, N.J., just 10 minutes away by NJTransit. I was on my way to witness first-hand Henri, the world’s greenest supercomputer one and a half years running, according to the Green500 list. On the train over, my guide, Henri’s architect and caregiver Ian Fisk told me: “Prepare to be disappointed.” Fisk is a former research physicist, a breed with which I am familiar, and among whom self-deprecation runs rampant. Knowing this, I did not heed his warning. After a 10-minute drive in Fisk’s self-proclaimed midlife-crisis vehicle, we arrived at an astonishingly cubic building and made our way to the security desk just past the front door. The building, Fisk explained, houses data centers for several financial institutions, so security is of the utmost importance. Henri itself is a less likely target for hackers—it is built and used by the Flatiron Institute, a research facility run by the Simons Foundation that focuses on computational biology, mathematics, quantum physics, neuroscience, and astrophysics. Yet, I had to present ID and get fingerprinted before being allowed to follow Fisk to the data warehouse. We walked into a huge, brightly lit room with about 10 rows of 12 or so racks each, filled floor to ceiling with computers. The fans were making so much noise I couldn’t quite make out anything Fisk was saying, but I gathered that most of the rows were just sharing space with Henri. We walked over to the back two rows, with black racks distinct from the white ones out front. These were run by the Flatiron institute. “Henri?” I mouthed. Fisk shook his head no. We walked into one of the rows until Fisk pointed out two undistinguished-looking racks filled with computers. “Henri” he nodded. Can’t say I wasn’t warned. As we walked out of the room and into a lounge with more favorable acoustics, I was about to learn that Henri’s comparatively unimpressive size is no coincidence. In our Q&A, Fisk explained how they achieved their greenest accolade, why they’ve been able to maintain the top spot, and what the future might hold for this and other supercomputers. The numbers in the first test run [November 2022] were not as good as the second time [June 2023]. The second time when there was a little bit more time to breathe, we upgraded the machine. It was bigger: it was 80 GPUs the first time and 144 the second time. It’s 2.7 petaflops, which for two racks of equipment is a reasonable size. It’s around 250 on the Top500 largest supercomputers list. And then No. 1 on the Green500 list. Can you explain your design decisions when building Henri? Why Nvidia’s H100s? Fisk: Our experience with Nvidia, which goes all the way back to K40s, was that every generation was about two to three times faster than its predecessor. And that was certainly true of all the things that led up to it, like the V100 and the A100. It’s about two and a half times better. We already had two racks of A100s, and when it came time to upgrade the facility, H100s were the thing to buy. The H100 at the time were only available in the PCI-connected version; they didn’t have the NV-link option yet. And they didn’t have any water-cooled ones, so we were using air-cooled systems again. The GPUs before that machine and after have all been water-cooled systems, because they’re just a little bit more efficient, and easier to operate because you can get rid of a lot more heat. But we chose it because we were expecting very nice performance numbers. And we got them, eventually. With Nvidia, the software and the hardware sort of come out at the same time. And the performance tends to get better over time as things get optimized properly. The thing that separates a computer from a supercomputer is the low-latency fabric. And on almost all systems right now, that low-latency fabric is InfiniBand. The only people who provide it is Mellanox [Technologies], which was recently acquired by the Nvidia Corp., so they own the whole stack. [What] has allowed us to stand on top has been that technology has evolved to use more power rather than be more efficient. We didn’t expect to win more than once. —Ian Fisk, Flatiron Institute There was one design choice that was sort of thrust upon us that we’re revisiting right now. When we bought the system, the only chassis that you could buy were PCI Gen 4, and the H100s use PCI Gen 5. Because it was Gen 4, we were limited by the communication speed to the GPUs and to the InfiniBand cards. When we started, we had HDR cards at 100 gigabits each. And we rapidly discovered that that wasn’t going to be sufficient to do a good test for the Green500. So, we upgraded to 400 gigabits of InfiniBand on each node, and that helped some. Had we had PCIe Gen 5, we could have had two times 400 gigabits, and that would have been even better. Back to top What optimizations did you have to do for the Green500 test? Fisk: I think doing the Green500 run is a little bit like being a hypermiler. You have a Honda Civic and you drive across the country getting 60 miles per gallon with the windows closed, AC off, and accelerating very slowly, but that’s not exactly the way you’d drive it in a rush to get somewhere. For instance, when you do the Green500 run, everything that doesn’t generate performance is turned down. There are big solid-state drives on all of the systems of this type when you’re running in production, because you need to serve training samples to machine-learning applications. But they use power, and they don’t give you any performance, so those get turned off. It’s a little bit like a hypermiler taking the spare tire out of their car because they wanted to get better mileage, but it’s not how they would actually drive it all the time. How have you been able to keep the No. 1 spot for almost two years? Fisk: Certainly, the thing that will knock Henri off its perch will be the next generation of hardware. But I think the thing that has allowed us to stand on top has been that technology has evolved to use more power rather than be more efficient. We didn’t expect to win more than once. We were expecting that people would come along with the water-cooled version of H100s and be more efficient than us, but that hasn’t happened so far. The H100 comes in two models, the PCI version that plugs into the board as a card and the motherboard mount, it’s called an SXM5. And the SXM5 is the NV-linked version. The big difference is that the SXM5 has a communication protocol between the GPUs that allows them to talk to each other at 900 gigabytes per second. It’s dramatically better than anything on InfiniBand. It’s really what allows them to solve problems like large language models, because when you’re doing these kinds of calculations, at each epoch, there can be a tremendous amount of information that has to flow back and forth. So those communication links are very important, but they also use more electricity. The LINPACK benchmark that they do for the Green500 test benefits from a good communication layer, but not at that level. The reason why no one has beaten the machine yet is that the SXM5s actually use a lot more electricity, they use 700 watts per GPU while ours only use 350, and the performance [on things like the LINPACK test] is not a factor of 2 different. Since the Green500 measures performance per watt, it doesn’t matter how fast you are, it matters how fast you are for how many watts you used. And that’s the thing that we see with those PCI-connected H100s. They are very hard to beat because they don’t use a lot of electricity and they have similar performance to the much higher wattage stuff on these kinds of calculations. Back to top Do you expect to be the greenest supercomputer again in May? Fisk: Well, we are building a new machine with 96 GPUs. These will be the SXM5s, water-cooled NV-linked devices. We will know soon if they will have better performance. As I mentioned, they may be faster, but they may not be more efficient. But, one thing we found with our A100s was that most of the performance is available in the first half the wattage, so you get 90 percent of the performance in the first 225 watts. So, one of the things that we’re going to try with the water-cooled system is to run it in power-capped mode, and see what kind of performance we get. The future is going to be expensive. And the future is going to be very high powered. —Ian Fisk, Flatiron Institute One nice thing about the water-cooled version is that it doesn’t need fans, because the fans count against your wattage. When these units are running, it’s about 4 kilowatts of power per three units of space (3U). So it’s like forty 100-watt lightbulbs in a small box. Cooling that down requires blowing a tremendous amount of air across it, so you can have a few hundred watts of fans. And with water cooling, you just have a central pump, which means significant savings. The heat capacity of water is about 4,000 times the heat capacity of air by volume, so you have to use a lot less of it. It’s going to be interesting to see the next Green500 list in May of this year. We’ll see who comes along and whether nobody beats us, or somebody beats us, or we beat ourselves. It’s all possible. What does the future look like for Henri or its successor? Fisk: The future is going to be expensive. And the future is going to be very high powered. When we started, the GPU was a specialized resource that was very good for machine learning and certain kinds of linear algebra calculations. At the beginning, everyone used a single GPU. Then they started using them together in groups where they would fit their computation across several nodes, up to eight nodes. Now, we’re seeing more and more people who want to do tightly connected large language models, where it requires 100 GPUs or several hundreds of GPUs connected in ways that we never would have imagined. For the next set of resources we’re buying, the network connectivity is 16 times better than the ones that came before that. It’s a similar set of equipment, but these ones have 1.6 terabits of communication per node, as compared to 100 gigabits. And it makes the machines very expensive, because suddenly the network fabric is a large factor in the purchase price, because you need lots and lots of InfiniBand switches and lots of cables. And these are 800 gigabit—exotic, very high performance cables. With tightly connected GPUs you can get models that have 10 to the power of 10 parameters. And this is what’s really driving that particular technology. —Ian Fisk, Flatiron Institute We expect there’ll be lots of people who are running conventional high-performance computing codes. But now there’s this new community that wants to use big chunks of very valuable resources, and we’re trying to support those people. It’s complicated, in part because we are competing with industries that do this, too. These kinds of resources are very hard to buy. They have long lead times, they’re very expensive, in part because it’s driven by the AI gold rush that is going on right now. We’re trying to figure out our place in that, and so we’re buying a medium-scale machine. And we don’t know what happens after that. Back to top What is the Flatiron Institute using Henri for? Fisk: It’s a mix. I would say, still 75 or 80 percent is what I would consider canned machine-learning applications. This is PyTorch primarily, where people are building models to make either simulation or prediction of various things, finding correlations. This runs across the whole spectrum. We’ve got people who are looking at how to understand the AI and build better models. We also have people who are working on things like structural systems biology, looking for correlations of microbiome in the gut. We have people working on protein structure, gene function, looking at gene sequences, and using machine-learning techniques to identify what’s going on. The most recent project is called Polymathic AI. A simplistic summary would be something like ChatGPT for science. The idea is to make a large enough foundation model for science, where you teach the AI algorithms a lot about physical processes, and then ask them to do things like fluid dynamics simulations. It’s a very ambitious project. And they’re trying to figure out how to get bigger, how to scale up their work. And the idea behind this is that with tightly connected GPUs you can get models that have 10 to the power of 10 parameters. And this is what’s really driving that particular technology. Henri is a workhorse machine. If you go into the queue right now, it’s entirely full. If I wanted to run another Green500 test and say: “I’m going to take this thing offline for two weeks,” I would have a riot on my hands. There would be pitchforks outside my office. So yes, it’s a very green efficient computer. But at the end of the day, its legacy is all of the amazing science it enables. Back to top

  • The 5 Spacecraft Behind China’s Moon Rock Sample Mission
    by Andrew Jones on 20. Marta 2024. at 14:49

    China launched a spacecraft today that is planned to be the first act in a complex, multistep campaign to achieve an unprecedented feat: Collecting samples from the far side of the moon and delivering the precious cargo to Earth. Queqiao-2 (“Magpie Bridge-2”), a communications relay satellite, launched at 8:31 a.m. local time on a Long March 8 rocket from the Wenchang Space Launch Site in Hainan province. If all goes well, its stable and highly elliptical lunar orbit will see Queqiao-2 spend vast portions of its 24-hour-period orbit out beyond the moon, able to view both the far side of the moon and the Earth. From here it will assist a series of landing, sampling, and lunar-ascent maneuvers needed to sample the lunar far side, which never faces the Earth, thanks to our planet’s gravity tidally locking with the moon’s rotation as it orbits Earth once every 27.3 days. “Landing is always the hardest, because everything is so time critical, you can’t just pause halfway through and try again later.” —Jonathan McDowell, Harvard-Smithsonian Center for Astrophysics With Queqiao-2 in place, the Chang’e-6 mission—a stack of four spacecraft—is scheduled to launch in May. It will target a landing in an Apollo crater on the far side of the moon to both scoop up and drill for up to 2,000 grams of rock samples. Chang’e-6 will launch on a larger Long March 5 rocket that arrived at the Wenchang spaceport on 15 March. After launch and separation from the rocket, according to a paper from 2021 authored by engineers from the Beijing Institute of Control Engineering, the mission’s service module will control the course of its five-day-journey to the moon, then fire its engines precisely to enter a polar lunar orbit. The lander will separate and prepare for its touchdown attempt. (The 2021 paper concerns China’s previous moon-sample mission, Chang’e-5 in 2020. However, the Chang’e-6 spacecraft are identical to Chang’e-5, and the often inscrutable China National Space Administration, CNSA, has indicated substantial similarities between the two missions in both public presentations and public remarks by prominent engineers. So, lacking a more definitive authority, the Chang’e-5 mission plan will be treated as providing sufficient indications of the course of Chang’e-6’s upcoming mission.) “Landing is always the hardest, because everything is so time critical, you can’t just pause halfway through and try again later,” says Jonathan McDowell, a Harvard-Smithsonian astronomer and a space-activity tracker and analyst. As has recently been seen with a lander from Japan (SLIM), which landed on its nose, and Intuitive Machines’ IM-1, which toppled on its side, even apparent successful landings can be rough. The Chang’e-4 soft-landed on the far side of the moon on 3 January 2019 and was later captured in this photo taken by the rover Yutu-2 (Jade Rabbit-2) on 11 January 2019. Xinhua/Getty Images China has however landed on the far side with Chang’e-4 and performed a near-side moon-rock sampling with Chang’e-5 in 2020. The Chang’e-6 lander and ascent module—initially made as backups in case Chang’e-5 failed—will likewise descend onto the surface in phases, according to the mission paper noted above. After an initial deceleration at 15 kilometers above the surface, the lander will reorient itself to a vertical position when 2 kilometers up. The lander will then use lidar—light detection and ranging—and optical cameras to guide it through a coarse hazard-avoidance phase and, starting at around 100 meters altitude, a hovering, fine hazard-avoidance phase to a safe landing spot. Launching into orbit from another planetary body is a rare and challenging event, with only the Soviet Union’s Luna program, the Apollo missions and Chang’e-5 having performed this before. The lander is also equipped with a laser altimeter and velocimeter, and a throttleable engine to provide adjustable thrust for the powered descent. It also utilizes a reconfigurable attitude-control system based on quaternion partition control to help estimate and compensate for disturbances from the spacecraft’s propellant sloshing around. On the surface, the lander will, like Chang’e-5, quickly get to collecting samples, with activities likely to be wrapped up within a couple of days, according to designers of the previous mission. These are loaded into the ascent vehicle sitting atop the lander. The ascent vehicle will lift off around a couple of days after landing. “The next hardest bit is the rendezvous,” says McDowell, noting the process should be similar to that used by Chang’e-5, although this time supported by Queqiao-2. Launching into orbit from another planetary body is a rare and challenging event, with only the Soviet Union’s Luna program, the Apollo missions and Chang’e-5 having performed this, with only the latter two having performed a lunar orbit rendezvous as well. According to the 2021 mission paper, the ascent vehicle will be guided by a central control unit, a sun sensor, and a star tracker. It is expected to enter a carefully planned circular lunar orbit. Both the ascent vehicle and waiting service module will begin phasing maneuvers needed to rendezvous and dock, made possible by inertial measurement units, radar and optical navigation, docking ports, capture mechanisms, and software algorithms, again using Chang’e-5’s mission profile as guide. The samples will be transferred to a reentry module aboard the service module, which will then prepare for the trip back to Earth. The reentry module will be released just prior to reaching Earth. The capsule, protected by ablative shielding, will perform a ballistic skip reentry, as performed during Chang’e-5, first bouncing off the atmosphere to kill some of the extra speed involved in lunar missions, before making a final, fiery plunge. The mission will take 53 days from launch to landing in Inner Mongolia. As to why this complex celestial performance is being orchestrated, Yuqi Qian, a postdoctoral fellow at the University of Hong Kong, says that the samples could contain material ejected from the lunar mantle, providing unprecedented insights into why the moon’s near and far sides are so different and the history of the Earth-moon system itself. “They may largely reshape humanity’s current understanding of the lunar early evolution from a never-sampled site,” Qian says.

  • Exploding Chips, Meta's AR Hardware, and More
    by Samuel K. Moore on 20. Marta 2024. at 09:00

    Stephen Cass: Hello and welcome to Fixing the Future, an IEEE Spectrum podcast where we look at concrete solutions to some big problems. I’m your host Stephen Cass, a senior editor at IEEE Spectrum. And before we start, I just want to tell you that you can get the latest coverage from some of Spectrum’s most important beats, including AI, climate change and robotics, by signing up for one of our free newsletters. Just go to spectrum.ieee.org/newsletters to subscribe. Today we’re going to be talking with Samuel K. Moore, who follows a semiconductor beat for us like a charge carrier in an electric field. Sam, welcome to the show. Samuel K. Moore: Thanks, Stephen. Good to be here. Cass: Sam, you recently attended the Big Kahuna Conference of the semiconductor research world, ISSCC. What exactly is that, and why is it so important? Moore: Well, besides being a difficult-to-say acronym, it actually stands for the IEEE International Solid State Circuits Conference. And this is really one of the big three of the semiconductor research world. It’s been going on for more than 70 years, which means it’s technically older than the IEEE in some ways. We’re not going to get into that. And it really is sort of the crème de la crème if you are doing circuits research. So there is another conference for inventing new kinds of transistors and other sorts of devices. This is the conference that’s about the circuits you can make from them. And as such, it’s got all kinds of cool stuff. I mean, we’re talking about like 200 or so talks about processors, memories, radio circuits, power circuits, brain-computer interfaces. There’s kind of really something for everybody. Cass: So while we’re there, we send you this monster thing and ask you to fish out— They’re not all going to be— Let’s be honest. They’re not all going to be gangbusters. What were the ones that really caught your eye? Moore: All right. So I’m going to tell you actually about a few things. First off, there’s a potential revolution in analog circuits that’s brewing. Just saw the beginnings of it. There’s a cool upcoming chip that does AI super efficiently by mixing its memory and computing resources. We had a peek at Meta’s future AR glasses or the chip for them anyways. And finally, there was a bunch of very cool security stuff, including a circuit that self-destructs. Cass: Oh, that sounds cool. Well, let’s start off with the analog stuff because you were saying this is like really a way of kind of almost saying bye-bye to some electronic analog stuff. So this is fascinating. Moore: Yeah. So this really kind of kicked the conference off with a bang because it was one of the plenary sessions. It was literally one of the first things that was said. And it had to come from the right person, and it kind of did. It was IEEE fellow and sort of analog establishment figure from the Netherlands Bram Nauta. And it was a kind of a real, like, “We’re doing it all wrong kind of moment,” but it was important because the stakes are pretty high. Basically, Moore’s Law has been really good for digital circuits, the stuff that you use to make the processing parts of CPUs and in its own way for memory but not so much for analog. Basically, you kind of look down the road and you are really not getting any better transistors and processes for analog going forward. And you’re starting to see this in places, even in high-end processors, the parts that kind of do the I/O. They’re just not advancing. They’re using super cutting-edge processes for the compute part and using the same I/O chiplet for like four or five generations. Cass: So this is like when you’re trying to see things from the outside world. So like your smartphone, it needs these converters to digitize your voice but also to handle the radio signal and so on. Moore: Exactly. Exactly. As they say, the world is analog. You have to make it digital to do the computing on it. So what you’re saying about a radio circuit is actually a great example because you’ve got the antenna and then you have to amplify, you have to mix in the carrier signal and stuff, but you have to amplify it. You have to amplify it really nicely quite linearly and everything like that. And then you feed it to your analog to digital converter. What Nauta is pointing out is that we’re not really going to get any better with this amplifier. It’s going to continue to burn tens or hundreds of times more power than any of the digital circuits. And so his idea is let’s get rid of it. No more linear amplifiers. Forget it. Instead, what he’s proposing is that we invent an analog-to-digital converter that doesn’t need one. So literally-- Cass: Well, why haven’t we done this before? It sounds very obvious. You don’t like a component. You throw it out. But obviously, it was doing something. And how do you make up that difference with the pure analog-to-digital converter? Moore: Well, I can’t tell you completely how it’s done, especially because he’s still working on it. But his math basically checks out. And this is really a question— this is really a question of Moore’s Law. It’s not so much, “Well, what are we doing now?” It’s, “What can we do in the future?” If we can’t get any better with our analog parts in the future, let’s make everything out of digital, digitize immediately. And let’s not worry about any of the amplification part. Cass: But is there some kind of trade-off being made here? Moore: There is. So right now, you’ve got your linear amplifier consuming milliwatts and your analog to digital converter, which is a thing that can take advantage of Moore’s Law going forward because it’s mostly just comparators and capacitors and stuff that you can deal with. And that consumes only microwatts. So what he’s saying is, “We’ll make the analog-to-digital converter a little bit worse. It’s going to consume a little more power. But the overall system is going to consume less if you take the whole system as a piece.” And that has been part of the problem is that the figures of merit, the things that you measure how good is your linear amplifier, is really just about the linear amplifier rather than worrying about like, “Well, what’s the whole system consuming?” And this looks like, if you care about the whole system, which is kind of what you have to, then this no longer really makes sense. Cass: This also sounds like it gets closer to the dream of the pure software-defined radio, which is you take basically an idea where you take your CPU, you connect one pin to an antenna, and then almost from DC to daylight, you’re able to handle everything in software-defined functions. Moore: That’s right. That’s right. Digital can take advantage of Moore’s Law. Moore’s Law is continuing. It’s slowing, but it’s continuing. And so that’s just sort of how things have been creeping along. And now it’s finally getting kind of to the edge, to that first amplifier. So anyways, he was kind of apprehensive about giving this talk because it is poo-pooing on quite a lot of things actually at this conference. So he told me he was actually pretty nervous about it. But it had some interest. I mean, there were some engineers from Apple and others that approached him that said, “Yeah, this kind of makes sense. And maybe we’ll take a look at this.” Cass: So fascinating. So it appears to be solving these bottlenecks and linear amplifier efficiencies of bottleneck. But there was another bottleneck that you mentioned, which is the memory wall. Moore: Yes. Cass: It’s a memory wall. Moore: Right. So the memory wall is this sort of longstanding issue in computing. Particularly, it started off in high-performance computing, but it’s kind of in all computing now, where the amount of time and energy needed to move a bit from memory to the CPU or the GPU is so much bigger than the amount of time and energy needed to move a bit from one part of the GPU or CPU to another part of the GPU or CPU, staying on the silicon, essentially. Cass: Going off silicon has a penalty. Moore: That’s a huge penalty. Cass: And this is why, in traditional CPUs, you have these like caches, L1. You hear these words, L1 cache, L2 cache, L3 cache. But this goes much further. What you’re talking about is much further than just having a little blob of memory near the CPU. Moore: Yes, yes. So the general memory wall is this problem. And people have been trying to solve this in all kinds of ways. And you just sort of see it in the latest NVIDIA GPUs basically has all of its DRAM is right on the same— is on like a silicon interposer with the GPU. They couldn’t be connected any more closely. You see it in that giant chip. If you remember, Cerebras has a wafer size chip. It’s as big as your face. And that is— Cass: Oh, that sounds an incredible chip. And we’ll definitely put the link to that in the show notes for this because there’s a great picture. It has to be kind of seen to be believed, I think. There’s a great picture of this monster, monster thing. But sorry. Moore: Yeah, and that is an extreme solution to the memory wall problem. But there’s all sorts of other cool research in this. And one of the best is sort of to bring the compute to the memory so that your bits just don’t have to move very far. There’s a bunch of different— well, a whole mess of different ways to do this. There were like nine talks or something on this when I was there, and there are even very cool ways that we’ve written about in Spectrum, where you can actually do you can do sort of AI calculations in memory using analog, where the-- Cass: Oh, so now we’re back to analog! Let’s creep it back in. Moore: Yeah, no, it’s cool. I mean, it’s cool that sort of coincidentally, the multiply and accumulate task, which is sort of the fundamental crux of all the matrix stuff that runs AI you can do in basically Ohm’s Law and Kirchhoff’s Law. They just kind of dovetail into this wonderful thing. But it’s very fiddly. Trying to do anything in analog is always [crosstalk]. Cass: So before digital computers, like right up into the ‘70s, analog computers were actually quite competitive, whereby you set up your problem using operational amplifiers, which is why they’re called operational amplifiers. Op amps are called op amps. And you set it all your equation all up, and then you produce results. And this is basically like taking one of those analog operations where the behavior of the components models a particular mathematical equation. And you’re taking a little bit of analog computing, and you’re putting it in because it matches with one particular calculation that’s used in AI. Moore: Exactly, yeah, yeah. So it’s a very fruitful field, and people are still chugging along at it. I met a guy at ISSCC. His name is Evangelos Eleftheriou. He is the CTO of a company called Axelera, and he is a veteran of one of these projects that was doing analog AI at IBM. And he came to the conclusion that it was just not ready for prime time. So instead, he found himself a digital way of doing the AI compute in memory. And it hinges on basically interleaving the compute so tightly with the cache memory that they’re kind of a part of each other. That required, of course, coming up with a sort of new kind of SRAM, which he was very hush-hush about, and also kind of doing things in integer math instead of floating point math. Most of what you see in the AI world, like NVIDIA and stuff like that, their primary calculations are in floating point numbers. Now, those floating point numbers are getting smaller and smaller. They’re doing more and more in just 8-bit floating point, but it’s still floating point. This depends on integers instead just because of the architecture depends on it. Cass: Yeah, no, I like integer math, actually, because I do a lot of this retrocomputing. A lot of that is in this where you actually end up doing a lot of integer math. And the truth is that you realize, oh, the Forth programming language also is famously very [integer]-based. And for a lot of real-world problems, you can find a perfectly acceptable scale factor that lets you use integers with no appreciable difference in precision. Floating points are kind of more general purpose. But this really had some impressive trade-offs in the benchmarks. Moore: Yeah, whatever they managed, despite any trade-offs they might have had to make for the math, they actually did very well. Now this is for— their aim is what’s called an edge computer. So it’s the kind of thing that would be running a bunch of cameras in sort of a traffic management situation or things like that. It was very machine-vision-oriented, but it’s like a computer or a card that you’d stick into a server that’s going to sit on-premises and do its thing. And when they ran a typical machine vision benchmark, they were able to do 2,500 frames per second. So that’s a lot of cameras potentially, especially when you consider most of these cameras are like— they’re not going 240. Cass: Even if you take it at a standard frame rate of, say, 20 frames per frame per second, that’s 100 cameras that you’re processing simultaneously. Moore: Yeah, yeah. And they were able to actually do this at like 353 frames per watt, which is a very good figure. And it’s performance per watt that really is kind of driving everything at the edge. If you ever want this sort of thing to go in a car or any kind of moving vehicle, everybody’s counting the watts. So that’s the thing. Anyways, I would really look, keep my eyes out for them. They are taping out this year. Should have some silicon later. Could be very cool. Cass: So speaking of that, getting into the chips and making differences, you can make changes sort of on the plane of the chips. But you and I have found some interesting stuff on 3D chip technology, which I know has been a thread of your coverage in recent years. Moore: Yeah, I’m all about the 3D chip technology. You’re finding 3D chip technology all the time pretty much in advanced processors. If you look at what Intel’s doing with its AI accelerators for supercomputers, if you look at what AMD is doing for basically all of its stuff now, they’re really taking advantage of being able to stack one chip on top of another. And this is, again, Moore’s Law slowing down, not getting as much in the two-dimensional shrinking as we used to. And we really can’t expect to get that much. And so if you want more transistors per square millimeter, which really is how you get more compute, you’ve got to start putting one slice of silicon on top of the other slice of silicon. Cass: So as we’re getting towards—instead of transistors per square millimeter, it’s going to be per cubic millimeter in the future. Moore: You could measure it that way. Thankfully, these things are so slim and sort of— Cass: Right. So it looks like a— Moore: Yeah, it looks basically the same form factor as a regular chip. So this 3D tech is powered by the most advanced part anyways is powered by something called hybrid bonding, which I’m afraid I have failed to understand where the word hybrid comes in at all. But really it is kind of making a cold weld between the copper pads on top of one chip and the copper pads on another one. Cass: Just explain what a cold well is because I have heard about a cold well is, but actually, when it comes to— it’s a problem when you’re building things in outer space. Moore: Oh, oh, that. Exactly that. So how it works here is— so picture you build your transistors on the plane of the silicon and then you’ve got layer upon layer of interconnects. And those terminate in a set of sort of pads at the top, okay? You’ve got the same thing on your other chip. And what you do is you put them face-to-face, and there’s going to be like a little bit of gap between the copper on one and the copper on the other, but the insulation around them will just stick together. Then you heat them up just a little bit and the copper expands and just kind of jams itself together and sticks. Cass: Oh, it’s almost like brazing, actually. Moore: I’ll take your word for it. I genuinely don’t know what that is. Cass: I could be wrong. I’m sure a nice metallurgist out there will correct me. But yes, but I see what you’re being with the magnet. You just need a little bit of whoosh. And then everything kind of sticks together. You don’t have to go into your soldering iron and do the heavy— Moore: There’s no solder involved. And that is actually really, really key because it means almost like an order of magnitude increase in the density you can have these connections. We’re talking about like having one connection every few microns. So that adds up to like 200,000 connections per square millimeter if my math is right. It’s actually quite a lot. And it’s really enough to make the distances between from one part of one piece of silicon to one part of another. The same kind of as if they were all just built on one piece of silicon. It’s like Cerebras did it all big in two dimensions. This is folding it up and getting essentially the same kind of connectivity, the same low energy per bit, the same low latency per bit. Cass: And this is where Meta came in. Moore: Yeah. So Meta has been showing up at this conference and other conferences sort of. I’ve noticed them on panels sort of talking about what they would want from chip technology for the ideal pair of augmented reality glasses. The talk they gave today was like— the point was you really just don’t want a shoebox walking around on your face. That’s just not how— Cass: That sounds like a very pointed jab at the moment, perhaps. Moore: Right, it does. Anyways, it turns out what they want is 3D technology because it allows them to pack in more performance, more silicon performance in an area that might actually fit into something that looks like a pair of glasses that you might actually want to wear. And again, flinging the bits around, it would probably reduce the power consumption of said chip, which is very important because you don’t want it to be really hot. You don’t want a really hot shoebox on your face. And you want it to last a long time. You don’t have to keep charging it. So what they showed for the first time, as far as I can tell, is sort of the silicon that they’ve been working on for this. This is a custom machine learning chip. It’s meant to do the kind of neural network stuff that you just absolutely need for augmented reality. And what they had was a four millimeter by four millimeter roughly chip that’s actually made up of two chips that are hybrid bonded together. Cass: And you need this stuff because you need the chip to be able to do all this computer vision processing to process what’s going on in the environment and reduce some sort of semantic stuff that you can overlay things on. This is why learning is so, so important. Machine learning is so important to these applications or AI in general. Yeah. Moore: Exactly, yeah. And you need that AI to be right there in your glasses as opposed to out in the cloud or even in a nearby server. Anything other than actually in the device is not going to give you enough latency and such, or it’s going to give you too much latency, excuse me. Anyway, so this chip was actually two 3D stacked chips. And what was very cool about this is they really made the 3D point because they had a version that was just the 2D, just like they had half of it. They tested the combined one, and they tested the half one. So the 3D stacked one was amazingly better. It wasn’t just twice as good. Basically, in their test, they tracked two hands, which is very important, obviously, for augmented reality. It has to know where your hands are. So that was the thing they tested. So the 3D chip was able to track two hands, and it used less energy than the ordinary 2D chip did when it was only tracking one hand. So 3D is a win for Meta clearly. We’ll see what the final project is like and whether anybody actually wants to use it. But it’s clear that this is the technology that’s going to get them there if they’re ever going to get there. Cass: So jumping to another track, you talked about you mentioned security at the top. And I love the security because there seems to be no limit to how paranoid you can be and yet still not always be able to keep up with the real world. Spectrum has had a long coverage of the history of electronic intelligence spying. We had this great piece on the Russian typewriter and how the Russians spied on American typewriters by putting this embedding circuitry directly into the covers of the typewriters. It’s a crazy story, but you entered the chip security track. And as I’m really eager to hear about the crazy ideas you heard there— or as it turns out, not so crazy ideas. Moore: Right. You’re not paranoid if they’re really trying to— they’re really out to get to you. So yeah, no, this was some real Mission Impossible stuff. I mean, you could kind of envision Ving Rhames and Simon Pegg hunched over a circuit board while Tom Cruise was running in the background. It was very cool. So I want to start with that vision of like somebody hunched over a circuit board that they’ve stolen and they’re trying to crack an encryption code or whatever and they’ve got a little probe on one exposed piece of copper. A group at Columbia and Intel came up with countermeasures for that. They invented a circuit that would reside basically on each pin going out of a processor, or you could have it on the memory side if you wanted. That can actually detect even the most advanced probe. So when you touch these probes to the line, there’s like a very, very slight change in capacitance. I mean, if you’re using a really high-end probe, it’s very, very slight. Larger probes, it’s huge. [laughter] You never think that the CPU is actually paying attention when you’re doing this. With this circuit, it could. It will know that you are actuall— that there’s a probe on a line, and it can take countermeasures like, “Oh, I’m just going to scramble everything. You’re never going to find any secrets from this.” So again, the countermeasures, what it triggers, they left up to you. But the circuit was very cool because now your CPU can know when someone’s trying to hack it. Cass: My CPU always knows I’m trying to hack it. It’s evil. But yes, I’m just trying to debug it, not everything else. But that’s actually pretty cool. And then there was another one where, yeah, again, you were going after another— University of Austin, Texas, were also doing this thing where even non-physical probes, I think, it could go after. Moore: So you don’t have to— you don’t always have to touch things. You can use the electromagnetic emissions from a chip as sort of what’s called a side channel attack. So it just sort of changes in the emissions from the chip when it’s doing particular things can leak information. So what the UT Austin team did was basically they made the circuitry that kind of does the encryption, the sort of key encryption circuitry. They modified it in a way so that the signature was just sort of a blur. And it still worked well. It did its job in a timely manner and stuff like that. But if you hold your EM sniffer up to it, you’re never going to figure out what the encryption key is. Cass: But I think you said you had one that was your absolute favorite. Moore: Yes. It’s totally my favorite. I mean, come on. How could I not like this? They invented a circuit that self-destructs. I got to tell you what the circuit is first because this is also a cool and— Cass: This is a different group. Moore: This is a group at University of Vermont and Marvell Technology. And what they came up with was a physical unclonable function circuit that— Cass: You’re going to have to go and unpack. Moore: Yeah, let me start with that. Physical and clonable function is basically there are always going to be very, very slight differences in each device on a chip, such that if you were to sort of take it, if you were to sort of measure those differences, every chip would be different. Every chip would have sort of its unique fingerprint. So these people have invented these physical and clonable function circuits. And they work great in some ways, but they’re actually very hard to make consistent. You don’t want to use this chip fingerprint as your security key if that fingerprint changes with temperature or as the chip ages. [laughter] So those are problems that different groups have come up with different solutions to solve. The Vermont group had their own solution. It was cool. But what I loved the most was that if the key is compromised or in danger of being compromised. For instance, somebody’s got a probe on it. [laughter] The circuit will actually destroy itself, literally destroy itself. Not in a sparks and smoke kind of way. Cass: Boo. Moore: I know. But at the micro level, it’s kind of like that. Basically, they just jammed the voltage up so high that there’s enough current in the long lines that copper atoms will actually be blown out of position. It will literally create voids and open circuits. At the same time, the voltage is again so high that the insulation in the transistors will start to get compromised, which is an ordinary aging effect, but they’re accelerating it greatly. And so you wind up basically with gobbledygook. Your fingerprint is gone. You could never countermeasure— sorry, you could never counterfeit this chip. You couldn’t say, well, “I got this,” because it’ll have a different fingerprint. It’s definitely not like— it won’t register as the same chip. Cass: So not only will it not work, but if you were to like-- because it’s not like blowing fuses because there are memory protection systems where you send a little-- because you don’t want someone downloading your firmware. You send a little pulse through blows a fuse. But if you really want to, you could crack open. You could decap that chip and see what’s going on. This is scorched Earth internally. Moore: Right, right. At least for the part that makes the physical unclonable function, that is essentially destroyed. And so if you encounter that chip and it doesn’t have the right fingerprint, which it won’t, you know it’s been compromised. Cass: Wow. Well, that is fascinating and very cool. But I’m afraid that’s all we have time today. So thanks so much for coming on and talking about IISSCC. Moore: ISSCC. Oh, yeah. Thanks, Stephen. It was a great time. Cass: So today on Fixing the Future, we were talking with Samuel K. Moore about the latest developments in semiconductor technology. For IEEE Spectrum‘s Fixing the Future, I’m Stephen Cass, and I hope you’ll join us next time.

  • These Courses Will Sharpen Your Knowledge on 6 Emerging Technologies
    by Angelique Parashis on 19. Marta 2024. at 18:00

    This year is shaping up to be an active one for new and developing technologies that are expected to impact how engineers work in the areas of data privacy, IoT security, Wi-Fi 6, and more. It’s crucial, therefore, for engineers to stay informed, be proactive, and invest in their career development to ensure that they can bring the most current information and best engineering practices to their workplace. Here are some of this year’s top technologies, along with the accompanying educational programs from IEEE. Data Privacy About two-thirds of U.S. adults surveyed about their views on data privacy had “little to no understanding of what companies do with the data they collect,” and 81 percent reported being concerned about how the data is being used, according to the results from a Pew Research Center survey conducted last year. With companies buying and selling our personal data, protecting privacy and securing the data remain important. IEEE Educational Activities in collaboration with the IEEE Digital Privacy Initiative, developed the four-course Protecting Privacy in the Digital Age program. It provides a framework for how to operationalize privacy in an organizational context, how to make the framework more helpful for end users, and how to address the technical challenges. Internet of Things Security Internet-enabled devices including connected lighting fixtures, voice-activated virtual assistants, doorbell cameras, and smart appliances “all serve as data-transferring endpoints in a system known as the Internet of Things,” expert Brooke Becher said in her article about the importance of IoT security. With 15 billion devices now connected to the IoT worldwide—and the number forecast to double by 2030—the need to protect cloud-based, Internet-connected hardware and associated networks has never been greater. The All About IoT Security six-course program covers malware and botnets, network monitoring, setting up testbeds, and applying blockchain technology. The courses were developed by IEEE Educational Activities with support from the IEEE Internet of Things Technical Community. Energy Efficiency/Sustainability Experts agree that the continued development of sustainable electricity sources will not only help with energy efficiency but also ensure greater accessibility to energy worldwide. “The fast-emerging category of energy networking, which combines the capabilities of software-defined networking and an electric power system made up of direct-current microgrids, will contribute to energy efficiency [and optimize] power usage, distribution, transmission, and storage,” Liz Centoni said in a Technology article about the year’s top 10 technology trends. Centoni is Cisco’s chief strategy officer and general manager of applications. “As more Wi-Fi 6–certified devices hit the market, from routers to laptops and more, it’s a good time for businesses to consider how their networks can accommodate Wi-Fi 6.—Shaun Carlson, Arvig The development of sustainable energy is critical for the estimated 800 million people worldwide who have no access to electricity. Two-thirds of this population live in sub-Saharan Africa. Minigrids offer a reliable, affordable solution to bring power to remote communities. The grids are designed to distribute electricity generated by solar panels, wind turbines, battery storage, hydropower, diesel generators, and other renewable sources. To help learners better understand the technologies, the four-course Minigrids in Africa program explores the context and roles that the grids play in Africa. The program covers maintenance, sustainability, and operational considerations for connecting the minigrids to national grids, as well as regulatory and public-policy concerns. High-Performance Computing Thanks to the variety of mission-critical applications for high-performance computing—including weather forecasting, health care, and drug development—quantum mechanics experts predict there will be an ongoing need for the utilization of computing power to process data and operations at high speeds. The five-course High Performance Computing Technologies, Solutions to Exascale Systems, and Beyond program developed by IEEE Educational Activities in partnership with IEEE Future Directions, focuses on how HPC can help provide solutions in the exascale era. The course identifies how to accelerate application performance while highlighting the anticipated software and hardware components and systems that will be prevalent in the near future. The program also dives into how artificial intelligence is likely to impact exascale-systems development. High-Efficiency Wi-Fi In a 2022 article about the advantages of Wi-Fi 6, telecom expert Shaun Carlson said the sixth generation of Wi-Fi networks—dubbed Wi-Fi 6 and technically known as IEEE 802.11ax promises “major improvements in the capacity and capability of wireless networks.” Carlson heads Arvig’s research and development. Benefits of Wi-Fi 6 include up to 40 percent faster connectivity and speed for supported devices, increased network capacity through the use of multiuser, multiple-input, multiple-output (MU-MIMO) technology, and greater efficiency that conserves battery power. “As more Wi-Fi 6–certified devices hit the market, from routers to laptops and more,” Carlson wrote, “it’s a good time for businesses to consider how their networks can accommodate Wi-Fi 6.” The two-course IEEE 802.11ax: An Overview of High-Efficiency Wi-Fi program provides an overview of the features introduced by Wi-Fi 6 to the physical and medium access control layers, which led to faster Wi-Fi. Configuration Management The threat of cyberattacks involving ransomware, malware, and computer worms continues to increase. An attack now occurs every 39 seconds. Experts report that 95 percent of cybersecurity breaches are the result of human error by users who unknowingly interact with nefarious websites that expose their system to malicious code. As a result, configuration management is becoming an increasingly standard approach that companies use to reduce their cyberthreat vulnerability. The process establishes configuration standards for each asset in a company’s network and automatically sends alerts about corrective actions that need to be taken, such as updates, patches, or reconfiguration. The five-course Software and Hardware Configuration Management in Systems Engineering program covers core concepts for building reliable software, best practices, and emerging applications. Display Your Knowledge Those who complete one of the course programs will receive an IEEE digital badge that can be shared on social media. Learners also earn continuing education credits. You can learn more on the IEEE Learning Network.

  • Nvidia Announces GR00T, a Foundation Model for Humanoids
    by Evan Ackerman on 18. Marta 2024. at 23:27

    Nvidia’s ongoing GTC developer conference in San Jose is, unsurprisingly, almost entirely about AI this year. But in between the AI developments, Nvidia has also made a couple of significant robotics announcements. First, there’s Project GR00T (with each letter and number pronounced individually so as not to invoke the wrath of Disney), a foundation model for humanoid robots. And secondly, Nvidia has committed to be the founding platinum member of the Open Source Robotics Alliance, a new initiative from the Open Source Robotics Foundation intended to make sure that the Robot Operating System (ROS), a collection of open-source software libraries and tools, has the support that it needs to flourish. GR00T First, let’s talk about GR00T (short for “Generalist Robot 00 Technology”). The way that Nvidia presenters enunciated it letter-by-letter during their talks strongly suggests that in private they just say “Groot.” So the rest of us can also just say “Groot” as far as I’m concerned. As a “general-purpose foundation model for humanoid robots,” GR00T is intended to provide a starting point for specific humanoid robots to do specific tasks. As you might expect from something being presented for the first time at an Nvidia keynote, it’s awfully vague at the moment, and we’ll have to get into it more later on. Here’s pretty much everything useful that Nvidia has told us so far: “Building foundation models for general humanoid robots is one of the most exciting problems to solve in AI today,” said Jensen Huang, founder and CEO of NVIDIA. “The enabling technologies are coming together for leading roboticists around the world to take giant leaps towards artificial general robotics.”Robots powered by GR00T... will be designed to understand natural language and emulate movements by observing human actions—quickly learning coordination, dexterity and other skills in order to navigate, adapt and interact with the real world. This sounds good, but that “will be” is doing a lot of heavy lifting. Like, there’s a very significant “how” missing here. More specifically, we’ll need a better understanding of what’s underlying this foundation model—is there real robot data under there somewhere, or is it based on a massive amount of simulation? Are the humanoid robotic companies involved contributing data to improve GR00T, or instead training their own models based on it? It’s certainly notable that Nvidia is name-dropping most of the heavy-hitters in commercial humanoids, including 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, Unitree Robotics, and XPENG Robotics. We’ll be able to check in with some of those folks directly this week to hopefully learn more. On the hardware side, Nvidia is also announcing a new computing platform called Jetson Thor: Jetson Thor was created as a new computing platform capable of performing complex tasks and interacting safely and naturally with people and machines. It has a modular architecture optimized for performance, power and size. The SoC includes a next-generation GPU based on NVIDIA Blackwell architecture with a transformer engine delivering 800 teraflops of 8-bit floating point AI performance to run multimodal generative AI models like GR00T. With an integrated functional safety processor, a high-performance CPU cluster and 100GB of ethernet bandwidth, it significantly simplifies design and integration efforts. Speaking of Nvidia’s Blackwell architecture—today the company also unveiled its B200 Blackwell GPU. And to round out the announcements, the chip foundry TSMC and Synopsys, an electronic design automation company, each said they will be moving Nvidia’s inverse lithography tool, cuLitho, into production. The Open Source Robotics Alliance The other big announcement is actually from the Open Source Robotics Foundation, which is launching the Open Source Robotics Alliance (OSRA), a “new initiative to strengthen the governance of our open-source robotics software projects and ensure the health of the Robot Operating System (ROS) Suite community for many years to come.” Nvidia is an inaugural platinum member of the OSRA, but they’re not alone—other platinum members include Intrinsic and Qualcomm. Other significant members include Apex, Clearpath Robotics, Ekumen, eProsima, PickNik, Silicon Valley Robotics, and Zettascale. “The [Open Source Robotics Foundation] had planned to restructure its operations by broadening community participation and expanding its impact in the larger ROS ecosystem,” explains Vanessa Yamzon Orsi, CEO of the Open Source Robotics Foundation. “The sale of [Open Source Robotics Corporation] was the first step towards that vision, and the launch of the OSRA is the next big step towards that change.” We had time for a brief Q&A with Orsi to better understand how this will affect the ROS community going forward. You structured the OSRA to have a mixed membership and meritocratic model like the Linux Foundation—what does that mean, exactly? Vanessa Yamzon Orsi: We have modeled the OSRA to allow for paths to participation in its activities through both paid memberships (for organizations and their representatives) and the community members who support the projects through their contributions. The mixed model enables participation in the way most appropriate for each organization or individual: contributing funding as a paying member, contributing directly to project development, or both. What are some benefits for the ROS ecosystem that we can look forward to through OSRA? Orsi: We expect the OSRA to benefit the OSRF’s projects in three significant ways. By providing a stable stream of funding to cover the maintenance and development of the ROS ecosystem. By encouraging greater community involvement in development through open processes and open, meritocratic status achievement. By bringing greater community involvement in governance and ensuring that all stakeholders have a voice in decision-making. Why will this be a good thing for ROS users? Orsi: The OSRA will ensure that ROS and the suite of open source projects under the stewardship of Open Robotics will continue to be supported and strengthened for years to come. By providing organized governance and oversight, clearer paths to community participation, and financial support, it will provide stability and structure to the projects while enabling continued development.

  • Nvidia Unveils Blackwell, Its Next GPU
    by Samuel K. Moore on 18. Marta 2024. at 01:15

    Today at Nvidia’s developer conference, GTC 2024, the company revealed its next GPU, the B200. The B200 is capable of delivering four times the training performance, up to 30 times the inference performance, and up to 25 times better energy efficiency, compared to its predecessor, the Hopper H100 GPU. Based on the new Blackwell architecture, the GPU can be combined with the company’s Grace CPUs to form a new generation of DGX SuperPOD computers capable of up to 11.5 billion billion floating point operations (exaflops) of AI computing using a new, low-precision number format. “Blackwell is a new class of AI superchip,” says Ian Buck, Nvidia’s vice president of high-performance computing and hyperscale. Nvidia named the GPU architecture for mathematician David Harold Blackwell, the first Black inductee into the U.S. National Academy of Sciences. The B200 is composed of about 1600 square millimeters of processor on two silicon dies that are linked in the same package by a 10 terabyte per second connection, so they perform as if they were a single 208-billion-transistor chip. Those slices of silicon are made using TSMC’s N4P chip technology, which provides a 6 percent performance boost over the N4 technology used to make Hopper architecture GPUs, like the H100. Like Hopper chips, the B200 is surrounded by high-bandwidth memory, increasingly important to reducing the latency and energy consumption of large AI models. B200’s memory is the latest variety, HBM3e, and it totals 192 GB (up from 141 GB for the second generation Hopper chip, H200). Additionally, the memory bandwidth is boosted to 8 terabytes per second from the H200’s 4.8 TB/s. Smaller Numbers, Faster Chips Chipmaking technology did some of the job in making Blackwell, but its what the GPU does with the transistors that really makes the difference. In explaining Nvidia’s AI success to computer scientists last year at IEEE Hot Chips, Nvidia chief scientist Bill Dally said that the majority came from using fewer and fewer bits to represent numbers in AI calculations. Blackwell continues that trend. It’s predecessor architecture, Hopper, was the first instance of what Nvidia calls the transformer engine. It’s a system that examines each layer of a neural network and determines whether it could be computed using lower-precision numbers. Specifically, Hopper can use floating point number formats as small as 8 bits. Smaller numbers are faster and more energy efficient to compute, require less memory and memory bandwidth, and the logic required to do the math takes up less silicon. “With Blackwell, we have taken a step further,” says Buck. The new architecture has units that do matrix math with floating point numbers just 4 bits wide. What’s more, it can decide to deploy them on parts of each neural network layer, not just entire layers like Hopper. “Getting down to that level of fine granularity is a miracle in itself,” says Buck. NVLink and Other Features Among the other architectural insights Nvidia revealed about Blackwell are that it incorporates a dedicated “engine” devoted to the GPU’s reliability, availability, and serviceability. According to Nvidia, it uses an AI-based system to run diagnostics and forecast reliability issues, with the aim of increasing up time and helping massive AI systems run uninterrupted for weeks at a time, a period often needed to train large language models. Nvidia also included systems to help keep AI models secure and to decompress data to speed database queries and data analytics. Finally, Blackwell incorporates Nvidia’s fifth generation computer interconnect technology NVLink, which now delivers 1.8 terabytes per second bidirectionally between GPUs and allows for high-speed communication among up to 576 GPUs. Hopper’s version of NVLink could only reach half that bandwidth. SuperPOD and Other Computers NVLink’s bandwidth is key to building large-scale computers from Blackwell, capable of crunching through trillion-parameter neural network models. The base computing unit is called the DGX GB200. Each of those include 36 GB200 superchips. These are modules that include a Grace CPU and two Blackwell GPUs, all connected together with NVLink. The Grace Blackwell superchip is two Blackwell GPUs and a Grace CPU in the same module.Nvidia Eight DGX GB200s can be connected further via NVLINK to form a 576-GPU supercomputer called a DGX SuperPOD. Nvidia says such a computer can blast through 11.5 exaflops using 4-bit precision calculations. Systems of tens of thousands of GPUs are possible using the company’s Quantum Infiniband networking technology. The company says to expect SuperPODs and other Nvidia computers to become available later this year. Meanwhile, chip foundry TSMC and electronic design automation company Synopsys each announced that they would be moving Nvidia’s inverse lithography tool, cuLitho, into production. Lastly, the Nvidia announced a new foundation model for humanoid robots called GR00T.

  • How Ultrasound Became Ultra Small
    by F. Levent Degertekin on 17. Marta 2024. at 14:00

    A startling change in medical ultrasound is working its way through hospitals and physicians’ offices. The long-standing, state-of-the-art ultrasound machine that’s pushed around on a cart, with cables and multiple probes dangling, is being wheeled aside permanently in favor of handheld probes that send images to a phone. These devices are small enough to fit in a lab coat pocket and flexible enough to image any part of the body, from deep organs to shallow veins, with sweeping 3D views, all with a single probe. And the AI that accompanies them may soon make these devices operable by untrained professionals in any setting—not just trained sonographers in clinics. The first such miniaturized, handheld ultrasound probe arrived on the market in 2018, from Butterfly Network in Burlington, Mass. Last September, Exo Imaging in Santa Clara, Calif., launched a competing version. Making this possible is silicon ultrasound technology, built using a type of microelectromechanical system (MEMS) that crams 4,000 to 9,000 transducers—the devices that convert electrical signals into sound waves and back again—onto a 2-by-3-centimeter silicon chip. By integrating MEMS transducer technology with sophisticated electronics on a single chip, these scanners not only replicate the quality of traditional imaging and 3D measurements but also open up new applications that were impossible before. How does ultrasound work? To understand how researchers achieved this feat, it’s helpful to know the basics of ultrasound technology. Ultrasound probes use transducers to convert electrical energy to sound waves that penetrate the body. The sound waves bounce off the body’s soft tissue and echo back to the probe. The transducer then converts the echoed sound waves to electrical signals, and a computer translates the data into an image that can be viewed on a screen. Conventional ultrasound probes contain transducer arrays made from slabs of piezoelectric crystals or ceramics such as lead zirconium titanate (PZT). When hit with pulses of electricity, these slabs expand and contract and generate high-frequency ultrasound waves that bounce around within them. Ultrasound technology has historically required bulky machinery with multiple probes. Julian Kevin Zakaras/Fairfax Media/Getty Images To be useful for imaging, the ultrasound waves need to travel out of the slabs and into the soft tissue and fluid of the patient’s body. This is not a trivial task. Capturing the echo of those waves is like standing next to a swimming pool and trying to hear someone speaking under the water. The transducer arrays are thus built from layers of material that smoothly transition in stiffness from the hard piezoelectric crystal at the center of the probe to the soft tissue of the body. The frequency of energy transferred into the body is determined mainly by the thickness of the piezoelectric layer. A thinner layer transfers higher frequencies, which allow smaller, higher-resolution features to be seen in an ultrasound image, but only at shallow depths. The lower frequencies of thicker piezoelectric material travel further into the body but deliver lower resolutions. As a result, several types of ultrasound probes are needed to image various parts of the body, with frequencies that range from 1 to 10 megahertz. To image large organs deep in the body or a baby in the womb, physicians use a 1- to 2-MHz probe, which can provide 2- to 3-millimeter resolution and can reach up to 30 cm into the body. To image blood flow in arteries in the neck, physicians typically use an 8- to 10-MHz probe. How MEMS transformed ultrasound The need for multiple probes along with the lack of miniaturization meant that conventional medical ultrasound systems resided in a heavy, boxy machine lugged around on a cart. The introduction of MEMS technology changed that. Over the last three decades MEMS has allowed manufacturers in an array of industries to create precise, extremely sensitive components at a microscopic scale. This advance has enabled the fabrication of high-density transducer arrays that can produce frequencies in the full 1- to 10-MHz range, allowing imaging of a wide range of depths in the body, all with one probe. MEMS technology also helped miniaturize additional components so that everything fits in the handheld probe. When coupled with the computing power of a smartphone, this eliminated the need for a bulky cart. The first MEMS-based silicon ultrasound prototypes emerged in the mid-1990s when the excitement of MEMS as a new technology was peaking. The key element of these early transducers was the vibrating micromachined membrane, which allowed the devices to generate vibrations in much the same way that banging on a drum creates sound waves in the air. Exo Imaging developed a handheld ultrasound machine using piezoelectric micromachined ultrasonic transducer (PMUT) technology.Exo Imaging Two architectures emerged. One of them, called the capacitive micromachined ultrasonic transducer, or CMUT, is named for its simple capacitor-like structures. Stanford University electrical engineer Pierre Khuri-Yakub and colleagues demonstrated the first versions. The CMUT is based on electrostatic forces in a capacitor formed by two conductive plates separated by a small gap. One plate—the micromachined membrane mentioned before—is made of silicon or silicon nitride with a metal electrode. The other—typically a micromachined silicon wafer substrate—is thicker and more rigid. When a voltage is applied, placing opposite charges on the membrane and substrate, attractive forces pull and flex the membrane toward the substrate. When an oscillating voltage is added, that changes the force, causing the membrane to vibrate, like a struck drumhead. When the membrane is in contact with the human body, the vibrations send ultrasound frequency waves into the tissue. How much ultrasound is generated or detected depends on the gap between the membrane and the substrate, which needs to be about one micrometer or less. Micromachining techniques made that kind of precision possible. The other MEMS-based architecture is called the piezoelectric micromachined ultrasonic transducer, or PMUT, and it works like a miniaturized version of a smoke alarm buzzer. These buzzers consist of two layers: a thin metal disk fixed around its periphery and a thin, smaller piezoelectric disk bonded on top of the metal disk. When voltages are applied to the piezoelectric material, it expands and contracts in thickness and from side to side. Because the lateral dimension is much larger, the piezo disk diameter changes more significantly and in the process bends the whole structure. In smoke alarms, these structures are typically 4 cm in diameter, and they’re what generates the shrieking sound of the alarm, at around 3 kilohertz. When the membrane is scaled down to 100 μm in diameter and 5 to 10 μm in thickness, the vibration moves up into megahertz frequencies, making it useful for medical ultrasound. Honeywell in the early 1980s developed the first micromachined sensors using piezoelectric thin films built on silicon diaphragms. The first PMUTs operating at ultrasound frequencies didn’t emerge until 1996, from the work of materials scientist Paul Muralt at the Swiss Federal Institute of Technology Lausanne (EPFL), in Switzerland. Early years of CMUT A big challenge with CMUTs was getting them to generate enough pressure to send sound waves deep into the body and receive the echoes coming back. The membrane’s motion was limited by the exceedingly small gap between the membrane and the substrate. This constrained the amplitude of the sound waves that could be generated. Combining arrays of CMUT devices with different dimensions into a single probe to increase the frequency range also compromised the pressure output because it reduced the probe area available for each frequency. Butterfly Network developed a handheld ultrasound machine using capacitive micromachined ultrasonic transducer (CMUT) technology.Butterfly The solution to these problems came from Khuri-Yakub’s lab at Stanford University. In experiments in the early 2000s, the researchers found that increasing the voltage on CMUT-like structures caused the electrostatic forces to overcome the restoring forces of the membrane. As a result, the center of the membrane collapses onto the substrate. A collapsed membrane seemed disastrous at first but turned out to be a way of making CMUTs both more efficient and more tunable to different frequencies. The efficiency increased because the gap around the contact region was very small, increasing the electric field there. And the pressure increased because the large doughnut-shaped region around the edge still had a good range of motion. What’s more, the frequency of the device could be adjusted simply by changing the voltage. This, in turn, allowed a single CMUT ultrasound probe to produce the entire ultrasound frequency range needed for medical diagnostics with high efficiency. Inside Butterfly Network’s CMUT ultrasound probe, the membrane collapses onto the substrate, generating an acoustic wave.Butterfly Network From there, it took more than a decade to understand and model the complicated electromechanical behavior of CMUT arrays and iron out the manufacturing. Modeling these devices was tricky because thousands of individual membranes interacted in each CMUT array. On the manufacturing side, the challenges involved finding the right materials and developing the processes needed to produce smooth surfaces and a consistent gap thickness. For example, the thin dielectric layer that separates the conductive membrane and the substrate must withstand about 100 volts at a thickness of 1 μm. If the layer has defects, charges can be injected into it, and the device can short at the edges or when the membrane touches the substrate, killing the device or at least degrading its performance. Eventually, though, MEMS foundries such as Philips Engineering Solutions in Eindhoven, Netherlands, and Taiwan Semiconductor Manufacturing Co. (TSMC), in Hsinchu, developed solutions to these problems. Around 2010, these companies began producing reliable, high-performance CMUTs. Early development of PMUTs Early PMUT designs also had trouble generating enough pressure to work for medical ultrasound. But they could bang out enough to be useful in some consumer applications, such as gesture detection and proximity sensors. In such “in-air ultrasound” uses, bandwidth isn’t critical, and frequencies can be below 1 MHz. In 2015, PMUTs for medical applications got an unexpected boost with the introduction of large 2D matrix arrays for fingerprint sensing in mobile phones. In the first demonstration of this approach, researchers at the University of California, Berkeley, and the University of California, Davis, connected around 2,500 PMUT elements to CMOS electronics and placed them under a silicone rubberlike layer. When a fingertip was pressed to the surface, the prototype measured the amplitudes of the reflected signals at 20 MHz to distinguish the ridges in the fingertip from the air pockets between them. This was an impressive demonstration of integrating PMUTs and electronics on a silicon chip, and it showed that large 2D PMUT arrays could produce a high enough frequency to be useful for imaging of shallow features. But to make the jump to medical ultrasound, PMUT technology needed more bandwidth, more output pressure, and piezoelectric thin films with better efficiency. Help came from semiconductor companies such as ST Microelectronics, based in Geneva, which figured out how to integrate PZT thin films on silicon membranes. These films require extra processing steps to maintain their properties. But the improvement in performance made the cost of the extra steps worthwhile. To achieve a larger pressure output, the piezoelectric layer needed to be thick enough to allow the film to sustain the high voltages required for good ultrasound images. But increased thickness leads to a more rigid membrane, which reduces the bandwidth. One solution was to use an oval-shaped PMUT membrane that effectively combined several membranes of different sizes into one. This is similar to changing the length of guitar strings to generate different tones. The oval membrane provides strings of multiple lengths on the same structure with its narrow and wide sections. To efficiently vibrate wider and narrower parts of the membrane at different frequencies, electrical signals are applied to multiple electrodes placed on corresponding regions of the membrane. This approach allowed PMUTs to be efficient over a wider frequency range. From academia to the real world In the early 2000s, researchers began to push CMUT technology for medical ultrasound out of the lab and into commercial development. Stanford University spun out several startups aimed at this market. And leading medical ultrasound imaging companies such as GE, Philips, Samsung, and Hitachi began developing CMUT technology and testing CMUT-based probes. But it wasn’t until 2011 that CMUT commercialization really began to make progress. That year, a team with semiconductor electronics experience founded Butterfly Network. The 2018 introduction of the IQ Probe was a transformative event. It was the first handheld ultrasound probe that could image the whole body with a 2D imaging array and generate 3D image data. About the size of a TV remote and only slightly heavier, the probe was initially priced at US $1,999—one-twentieth the cost of a full-size, cart-carried machine. Around the same time, Hitachi in Tokyo and Kolo Medical in Suzhou, China (formerly in San Jose, Calif.), commercialized CMUT-based probes for use with conventional ultrasound systems. But neither has the same capabilities as Butterfly’s. For example, the CMUT and electronics aren’t integrated on the same silicon chip, which means the probes have 1D arrays rather than 2D. That limits the system’s ability to generate images in 3D, which is necessary in advanced diagnostics, such as determining bladder volume or looking at simultaneous orthogonal views of the heart. Exo Imaging’s September 2023 launch of its handheld probe, the Exo Iris, marked the commercial debut of PMUTs for medical ultrasound. Developed by a team with experience in semiconductor electronics and integration, the Exo Iris is about the same size and weight as Butterfly’s IQ Probe. Its $3,500 price is comparable to Butterfly’s latest model, the IQ+, priced at $2,999. The ultrasound MEMS chips in these probes, at 2 by 3 cm, are some of the largest silicon chips with combined electromechanical and electronic functionality. The size and complexity impose production challenges in terms of the uniformity of the devices and the yield. These handheld devices operate at low power, so the probe’s battery is lightweight, lasts for several hours of continuous use while the device is connected to a cellphone or tablet, and has a short charging time. To make the output data compatible with cellphones and tablets, the probe’s main chip performs digitization and some signal processing and encoding. Two MEMS ultrasound architectures have emerged. In the capacitive micromachined ultrasonic transducer (CMUT) design, attractive forces pull and flex the membrane toward the substrate. When an oscillating voltage is added, the membrane vibrates like a struck drumhead. Increasing the voltage causes the electrostatic forces to overcome the restoring forces of the membrane, causing the membrane to collapse onto the substrate. In the piezoelectric micromachined ultrasonic transducer (PMUT) architecture, voltages applied to the piezoelectric material cause it to expand and contract in thickness and from side to side. Because the lateral dimension is much larger, the piezo disk diameter changes significantly, bending the whole structure. To provide 3D information, these handheld probes take multiple 2D slices of the anatomy and then use machine learning and AI to construct the necessary 3D data. Built-in AI-based algorithms can also help doctors and nurses precisely place needles in desired locations, such as in challenging vasculature or in other tissue for biopsies. The AI developed for these probes is so good that it may be possible for professionals untrained in ultrasound, such as nurse midwives, to use the portable probes to determine the gestational age of a fetus, with accuracy similar to that of a trained sonographer, according to a 2022 study in NEJM Evidence. The AI-based features could also make the handheld probes useful in emergency medicine, in low-income settings, and for training medical students. Just the beginning for MEMS ultrasound This is only the beginning for miniaturized ultrasound. Several of the world’s largest semiconductor foundries, including TSMC and ST Microelectronics, now do MEMS ultrasound chip production on 300 and 200 mm wafers, respectively. In fact, ST Microelectronics recently formed a dedicated “Lab-in-Fab” in Singapore for thin-film piezoelectric MEMS, to accelerate the transition from proofs of concept to volume production. Philips Engineering Solutions offers CMUT fabrication for CMUT-on-CMOS integration, and Vermon in Tours, France, offers commercial CMUT design and fabrication. That means startups and academic groups now have access to the base technologies that will make a new level of innovation possible at a much lower cost than 10 years ago. With all this activity, industry analysts expect ultrasound MEMS chips to be integrated into many different medical devices for imaging and sensing. For instance, Butterfly Network, in collaboration with Forest Neurotech, is developing MEMS ultrasound for brain-computer interfacing and neuromodulation. Other applications include long-term, low-power wearable devices, such as heart, lung, and brain monitors, and muscle-activity monitors used in rehabilitation. In the next five years, expect to see miniature passive medical implants with ultrasound MEMS chips, in which power and data are remotely transferred using ultrasound waves. Eventually, these handheld ultrasound probes or wearable arrays could be used not only to image the anatomy but also to read out vital signs like internal pressure changes due to tumor growth or deep-tissue oxygenation after surgery. And ultrasound fingerprint-like sensors could one day be used to measure blood flow and heart rate. One day, wearable or implantable versions may enable the generation of passive ultrasound images while we sleep, eat, and go about our lives.

  • China and Norway Lead the World’s EV Switchover
    by Willie Jones on 16. Marta 2024. at 17:14

    The U.S. government recently backed down from enacting tough new measures that would have forced automakers to quadruple their sales of electric vehicles by 2030. Even if Washington hadn’t buckled to outside pressure, the U.S. ambitions for 2030 would not have been exceptional. The move would have raised market share of all-electric vehicles in the U.S. to a level still well below 20 percent. Meanwhile, there is a growing group of nations with their sights set much higher. China, for one, is expected to meet its own 2030 EV adoption target: 40 percent of vehicles sold. By decade’s end, China is expected to be selling only EVs in regions like the island province of Hainan. Norway, more ambitiously still, aims to eliminate sales of new ICE vehicles by 2025. (Eighty percent of new vehicles sold there, as of 2022, are EVs.) It stands to reason that Norway is far ahead of the rest of the world in terms of EV adoption. Norway has been working, with a consistent program of government funding and incentives, toward getting EVs on its roads since the 1990s. Early government investment in charging infrastructure went a long way toward soothing the range anxiety that made car buyers in other places reluctant to make the switch to battery power from gasoline or diesel. Globally, according to research by the Rocky Mountain Institute, EVs will comprise two-thirds of the world’s car sales by 2030. However, according to the World Resources Institute, “EVs need to account for 75 percent to 95 percent of passenger vehicle sales by 2030 in order to meet international climate goals aimed at keeping global warming to 1.5 degrees C (2.7 degrees F).” According to the WRI’s analysis, above, Iceland, Sweden, the Netherlands, and China are the leading EV adopters after Norway. But as of 2022, there was still a major gap between the top spot and the countries trailing behind, the WRI found. Forty-one percent of Iceland’s auto sales, 32 percent of Sweden’s, 24 percent of the Netherlands’, and 22 percent of China’s were EVs. The nations in this group, however, have made pledges that would narrow the gap by 2030. Analysts are optimistic that electric vehicle sales will reach the levels necessary to help avert climate disaster. WRI adds that because the average annual growth rate in EV sales was 65 percent over the past five years, the world needs an average annual growth rate of only 31 percent through 2032. Who’s aiming to achieve what by decade’s end? Thirty-three countries are signatories of the Global Commercial Vehicle Drive to Zero agreement for heavy- and medium-duty vehicles like tractor-trailers, buses, and box trucks. The group’s member states are “working together to enable 100 percent zero-emission new truck and bus sales by 2040 with an interim goal of 30 percent zero-emission vehicle sales by 2030.” More than a dozen European nations are signatories of the pact; their membership dovetails with the European Union’s promise to reduce the continent’s average vehicle CO2 emissions by 45 percent by 2030 and 90 percent by 2040. Germany has not signed on to the Drive to Zero agreement, But that hasn’t stopped it from pursuing its own set of ambitious goals. The German government wants all new vehicles for its government-owned fleet to be “environmentally friendly drive technologies” by 2030, and has set a 2025 deadline by which at least half of those vehicles will be EVs. (For more on what other countries’ plans are, check out the International Energy Agency’s Global EV Policy Explorer page.) Demand for battery-powered vehicles has risen steadily as advances in battery technology and production have brought the purchase prices of EVs down. EVs are at the point where their sticker prices have fallen or will soon fall below those of comparable vehicles with internal combustion engines. According to analysis by the Energy Innovation and System Transition project, that milestone will likely be reached this year in Europe. Cost parity between battery- and petrol-powered vehicles will happen by 2026 in the U.S. and 2027 in India, say EEIST researchers. If those sunny forecasts hold up, they certainly won’t hinder other national EV adoption goals. The EU has targeted a fivefold boost of EV presence on its roads, from roughly 8 million today to 40 million by 2030. To make sure EV availability won’t fall short, Europe is planning to turn to Chinese manufacturers as a backstop. The EU says its countries will import more than a million EVs a year from China in order to help the continent reach its environmental targets. Meanwhile, strong government policy and financial incentives from these countries are laying the groundwork for a more robust EV industry to hit the marketplace as the cost of EV ownership continues to fall. Not to be outdone, India’s government has enacted an enhanced EV adoption strategy featuring generous incentives it predicts will allow electric vehicle sales there to catch up with those in China and EU nations by 2030. Downsides to the forecasts But not all skies are sunny. Though it’s clear that ess carbon in the atmosphere coming from tailpipe exhaust is a win for the planet, not everyone shares the belief that all-electric transportation is the panacea it is chalked up to be. Among those suggesting a measured approach that takes factors such as the local availability of natural resources into account is economist David S. Rapson, a professor at the University of California, Davis. “It is quite possible that, absent technological advancement, the costs of mitigating greenhouse gasses through electrification can rise above current estimates of the social cost of carbon or, more significantly, above alternative approaches to mitigating climate change,” says Rapson. “If such an outcome does arise, policies that rigidly adhere to 100 percent [EV adoption] targets could prove extremely costly and ultimately counterproductive.” Pushback against such government targets is happening beyond U.S. borders. Canada ‘s Liberal government issued a draft in 2022 calling for 20 percent of new light vehicles sold there to be zero-emission vehicles by 2026. The plan seeks to raise that figure to 60 percent by 2030. But Canada might have a fight on its hands that mirrors what the U.S. government was up against before its about-face. Tim Reuss, president of the Canadian Automobile Dealers Association, told Wards Auto that, “With the current high interest rates and high inflation severely impacting consumer affordability, many consumers lack the means to purchase EVs, as evidenced by the rising inventory levels on our members’ lots. Instead of attempting to dictate what individuals have to purchase, we suggest government focus on creating the right set of circumstances to stimulate demand.” Canada might be forced to lower its EV adoption trajectory before all is said and done. Russia’s government, which will likely see little if any pushback, says it is instituting measures that will result in electric vehicles comprising 10 percent of the country’s overall vehicle production by 2030.

  • How Zipline Designed Its Droid Delivery System
    by Evan Ackerman on 15. Marta 2024. at 20:08

    About a year ago, Zipline introduced Platform 2, an approach to precision urban drone delivery that combines a large hovering drone with a smaller package-delivery “Droid.” Lowered on a tether from the belly of its parent Zip drone, the Droid contains thrusters and sensors (plus a 2.5- to 3.5-kilogram payload) to reliably navigate itself to a delivery area of just one meter in diameter. The Zip, meanwhile, safely remains hundreds of meters up. After depositing its payload, the Droid rises back up to the drone on its tether, and off they go. At first glance, the sensor and thruster-packed Droid seems complicated enough to be bordering on impractical, especially when you consider the relative simplicity of other drone delivery solutions, which commonly just drop the package itself on a tether from a hovering drone. I’ve been writing about robots long enough that I’m suspicious of robotic solutions that appear to be overengineered, since that’s always a huge temptation with robotics. Like, is this really the best way of solving a problem, or is it just the coolest way? We know the folks at Zipline pretty well, though, and they’ve certainly made creative engineering work for them, as we saw when we visited one of their “nests” in rural Rwanda. So as Zipline nears the official launch of Platform 2, we spoke with Zipline cofounder and CTO Keenan Wyrobek, Platform 2 lead Zoltan Laszlo, and industrial designer Gregoire Vandenbussche to understand exactly why they think this is the best way of solving precision urban drone delivery. First, a quick refresher. Here’s what the delivery sequence with the vertical takeoff and landing (VTOL) Zip and the Droid looks like: The system has a service radius of about 16 kilometers (10 miles), and it can make deliveries to outdoor spaces of “any meaningful size.” Visual sensors on the Droid find the delivery site and check for obstacles on the way down, while the thrusters compensate for wind and movement of the parent drone. Since the big VTOL Zip remains well out of the way, deliveries are fast, safe, and quiet. But it takes two robots to pull off the delivery rather than just one. On the other end is the infrastructure required to load and charge these drones. Zipline’s Platform 1 drones require a dedicated base with relatively large launch and recovery systems. With Platform 2, the drone drops the Droid into a large chute attached to the side of a building so that the Droid can be reloaded, after which it pulls the Droid out again and flies off to make the delivery: “We think it’s the best delivery experience. Not the best drone delivery experience, the best delivery experience,” Zipline’s Wyrobek tells us. That may be true, but the experience also has to be practical and sustainable for Zipline to be successful, so we asked the Zipline team to explain the company’s approach to precision urban delivery. Zipline on: Approach to drone delivery Concept for Droid design Designing for cuteness Making pinpoint deliveries IEEE Spectrum: What problems is Platform 2 solving, and why is it necessary to solve those problems in this specific way? Keenan Wyrobek: There are literally billions of last-mile deliveries happening every year in [the United States] alone, and our customers have been asking for years for something that can deliver to their homes. With our long-range platform, Platform 1, we can float a package down into your yard on a parachute, but that takes some space. And so one half of the big design challenge was how to get our deliveries precise enough, while the other half was to develop a system that will bolt on to existing facilities, which Platform 1 doesn’t do. Zoltan Laszlo: Platform 1 can deliver within an area of about two parking spaces. As we started to actually look at the data in urban areas using publicly available lidar surveys, we found that two parking spaces serves a bit more than half the market. We want to be a universal delivery service. But with a delivery area of 1 meter in diameter, which is what we’re actually hitting in our delivery demonstrations for Platform 2, that gets us into the high 90s for the percentage of people that we can deliver to. Wyrobek: When we say “urban,” what we’re talking about is three-story sprawl, which is common in many large cities around the world. And we wanted to make sure that our deliveries could be precise enough for places like that. There are some existing solutions for precision aerial delivery that have been operating at scale with some success, typically by winching packages to the ground from a VTOL drone. Why develop your own technique rather than just going with something that has already been shown to work? Laszlo: Winching down is the natural extension of being able to hover in place, and when we first started, we were like, “Okay, we’re just going to winch down. This will be great, super easy.” So we went to our test site in Half Moon Bay [on the Northern California coast] and built a quick prototype of a winch system. But as soon as we lowered a box down on the winch, the wind started blowing it all over the place. And this was from the height of our lift, which is less than 10 meters up. We weren’t even able to stay inside two parking spaces, which told us that something was broken with our approach. The aircraft can sense the wind, so we thought we’d be able to find the right angle for the delivery and things like that. But the wind where the aircraft is may be different from the wind nearer the ground. We realized that unless we’re delivering to an open field, a package that does not have active wind compensation is going to be very hard to control. We’re targeting high-90th percentile in terms of availability due to weather—even if it’s a pretty blustery day, we still want to be able to deliver. Wyrobek: This was a wild insight when we really understood that unless it’s a perfect day, using a winch actually takes almost as much space as we use for Platform 1 floating a package down on a parachute. Engineering test footage of Zipline’s Platform 2 docking system at their test site in Half Moon Bay in California. How did you arrive at this particular delivery solution for Platform 2? Laszlo: I don’t remember whose idea it was, but we were playing with a bunch of different options. Putting thrusters on the tether wasn’t even the craziest idea. We had our Platform 1 aircraft, which was reliable, so we started with looking at ways to just make that aircraft deliver more precisely. There was only so much more we could do with passive parachutes, but what does an active, steerable parachute look like? There are remote-controlled paragliding toys out there that we tested, with mixed results—the challenge is to minimize the smarts in your parachute, because there’s a chance you won’t get it back. So then we started some crazy brainstorming about how to reliably retrieve the parachute. Wyrobek: One idea was that the parachute would come with a self-return envelope that you could stick in the mail. Another idea was that the parachute would be steered by a little drone, and when the package got dropped off, the drone would reel the parachute in and then fly back up into the Zip. Laszlo: But when we realized that the package has to be able to steer itself, that meant the Zip doesn’t need to be active. The Zip doesn’t need to drive the package, it doesn’t even need to see the package, it just needs to be a point up in the sky that’s holding the package. That let us move from having the Zip 50 feet up, to having it 300 feet up, which is important because it’s a big, heavy drone that we don’t want in our customer’s space. And the final step was adding enough smarts to the thing coming down into your space to figure out where exactly to deliver to, and of course to handle the wind. Once you knew what you needed to do, how did you get to the actual design of the droid? Gregoire Vandenbussche: Zipline showed me pretty early on that they were ready to try crazy ideas, and from my experience, that’s extremely rare. When the idea of having this controllable tether with a package attached to it came up, one of my first thoughts was that from a user standpoint, nothing like this exists. And the difficulty of designing something that doesn’t exist is that people will try to identify it according to what they know. So we had to find a way to drive that thinking towards something positive. Early Droid concept sketches by designer Gregoire Vandenbussche featured legs that would fold up after delivery.Zipline First we thought about putting words onto it, like “hello” or something, but the reality is that we’re an international company and we need to be able to work everywhere. But there’s one thing that’s common to everyone, and that’s emotions—people are able to recognize certain things as being approachable and adorable, so going in that direction felt like the right thing to do. However, being able to design a robot that gives you that kind of emotion but also flies was quite a challenge. We took inspiration from other things that move in 3D, like sea mammals—things that people will recognize even without thinking about it. Vandenbussche’s sketches show how the design of the Droid was partially inspired by dolphins.Zipline Now that you say it, I can definitely see the sea mammal inspiration in the drone. Vandenbussche: There are two aspects of sea mammals that work really well for our purpose. One of them is simplicity of shape; sea mammals don’t have all that many details. Also, they tend to be optimized for performance. Ultimately, we need that, because we need to be able to fly. And we need to be able to convey to people that the drone is under control. So having something you can tell is moving forward or turning or moving away was very helpful. Wyrobek: One other insight that we had is that Platform 2 needs to be small to fit into tight delivery spaces, and it needs to feel small when it comes into your personal space, but it also has to be big enough inside to be a useful delivery platform. We tried to leverage the chubby but cute look that baby seals have going on. The design journey was pretty fun. Gregoire would spend two or three days coming up with a hundred different concept sketches. We’d do a bunch of brainstorming, and then Gregoire would come up with a whole bunch of new directions, and we’d keep exploring. To be clear, no one would describe our functional prototypes from back then as “cute.” But through all this iteration eventually we ended up in an awesome place. And how do you find that place? When do you know that your robot is just cute enough? One iteration of the Droid, Vandenbussche determined, looked too technical and intimidating.Zipline Vandenbussche: It’s finding the balance around what’s realistic and functional. I like to think of industrial design as taking all of the constraints and kind of playing Tetris with them until you get a result that ideally satisfies everybody. I remember at one point looking at where we were, and feeling like we were focusing too much on performance and missing that emotional level. So, we went back a little bit to say, where can we bring this back from looking like a highly technical machine to something that can give you a feeling of approachability? Laszlo: We spent a fair bit of time on the controls and behaviors of the droid to make sure that it moves in a very approachable and predictable way, so that you know where it’s going ahead of time and it doesn’t behave in unexpected ways. That’s pretty important for how people perceive it. We did a lot of work on how the droid would descend and approach the delivery site. One concept had the droid start to lower down well before the Zip was hovering directly overhead. We had simulations and renderings, and it looked great. We could do the whole delivery in barely over 20 seconds. But even if the package is far away from you, it still looks scary because [the Zip is] moving faster than you would expect, and you can’t tell exactly where it’s going to deliver. So we deleted all that code, and now it just comes straight down, and people don’t back away from the Droid anymore. They’re just like, “Oh, okay, cool.” How did you design the thrusters to enable these pinpoint deliveries? Early tests of the Droid centered around a two-fan version.Zipline Laszlo: With the thrusters, we knew we wanted to maximize the size of at least one of the fans, because we were almost always going to have to deal with wind. We’re trying to be as quiet as we can, so the key there is to maximize the area of the propeller. Our leading early design was just a box with two fans on it: Two fans with unobstructed flow meant that it moved great, but the challenge of fitting it inside another aircraft was going to be painful. And it looked big, even though it wasn’t actually that big. Vandenbussche: It was also pretty intimidating when you had those two fans facing you and the Droid coming toward you. A single steerable fan [left] that acted like a rudder was simpler in some ways, but as the fan got larger, the gyroscopic effects became hard to manage. Instead of one steerable fan, how about two steerable fans? [right] Omnidirectional motion was possible with this setup, but packaging it inside of a Zip didn’t work.Zipline Laszlo: We then started looking at configurations with a main fan and a second smaller fan, with the bigger fan at the back pushing forward and the smaller fan at the front providing thrust for turning. The third fan we added relatively late because we didn’t want to add it at all. But we found that [with two fans] the droid would have to spin relatively quickly to align to shifting winds, whereas with a third fan we can just push sideways in the direction that we need. What kind of intelligence does the Droid have? The current design of Zipline’s Platform 2 Droid is built around a large thruster in the rear and two smaller thrusters at the front and back.Zipline Wyrobek: The Droid has its own little autopilot, and there’s a very simple communications system between the two vehicles. You may think that it’s a really complex coordinated control problem, but it’s not: The Zip just kind of hangs out, and the Droid takes care of the delivery. The sensing challenge is for the Droid to find trees and powerlines and things like that, and then find a good delivery site. Was there ever a point at which you were concerned that the size and weight and complexity would not be worth it? Wyrobek: Our mindset was to fail fast, to try things and do what we needed to do to convince ourselves that it wasn’t a good path. What’s fun about this kind of iterative process is oftentimes, you try things and you realize that actually, this is better than we thought. Laszlo: We first thought about the Droid as a little bit of a tax, in that it’s costing us extra weight. But if your main drone can stay high enough up that it avoids trees and buildings, then it can just float around up there. If it gets pushed around by the wind, it doesn’t matter because the Droid can compensate. Wyrobek: Keeping the Zip at altitude is a big win in many ways. It doesn’t have to spend energy station-keeping, descending, and then ascending again. We just do that with the much smaller Droid, which also makes the hovering phase much shorter. It’s also much more efficient to control the small droid than the large Zip. And having all of the sensors on the Droid very close to the area that you’re delivering to makes that problem easier as well. It may look like a more complex system from the outside, but from the inside, it’s basically making all the hardest problems much easier. Over the past year, Zipline has set up a bunch of partnerships to make residential deliveries to consumers using Droid starting in 2024, including prescriptions from Cleveland Clinic in Ohio, medical products from WellSpan Health in Pennsylvania, tasty food from Mendocino Farms in California, and a little bit of everything from Walmart starting in Dallas. Zipline’s plan is to kick things off with Platform 2 later this year.

  • The Heart and the Chip: What Could Go Wrong?
    by Daniela Rus on 15. Marta 2024. at 19:30

    At several points in this book I’ve mentioned the fictional character Tony Stark, who uses technology to transform himself into the superhero Iron Man. To me this character is a tremendous inspiration, yet I often remind myself that in the story, he begins his career as an MIT-­trained weapons manufacturer and munitions developer. In the 2008 film Iron Man, he changes his ways because he learns that his company’s specialized weapons are being used by terrorists. Legendary MIT roboticist Daniela Rus has published a new book called The Heart and the Chip: Our Bright Future with Robots. “There is a robotics revolution underway,” Rus says in the book’s introduction, “one that is already causing massive changes in our society and in our lives.” She’s quite right, of course, and although some of us have been feeling that this is true for decades, it’s arguably more true right now than it ever has been. But robots are difficult and complicated, and the way that their progress is intertwined with the humans that make them and work with them means that these changes won’t come quickly or easily. Rus’ experience gives her a deep and nuanced perspective on robotics’ past and future, and we’re able to share a little bit of that with you here. The following excerpt is from Chapter 14, entitled “What Could Go Wrong?” Which, let’s be honest, is the right question to ask (and then attempt to conclusively answer) whenever you’re thinking about sending a robot out into the real world. —Evan Ackerman Daniela Rus: Should roboticists consider subscribing to their own Hippocratic oath? Remember, robots are tools. Inherently, they are neither good nor bad; it’s how we choose to use them that matters. In 2022, aerial drones were used as weapons on both sides of devastating wars. Anyone can purchase a drone, but there are regulations for using drones that vary between and within different countries. In the United States, the Federal Aviation Administration requires that all drones be registered, with a few exceptions, including toy models weighing less than 250 grams. The rules also depend on whether the drone is flown for fun or for business. Regardless of regulations, anyone could use a flying robot to inflict harm, just like anyone can swing a hammer to hurt someone instead of driving a nail into a board. Yet drones are also being used to deliver critical medical supplies in hard-­to-­reach areas, track the health of forests, and help scientists like Roger Payne monitor and advocate for at-­risk species. My group collaborated with the modern dance company Pilobolus to stage the first theatrical performance featuring a mix of humans and drones back in 2012, with a robot called Seraph. So, drones can be dancers, too. In Kim Stanley Robinson’s prescient science fiction novel The Ministry for the Future, a swarm of unmanned aerial vehicles is deployed to crash an airliner. I can imagine a flock of these mechanical birds being used in many good ways, too. At the start of its war against Ukraine, Russia limited its citizens’ access to unbiased news and information in hopes of controlling and shaping the narrative around the conflict. The true story of the invasion was stifled, and I wondered whether we could have dispatched a swarm of flying video screens capable of arranging themselves into one giant aerial monitor in the middle of popular city squares across Russia, showing real footage of the war, not merely clips approved by the government. Or, even simpler: swarms of flying digital projectors could have broadcasted the footage on the sides of buildings and walls for all to see. If we had deployed enough, there would have been too many of them to shut down. There may be variations of Tony Stark passing through my university or the labs of my colleagues around the world, and we need to do whatever we can to ensure these talented young individuals endeavor to have a positive impact on humanity. The Tony Stark character is shaped by his experiences and steered toward having a positive impact on the world, but we cannot wait for all of our technologists to endure harrowing, life-­changing experiences. Nor can we expect everyone to use these intelligent machines for good once they are developed and moved out into circulation. Yet that doesn’t mean we should stop working on these technologies—­the potential benefits are too great. What we can do is think harder about the consequences and put in place the guardrails to ensure positive benefits. My contemporaries and I can’t necessarily control how these tools are used in the world, but we can do more to influence the people making them. There may be variations of Tony Stark passing through my university or the labs of my colleagues around the world, and we need to do whatever we can to ensure these talented young individuals endeavor to have a positive impact on humanity. We absolutely must have diversity in our university labs and research centers, but we may be able to do more to shape the young people who study with us. For example, we could require study of the Manhattan Project and the moral and ethical quandaries associated with the phenomenal effort to build and use the atomic bomb. At this point, ethics courses are not a widespread requirement for an advanced degree in robotics or AI, but perhaps they should be. Or why not require graduates to swear to a robotics-­ and AI-­attuned variation on the Hippocratic oath? The oath comes from an early Greek medical text, which may or may not have been written by the philosopher Hippocrates, and it has evolved over the centuries. Fundamentally, it represents a standard of medical ethics to which doctors are expected to adhere. The most famous of these is the promise to do no harm, or to avoid intentional wrongdoing. I also applaud the oath’s focus on committing to the community of doctors and the necessity of maintaining the sacred bond between teacher and pupils. The more we remain linked as a robotics community, the more we foster and maintain our relationships as our students move out into the world, the more we can do to steer the technology toward a positive future. Today the Hippocratic oath is not a universal requirement for certification as a doctor, and I do not see it functioning that way for roboticists, either. Nor am I the first roboticist or AI leader to suggest this possibility. But we should seriously consider making it standard practice. In the aftermath of the development of the atomic bomb, when the potential of scientists to do harm was made suddenly and terribly evident, there was some discussion of a Hippocratic oath for scientific researchers. The idea has resurfaced from time to time and rarely gains traction. But science is fundamentally about the pursuit of knowledge; in that sense it is pure. In robotics and AI, we are building things that will have an impact on the world and its people and other forms of life. In this sense, our field is somewhat closer to medicine, as doctors are using their training to directly impact the lives of individuals. Asking technologists to formally recite a version of the Hippocratic oath could be a way to continue nudging our field in the right direction, and perhaps serve as a check on individuals who are later asked to develop robots or AI expressly for nefarious purposes. Of course, the very idea of what is good or bad, in terms of how a robot is used, depends on where you sit. I am steadfastly opposed to giving armed or weaponized robots autonomy. We cannot and should not trust machine intelligences to make decisions about whether to inflict harm on a person or group of people on their own. Personally, I would prefer that robots never be used to do harm to anyone, but this is now unrealistic. Robots are being used as tools of war, and it is our responsibility to do whatever we can to shape their ethical use. So, I do not separate or divorce myself from reality and operate solely in some utopian universe of happy, helpful robots. In fact, I teach courses on artificial intelligence to national security officials and advise them on the strengths, weaknesses, and capabilities of the technology. I see this as a patriotic duty, and I’m honored to be helping our leaders understand the limitations, strengths, and possibilities of robots and other AI-­enhanced physical systems—­what they can and cannot do, what they should and should not do, and what I believe they must do. Ultimately, no matter how much we teach and preach about the limitations of technology, the ethics of AI, or the potential dangers of developing such powerful tools, people will make their own choices, whether they are recently graduated students or senior national security leaders. What I hope and teach is that we should choose to do good. Despite the efforts of life extension companies, we all have a limited time on this planet, what the scientist Carl Sagan called our “pale blue dot,” and we should do whatever we can to make the most of that time and have a positive impact on our beautiful environment, and the many people and other species with which we share it. My decades-­long quest to build more intelligent and capable robots has only strengthened my appreciation for—­no, wonder at—­the marvelous creatures that crawl, walk, swim, run, slither, and soar across and around our planet, and the fantastic plants, too. We should not busy ourselves with the work of developing robots that can eliminate these cosmically rare creations. We should focus instead on building technologies to preserve them, and even help them thrive. That applies to all living entities, including the one species that is especially concerned about the rise of intelligent machines. Excerpted from “The Heart and the Chip: Our Bright Future with Robots”. Copyright 2024 by Daniela Rus, Gregory Mone. Used with permission of the publisher, W.W. Norton & Company. All rights reserved.

  • Andrew Ng: Unbiggen AI
    by Eliza Strickland on 9. Februara 2022. at 15:31

    Andrew Ng has serious street cred in artificial intelligence. He pioneered the use of graphics processing units (GPUs) to train deep learning models in the late 2000s with his students at Stanford University, cofounded Google Brain in 2011, and then served for three years as chief scientist for Baidu, where he helped build the Chinese tech giant’s AI group. So when he says he has identified the next big shift in artificial intelligence, people listen. And that’s what he told IEEE Spectrum in an exclusive Q&A. Ng’s current efforts are focused on his company Landing AI, which built a platform called LandingLens to help manufacturers improve visual inspection with computer vision. He has also become something of an evangelist for what he calls the data-centric AI movement, which he says can yield “small data” solutions to big issues in AI, including model efficiency, accuracy, and bias. Andrew Ng on... What’s next for really big models The career advice he didn’t listen to Defining the data-centric AI movement Synthetic data Why Landing AI asks its customers to do the work The great advances in deep learning over the past decade or so have been powered by ever-bigger models crunching ever-bigger amounts of data. Some people argue that that’s an unsustainable trajectory. Do you agree that it can’t go on that way? Andrew Ng: This is a big question. We’ve seen foundation models in NLP [natural language processing]. I’m excited about NLP models getting even bigger, and also about the potential of building foundation models in computer vision. I think there’s lots of signal to still be exploited in video: We have not been able to build foundation models yet for video because of compute bandwidth and the cost of processing video, as opposed to tokenized text. So I think that this engine of scaling up deep learning algorithms, which has been running for something like 15 years now, still has steam in it. Having said that, it only applies to certain problems, and there’s a set of other problems that need small data solutions. When you say you want a foundation model for computer vision, what do you mean by that? Ng: This is a term coined by Percy Liang and some of my friends at Stanford to refer to very large models, trained on very large data sets, that can be tuned for specific applications. For example, GPT-3 is an example of a foundation model [for NLP]. Foundation models offer a lot of promise as a new paradigm in developing machine learning applications, but also challenges in terms of making sure that they’re reasonably fair and free from bias, especially if many of us will be building on top of them. What needs to happen for someone to build a foundation model for video? Ng: I think there is a scalability problem. The compute power needed to process the large volume of images for video is significant, and I think that’s why foundation models have arisen first in NLP. Many researchers are working on this, and I think we’re seeing early signs of such models being developed in computer vision. But I’m confident that if a semiconductor maker gave us 10 times more processor power, we could easily find 10 times more video to build such models for vision. Having said that, a lot of what’s happened over the past decade is that deep learning has happened in consumer-facing companies that have large user bases, sometimes billions of users, and therefore very large data sets. While that paradigm of machine learning has driven a lot of economic value in consumer software, I find that that recipe of scale doesn’t work for other industries. Back to top It’s funny to hear you say that, because your early work was at a consumer-facing company with millions of users. Ng: Over a decade ago, when I proposed starting the Google Brain project to use Google’s compute infrastructure to build very large neural networks, it was a controversial step. One very senior person pulled me aside and warned me that starting Google Brain would be bad for my career. I think he felt that the action couldn’t just be in scaling up, and that I should instead focus on architecture innovation. “In many industries where giant data sets simply don’t exist, I think the focus has to shift from big data to good data. Having 50 thoughtfully engineered examples can be sufficient to explain to the neural network what you want it to learn.” —Andrew Ng, CEO & Founder, Landing AI I remember when my students and I published the first NeurIPS workshop paper advocating using CUDA, a platform for processing on GPUs, for deep learning—a different senior person in AI sat me down and said, “CUDA is really complicated to program. As a programming paradigm, this seems like too much work.” I did manage to convince him; the other person I did not convince. I expect they’re both convinced now. Ng: I think so, yes. Over the past year as I’ve been speaking to people about the data-centric AI movement, I’ve been getting flashbacks to when I was speaking to people about deep learning and scalability 10 or 15 years ago. In the past year, I’ve been getting the same mix of “there’s nothing new here” and “this seems like the wrong direction.” Back to top How do you define data-centric AI, and why do you consider it a movement? Ng: Data-centric AI is the discipline of systematically engineering the data needed to successfully build an AI system. For an AI system, you have to implement some algorithm, say a neural network, in code and then train it on your data set. The dominant paradigm over the last decade was to download the data set while you focus on improving the code. Thanks to that paradigm, over the last decade deep learning networks have improved significantly, to the point where for a lot of applications the code—the neural network architecture—is basically a solved problem. So for many practical applications, it’s now more productive to hold the neural network architecture fixed, and instead find ways to improve the data. When I started speaking about this, there were many practitioners who, completely appropriately, raised their hands and said, “Yes, we’ve been doing this for 20 years.” This is the time to take the things that some individuals have been doing intuitively and make it a systematic engineering discipline. The data-centric AI movement is much bigger than one company or group of researchers. My collaborators and I organized a data-centric AI workshop at NeurIPS, and I was really delighted at the number of authors and presenters that showed up. You often talk about companies or institutions that have only a small amount of data to work with. How can data-centric AI help them? Ng: You hear a lot about vision systems built with millions of images—I once built a face recognition system using 350 million images. Architectures built for hundreds of millions of images don’t work with only 50 images. But it turns out, if you have 50 really good examples, you can build something valuable, like a defect-inspection system. In many industries where giant data sets simply don’t exist, I think the focus has to shift from big data to good data. Having 50 thoughtfully engineered examples can be sufficient to explain to the neural network what you want it to learn. When you talk about training a model with just 50 images, does that really mean you’re taking an existing model that was trained on a very large data set and fine-tuning it? Or do you mean a brand new model that’s designed to learn only from that small data set? Ng: Let me describe what Landing AI does. When doing visual inspection for manufacturers, we often use our own flavor of RetinaNet. It is a pretrained model. Having said that, the pretraining is a small piece of the puzzle. What’s a bigger piece of the puzzle is providing tools that enable the manufacturer to pick the right set of images [to use for fine-tuning] and label them in a consistent way. There’s a very practical problem we’ve seen spanning vision, NLP, and speech, where even human annotators don’t agree on the appropriate label. For big data applications, the common response has been: If the data is noisy, let’s just get a lot of data and the algorithm will average over it. But if you can develop tools that flag where the data’s inconsistent and give you a very targeted way to improve the consistency of the data, that turns out to be a more efficient way to get a high-performing system. “Collecting more data often helps, but if you try to collect more data for everything, that can be a very expensive activity.” —Andrew Ng For example, if you have 10,000 images where 30 images are of one class, and those 30 images are labeled inconsistently, one of the things we do is build tools to draw your attention to the subset of data that’s inconsistent. So you can very quickly relabel those images to be more consistent, and this leads to improvement in performance. Could this focus on high-quality data help with bias in data sets? If you’re able to curate the data more before training? Ng: Very much so. Many researchers have pointed out that biased data is one factor among many leading to biased systems. There have been many thoughtful efforts to engineer the data. At the NeurIPS workshop, Olga Russakovsky gave a really nice talk on this. At the main NeurIPS conference, I also really enjoyed Mary Gray’s presentation, which touched on how data-centric AI is one piece of the solution, but not the entire solution. New tools like Datasheets for Datasets also seem like an important piece of the puzzle. One of the powerful tools that data-centric AI gives us is the ability to engineer a subset of the data. Imagine training a machine-learning system and finding that its performance is okay for most of the data set, but its performance is biased for just a subset of the data. If you try to change the whole neural network architecture to improve the performance on just that subset, it’s quite difficult. But if you can engineer a subset of the data you can address the problem in a much more targeted way. When you talk about engineering the data, what do you mean exactly? Ng: In AI, data cleaning is important, but the way the data has been cleaned has often been in very manual ways. In computer vision, someone may visualize images through a Jupyter notebook and maybe spot the problem, and maybe fix it. But I’m excited about tools that allow you to have a very large data set, tools that draw your attention quickly and efficiently to the subset of data where, say, the labels are noisy. Or to quickly bring your attention to the one class among 100 classes where it would benefit you to collect more data. Collecting more data often helps, but if you try to collect more data for everything, that can be a very expensive activity. For example, I once figured out that a speech-recognition system was performing poorly when there was car noise in the background. Knowing that allowed me to collect more data with car noise in the background, rather than trying to collect more data for everything, which would have been expensive and slow. Back to top What about using synthetic data, is that often a good solution? Ng: I think synthetic data is an important tool in the tool chest of data-centric AI. At the NeurIPS workshop, Anima Anandkumar gave a great talk that touched on synthetic data. I think there are important uses of synthetic data that go beyond just being a preprocessing step for increasing the data set for a learning algorithm. I’d love to see more tools to let developers use synthetic data generation as part of the closed loop of iterative machine learning development. Do you mean that synthetic data would allow you to try the model on more data sets? Ng: Not really. Here’s an example. Let’s say you’re trying to detect defects in a smartphone casing. There are many different types of defects on smartphones. It could be a scratch, a dent, pit marks, discoloration of the material, other types of blemishes. If you train the model and then find through error analysis that it’s doing well overall but it’s performing poorly on pit marks, then synthetic data generation allows you to address the problem in a more targeted way. You could generate more data just for the pit-mark category. “In the consumer software Internet, we could train a handful of machine-learning models to serve a billion users. In manufacturing, you might have 10,000 manufacturers building 10,000 custom AI models.” —Andrew Ng Synthetic data generation is a very powerful tool, but there are many simpler tools that I will often try first. Such as data augmentation, improving labeling consistency, or just asking a factory to collect more data. Back to top To make these issues more concrete, can you walk me through an example? When a company approaches Landing AI and says it has a problem with visual inspection, how do you onboard them and work toward deployment? Ng: When a customer approaches us we usually have a conversation about their inspection problem and look at a few images to verify that the problem is feasible with computer vision. Assuming it is, we ask them to upload the data to the LandingLens platform. We often advise them on the methodology of data-centric AI and help them label the data. One of the foci of Landing AI is to empower manufacturing companies to do the machine learning work themselves. A lot of our work is making sure the software is fast and easy to use. Through the iterative process of machine learning development, we advise customers on things like how to train models on the platform, when and how to improve the labeling of data so the performance of the model improves. Our training and software supports them all the way through deploying the trained model to an edge device in the factory. How do you deal with changing needs? If products change or lighting conditions change in the factory, can the model keep up? Ng: It varies by manufacturer. There is data drift in many contexts. But there are some manufacturers that have been running the same manufacturing line for 20 years now with few changes, so they don’t expect changes in the next five years. Those stable environments make things easier. For other manufacturers, we provide tools to flag when there’s a significant data-drift issue. I find it really important to empower manufacturing customers to correct data, retrain, and update the model. Because if something changes and it’s 3 a.m. in the United States, I want them to be able to adapt their learning algorithm right away to maintain operations. In the consumer software Internet, we could train a handful of machine-learning models to serve a billion users. In manufacturing, you might have 10,000 manufacturers building 10,000 custom AI models. The challenge is, how do you do that without Landing AI having to hire 10,000 machine learning specialists? So you’re saying that to make it scale, you have to empower customers to do a lot of the training and other work. Ng: Yes, exactly! This is an industry-wide problem in AI, not just in manufacturing. Look at health care. Every hospital has its own slightly different format for electronic health records. How can every hospital train its own custom AI model? Expecting every hospital’s IT personnel to invent new neural-network architectures is unrealistic. The only way out of this dilemma is to build tools that empower the customers to build their own models by giving them tools to engineer the data and express their domain knowledge. That’s what Landing AI is executing in computer vision, and the field of AI needs other teams to execute this in other domains. Is there anything else you think it’s important for people to understand about the work you’re doing or the data-centric AI movement? Ng: In the last decade, the biggest shift in AI was a shift to deep learning. I think it’s quite possible that in this decade the biggest shift will be to data-centric AI. With the maturity of today’s neural network architectures, I think for a lot of the practical applications the bottleneck will be whether we can efficiently get the data we need to develop systems that work well. The data-centric AI movement has tremendous energy and momentum across the whole community. I hope more researchers and developers will jump in and work on it. Back to top This article appears in the April 2022 print issue as “Andrew Ng, AI Minimalist.”

  • How AI Will Change Chip Design
    by Rina Diane Caballar on 8. Februara 2022. at 14:00

    The end of Moore’s Law is looming. Engineers and designers can do only so much to miniaturize transistors and pack as many of them as possible into chips. So they’re turning to other approaches to chip design, incorporating technologies like AI into the process. Samsung, for instance, is adding AI to its memory chips to enable processing in memory, thereby saving energy and speeding up machine learning. Speaking of speed, Google’s TPU V4 AI chip has doubled its processing power compared with that of its previous version. But AI holds still more promise and potential for the semiconductor industry. To better understand how AI is set to revolutionize chip design, we spoke with Heather Gorr, senior product manager for MathWorks’ MATLAB platform. How is AI currently being used to design the next generation of chips? Heather Gorr: AI is such an important technology because it’s involved in most parts of the cycle, including the design and manufacturing process. There’s a lot of important applications here, even in the general process engineering where we want to optimize things. I think defect detection is a big one at all phases of the process, especially in manufacturing. But even thinking ahead in the design process, [AI now plays a significant role] when you’re designing the light and the sensors and all the different components. There’s a lot of anomaly detection and fault mitigation that you really want to consider. Heather GorrMathWorks Then, thinking about the logistical modeling that you see in any industry, there is always planned downtime that you want to mitigate; but you also end up having unplanned downtime. So, looking back at that historical data of when you’ve had those moments where maybe it took a bit longer than expected to manufacture something, you can take a look at all of that data and use AI to try to identify the proximate cause or to see something that might jump out even in the processing and design phases. We think of AI oftentimes as a predictive tool, or as a robot doing something, but a lot of times you get a lot of insight from the data through AI. What are the benefits of using AI for chip design? Gorr: Historically, we’ve seen a lot of physics-based modeling, which is a very intensive process. We want to do a reduced order model, where instead of solving such a computationally expensive and extensive model, we can do something a little cheaper. You could create a surrogate model, so to speak, of that physics-based model, use the data, and then do your parameter sweeps, your optimizations, your Monte Carlo simulations using the surrogate model. That takes a lot less time computationally than solving the physics-based equations directly. So, we’re seeing that benefit in many ways, including the efficiency and economy that are the results of iterating quickly on the experiments and the simulations that will really help in the design. So it’s like having a digital twin in a sense? Gorr: Exactly. That’s pretty much what people are doing, where you have the physical system model and the experimental data. Then, in conjunction, you have this other model that you could tweak and tune and try different parameters and experiments that let sweep through all of those different situations and come up with a better design in the end. So, it’s going to be more efficient and, as you said, cheaper? Gorr: Yeah, definitely. Especially in the experimentation and design phases, where you’re trying different things. That’s obviously going to yield dramatic cost savings if you’re actually manufacturing and producing [the chips]. You want to simulate, test, experiment as much as possible without making something using the actual process engineering. We’ve talked about the benefits. How about the drawbacks? Gorr: The [AI-based experimental models] tend to not be as accurate as physics-based models. Of course, that’s why you do many simulations and parameter sweeps. But that’s also the benefit of having that digital twin, where you can keep that in mind—it’s not going to be as accurate as that precise model that we’ve developed over the years. Both chip design and manufacturing are system intensive; you have to consider every little part. And that can be really challenging. It’s a case where you might have models to predict something and different parts of it, but you still need to bring it all together. One of the other things to think about too is that you need the data to build the models. You have to incorporate data from all sorts of different sensors and different sorts of teams, and so that heightens the challenge. How can engineers use AI to better prepare and extract insights from hardware or sensor data? Gorr: We always think about using AI to predict something or do some robot task, but you can use AI to come up with patterns and pick out things you might not have noticed before on your own. People will use AI when they have high-frequency data coming from many different sensors, and a lot of times it’s useful to explore the frequency domain and things like data synchronization or resampling. Those can be really challenging if you’re not sure where to start. One of the things I would say is, use the tools that are available. There’s a vast community of people working on these things, and you can find lots of examples [of applications and techniques] on GitHub or MATLAB Central, where people have shared nice examples, even little apps they’ve created. I think many of us are buried in data and just not sure what to do with it, so definitely take advantage of what’s already out there in the community. You can explore and see what makes sense to you, and bring in that balance of domain knowledge and the insight you get from the tools and AI. What should engineers and designers consider when using AI for chip design? Gorr: Think through what problems you’re trying to solve or what insights you might hope to find, and try to be clear about that. Consider all of the different components, and document and test each of those different parts. Consider all of the people involved, and explain and hand off in a way that is sensible for the whole team. How do you think AI will affect chip designers’ jobs? Gorr: It’s going to free up a lot of human capital for more advanced tasks. We can use AI to reduce waste, to optimize the materials, to optimize the design, but then you still have that human involved whenever it comes to decision-making. I think it’s a great example of people and technology working hand in hand. It’s also an industry where all people involved—even on the manufacturing floor—need to have some level of understanding of what’s happening, so this is a great industry for advancing AI because of how we test things and how we think about them before we put them on the chip. How do you envision the future of AI and chip design? Gorr: It’s very much dependent on that human element—involving people in the process and having that interpretable model. We can do many things with the mathematical minutiae of modeling, but it comes down to how people are using it, how everybody in the process is understanding and applying it. Communication and involvement of people of all skill levels in the process are going to be really important. We’re going to see less of those superprecise predictions and more transparency of information, sharing, and that digital twin—not only using AI but also using our human knowledge and all of the work that many people have done over the years.

  • Atomically Thin Materials Significantly Shrink Qubits
    by Dexter Johnson on 7. Februara 2022. at 16:12

    Quantum computing is a devilishly complex technology, with many technical hurdles impacting its development. Of these challenges two critical issues stand out: miniaturization and qubit quality. IBM has adopted the superconducting qubit road map of reaching a 1,121-qubit processor by 2023, leading to the expectation that 1,000 qubits with today’s qubit form factor is feasible. However, current approaches will require very large chips (50 millimeters on a side, or larger) at the scale of small wafers, or the use of chiplets on multichip modules. While this approach will work, the aim is to attain a better path toward scalability. Now researchers at MIT have been able to both reduce the size of the qubits and done so in a way that reduces the interference that occurs between neighboring qubits. The MIT researchers have increased the number of superconducting qubits that can be added onto a device by a factor of 100. “We are addressing both qubit miniaturization and quality,” said William Oliver, the director for the Center for Quantum Engineering at MIT. “Unlike conventional transistor scaling, where only the number really matters, for qubits, large numbers are not sufficient, they must also be high-performance. Sacrificing performance for qubit number is not a useful trade in quantum computing. They must go hand in hand.” The key to this big increase in qubit density and reduction of interference comes down to the use of two-dimensional materials, in particular the 2D insulator hexagonal boron nitride (hBN). The MIT researchers demonstrated that a few atomic monolayers of hBN can be stacked to form the insulator in the capacitors of a superconducting qubit. Just like other capacitors, the capacitors in these superconducting circuits take the form of a sandwich in which an insulator material is sandwiched between two metal plates. The big difference for these capacitors is that the superconducting circuits can operate only at extremely low temperatures—less than 0.02 degrees above absolute zero (-273.15 °C). Superconducting qubits are measured at temperatures as low as 20 millikelvin in a dilution refrigerator.Nathan Fiske/MIT In that environment, insulating materials that are available for the job, such as PE-CVD silicon oxide or silicon nitride, have quite a few defects that are too lossy for quantum computing applications. To get around these material shortcomings, most superconducting circuits use what are called coplanar capacitors. In these capacitors, the plates are positioned laterally to one another, rather than on top of one another. As a result, the intrinsic silicon substrate below the plates and to a smaller degree the vacuum above the plates serve as the capacitor dielectric. Intrinsic silicon is chemically pure and therefore has few defects, and the large size dilutes the electric field at the plate interfaces, all of which leads to a low-loss capacitor. The lateral size of each plate in this open-face design ends up being quite large (typically 100 by 100 micrometers) in order to achieve the required capacitance. In an effort to move away from the large lateral configuration, the MIT researchers embarked on a search for an insulator that has very few defects and is compatible with superconducting capacitor plates. “We chose to study hBN because it is the most widely used insulator in 2D material research due to its cleanliness and chemical inertness,” said colead author Joel Wang, a research scientist in the Engineering Quantum Systems group of the MIT Research Laboratory for Electronics. On either side of the hBN, the MIT researchers used the 2D superconducting material, niobium diselenide. One of the trickiest aspects of fabricating the capacitors was working with the niobium diselenide, which oxidizes in seconds when exposed to air, according to Wang. This necessitates that the assembly of the capacitor occur in a glove box filled with argon gas. While this would seemingly complicate the scaling up of the production of these capacitors, Wang doesn’t regard this as a limiting factor. “What determines the quality factor of the capacitor are the two interfaces between the two materials,” said Wang. “Once the sandwich is made, the two interfaces are “sealed” and we don’t see any noticeable degradation over time when exposed to the atmosphere.” This lack of degradation is because around 90 percent of the electric field is contained within the sandwich structure, so the oxidation of the outer surface of the niobium diselenide does not play a significant role anymore. This ultimately makes the capacitor footprint much smaller, and it accounts for the reduction in cross talk between the neighboring qubits. “The main challenge for scaling up the fabrication will be the wafer-scale growth of hBN and 2D superconductors like [niobium diselenide], and how one can do wafer-scale stacking of these films,” added Wang. Wang believes that this research has shown 2D hBN to be a good insulator candidate for superconducting qubits. He says that the groundwork the MIT team has done will serve as a road map for using other hybrid 2D materials to build superconducting circuits.

640px-NATO_OTAN_landscape_logo.svg-2627358850
BHTelecom_Logo
roaming
RIPE_NCC_Logo2015-1162707916
MON_4
mibo-logo
intel_logo-261037782
infobip
bhrt-zuto-logo-1595455966
elektro
eplus_cofund_text_to_right_cropped-1855296649
fmon-logo
h2020-2054048912
H2020_logo_500px-3116878222
huawei-logo-vector-3637103537
Al-Jazeera-Balkans-Logo-1548635272
previous arrowprevious arrow
next arrownext arrow