Toolbox

I climbed into the front seat of the air taxi, buckled the seat belt and braced as the aircraft lifted off. The futuristic cityscape of Busan, South Korea, dropped away, and a digital avatar popped up on the windscreen with a message.

I couldn't answer as a wave of motion sickness hit me. The virtual reality goggles combined with motion-simulating seats pitching back and forth and side to side made it feel like I was actually hovering and maneuvering in the air. They also made me so nauseous I had to close my eyes for the rest of the three-minute journey.

Welcome to the metaverse — sort of.

South Korean company SK Telecom's air taxi mockup was one of the eye-catching demonstrations at MWC, or Mobile World Congress, the world's biggest telecom industry trade show. Tech companies and wireless carriers at this week's expo in Barcelona displayed advancements to connect people and businesses online, increasingly in new virtual reality worlds dubbed the metaverse.

Visitor Mark Varahona also felt woozy after trying the flight experience but is still considering buying a virtual reality headset, the hardware needed to enter any immersive digital universe.

"I was thinking to buy it before coming here. And maybe now I will buy them," he said. "They look quite nice."

The metaverse exploded in popularity after Facebook founder Mark Zuckerberg in late 2021 pronounced it the next big thing for the internet, renaming his social media empire and socking tens of billions into the idea.

He portrayed it as 3D community where people can meet, work and play — doing everything from trying on digital clothes, holding a virtual meeting or taking a trip online.

But doubts about the viability of the metaverse have been creeping in as the initial hype wears off. Sales of virtual reality headsets in the U.S. slipped 2% by December from the previous year, according to NPD Research. Reality Labs, which makes Meta Quest headsets, posted an operating loss of $13.7 billion in 2022.

Meta has said it plans to hire 10,000 engineers in Europe to work on the metaverse. When asked for an update, the company said, "Our expansion in Europe was always a long-term one planned over a number of years. We remain committed to Europe."

The "metaverse has not gone away," said Ben Wood, principal analyst at CCS Insight. "But I think there's a lot more skepticism about what role it's going to play, particularly in the consumer domain beyond the obvious areas of things like gaming."

The definition of the metaverse has been hard to pin down, adding to the skepticism. It is not the same as virtual reality and its cousin, augmented reality, said Tuong Nguyen, a Gartner analyst specializing in emerging technologies.

"So AR and VR very closely related to the metaverse in the same way that computers are related to the Internet," he said. "Think of it instead as evolution of the Internet, which changes the way that we interact with the world."

So how should SK Telecom's flight simulator be defined?

"Technically, it's not metaverse, but kind of metaverse," said Ken Wohn, a company manager.

South Korea's biggest telecom provider teamed up last year with California's Joby Aviation to develop an electric air taxi service to the country.

One day the air taxis might operate autonomously, using high-speed 5G wireless connections, Wohn said.

It was a different experience at French wireless company Orange's metaverse demonstration, where users were transported to a futuristic neon-hued technoscape with lightning bolts, giant robots and a falcon carrying a green orb in its talons.

A dancing figure appeared, representing the movements of a real-life dancer wearing motion-capture gear. It was a dazzling display, though what consumer purpose it had was not immediately clear.

Miguel Angel Almonacid, Orange's network strategy director for Spain, said it demonstrates how new 5G networks will eliminate lag for metaverse users watching something happening far away.

The metaverse might be more suited to practical purposes in the workplace, analysts said.

"That's where we'll see traction first because the barriers aren't as high," said Gartner's Nguyen. For example, a worker could use augmented reality glasses to pull up diagnostics or an instruction manual.

Spanish startup La Frontera uses the metaverse to provide virtual meetings with "realistic avatars," Marta Ortiz, a business development executive, said as she guided me through the company's metaverse.

We arrived first on a beach, with boulders, palm trees and a light blue sea. Her avatar appeared, a head and shoulders with disembodied hands hovering in front of her chest. We entered a nearby conference room with a boardroom table, where I used handheld controllers to pick up 3D objects like a toy ray gun and a bottle of Champagne.

Other metaverse applications include training for risky, repetitive or highly detailed procedures, like surgery.

The beach disappeared, replaced by an overturned tanker truck on fire. A fire extinguisher hung in midair. Ortiz told me to grab it with my virtual hand and spray it at the flames, which started to die down.

The virtual world also could be useful for showcasing products too big to move easily, like private jets.

Or they could be too small for humans.

The scene changed to a sci-fi setting, with crimson walls rising up around us, representing the inside of a blood vessel. Reddish-brown doughnut-shaped blood cells floated past, followed by spiky orbs. The blood vessel's wall opened up, exposing pulsing white streaks on a blue background, depicting neurons in a brain.

La Frontera works with pharmaceutical companies to "show how a drug works in the body at a cellular level," Ortiz said. In this case, it was a medicine to treat multiple sclerosis, which attacks brain neurons.

Speed around a French village in the video game Gran Turismo and you might spot a Corvette behind you trying to catch your slipstream.

The technique of using the draft of an opponent's racecar to speed up and overtake them is one favored by skilled players of PlayStation's realistic racing game.

But this Corvette driver is not being controlled by a human — it's GT Sophy, a powerful artificial intelligence agent built by PlayStation-maker Sony.

Gran Turismo players have been competing against computer-generated racecars since the franchise launched in the 1990s, but the new AI driver that was unleashed last week on Gran Turismo 7 is smarter and faster because it's been trained using the latest AI methods.

"Gran Turismo had a built-in AI existing from the beginning of the game, but it has a very narrow band of performance and it isn't very good," said Michael Spranger, chief operating officer of Sony AI. "It's very predictable. Once you get past a certain level, it doesn't really entice you anymore."

But now, he said, "this AI is going to put up a fight."

Visit an artificial intelligence laboratory at universities and companies like Sony, Google, Meta, Microsoft and ChatGPT-maker OpenAI and it's not unusual to find AI agents like Sophy racing cars, slinging angry birds at pigs, fighting epic interstellar battles or helping human gamers build new Minecraft worlds -- all part of the job description for computer systems trying to learn how to get smarter in games.

But in some instances, they are also trying to learn how to get smarter in the real world. In a January paper, a University of Cambridge researcher who built an AI agent to control Pokémon characters argued it could "inspire all sorts of applications that require team management under conditions of extreme uncertainty, including managing a team of doctors, robots or employees in an ever-changing environment, like a pandemic-stricken region or a war zone."

And while that might sound like a kid making a case for playing three more hours of Pokémon Violet, the study of games has been used to advance AI research — and train computers to solve complex problems — since the mid-20th century.

Initially, AI was used on games like checkers and chess to test at winning strategy games. Now a new branch of research is more focused on performing open-ended tasks in complex worlds and interacting with humans, not just for the purpose of beating them.

"Reality is like a super-complicated game," said Nicholas Sarantinos, who authored the Pokémon paper and recently turned down a doctoral offer at Oxford University to start an AI company aiming to help corporate workplaces set up more collaborative teams.

In the web-based Pokémon Showdown battle simulator, Sarantinos developed an algorithm to analyze a team of six Pokémon — predicting how they would perform based on all the possible battle scenarios ahead of them and their comparative strengths and weaknesses.

Microsoft, which owns the popular Minecraft game franchise as well as the Xbox game system, has tasked AI agents with a variety of activities — from steering clear of lava to chopping trees and making furnaces. Researchers hope some of their learnings could eventually play a role in real-world technology, such as how to get a home robot to take on certain chores without having to program it to do so.

While it "goes without stating" that real humans behave quite differently from fictional video game creatures, "the core ideas can still be used," Sarantinos said. "If you use psychology tests, you can take this information to conclude how well they can work together."

Amy Hoover, an assistant professor of informatics at the New Jersey Institute of Technology who's built algorithms for the digital card game Hearthstone, said "there really is a reason for studying games" but it is not always easy to explain.

"People aren't always understanding that the point is about the optimization method rather than the game," she said.

Games also offer a useful testbed for AI — including for some real-world applications in robotics or health care — that's safer to try in a virtual world, said Vanessa Volz, an AI researcher at the Danish startup Modl.ai, which builds AI systems for game development.

But, she adds, "it can get overhyped."

"It's probably not going to be one big breakthrough and that everything is going to be shifted to the real world," Volz said.

Japanese electronics giant Sony launched its own AI research division in 2020 with entertainment in mind, but it's nonetheless attracted broader academic attention. Its research paper introducing Sophy last year made it on the cover of the prestigious science journal Nature, which said it could potentially have effects on other applications such as drones and self-driving vehicles.

The technology behind Sophy is based on an algorithmic method known as reinforcement learning, which trains the system by rewarding it when it gets something right as it runs virtual races thousands of times.

"The reward is going to tell you that, 'You're making progress. This is good,' or, 'You're off the track. Well, that's not good,'" Spranger said.

The world's best Gran Turismo players are still finishing ahead of Sophy at tournaments, but average players will find it hard to beat — and can adjust difficulty settings depending on how much they want to be challenged.

PlayStation players will only get to try racing against Sophy until March 31, on a limited number of circuits, so it can get some feedback and go back into testing. Peter Wurman, director of Sony AI America and project lead on GT Sophy, said it takes about two weeks for AI agents to train on 20 PlayStations.

"To get it spread throughout the whole game, it takes some more breakthroughs and some more time before we're ready for that," he said.

And to get it onto real streets or Formula One tracks? That could take a lot longer.

Self-driving car companies adopt similar machine-learning techniques, but "they don't hand over complete control of the car the way we are able to," Wurman said. "In a simulated world, there's nobody's life at risk. You know exactly the kinds of things you're going to see in the environment. There's no people crossing the road or anything like that."

Matt O'Brien is an AP technology writer

Actor Simu Liu will host the Academy of Motion Picture Arts and Sciences’ Scientific and Technical Awards presentation on Friday, February 24, at the Academy Museum of Motion Pictures. He will present nine awards, including an Oscar® statuette, to 20 individual award recipients during the evening. 

Liu made history as the star of “Shang-Chi and the Legend of the Ten Rings,” the first Marvel movie to center on an Asian lead character.  His previous credits include the television series “Kim’s Convenience.”  His upcoming projects include “Barbie,” with Margot Robbie and Ryan Gosling.

Achievements receiving Academy Scientific and Technical Awards need not to have been developed and introduced during a specified period of time. Rather, the achievements must demonstrate a proven record of contributing significant value to the scientific and technical processes of making motion pictures.

The 95th Oscars® will be held on Sunday, March 12, at the Dolby® Theatre at Ovation Hollywood and will be televised live on ABC and in more than 200 territories worldwide.

The Academy of Motion Picture Arts and Sciences will honor eight scientific and technical achievements represented by 19 individual award recipients at its annual Scientific and Technical Awards presentation on Friday, February 24, at the Academy Museum of Motion Pictures, marking its return to an in-person event for the first time since 2019. 

In addition, Iain Neil will receive the Gordon E. Sawyer Award (an Oscar® statuette) for his extraordinary technological contributions that have brought credit to the industry.

“Since 1931, the Academy has recognized the most important innovations in filmmaking; inventors and engineers have been advancing the art and science of motion pictures ever since,” said Barbara Ford Grant, chair of the Scientific and Technical Awards Committee.  “Their efforts have not only served to enrich the art form but inspire a global industry to engineer, create, change, and push the boundaries of our craft.  This year we honor achievements spanning accomplishments from pioneering methods in practical rain effects to career-long contributions in optical design for cinematography to humanistic-driven AI techniques.  This outstanding work has enabled new and exciting ways of creating and further expanding how we experience motion pictures.”

Unlike other Academy Awards® to be presented this year, achievements receiving Scientific and Technical Awards need not have been developed and introduced during a specified period of time.  Rather, the achievements must demonstrate a proven record of contributing significant value to the process of making motion pictures. 

The Academy Awards for scientific and technical achievements are: 
 

TECHNICAL ACHIEVEMENT AWARDS (ACADEMY CERTIFICATES)
To Howard Jensen and Danny Cangemi for the concept and creation, and to John Frazier for the development of the 60- and 100-foot Rain Bars.

The Rain Bars provide a portable system for the creation of realistic, large-scale, adjustable, practical rain for motion pictures. Their rapid setup and relocation capabilities enable the efficient production of effects ranging from misting drizzles to torrential downpours.

To Mark Hills and Jim Vanns for the design and engineering of the FQ render farm management system.

FQ’s highly efficient scheduler and sophisticated prioritization algorithms reflect a deep understanding of render farm management. With an architecture that has remained largely unchanged for more than a decade, FQ continues to support substantial growth in computational complexity at Framestore.

To Matt Chambers for his contributions to modern render farm management system design as exemplified in the scheduling architectures of Cue3 and Plow.

These design contributions have resulted in robust, versatile, extensible and highly scalable render farm management systems that have supported substantial growth in computational complexity at Sony Pictures Imageworks and Weta Digital.

To Sébastien Deguy and Christophe Soum for the concept and original implementation of Substance Engine, and to Sylvain Paris and Nicolas Wirrmann for the design and engineering of Substance Designer.

Adobe Substance 3D Designer provides artists with a flexible and efficient procedural workflow for designing complex textures. Its sophisticated and art-directable pattern generators, intuitive design, and renderer-agnostic architecture have led to widespread adoption in motion picture visual effects and animation.

To David Eberle, Theodore Kim, Fernando de Goes and Audrey Wong for the design and development of the Fizt2 elastic simulation system.

Fizt2 provides a high-performance solver with novel and stable implicit physics and robust collision detection. The design of this system enables artist workflows to easily apply soft-body dynamics to a broad range of interacting animated characters and objects.

SCIENTIFIC AND ENGINEERING AWARDS (ACADEMY PLAQUES)
To Larry Barton for the pioneering design, development and engineering, and to Ben Wilcox for the electronic engineering and software development, of the Cinematography Electronics CineTape. 

The CineTape distance measurement system provides focus-pullers with continuous, accurate, real-time distance information to the subject, either at the camera or remotely. This high resolution distance data has enabled the reliable execution of shots that previously were impossible to judge accurately or had required multiple takes to achieve.

To Howard Preston for the concept, design and engineering, and to Bernie Butler-Smith for the design and implementation of electronic circuitry and software, of the Preston Cinema Systems Light Ranger 2.

The Light Ranger 2 provides precise real-time focus distance information by continuously tracking subjects in sixteen discrete zones. The distance and depth of field indicators are superimposed on the camera image, enabling the focus-puller to intuitively judge focus, even in formerly impossible and extremely challenging situations.

AWARD OF COMMENDATION (SPECIAL PLAQUE)
To Ryan Laney for his innovative adaptation and deployment of AI-driven facial veiling technology used to protect the identities while preserving the visual relatability of subjects in documentary filmmaking as exemplified in Welcome to Chechnya (2020).

GORDON E. SAWYER AWARD (OSCAR STATUETTE)
To Iain Neil for his substantial, extensive and innovative lens designs which have had lasting impact in motion picture cinematography.

The 95th Oscars® will be held on Sunday, March 12, 2023, at the Dolby® Theatre at Ovation Hollywood and will be televised live on ABC and in more than 200 territories worldwide.