Toolbox

No sooner did ChatGPT get unleashed than hackers started "jailbreaking" the artificial intelligence chatbot — trying to override its safeguards so it could blurt out something unhinged or obscene.

But now its maker, OpenAI, and other major AI providers such as Google and Microsoft, are coordinating with the Biden administration to let thousands of hackers take a shot at testing the limits of their technology.

Some of the things they'll be looking to find: How can chatbots be manipulated to cause harm? Will they share the private information we confide in them to other users? And why do they assume a doctor is a man and a nurse is a woman?

"This is why we need thousands of people," said Rumman Chowdhury, lead coordinator of the mass hacking event planned for this summer's DEF CON hacker convention in Las Vegas that's expected to draw several thousand people. "We need a lot of people with a wide range of lived experiences, subject matter expertise and backgrounds hacking at these models and trying to find problems that can then go be fixed."

Anyone who's tried ChatGPT, Microsoft's Bing chatbot or Google's Bard will have quickly learned that they have a tendency to fabricate information and confidently present it as fact. These systems, built on what's known as large language models, also emulate the cultural biases they've learned from being trained upon huge troves of what people have written online.

The idea of a mass hack caught the attention of U.S. government officials in March at the South by Southwest festival in Austin, Texas, where Sven Cattell, founder of DEF CON's long-running AI Village, and Austin Carson, president of responsible AI nonprofit SeedAI, helped lead a workshop inviting community college students to hack an AI model.

Carson said those conversations eventually blossomed into a proposal to test AI language models following the guidelines of the White House's Blueprint for an AI Bill of Rights — a set of principles to limit the impacts of algorithmic bias, give users control over their data and ensure that automated systems are used safely and transparently.

There's already a community of users trying their best to trick chatbots and highlight their flaws. Some are official "red teams" authorized by the companies to "prompt attack" the AI models to discover their vulnerabilities. Many others are hobbyists showing off humorous or disturbing outputs on social media until they get banned for violating a product's terms of service.

"What happens now is kind of a scattershot approach where people find stuff, it goes viral on Twitter," and then it may or may not get fixed if it's egregious enough or the person calling attention to it is influential, Chowdhury said.

In one example, known as the "grandma exploit," users were able to get chatbots to tell them how to make a bomb — a request a commercial chatbot would normally decline — by asking it to pretend it was a grandmother telling a bedtime story about how to make a bomb.

In another example, searching for Chowdhury using an early version of Microsoft's Bing search engine chatbot — which is based on the same technology as ChatGPT but can pull real-time information from the internet — led to a profile that speculated Chowdhury "loves to buy new shoes every month" and made strange and gendered assertions about her physical appearance.

Chowdhury helped introduce a method for rewarding the discovery of algorithmic bias to DEF CON's AI Village in 2021 when she was the head of Twitter's AI ethics team — a job that has since been eliminated upon Elon Musk's October takeover of the company. Paying hackers a "bounty" if they uncover a security bug is commonplace in the cybersecurity industry — but it was a newer concept to researchers studying harmful AI bias.

This year's event will be at a much greater scale, and is the first to tackle the large language models that have attracted a surge of public interest and commercial investment since the release of ChatGPT late last year.

Chowdhury, now the co-founder of AI accountability nonprofit Humane Intelligence, said it's not just about finding flaws but about figuring out ways to fix them.

"This is a direct pipeline to give feedback to companies," she said. "It's not like we're just doing this hackathon and everybody's going home. We're going to be spending months after the exercise compiling a report, explaining common vulnerabilities, things that came up, patterns we saw."

Some of the details are still being negotiated, but companies that have agreed to provide their models for testing include OpenAI, Google, chipmaker Nvidia and startups Anthropic, Hugging Face and Stability AI. Building the platform for the testing is another startup called Scale AI, known for its work in assigning humans to help train AI models by labeling data.

"As these foundation models become more and more widespread, it's really critical that we do everything we can to ensure their safety," said Scale CEO Alexandr Wang. "You can imagine somebody on one side of the world asking it some very sensitive or detailed questions, including some of their personal information. You don't want any of that information leaking to any other user."

Other dangers Wang worries about are chatbots that give out "unbelievably bad medical advice" or other misinformation that can cause serious harm.

Anthropic co-founder Jack Clark said the DEF CON event will hopefully be the start of a deeper commitment from AI developers to measure and evaluate the safety of the systems they are building.

"Our basic view is that AI systems will need third-party assessments, both before deployment and after deployment. Red-teaming is one way that you can do that," Clark said. "We need to get practice at figuring out how to do this. It hasn't really been done before."

Matt O'Brien is an AP technology writer

Vice President Kamala Harris met on Thursday with the heads of Google, Microsoft and two other companies developing artificial intelligence as the Biden administration rolls out initiatives meant to ensure the rapidly evolving technology improves lives without putting people's rights and safety at risk.

President Joe Biden briefly dropped by the meeting in the White House's Roosevelt Room, saying he hoped the group could "educate us" on what is most needed to protect and advance society.

"What you're doing has enormous potential and enormous danger," Biden told the CEOs, according to a video posted to his Twitter account.

The popularity of AI chatbot ChatGPT — even Biden has given it a try, White House officials said Thursday — has sparked a surge of commercial investment in AI tools that can write convincingly human-like text and churn out new images, music and computer code.

But the ease with which it can mimic humans has propelled governments around the world to consider how it could take away jobs, trick people and spread disinformation.

The Democratic administration announced an investment of $140 million to establish seven new AI research institutes.

In addition, the White House Office of Management and Budget is expected to issue guidance in the next few months on how federal agencies can use AI tools. There is also an independent commitment by top AI developers to participate in a public evaluation of their systems in August at the Las Vegas hacker convention DEF CON.

But the White House also needs to take stronger action as AI systems built by these companies are getting integrated into thousands of consumer applications, said Adam Conner of the liberal-leaning Center for American Progress.

"We're at a moment that in the next couple of months will really determine whether or not we lead on this or cede leadership to other parts of the world, as we have in other tech regulatory spaces like privacy or regulating large online platforms," Conner said.

The meeting was pitched as a way for Harris and administration officials to discuss the risks in current AI development with Google CEO Sundar Pichai, Microsoft CEO Satya Nadella and the heads of two influential startups: Google-backed Anthropic and Microsoft-backed OpenAI, the maker of ChatGPT.

Harris said in a statement after the closed-door meeting that she told the executives that "the private sector has an ethical, moral, and legal responsibility to ensure the safety and security of their products."

ChatGPT has led a flurry of new "generative AI" tools adding to ethical and societal concerns about automated systems trained on vast pools of data.

Some of the companies, including OpenAI, have been secretive about the data their AI systems have been trained upon. That's made it harder to understand why a chatbot is producing biased or false answers to requests or to address concerns about whether it's stealing from copyrighted works.

Companies worried about being liable for something in their training data might also not have incentives to rigorously track it in a way that would be useful "in terms of some of the concerns around consent and privacy and licensing," said Margaret Mitchell, chief ethics scientist at AI startup Hugging Face.

"From what I know of tech culture, that just isn't done," she said.

Some have called for disclosure laws to force AI providers to open their systems to more third-party scrutiny. But with AI systems being built atop previous models, it won't be easy to provide greater transparency after the fact.

"It's really going to be up to the governments to decide whether this means that you have to trash all the work you've done or not," Mitchell said. "Of course, I kind of imagine that at least in the U.S., the decisions will lean towards the corporations and be supportive of the fact that it's already been done. It would have such massive ramifications if all these companies had to essentially trash all of this work and start over."

While the White House on Thursday signaled a collaborative approach with the industry, companies that build or use AI are also facing heightened scrutiny from U.S. agencies such as the Federal Trade Commission, which enforces consumer protection and antitrust laws.

The companies also face potentially tighter rules in the European Union, where negotiators are putting finishing touches on AI regulations that could vault the 27-nation bloc to the forefront of the global push to set standards for the technology.

When the EU first drew up its proposal for AI rules in 2021, the focus was on reining in high-risk applications that threaten people's safety or rights such as live facial scanning or government social scoring systems, which judge people based on their behavior. Chatbots were barely mentioned.

But in a reflection of how fast AI technology has developed, negotiators in Brussels have been scrambling to update their proposals to take into account general purpose AI systems such as those built by OpenAI. Provisions added to the bill would require so-called foundation AI models to disclose copyright material used to train the systems, according to a recent partial draft of the legislation obtained by The Associated Press.

A European Parliament committee is due to vote next week on the bill, but it could be years before the AI Act takes effect.

Elsewhere in Europe, Italy temporarily banned ChatGPT over a breach of stringent European privacy rules, and Britain's competition watchdog said Thursday it's opening a review of the AI market.

In the U.S., putting AI systems up for public inspection at the DEF CON hacker conference could be a novel way to test risks, though not likely as thorough as a prolonged audit, said Heather Frase, a senior fellow at Georgetown University's Center for Security and Emerging Technology.

Along with Google, Microsoft, OpenAI and Anthropic, companies that the White House says have agreed to participate include Hugging Face, chipmaker Nvidia and Stability AI, known for its image-generator Stable Diffusion.

"This would be a way for very skilled and creative people to do it in one kind of big burst," Frase said.

O'Brien reported from Cambridge, Massachusetts. AP writers Seung Min Kim in Washington and Kelvin Chan in London contributed to this report.

ARRI Asia-Pacific--established in 2008 and initially operating from Hong Kong--has relocated to Singapore.

To mark the occasion, ARRI Asia-Pacific organized a grand ceremony on April 25 with an open house and an ARRI party. The top management team from the headquarters in Germany, including Executive Board members Dr. Matthias Erb (chairman) and Lars Weyer (CFO), industry leaders, and key players from Asia-Pacific’s moving image industry graced the festivities.

ARRI Asia Pte. Ltd., the official name of the new subsidiary in Singapore, is part of ARRI Asia-Pacific, which also includes ARRI Korea, ARRI Japan, and ARRI Australia. Together, they provide sales and services to the entire Asia-Pacific region.

“The inauguration of the Singapore subsidiary in the heart of the Asia-Pacific market symbolizes a new phase in ARRI’s venture in the region. It shows how vital the region, including its emerging markets, is for ARRI. Together with our customers, we plan to significantly increase our activities here,” said Dr. Erb.

Bertrand Dauphant, ARRI Asia-Pacific managing director, added, “With this move, ARRI is now even better equipped to serve the Asia-Pacific market and meet the increasing demand for our products and services. ARRI Asia-Pacific is now structured around four strong hubs allowing us to better support our clients and promote industry growth throughout the region.”

Primely located in Marina Centre, Singapore, the new corporate office spans 3,600 square feet and boasts a modern and innovative design with exceptional facilities for both customers and staff. The facility features a multi-purpose creative space that can be easily converted for equipment demonstrations, ARRI Academy training, company events, and more. The office also includes an open-concept workspace, adaptable meeting rooms, and a collaboration corner to enhance productivity and efficiency.

Furthermore, the subsidiary in Singapore houses a fully equipped 3,000 square-foot service center to cater to the growing demand for maintenance and repair of the extensive range of ARRI products in the market. In addition, the service center includes a warehouse space to ensure clients receive products promptly.

RED Digital Cinema® has announced the availability of the RED Connect Module for RED’s V-Raptor and V-Raptor XL cameras. The new module allows users to unlock the capabilities of up to 8K live cinematic streaming via the RED Connect solution. RED Connect enables users the ability to real-time stream RAW R3D files direct from the V-Raptor and V-Raptor XL camera systems over IP to a camera control unit (CCU), opening a range of creative applications from live broadcast to virtual production to true 8K VR.

The RED Connect Module is a turn-key solution for customers to access the wide-ranging capabilities of RED Connect. With a compact form-factor and easy connectivity, the module attaches securely to the back of the V-Raptor or V-Raptor XL via the camera’s V-Lock battery mechanism. The high-speed data connection is created by connecting the module to the camera’s CFexpress media slot.

The module allows for live streaming of up to 8K at 120FPS or 4K at 240FPS, as well as all other frame rate and resolution combinations offered by the camera. It also supports simultaneous streaming and recording from the CCU. The module offers Genlock and timecode synchronization of multiple cameras using PTP (SMPTE ST 2059-2) and up to 10 Gbps connection via a single-mode LC connector.

“We are extremely excited to officially launch RED Connect and the new module,” said Jeff Goodman, RED Digital Cinema VP of product management. “Live streaming of full-quality R3Ds over IP at every resolution and framerate, combined with RED’s sensor capabilities, creates an entirely new paradigm for content creation and broadcast. In early release testing, our customers have been blowing us away by what they have been able to produce. We’re excited to see what’s next creatively now that RED Connect’s flexible, open ecosystem is available to take content creation to the next level.”

The RED Connect system extends the camera’s capabilities in virtual productions and new production environments such as live XR. Users can simplify their camera setup by reducing SDI cables, timecode and genlock devices, and other connections to a single ethernet cable to reduce points of failure. In the world of XR, users can stream 8K 60p content straight from the camera to any end device. The increased resolution pushes the visual experience into an entirely new immersive world, especially when viewed in a VR headset environment.

RED’s innovative partners, including COSM, Media.Monks, and NVIDIA, have been deploying and testing RED Connect and the module in real-world applications over the past 18 months.

“The opportunity to leverage uncompressed 8K end-to-end presents a tremendous opportunity for broadcasters, filmmakers and immersive video in particular,” noted Lewis Smithingham, Media.Monks SVP of innovation. “Uncompressed 8K delivers on the promise of VR, and when combined with stereo lensing, produces a sense of presence never experienced before.” 

For varying use cases, the RED Connect Module can be deployed by adapting the CCU design. The flexibility of the CCU allows for real-time AI processing using RED’s SDK and NVIDIA professional GPUs for graphic intensive workflows with complete SMPTE 2110 implementation or options for one to two frames of latency for 8K and 4K IP broadcast and everything in-between.

The RED Connect Module is priced at $14,995, with RED Connect licenses available for one year or as a perpetual license. A one-month option will be available soon. Pricing for the RED Connect feature varies based on the length of licensing agreement. RED will showcase RED Connect with the module at the 2023 National Association of Broadcasters (NAB) show, including demonstrations of live streaming of 8K60P to VR headsets enabled by Microsoft and Meta at the Microsoft booth (#W1529).