DARPA Taps RTX to Attune AI Decisions to Human Values
August 11, 2023 | Raytheon TechnologiesEstimated reading time: 2 minutes
RTX's BBN division received a contract award to support DARPA's "In The Moment" program. ITM aims to develop the foundations needed for algorithms that are trusted to independently make decisions in scenarios like mass casualty triage and disaster relief, where complex and rapid decisions are needed in dynamic situations where there is often no human consensus and no clear right answer.
"ITM is about more than getting AI to provide the correct answer in very controlled scenarios. We're not talking about training AI on labelled data to help identify a cancerous tumor on an X-Ray," said Alice Leung, Raytheon BBN principal investigator. "What we're trying to accomplish instead is the ability to create AI systems that humans would allow to make decisions independently in uncontrolled environments. To accomplish this, we need to determine how human experts make really difficult decisions and assess whether to trust the decisions of others. We'll be conducting both decision-making research and trust research."
The Raytheon BBN-led team, which includes Kairos Research, MacroCognition, and Valkyries Austere Medical Solutions, will use a cognitive interviewing technique to understand how experts—in this case, medical professionals and first responders—evaluate information and make tough trade-offs to act decisively at critical decision points. This qualitative information will be used to design scenario-based experiments to study how differences in an individual's decision-making attributes can explain their choices, and how the alignment of attributes between two different people impacts the willingness to delegate decisions to another. This will enable AI to be tuned to match an expert population, or even to be tuned to match an individual expert.
"Because the way we make decisions varies from person to person, it's unlikely that a one-size fits all trusted AI model exists," said Leung. "Instead, in theory, we should be able to create AI systems that adapt to the user and domain. Decisions are difficult because of uncertainty and trade-offs between competing goals. We want to be able to tune an AI's attributes such as risk tolerance, process focus, or willingness to change plans to better match a user or a group of users."
DARPA is bringing together multiple teams to collaborate on this program. Other teams will focus on the development of prototype AI decision-makers that start with baseline knowledge and can then be tuned to match a set of target attributes. The research products from this program will be integrated and evaluated to determine how well the algorithmic agents were able to make decisions consistent with the target human attributes when faced with difficult scenarios. The program will also test whether human experts trust these aligned agents over the baseline agents or other actual humans. In these program evaluations of trust, the human experts will be shown a record of decisions in difficult scenarios without knowing whether the decision-maker was an AI or a human.
Work on this contract, which is sponsored by DARPA and the Air Force Research Lab, is being performed in Cambridge, Massachusetts; Dayton, Ohio; and Anniston, Alabama.
Suggested Items
Chinese Smartphone Market Maintains its Recovery Momentum at 6.5% Growth in 1Q24,
04/26/2024 | IDCAccording to preliminary data from the International Data Corporation (IDC) Worldwide Quarterly Mobile Phone Tracker, China smartphone shipments grew 6.5% year over year (YoY) to 69.3 million units in 1Q24.
Boeing Opens Research & Technology Center in Japan
04/23/2024 | BoeingBoeing today opened a Boeing Research & Technology (BR&T) Center in Japan that will focus on innovation to enable the commercial aviation industry meet its goal of net zero carbon emissions by 2050.
Gartner Forecasts Worldwide IT Spending to Grow 8% in 2024
04/17/2024 | Gartner, Inc.Worldwide IT spending is expected to total $5.06 trillion in 2024, an increase of 8% from 2023, according to the latest forecast by Gartner, Inc. This is an increase from the previous quarter’s forecast of 6.8% growth and puts worldwide IT spending on track to surpass $8 trillion well before the end of the decade.
Argonne, RIKEN Sign a Memorandum of Understanding in Support of AI for Science
04/16/2024 | BUSINESS WIRELeaders in high performance computing in the U.S. and Japan have signed a memorandum of understanding (MOU) establishing a cooperative relationship in support of artificial intelligence (AI) computing projects.
RAICo Fellowship to Widen Participation Within the Field of Robotics and Artificial Intelligence
04/12/2024 | RAICoWOMEN and those early in their academia career who are passionate about robotics and artificial intelligence are encouraged to apply for a RAICo Fellowship.