Making AI Research Transparent and Accessible

Having a supportive workplace can mean having the opportunity to pursue what you’re passionate about. For Booz Allen Lead Scientist Stella Biderman, this involves studying artificial intelligence (AI), exploring its potential and limitations, and helping to make understanding of this transformative technology open to all.

At Booz Allen, Stella is researching large language models and algorithms like ChatGPT and DALL-E that can recognize, summarize, translate, predict, and generate content based on the knowledge they gain from huge datasets.

“As large language models proliferate throughout society, it’s really important to understand what they can and cannot do,” Stella says. “One critical way to ensure this happens is through training and publicly releasing models that can be studied by people or organizations that can’t afford to train their own.”

Read more about Stella’s contributions at Booz Allen and how she’s working with a non-profit AI research lab to make knowledge and understanding of this technology open to all, while fostering collaboration between private industry and the AI research community.

After completing her university studies in mathematics and philosophy, Stella applied for data science jobs and joined Booz Allen.

Her initial data scientist role led to work with a research group providing government agencies with outside expertise in math, analytics, statistics, and model building. “We worked with analysts to gain an understanding of their missions and goals and look at what tech they might be deploying in 2 to 5 years,” Stella says.

In her free time, she avidly followed the evolution of natural language processing (NLP)—the ability of a computer to understand human language—and large language models. In 2020, GPT-3—a large language model that uses deep learning to generate human-like text—was released. “It quickly became a big fascination of mine,” Stella says.

“I explained to my leadership that I thought this would be a high-impact area,” she says. “Now most of my work is focused on large language models and similar tech.” 

Once unsupervised learning models known as “transformers” were introduced, they “became state of the art for NLP and made a big splash in computer vision,” Stella explains. “This changed NLP forever.”

“Transformers are extremely powerful and versatile,” she says. “We can now do protein modeling in the same way we do NLP—just change out the data.”

AI models like ChatGPT, DALL-E, and stable diffusion all use transformers. “They’re spiritually based on the same revolution,” Stella points out.

The next step involves scaling models up, which is easier said than done. “Money is the biggest limitation,” she says. “Training these models is very expensive—often hundreds of thousands or millions of dollars per model.”  

This is where EleutherAI comes in. Founded “by three people hanging out talking about how cool AI is,” the volunteer group builds and trains large language models for anyone to use for research, provided their computers have adequate hardware and are configured appropriately.

As the group’s executive director, Stella has played an instrumental role. She has co-authored research, built an evaluation framework for the models, trained models, and helped advise people on their use. She also created a data set that helps inform research in areas such as bias and online discourse.

“I feel supported by Booz Allen in my research,” she says. “Very few places would allow me to take what I was doing outside of the company and support me in pursuing it during work hours.”

Especially as generative AI is increasingly used to create new text, audio, image, and video content, “There’s a lot of confusion, limitations, and things you need to be careful about,” Stella says. “This technology is important for people to study and try to understand.”