Pyrra, a startup using artificial intelligence (AI) to identify threats to people and companies online.

Created an AI platform that identifies hate speech, threats, and disinformation on social media sites, which can alert companies and individuals to security threats.

Using natural language processing, a type of AI that can interpret human language, Pyrra’s AI models process posts from social media sites and score them for metrics like hate speech, offensive content, and threats of violence. Using a zero-shot classification model, a type of computer visionmodel that can categorize new images with little training, the system can also analyze images for negative sentiments, like racist, anti-semitic, anti-LGBT, or anti-Muslim content. The AI system flags any troubling content for deeper investigation, either by law enforcement or through an organization’s internal processes.

Pyrra was initially created as a research and development project inside Human Rights First, a non-governmental organization focused on promoting human rights worldwide. The platform got its first test ahead of January 6, 2021, when supporters of former president Donald Trump overtook the U.S. Capitol in Washington D.C. In the lead up to January 6, Welton Chang, one of Pyrra’s founders and its CEO, had been monitoring conversations on small and alternative social media sites like Parler and Discord. “I knew that there was going to be a major event,” he tells MIT Horizon.

Afterwards, it became even clearer that conversations happening on small-scale social media sites, as well as more mainstream sites, could provide important insights on potentially dangerous social movements. Pyrra would officially launch as an independent entity about a year later. “The more we looked at what was in front of us, the more we realized that we really needed to get our technology out into the market, to help companies, to help governments figure out what was going on in the smaller forums on the internet,” Chang says. “There was a market gap that wasn’t being covered by existing technologies.”

Pyrra currently works most often through subscriptions with big companies and nonprofits, and Chang says researchers also use the platform to understand how hate speech and discussions of political violence are changing on social media sites over time.

To identify potential threats online, companies, public figures, and individuals would ordinarily need to search manually for relevant posts on social media sites to understand how they were being talked about. That can be time-consuming, and company analysts can only get through a small fraction of all relevant posts in a day. “There’s so much content out there. It’s almost an infinite pool of data that’s being created every single day,” Chang says. “How could you possibly look at everything?” Pyrra processes millions of records every day, trawling more than 30 social media sites. The software is capable of analyzing several different languages.

Some companies also don’t have dedicated staff focused on online threats, which means such monitoring is not always a priority. Only a machine is capable of keeping tabs on all that data in real time, Chang says. “You don’t really need a master’s degree to be able to go and conduct searches on these platforms. So you want to offload that tedious task to a machine,” he says.

Pyrra now has over 1,000 active projects on its system, such as monitoring mentions of a company’s executives or how a brand is being discussed online. Companies may select to track those discussions over time, send them to their legal team, or even forward them to law enforcement for potential action if there’s concern about illegal or dangerous activities.

AI can handle repetitive tasks—like monitoring social media sites and flagging posts—more consistently than humans. That’s important in cases of safety, because missing a post, in some circumstances, can mean life or death. “Humans have to go to the bathroom, take vacations, take care of their kids, eat food,” Chang says. “But a machine doesn’t need to do any of that.” In the case of upsetting or hateful content, such posts can also have an emotional toll on a human reviewer, whereas a machine can process those posts continually and without being affected.

For more on how the models Pyrra uses can process and understand content, see How Natural Language Processing Works.