Addressing Online Threats: AI's Role in Countering Harmful Social Media Content

The talk traces how social media has fragmented from a handful of mainstream platforms into dozens of smaller communities where like-minded users reinforce each other's beliefs. The speaker cites survey research suggesting that roughly four to five percent of American adults get nearly all their news from such sites, and argues that weak moderation on these platforms lets extreme material persist and spread.

The middle of the talk turns to case studies that link online activity to physical harm. A Buffalo supermarket attacker was exposed to a livestreamed massacre clip on an imageboard, and a plot against electrical substations in Baltimore was promoted on Telegram. After a mass shooting in Lewiston, Maine, an innocent man was publicly identified on fringe sites, and threats against a private employee were shared using a photo rather than a name, which shows why image analysis matters.

The speaker explains that monitoring millions of daily posts far exceeds what human analysts can review, so the company uses AI classifiers for text and images, including facial matching, while acknowledging that even strong models are far from perfect. The Q&A covered analyst wellbeing, the fact that clients handle law enforcement referrals, and the limits of relying only on open data. The takeaway is that AI can narrow the workload, but human review remains essential.