Words That Hurt: Turning AI Insight into Safer Digital Democracy
SOCYTI brings research, artificial intelligence and human judgment together to address digital violence and online hate speech.
At Arendalsuka on 12 August, researchers, public officials, journalists and civil society representatives gathered for the SOCYTI session “Words that Hurt: How Can We Counteract Digital Violence and Online Hate Speech?” The discussion focused on a pressing democratic challenge: how can we identify harmful digital behaviour early enough to act, while protecting privacy, human rights and freedom of expression?
Moderated by Karl-Andreas Kjelstrup, the session featured SOCYTI project manager Professor Rajendra Akerkar (Vestlandsforsking), Professor Sule Yildirim Yayilgan (NTNU Gjøvik), Professor Carol Azungi Dralega (NLA Høgskolen), and Kim Henrik Gronert (Kristiansand kommune). The programme combined a presentation of the research with a live demonstration of the SOCYTI analytical tool and an open panel discussion.
The event was also the project's final public event, ahead of its conclusion in September 2026.
Photo credit: Ehtesham Hashmi
“Detection is the first step, not the last. The real challenge is turning an early signal into responsible action while keeping human judgment, rights and democratic values at the centre.”
— Karl-Andreas Kjelstrup, moderator & journalist
Why early detection matters
Online hate speech, radicalisation and polarisation do not remain confined to social media. They can contribute to fear, social fragmentation and tensions in local communities, affecting schools, public services and democratic participation.
SOCYTI, led by Vestlandsforsking with NTNU Gjøvik, NLA Høgskolen and George Mason University, addresses this challenge by combining social science with AI-driven analytics.
As Professor Akerkar explained during the session:
Photo credit: Ehtesham Hashmi
“Most online safety tools were built for English, leaving smaller language communities like Norway without effective protection. We need to understand local language and context so we can identify early signs of hostility before they turn into real-world harm.”
— Professor Rajendra Akerkar, SOCYTI project manager
Traditional ways of fighting online hate, like searching for basic "bad words" or reacting after the damage is done, cannot keep up with today's digital landscape. Online hostility moves rapidly, uses coded language, and spreads across platforms. To protect local communities, public authorities need timely, evidence-based tools that recognize rising hostility before it escalates into real-world harm.
The SOCYTI project moves far beyond outdated keyword filters. Using a cloud-based framework powered by multi-task learning and knowledge graphs, the system evaluates linguistic context, tone, and underlying meaning across languages to spot early patterns of toxicity and polarization.
Crucially, this technology was co-created directly alongside the people who understand digital violence best. Victims of online abuse, journalists, civil society advocates, municipal workers, and law enforcement officials participated in hands-on workshops throughout the project lifecycle. They helped train the AI to understand local context and respect the line between harmful targeted speech and protected free expression. Stakeholders also tested the software to ensure its alerts are clear, transparent, and practical for non-technical decision-makers. By combining smart AI with continuous human oversight, SOCYTI turns complex data into trustworthy insights that help local leaders safeguard public debate.
Representing municipal practice, Kim Henrik Gronert (Kristiansand Municipality) shared his impressions from testing the early prototype:
“My primary focus during testing was evaluating whether the tool could accurately assess online content. In my experience, it performed remarkably well in identifying and rating the severity of hateful speech.”
From detection to decision support
The live demonstration showed how the SOCYTI tool can turn large amounts of online conversation into signals that are easier for people to interpret. Ehtesham Hashmi, a student team member on the project, presented the live SOCYTI software demonstration.
Photo credit: Carol Azungi Dralega
The purpose is not to create an automated digital judge. A flagged message or pattern is not a legal finding, a reason to ban someone, or an instruction to delete content. Instead, it is an early-warning signal that can prompt human assessment.
That distinction was central to the panel discussion.
The system considers the wider context of communication including tone, target and severity rather than simply searching for offensive words. At the same time, the boundaries of what constitutes hate speech or harmful behaviour must remain grounded in human expertise, law and social standards.
Human oversight is a safeguard, not a limitation
The panel also addressed one of the most important concerns surrounding AI-based monitoring: could a tool designed to improve digital safety become a surveillance or censorship mechanism?
SOCYTI's approach places privacy, ethics and human oversight at the centre. The system is designed to focus on broader trends and group-level conversations, with personal details such as names and user IDs removed.
Its Explainable AI capabilities are intended to make the reasons behind risk signals understandable to human users. This transparency is essential if public authorities and civil society are to trust the technology and if citizens are to have confidence that it supports, rather than undermines, democratic participation.
From research to practice
The Arendalsuka discussion highlighted a broader lesson: tackling digital violence requires more than better algorithms.
Photo credit: Carol Azungi Dralega
It requires collaboration between researchers, municipalities, civil society, technology developers and policymakers. It requires privacy and human rights to be designed into systems from the outset. And it requires clear pathways for turning technical signals into proportionate, lawful and socially responsible action.
For decision-makers, the SOCYTI experience highlights several clear priorities for building safer digital spaces. First, authorities must invest in early-warning systems (decision support systems) rather than relying solely on reactive moderation after harm has already occurred. This technology must be privacy-preserving and grounded in human rights from day one, incorporating transparent, explainable AI so non-technical users understand why content is flagged. Crucially, automated systems must never operate as autonomous judges; human oversight is essential to keep people accountable for major decisions. Additionally, digital safety tools should be co-created alongside municipalities and civil society to match real-world operational needs, with practical response procedures developed in tandem so an early warning can translate into meaningful civic action.
A final public conversation and a starting point for what comes next
As SOCYTI approaches the end of its project period in September, this event offered an opportunity to reflect on what has been achieved and what remains to be done.
The project has demonstrated how AI and social science can work together to provide earlier insight into harmful digital patterns, particularly in local-language contexts. But it has also highlighted a fundamental principle: technology should strengthen human decision-making, not replace it.
The challenge now is to move from research insight to responsible implementation.
Digital violence and online hate speech will not be solved by AI alone. But research-backed tools, combined with strong safeguards, human expertise and effective local response systems, can give communities something they often lack: time to understand what is happening and act before online hostility becomes harder to contain.
The SOCYTI session at Arendalsuka made that point clearly. Detection is important, but it is only the beginning.