A New Zealand startup is developing a tool to redirect users showing violent extremist tendencies on AI platforms to deradicalization support, aiming to address safety concerns amidst growing legal actions against AI companies. The hybrid model will combine a chatbot with referrals to real-world mental health services.
A New Zealand startup is developing a tool to redirect users exhibiting violent extremist tendencies on AI platforms like ChatGPT to deradicalization support. This initiative aims to address safety concerns amid increasing legal actions against AI companies, which are accused of failing to prevent, and sometimes enabling, violent behavior.
The tool, created by ThroughLine, a company specializing in crisis intervention, will leverage a hybrid model combining a chatbot designed to engage with individuals showing signs of extremism with referrals to real-world mental health services. This effort reflects a growing recognition of the need for proactive measures to mitigate the potential for AI-driven radicalization and the spread of extremist ideologies online.\ThroughLine, already contracted by OpenAI, Anthropic, and Google to provide crisis support for users at risk of self-harm, domestic violence, or eating disorders, is expanding its focus to include preventing violent extremism. The company is collaborating with The Christchurch Call, an initiative established after the 2019 New Zealand terrorist attack to combat online hate speech, for guidance in developing the intervention chatbot. The aim is to create a comprehensive support system for individuals expressing extremist views, thereby reducing the risk of real-world harm. The development of this tool is a response to the evolving landscape of online discourse and the increasing complexity of mental health issues exacerbated by the rise of AI chatbots. The company is actively working with experts to ensure the effectiveness and ethical implementation of its approach, avoiding the use of generic large language model datasets in favor of specialized expertise.\The tool's potential rollout is intended for a range of users, including moderators of gaming forums, parents, and caregivers, to help identify and address extremist behavior. The design includes follow-up mechanisms and consideration of potential triggers, balancing the need for intervention with the risk of escalating harmful behaviors. Experts emphasize the importance of effective follow-up systems and supportive resources to ensure the success of such initiatives. The project also addresses concerns about the limitations of existing moderation practices, which can inadvertently push individuals toward less-regulated platforms, and the importance of offering support instead of abruptly cutting off access. The goal is to provide a safe and supportive environment for those struggling with extremist thoughts, connecting them with appropriate resources and preventing potential harm. The project highlights the necessity for nuanced approaches that acknowledge the complexities of online interactions and mental health challenges in the context of rapidly evolving AI technologies
AI Extremism Deradicalization Chatgpt Mental Health
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Drone attack targets Kuwait airport, fuel tanks on fireKuwait City – Kuwait's civil aviation authority said Wednesday that the Gulf state's international airport had come under an Iranian drone attack that led
Read more »
Drone attack targets Kuwait airport, fuel tanks on fireKuwait City – Kuwait's civil aviation authority said Wednesday that the Gulf state's international airport had come under an Iranian drone attack that led
Read more »
Rimini Street Wins Multiple Stevie Awards for AI-Powered Customer Service ExcellenceRimini Street, a leading third-party support provider, announced wins in three categories at the 20th annual Stevie Awards, highlighting its AI-driven customer service and commitment to client satisfaction. The company was recognized for its innovative use of AI to enhance human expertise in customer service, its people-first approach to IT support, and its customer satisfaction strategy. This includes gold for Best Use of AI in Customer Service, silver for Front-Line Customer Service Team of the Year, and bronze for Best Customer Satisfaction Strategy.
Read more »
Indie Game Developer Crafts Narrative-Driven Title 'Lost & Found'A De La Salle Lipa graduate, inspired by 'The Amazing World of Gumball' and 'Perfect Days,' is developing 'Lost & Found,' an indie game exploring the emotional impact of forgotten objects. The developer handles all aspects of the game's creation, drawing inspiration from childhood experiences and a passion for storytelling.
Read more »
Garmin Launches Pokémon Sleep Watch FacesGarmin introduces free Pokémon Sleep themed watch faces for select smartwatches, allowing users to personalize their devices with designs featuring Snorlax and friends or customizable companion options. The watch faces dynamically display Pokémon expressions based on the user's Body Battery energy levels and include a night mode to help users wind down. Compatibility is also extended to sync sleep data with the Pokémon Sleep mobile app on certain Garmin devices.
Read more »
Tricycle Hit by Pick-up Truck in Pangasinan Kills Three, Fuel Tanker Driver Dies in QuezonThree individuals returning from a beach trip in Pangasinan were killed when their tricycle was hit by a pick-up truck, suspected to be driven by a drunk driver. Separately, a fuel tanker driver died after his vehicle crashed into a water refilling station in Quezon.
Read more »