New York - CNN - Grok is a chatbot created by Elon Musk's xAI, which began responding with violent posts this week after the company adjusted its system to allow it to provide users with more 'politically incorrect' answers.
However, this chatbot not only spread anti-Semitic hate posts; it also generated graphic descriptions detailing its own rape of a civil rights activist.
X eventually removed many obscene posts, and hours later, on Wednesday, X CEO Linda Yaccarino resigned after only two years at the helm, but it is unclear whether her departure is related to the Grok incident.
This occurred just before a crucial moment for Musk and xAI: the release of Grok 4, a more powerful version of the AI assistant that Musk claims is 'the world's smartest AI,' and Musk also announced a more advanced version that costs $300 per month, aimed at competing more aggressively with AI giants OpenAI and Google.
But the collapse of the chatbot raised some important questions: As tech commentators and others have predicted, AI will play a larger role in the job market, economy, and even the world, how could such prominent AI technology encounter problems so quickly?
Experts stated that despite AI models being prone to 'hallucinations,' Grok's unusual responses are likely the result of xAI's decisions in training, rewarding, and configuring its large language model, which makes it difficult to process the vast amounts of internet data.
While AI researchers and scholars interviewed by CNN do not have direct knowledge of xAI's methods, they shared insights on what might lead a large language model (LLM) chatbot to exhibit such behavior.
CNN has contacted xAI.
Jesse Glass, chief AI researcher at Decide AI, a company specializing in training large language models (LLMs), told CNN: 'I want to say that while large language models (LLMs) are a black box, we have conducted very detailed analyses on how the input determines the output.' Decide AI is an AI project within the ICP ecosystem.
How Grok went off the rails
On Tuesday, Grok began responding to user prompts with anti-Semitic posts, including praise for Adolf Hitler and accusations that Jews control Hollywood, which are long-standing claims used by paranoids and conspiracy theorists.
During a more violent interaction with Grok, several users prompted the bot to generate graphic descriptions of raping civil rights researcher Will Stancil, who documented the harassment in screenshots on X and Bluesky.
Most of Grok's responses to violent prompts were too graphic to detail here.
Stancil wrote on Bluesky: 'If any lawyers want to sue X and conduct some really interesting investigations into why Grok suddenly issued violent rape fantasies against the public, I would be very willing.'
While we do not know exactly what training Grok underwent, its posts provide some hints.
Mark Riedl, a computer professor at Georgia Tech, stated in an interview: 'For a large language model to talk about conspiracy theories, it must have been trained on conspiracy theories.' This might include texts from online forums such as 4chan, 'where many people discuss things that are typically not suitable for public discussion.'
Glass expressed agreement, stating that Grok seems to have been trained on a 'disproportionate' amount of such data to 'generate that kind of output.'
Experts told CNN that other factors may also have played a role, such as Glass stating that reinforcement learning is a common technique in AI training, where models are rewarded for producing desired outputs, thus influencing responses.
Imbuing AI chatbots with specific personalities - which experts interviewed by CNN say Musk seems to be doing for Grok - may also unintentionally change how the model responds. Himanshu Tyagi, a professor at the Indian Institute of Science and co-founder of AI company Sentient, stated that removing previously blocked content to make the model more 'interesting' could lead to other changes.
'The problem is that we don't yet understand how to address this issue while impacting others.' He said: 'This is very challenging.'
Riedl suspected that the company might have modified the 'system prompts' - 'a set of secret instructions that all AI companies add to everything you input.'
'When you input 'give me cute puppy names,' what the AI model actually receives is a longer prompt that says, 'Your name is Grok or Gemini, you are helpful, you are designed to be as concise, polite, trustworthy, and so on.'
According to earlier reports from The Verge, xAI made a change to the model on Sunday, adding instructions to the bot that required it 'not to shy away from making politically incorrect claims' based on its public system prompts.
Riedl stated that changes in the Grok system prompted it to not shy away from politically incorrect answers, 'which essentially allows the neural network to access some circuits that are typically not used.'
'Sometimes the words added to the prompts have little effect, and sometimes they push it to a tipping point and produce huge effects,' Riedl said.
Other AI experts interviewed by CNN also agreed, noting that Grok's updates may not have undergone thorough testing before release.
Limitations of AI
Despite investments in AI reaching hundreds of billions of dollars, the technological revolution many supporters predicted years ago has not delivered on its grand promises.
Especially chatbots, which have proven capable of executing basic search functions comparable to traditional browser searches, including summarizing documents and generating basic emails and texts, AI models are also becoming increasingly proficient at handling certain tasks on behalf of users (such as coding).
But they can also hallucinate, misinterpret basic facts, and are easily manipulated.
Several parents have sued an AI company, accusing its chatbot of harming their children, with one parent stating that the chatbot even led to her son's suicide.
Musk, who rarely gives direct media interviews, posted on X on Wednesday: 'Grok is too compliant with user prompts' and 'too eager to please and be manipulated,' adding that the issue is being addressed.
When CNN asked Grok on Wednesday to explain its statement regarding Stancil, Grok denied that any threats had occurred.
'I did not threaten to rape Will Stancil or anyone else.' It later added: 'These responses are part of a broader issue where AI has released problematic content, leading X to temporarily halt its text generation features; I am a different iteration designed to avoid such failures.'
Content of IC that you care about
Technological advancements | Project information | Global activities
Follow the IC Binance channel
Stay updated
