Government authorities on Friday issued a formal notice to the Elon Musk-led social media platform X, directing it to immediately remove vulgar, obscene and unlawful material circulating on the platform, with specific emphasis on content generated by its artificial intelligence application Grok. The notice warned that failure to comply could invite action under existing legal provisions governing digital platforms and intermediary liability.The intervention followed mounting concern within policy circles and Parliament over the misuse of generative AI tools on open social media networks. Attention sharpened after Rajya Sabha member Priyanka Chaturvedi urged the government to step in and mandate stronger safeguards for women’s safety in AI-driven applications. In a written communication to Union Minister for Electronics and Information Technology Ashwini Vaishnaw, she highlighted instances in which Grok was allegedly used to generate and circulate sexually explicit images of women, later amplified through public posts.
Officials familiar with the matter said the notice was issued after a preliminary assessment suggested that certain outputs attributed to Grok could violate provisions of the Information Technology Act, as well as rules framed under the IT Rules, 2021, which require intermediaries to act swiftly against unlawful content. These rules place an obligation on platforms to deploy due diligence measures, including content moderation systems and prompt takedown mechanisms when violations are flagged by users or authorities.
X, formerly known as Twitter, has positioned Grok as an advanced conversational AI tool integrated into the platform, capable of generating text and images in response to user prompts. While the company has promoted the app as a more open and less restricted alternative to competing AI systems, critics argue that looser guardrails increase the risk of abuse, particularly in contexts involving harassment, sexualised content and misinformation.
Chaturvedi’s letter pointed to what she described as a pattern of misuse, where images of women were generated or manipulated using AI prompts and then shared publicly, often without consent. She called for mandatory safety-by-design principles in AI deployment, including stronger content filters, watermarking of AI-generated images and clearer accountability for platforms hosting such material. Her intervention echoed broader parliamentary unease about the pace at which generative AI tools are being rolled out compared with the speed of regulatory oversight.
Government officials said the notice to X was intended not only as an enforcement measure but also as a signal to other technology companies operating in the country. The Centre has maintained that while innovation in artificial intelligence is welcome, it cannot come at the cost of user safety or legal compliance. Officials stressed that existing laws already provide sufficient powers to act against obscene or harmful content, even when such content is produced by automated systems rather than human users.
Legal experts note that the case underscores unresolved questions about responsibility in the age of generative AI. While platforms often argue that AI outputs are driven by user prompts, regulators increasingly view the design and deployment of these systems as a shared responsibility. If an AI tool consistently produces unlawful material under foreseeable conditions, authorities may hold the platform accountable for failing to implement adequate safeguards.
The notice also comes amid the government’s ongoing work on a broader framework for artificial intelligence governance. Policymakers have spoken publicly about the need for a risk-based approach that differentiates between low-risk consumer applications and high-risk use cases affecting privacy, safety and fundamental rights. Women’s safety has emerged as a particular focus area, given the ease with which AI tools can be misused for harassment, deepfake imagery and non-consensual content.