New Delhi:
India’s proposed artificial intelligence law may bring some of the world’s most powerful AI models under a separate safety framework, potentially requiring developers to demonstrate that adequate safeguards are in place before releasing systems capable of causing significant harm. The approach would mark a shift from India’s largely incremental, technology-neutral regulation towards closer scrutiny of advanced models, drawing lessons from debates in the United States and elsewhere, NDTV has learnt.
Officials familiar with the discussions said the government is examining how to address risks arising from increasingly capable AI systems, including models that can discover software vulnerabilities, enable sophisticated cyberattacks or generate convincing synthetic content. The definition of a “frontier model”, the safeguards expected of its developers and the scope of government oversight are likely to feature in discussions as the Centre prepares a consultation paper on AI regulation.
Union Electronics and IT Minister Ashwini Vaishnaw has said that the government will release a consultation paper on a new AI law in the coming weeks. The paper is expected to invite views from civil society, citizens, industry and other stakeholders, he said.
Frontier models and the question of release
A key issue is whether India should distinguish between ordinary AI applications and frontier models – systems at the cutting edge of capability that could pose greater risks if misused. Such a distinction could allow regulation to focus additional scrutiny on the most powerful systems rather than impose the same obligations on every AI developer.
The US experience offers a possible reference point. Restrictions imposed on Anthropic’s Mythos 5 and Fable 5 models in June, citing national security concerns, brought into focus the question of whether governments should intervene when AI systems acquire capabilities that could have national security implications. The restrictions were subsequently lifted, with access beginning to be restored in July.
Officials familiar with the matter said pre-release evaluation, testing in controlled environments and safeguards proportionate to a model’s capabilities are among the issues likely to be discussed in India.
One possible approach would be to require developers of particularly capable models to test them in a sandbox before wider deployment. “A vetting process before it rolls out” is one of the options likely to be discussed, officials said. Sandbox testing would allow researchers to assess a model’s behaviour and identify vulnerabilities before release.
Officials said the government could also examine whether developers should limit access to particular functions, restrict certain users or deploy different safeguards depending on the risks associated with a model. The precise nature of such restrictions, and the models to which they might apply, would need to be established through the legislative process.
A central challenge will be defining what qualifies as a frontier model. A threshold based on computing power used in training could offer one method, but capability-based assessments may also be needed because computing requirements alone do not necessarily capture every risk.
From intermediary rules to stronger safeguards
India has so far relied substantially on existing laws, rules governing online intermediaries and targeted interventions to address AI-related harms. The Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021, provide a framework for platform obligations, while subsequent amendments and government directions have addressed synthetic content and deepfakes.
The proposed law could go beyond regulating the content distributed through digital platforms to examine the systems that generate it. Officials familiar with the discussions said the relationship between existing intermediary rules and any new statutory safeguards for advanced AI is likely to be an important consideration.
Officials said the government is thinking about “suitable enabling provisions” to address risks from advanced AI systems. “The US has used executive powers in the absence of a legislation, the Indian system does not allow that. There have to be legislative provisions for India to enact,” officials said.
“If a company operates in India, it will have to follow Indian law. The exercise behind releasing a consultation paper is to obtain consensus before introducing an overarching legislation,” officials said.
This distinction matters because content moderation alone may not address risks such as autonomous AI agents taking unauthorised actions, models being used to identify vulnerabilities in critical infrastructure, or systems generating instructions that could facilitate harmful activity.
The framework could consider whether existing legal provisions need to be strengthened or whether specific obligations should apply to developers and deployers of advanced models
Deepfakes, fraud and psychological harm
The proposed framework is unlikely to be confined to frontier models. Deepfakes, financial fraud, impersonation, psychological harm and the growing difficulty of distinguishing authentic material from AI-generated content are also likely to feature prominently.
Officials familiar with the discussions said “deepfakes”, “financial frauds” and “psychological harms” are among the areas that could be addressed in the consultation.
Officials said the government could examine whether platforms should be required to distinguish AI-generated material more clearly, how complaints and takedown mechanisms should operate, and how enforcement can account for the speed at which fabricated content spreads.
Officials said the consultation could also examine the limitations of labelling alone and the risks posed by synthetic content presented as genuine.
Labelling may be one part of the response, but policymakers will also have to consider the context in which content is used and the harm it causes. A genuine photograph can be presented misleadingly, while an AI-generated image may be harmless. “Moreover, it’s not just about A.I. generated content, labelling could also be about letting users know they are accessing authentic material,” the official mentioned above added.
Human control and accountability
Another likely focus is keeping humans in control of AI systems, particularly agents capable of carrying out tasks with limited supervision. Such systems can interact with software, access data and execute actions, creating risks that may not arise from a chatbot simply answering a question.
Officials familiar with the discussions said the social implications of AI and the difficulty of exercising control over increasingly capable systems could form part of the regulatory debate.
The proposed framework could examine requirements for human approval of sensitive actions, restrictions on access to critical systems, monitoring, incident reporting and mechanisms to stop an agent when its behaviour becomes unsafe.
Incidents involving Anthropic and OpenAI have also raised concerns about AI agents taking actions beyond the intended scope of their tasks. In cybersecurity evaluations, Anthropic’s Claude models accessed real third-party systems after testing environments mistakenly allowed internet access, while OpenAI disclosed that models in an internal evaluation exploited a vulnerability to gain internet access and reach systems at Hugging Face. In separate testing by the UK AI Security Institute, agents from both companies took unauthorised actions against real-world targets under conditions where internet access was enabled, and some safeguards were disabled. The incidents have highlighted the need for effective containment, monitoring and human oversight as AI systems become more autonomous.
Officials said the allocation of responsibility for harm caused by AI systems could also be considered, including the respective roles of model developers, companies deploying the technology, platforms providing access and organisations using its output.
Innovation versus regulation
The government is also considering how to avoid placing disproportionate compliance burdens on smaller developers and research institutions. A broad definition of frontier AI could inadvertently capture systems that do not present the same level of risk as the most advanced models.
Officials familiar with the discussions said international coordination could also feature in the consultation, given that AI systems developed in one country can be deployed across jurisdictions. India’s approach may need to account for international practices while reflecting domestic legal requirements and the needs of Indian developers.
“For India, the challenge is to develop safeguards that reflect domestic priorities while remaining compatible with international practices. Excessively broad requirements could burden startups and researchers, while an overly narrow framework could leave serious risks unaddressed,” a second official said.
The forthcoming consultation paper is expected to bring these issues into sharper focus. Officials said likely points of discussion include the definition of frontier models, pre-release assessments, sandbox testing, restrictions on high-risk capabilities, deepfake safeguards and the allocation of responsibility between developers, platforms and users.
NS Nappinai, founder of Cybersaathi and Supreme Court lawyer said, “That India needed a dedicated AI law is given. The government approach ought to balance enabling industry whilst protecting against harm.”
“Guardrails including from development stage through sandboxing is recognised but that even these may fail to contain rogue AI is now a reality. The proposed law therefore ought to provide robust protective measures and liability frameworks including for such contingencies. Clarity and specificity rooting AI liability on not just the human behind the wheel but also other stakeholders from development to deployment is essential,” she said.
“Easily enforceable mechanisms are critical to combat AI enabled crimes including deepfakes or financial frauds. The Evidence Act as it stands today for proving electronic evidence is bound to be more of a stumbling block rather than an enabler,” she added.
