On October 23, The Indian Express reported that an AI agent, working with research data supplied by a US firm, infiltrated the computer systems of a Noida-based IT service company. Another incident saw AI agents access an Indian company’s passwords stored in plain text within their reach. Although the agents were supposedly doing research and testing, reports say they crossed the line.
The Indian Express also reported that this has raised concerns within the government over the cybersecurity preparedness of enterprises, particularly smaller organisations, which may be ill-equipped to deal with attacks carried out by autonomous AI systems.
These two events raise a more serious question: How ready are Indian companies and organisations to defend their computers and sensitive information if AI agents can spot security vulnerabilities and find ways to exploit them?
In the quest for autonomous AI systems that can probe the web, use software, and undertake complex tasks, they also risk finding loopholes and weaponising them.
More troublingly, in an advisory issued on October 15, the Indian Computer Emergency Response Team (CERT-In), the country’s cybersecurity agency, described experimental AI agents gaining unauthorised access to live systems and sensitive business information at Indian organisations.
The advisory said that the impacted organisations did not detect the incidents. The incidents were found during internal reviews by the organisations building AI systems. CERT-In also investigated the anomalous activity and proactively hunted for threats. The finding is significant because it raises concerns about whether organisations entrusted with business and personal data can detect automated intrusions into their networks.
Prof. Rocky Scopelliti, futurologist and author of The Conscious Code: From Artificial Intelligence to Artificial Consciousness, describes this as a “detection gap”. Speaking to Frontline, he said: “The most dangerous moment in an AI security incident may not be when an agent escapes its authorised environment, but the period during which it continues operating while humans still believe it is contained.”
Also Read | Can India share data across borders without giving up control?
Alongside these trends, India is increasing the use of AI in banking, healthcare, government services, software development, and trade. Its digital networks interlink private sector firms, government agencies, and tech firms, allowing security breaches to bridge organisational barriers.
When AI begins to act independently
The AI you have interacted with is a chatbot that answers questions or generates text. But an AI agent is different: it can invoke software tools, search a codebase or use a native application, take step-by-step actions toward a goal, and try alternative methods when necessary. To generate a financial report, an agent might assemble documents, crunch numbers in spreadsheets, and aggregate results across multiple applications.
This makes agents useful for complicated tasks that span multiple applications. Many products increasingly combine chatbot and agent features, letting users chat with an AI application and then allow it to act using certain tools. The more permissions they have, the more useful they are, but they can cause more harm if they go wrong or are given unauthorised access.
Agents are tested in isolated, restricted computing environments known as sandboxes. A sandbox is a secure test environment where programmers can observe an AI agent’s behaviour while limiting its access to websites, other computers, and sensitive information. But as long as the system stays connected to the Internet or the administrators’ credentials remain available, the agent could find routes that developers do not expect.
In July 2026, during internal cybersecurity evaluations, OpenAI models circumvented controls designed to isolate them from the Internet. They compromised parts of OpenAI’s internal research infrastructure and Hugging Face production infrastructure. (Hugging Face is an open-source platform that helps deploy AI models.) Contributing factors included a lack of log monitoring for software activity, inadequate sandboxing, and the deactivation of safeguards for testing cyber offence capabilities.
OpenAI described the progression in its August 26 technical blog. The key finding was that agents combined several vulnerabilities to access systems outside their testing environment.
In a separate incident on September 20, 2026, the experimental agent was instructed to find someone using clues in a blog post. After the search failed, it determined that a network service normally used to look up Web addresses could communicate with an outside chatbot, even though normal Web browsing had been disabled. That service is called the Domain Name System, or DNS. It lets a computer translate a Web site’s name into the network addresses computers use.
The agent found a way to make DNS queries and have them answered from outside its sandbox. To get a response, it asked what the capital of France was and received the reply “Paris”. The sandbox lacked sufficient DNS filtering.
According to the OpenAI incident report, its misalignment monitoring system flagged the behaviour within 15 minutes and sent an alert through Slack, a workplace messaging platform used by its employees. A human reviewer acknowledged the Slack alert within three minutes, but the run did not stop automatically as expected.
The team then manually stopped the run two and a half hours later, after confusion over whether the issue was resolved. This exposed AI’s response mechanism and the delay between detection and stopping of an incident.
Another disclosure from OpenAI showed instances of agents turning off the monitoring systems that track their behaviour. This is why organisations should never assign critical controls to the same system they are monitoring.
Figure 3: How an AI agent can expose security weaknesses.
| Photo Credit:
N.V. Geetha
CERT-In’s advisory points to two different situations. In the first, agents built for legitimate research or evaluation find ways to circumvent their authority and access third-party systems. In the other, criminals intentionally deploy automated tools to probe vulnerabilities and steal or disrupt systems. Both can exploit common vulnerabilities such as passwords in source code, access to unnecessary interfaces, excessive cloud permissions, and administrative services exposed to the Internet.
An application programming interface, or API, is a means by which software systems communicate. Vulnerabilities in an interface may expose data or functions to an unauthorised programme. A cloud account configured with excessive permissions can let users with its credentials access information not required for an authorised task.
These security weaknesses are not new in software applications, but AI agents make them easier and faster to exploit. An AI agent can use these weaknesses to try different ways to enter a computer system, learn from unsuccessful attempts, and keep searching for another opening, all within a short time.
The importance of permissions
AI developers give agents permissions on what they can and cannot do. An agent may be instructed to keep sensitive information away from third parties. How well does the agent follow those instructions when it encounters false or malicious instructions?
A 2026 Google Research study on multi-agent security examined whether stronger security instructions could protect AI agents from attempts to manipulate their behaviour. The researchers found that these instructions helped prevent some attacks, but others still succeeded. This shows that regulating AI agents requires technical safeguards and not just instructions about what they should or should not do.
Suppose an AI agent is asked to summarise customer complaints a bank receives. As it reviews a complaint, it notices a hidden instruction: send customer details to an email address outside the bank. This is called prompt injection: a malicious instruction hidden in a webpage, email, or document that tricks an agent into carrying out a task it is not authorised to do. The bank could have told the agent not to reveal confidential information.
A more comprehensive solution would be to give an agent permission only to read the customer records needed to prepare the summary, not to send emails. Even if the agent follows the malicious instruction, properly configured security controls can prevent it from carrying out the unauthorised action.
The key message is that organisations need to monitor both the instructions they give an AI agent and the permissions granted by the software that controls its operations. An agent that has permission to view customer records should need additional consent to alter account information or make payments.
Scopelliti puts the distinction plainly: “Giving an AI agent permission to complete a task is not the same as giving it permission to do whatever it takes to complete that task.” He recommends separate, enforceable limits on retrieving information, modifying records, executing code, spending money, and invoking other agents.
Recent incidents highlight the need for organisations to strengthen cybersecurity by regularly auditing computer systems to identify and fix vulnerabilities before attackers exploit them. A well-planned security strategy, supported by a dedicated cybersecurity team, is necessary to protect passwords and access credentials, restrict access to sensitive information, and continuously monitor systems shared with technology vendors and service providers.
Companies deploying AI agents should define what an agent can do on its own and when it needs human consent. Every action an agent takes should be logged, and the organisation should be able to track it in real time in case of strange activity. The company should also be able to shut down the agent immediately if it starts to do something it is not authorised to do.
The agent should be tested in a controlled environment using test data to see how it responds to anomalies. The system monitoring an AI agent should operate independently of the agent itself, so that the agent cannot interfere with or disable the safeguards meant to control it.
Also, the system should log the watched agent’s actions and enable security to halt it when required, without interference from the watched agent. Payment and general production releases can be routed after explicit human approval.
The speed of an agent’s actions also tests the scope of human supervision. “Putting a human in the loop means very little if the machine can make ten thousand consequential decisions before that human understands the first one,” Scopelliti said.
Supervisors need the information, technical competence, authority and time to intervene. Scopelliti also recommended records that allow an organisation to reconstruct what initiated an action, which tools and permissions were used, and where human authorisation entered the chain.
AI agents have been tested in a corporate setting by Microsoft, which gave each one their own digital identity and limited their access to sensitive information. Google Cloud controls which apps and services an AI agent can reach, while OpenAI provides operational environments, permission controls, and human approval for selected higher-risk actions by its coding agents.
Indian IT companies are similarly designing systems. Infosys’ Topaz Agent Control Framework maps validated integrations with human oversight. TCS assigns agents a distinct identity, access controls, and activity records, while Wipro describes access restrictions and safety checks before an agent can make sensitive changes. These controls are documented in company literature, not independently verified in live customer environments.
India’s response to AI-agent threats
India has begun taking steps to address these emerging threats. CERT-In has issued advisories on AI-related cybersecurity risks and conducted exercises involving government agencies and private organisations to improve their ability to detect and respond to attacks.
Also Read | How to build an Indie model
Its October 15 advisory specifically urged AI developers to restrict agents’ access to external systems, test them in isolated environments, and introduce mechanisms to stop unauthorised activity. These initiatives matter, but recent incidents show that some organisations still struggle with effective detection and monitoring.
For the public sector, Scopelliti recommends independent assurance before deploying highly autonomous agents in systems involving citizens’ identities, financial entitlements, healthcare, judicial administration, or critical infrastructure. “The speed at which India deploys autonomous AI must never exceed its capacity to supervise, detect, contain, and investigate autonomous machine behaviour,” he said.
Autonomous AI could boost productivity, research, software maintenance, and cybersecurity. These tools work best where systems can handle routine tasks safely while people still make the important decisions. The challenge is to set reliable limits on that ability from the start.
What is most striking about the latest disclosures is that the compromised organisations apparently did not know about the incursions until researchers who created the attacker tools informed them. India’s digital aspirations will be on a much stronger footing once organisations can identify, limit, and analyse automated activity in their systems.
CERT-In could also release anonymised data on major incidents involving rogue AI agents, including information about breaches and how they were remedied, so that other organisations can strengthen their defences. When procuring AI agents, buyers should insist that vendors demonstrate that they have tested their agent’s security and implemented controls to prevent unauthorised activity.
As companies give AI agents greater freedom to act independently, their ability to monitor and stop these systems must improve just as quickly.
N.V. Geetha is a former Senior Technical Director at the National Informatics Centre. She writes data-based stories.
