Introduction: AI Adoption Brings a New Set of Security Responsibilities

Artificial Intelligence is rapidly becoming part of everyday business operations. From AI-powered customer service and fraud detection to predictive analytics, automated decision-making and generative AI applications, organisations are integrating AI into their products, services and internal processes.

However, increased AI adoption also introduces security risks that traditional cybersecurity controls may not fully address.

Manipulated training data, prompt injection, sensitive information leakage, model theft, algorithmic bias, hallucinations, insecure third-party dependencies and excessive AI autonomy are some of the emerging risks organisations must account for.

Recognising these challenges, the Indian Computer Emergency Response Team (CERT-In) Working Group has developed the Artificial Intelligence (AI) Security Auditing Guidelines v1.0, dated 4 November 2026.

The Guidelines establish a risk-based, lifecycle-oriented framework for assessing the security, governance, privacy, fairness, transparency, resilience and auditability of AI systems.

For organisations developing, procuring, deploying or consuming AI, this is an important opportunity to evaluate whether their existing security and governance practices are equipped to address AI-specific risks.

1. Who Should Pay Attention to These Guidelines?

The Guidelines cover AI systems developed, procured, deployed or operated by organisations in India, as well as systems processing Indian data or providing AI-enabled services to Indian users.

Their intended audience includes:

• BFSI and regulated organisations: Using AI in customer-facing services, fraud detection, credit assessment and other business-critical processes.
• Enterprises and technology companies: Integrating AI into applications, workflows and business operations.
• AI developers and service providers: Building, hosting or offering AI-powered products and platforms.
• Government and public sector organisations: Using AI in citizen services, public administration and critical infrastructure.
• Startups and MSMEs: Developing or adopting AI-enabled solutions.

Importantly, the framework considers AI systems regardless of whether they are developed internally, obtained from third parties, consumed through APIs or cloud services, or built using open-source models.

2. A Risk-Based Approach to AI Security Auditing

One of the key aspects of the Guidelines is the introduction of a four-tier risk classification model.

Instead of applying identical audit requirements to every AI deployment, the framework determines audit depth, frequency, documentation and independence based on the potential impact of an AI system.

The four risk tiers are:

The risk classification considers factors such as:

• Impact severity and operational criticality.
• Level of AI autonomy and decision-making.
• Sensitivity of data being processed.
• Scale of deployment and external exposure.
• Adversarial exposure and system complexity.
• Explainability and transparency.
• Incident response readiness.
• Reversibility and recoverability.

An important consideration is that the same AI technology may fall into different risk tiers depending on its application. A recommendation engine used for content personalisation, for example, may have a very different risk profile from an AI system influencing financial or safety-critical decisions.

What organisations should do: Identify each AI system, document its intended purpose and deployment context, and establish a risk classification based on its actual usage and potential impact.

3. Nine Core Domains Every AI Security Programme Should Address

The Guidelines establish nine core audit domains covering the entire AI lifecycle, from design and procurement to deployment, monitoring and eventual decommissioning.

Organisations should evaluate their existing practices across these domains:

These domains make it clear that AI security is not limited to testing the model or application. It requires coordinated governance across technology, data, security, risk, compliance and business functions.

4. New AI Architectures Require Specialised Security Assessments

Traditional application security testing alone may not adequately address the risks introduced by newer AI architectures.

The Guidelines provide additional audit requirements for Generative AI, Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Agentic AI, Multimodal AI, Multi-Agent AI and Autonomous AI systems.

Some important risk areas include:

Generative AI and LLMs

• Prompt injection and attempts to override system instructions.
• Leakage of confidential information and sensitive context.
• Hallucinations and misleading or unsupported outputs.
• Unsafe output handling and inappropriate content generation.

RAG-based applications

• Unauthorised access to retrieved documents.
• Poisoned or malicious knowledge sources.
• Cross-tenant data leakage.
• Weaknesses in vector databases, embeddings and retrieval controls.

Agentic AI systems

• Excessive permissions and unauthorised tool usage.
• Autonomous execution of code, transactions or external actions.
• Inadequate human approval mechanisms.
• Risks arising from persistent memory and multi-step execution.

Multimodal and Multi-Agent AI

• Malicious instructions embedded in images, audio or documents.
• Cross-modal prompt injection.
• Insecure communication and delegation between agents.
• Cascading failures across interconnected agents.

The Guidelines specifically emphasise least-privilege access, approved tool connections, human approval gates, execution logging, emergency interruption and rollback capabilities for agentic systems.

What organisations should do: Move beyond conventional VAPT and include AI-specific security testing based on the architecture, functionality, exposure and autonomy of each AI deployment.

5. Cybersecurity and Resilience Must Extend to AI Systems

AI systems depend on applications, APIs, infrastructure, datasets, models, cloud services and third-party components. Weaknesses in any of these dependencies can compromise the overall system.

The Guidelines emphasise security and resilience across these interconnected components.

Organisations should pay particular attention to:

• AI-specific threat modelling and attack surface identification.
• Secure access management and privilege controls.
• Data and model integrity.
• Vulnerability management and adversarial testing.
• API, infrastructure and supply-chain security.
• Logging, monitoring and forensic readiness.
• Incident response and recovery mechanisms.
• Business continuity, rollback and safe shutdown capabilities.

Security assessments should also consider how AI systems behave under malicious inputs, unexpected conditions and attempted exploitation.

This makes periodic technical testing and continuous security monitoring important parts of an effective AI assurance programme.

6. AI Security Auditing Is a Continuous Process

The Guidelines outline a structured audit lifecycle rather than treating an audit as a one-time exercise.

The process broadly covers:

1. Engagement initiation: Establish the audit objectives, stakeholders and scope.
2. Risk assessment and scoping: Identify AI systems, determine risk tiers and define applicable audit requirements.
3. Evidence collection: Review policies, architecture documents, model records, configurations, logs and other relevant evidence.
4. Technical assessment: Evaluate security controls and conduct applicable AI-specific and cybersecurity testing.
5. Finding classification: Categorise identified gaps based on their severity and potential impact.
6. Reporting and remediation: Document findings, corrective actions, ownership and escalation requirements.
7. Follow-up verification: Validate remediation and confirm that identified gaps have been addressed.

Audit findings are classified as Critical, High, Moderate or Low/Observation, with corresponding containment, remediation and escalation requirements.

The framework also introduces a five-level AI governance maturity model, allowing organisations to assess their current capabilities and develop a structured improvement roadmap.

The larger message is clear: AI security assurance must continue throughout the system’s lifecycle, including after deployment and following material changes.

7. A Practical Readiness Checklist for Organisations

Organisations can begin preparing by reviewing the following areas:

• AI inventory: Have all AI systems, applications, models, APIs and third-party AI services been identified and documented?
• Risk classification: Has each AI system been assessed and assigned an appropriate risk tier?
• AI governance: Are AI policies, ownership, roles and accountability mechanisms defined?
• Data governance: Are data sources, provenance, permissions, retention and privacy controls documented?
• Model management: Is there a central inventory of models with versioning, validation and performance records?
• AI security testing: Have AI-specific vulnerabilities, adversarial attacks and prompt injection scenarios been assessed?
• Human oversight: Are appropriate approval, intervention and override mechanisms in place?
• Third-party assurance: Are AI vendors, model providers, APIs and external dependencies assessed?
• Monitoring and incident response: Are AI-specific incidents, model drift and security events monitored?
• Audit readiness: Is sufficient evidence available to demonstrate that security and governance controls are implemented and operating effectively?

This checklist is a starting point for internal readiness reviews, not a substitute for the complete audit requirements in the Guidelines.

Conclusion: Build Security into Every Stage of AI Adoption

As organisations accelerate their AI adoption, security and governance must evolve alongside innovation.

The CERT-In AI Security Auditing Guidelines provide a structured foundation for organisations to understand their AI risk exposure, strengthen accountability, implement appropriate controls and establish continuous assurance.

The most effective approach is to start early: build an inventory of AI systems, understand their risk profiles, assess existing controls, identify gaps and establish a roadmap for improvement.

AI adoption should not come at the cost of security, trust or accountability. The time to build AI security into your organisation is now.

How CyRAACS Can Help

At CyRAACS, we bring together cybersecurity expertise, AI security assessment capabilities and governance experience to help organisations strengthen their security posture.

Our services can support organisations through AI security assessments, AI Red Teaming, vulnerability assessments, governance and compliance readiness, and security testing of AI-enabled applications.

Whether you are developing AI-powered products or integrating AI into existing business operations, a structured assessment can help identify weaknesses before they become business risks.

Talk to our experts to understand your AI security readiness and build a more secure, resilient and trustworthy AI environment.