Autonomous AI attacks unlock new cybersecurity nightmares for enterprises

Even as companies worldwide race to integrate Artificial Intelligence (AI) tools into their processes to reduce costs and drive growth, the technology has brought forth its own set of challenges, especially in cybersecurity.

AI has changed the way how enterprises and individuals face cyberattacks as these have the speed, scale and sophistication much higher than the traditional threats. Today, there are AI attack agents that can attack the computer system without human intervention. This challenge is only going to get bigger, more complex and dangerous.

Inspira Enterprise, a Mumbai-based technology company specialising in cybersecurity services, believes enterprises need to follow the fundamentals to guard themselves against such attacks. Heightened awareness among employees and businesses can mitigate some of these challenges.

Founded in 2008, Inspira Enterprise is among the few Indian technology companies specialising in cybersecurity, and now it has its presence across key markets globally.

“Businesses have to invest in R&D to continuously strengthen their cybersecurity solutions in today’s rapidly evolving threat landscape,” Chetan Jain, Managing Director, Inspira Enterprise, says.

Edited excerpts:

Enterprise Story: What are the new kinds of cybersecurity threats emerging through AI?

Chetan Jain (CJ): AI is leveraged to enhance defenses by cybersecurity teams. Unfortunately, the same technology is also used to create sophisticated attacks benefiting cybercriminals, and transforming the threat landscape rapidly. Cybersecurity threats emerging through AI are many. The Autonomous AI Attacks are AI agents that identify vulnerabilities, plan, and execute an attack, all on their own without human intervention.

Within minutes, a malicious AI can identify outdated software and exploit it across several thousand websites. Prompt Injection tricks AI systems into not heeding their safety rules, leading to the revealing of sensitive information.

AI-Powered Malware tools, such as FraudGPT, enable cyber attackers to create malware that transforms its appearance every time, and becomes a challenge. Deepfake scams are highly realistic AI-generated voices and videos used to impersonate CEOs or government officials.

In smarter phishing, AI writes highly convincing personalised phishing messages where an email referencing a recent purchase is received by a victim rather than a generic bank alert.

In Shadow AI, the AI tools are used by employees within organisations without actual approval. Sensitive customer data can be fed into an unapproved AI tool, leading to compliance and data privacy risks.

AI-Enhanced Hacking can speed up reverse engineering, crack passwords, or identify vulnerabilities at unprecedented speed. In Stealth Attacks, AI can help attackers operate without files by hiding in legitimate tools.

Enterprise Story: Are these AI cyber threats very different from the existing ones?

CJ: AI-powered threats are not entirely new, but they change the speed, scale, and sophistication of traditional threats. Cyberthreats such as malware, social engineering, and phishing have existed for decades, and AI technology enables anyone to deploy these threats with ease.

Cybercriminals can generate them rapidly, while cybersecurity teams struggle to detect them in time. Earlier, a phishing campaign could have taken several days to get designed and tested, with grammatical mistakes while formatting often exposing them. Today, by leveraging AI technology, threat actors can generate the same campaign in minutes, personalised for each recipient (victim), and make it look like a legitimate message.

There are some of the evolved threats driven by AI capabilities, such as prompt Injection attacks and autonomous AI attacks.

Enterprise Story: What do you mean by AI governance? Is it a framework that needs to be set up by an enterprise or broad industry guidelines that need to be followed?

CJ: AI governance refers to a set of principles, processes, standards, policies, practices, and tools that support the management of AI usage within organisations in the most responsible way.

This framework serves as a guideline to reduce potential risks of AI by developing, deploying, and monitoring the technology, ensuring they align with the original business goals as well as meet regulatory compliance.

Regulatory and industry bodies such as the EU Act, NIST AI Risk Management Framework, and ISO/IEC 42001 introduced these guidelines for the responsible use of AI.

Furthermore, every organisation has to translate these industry guidelines into internal governance structures customized to its context. This can include defining AI usage policies, creating model risk assessment checklists, setting up bias detection and explainability reviews, and ensuring incident response for AI-related failures.

Enterprise Story: How can enterprises protect themselves from AI cybersecurity threats?

CJ: Organisations must have a good understanding of the various tactics attackers use to strengthen their skills that lead to data breaches. In addition to traditional cybersecurity controls, organisations have to implement AI-specific security measures. Super-strong passwords are a must, and they should not be reused across other documents. Software should be updated and vulnerabilities patched.

Multi-factor authentication provides an additional layer of protection against unauthorized access. AI-driven security solutions, such as AI-powered endpoint security and AI-powered security information and event management (SIEM), among others, enhance the effectiveness of cybersecurity tools and have to be leveraged extensively.

AI-monitoring to identify unusual behavior, model hardening to safeguard AI systems from attacks such as data poisoning, AI red teaming, where teams simulate adversarial attacks on AI systems to spot vulnerabilities enhance protection from AI cybersecurity threats.

GenAI can help create realistic simulations of cyberattacks, enabling security teams to test their cybersecurity defenses and incident response measures. Employee training and awareness have to be enhanced to establish a vigilant workforce. Businesses have to invest in R&D to continuously strengthen their cybersecurity solutions in today’s rapidly evolving threat landscape.

Enterprise Story: What policy initiatives are needed, especially from the government, to combat these AI threats?

CJ: The government’s role in policies and regulations in AI is to ensure AI technologies are built and leveraged responsibly and ethically. Organisations should be required to disclose the use of AI, especially in decision-making processes that have an impact on individuals.

Transparency and accountability should be mandatory in the explainability of AI outputs, traceability of training data, and clear lines of responsibility in the case of AI failures. As AI advances further, transparency and accountability should continue to remain top priorities for policymakers.

Clear AI security standards should be established, where minimum security requirements for AI systems are well defined.

To ensure all AI policies are well-rounded and address different concerns, it is important to engage with different stakeholders, the public, corporates, as well as academic institutions. The government should establish AI security centers that can collaborate with the industry to share best practices and threat intelligence, and rapid-response capabilities for AI-driven cyber incidents.