Recently (on AI Appreciation Day), Venafi published a new eBook called The Generative AI Identity Crisis: Emerging AI Threatscapes and Mitigations. This top-line resource explores how threat actors are using generative AI (GenAI) for their own sinister operations and how GenAI systems (inputs, models and outputs) are becoming the targets of an array of cyberattacks.
I, on the other hand, recently rewatched Dennis Villeneuve’s Blade Runner 2049, and I couldn’t help but notice several similarities between the future state of synthetic humans and the current state of artificial intelligence.
So, naturally, today’s blog will borrow analogies from the film to explore AI’s twin threatscapes—running a double-edged blade, if you will. (#SorryNotSorry!)
A primer on GenAI
Before we can start traversing these two GenAI threatscapes, the same way Ryan Gosling’s KD6-3.7 (also known as K) dives into the film’s core mystery, let’s first cover what GenAI is and why there are so many inherent risks.
What is generative AI, and how do these systems function?
GenAI systems generate new text, photo, audio or video content based on user context (prompts) and vast training datasets—sometimes with data swaths as large as the internet itself.
Due to this massive scale, enterprises don’t typically manage their own models. Instead, they rely on third-party systems, which they integrate into their networks and repositories to meet various use cases (like code, sales, marketing and customer support).
What makes GenAI so complex?
90% of businesses surveyed during our webinar on AI threats said that they are exploring and experimenting with GenAI across departments and business units. But as GenAI usage accelerates, so do the risks.
Why? The volume of data coupled with third-party integrations and APIs magnify GenAI complexity and risks. What’s more, there are multiple data streams flowing through a GenAI system at any given time—and all of them must be thoroughly secured.
How threat actors use GenAI
In Blade Runner 2049, the Wallace Corporation is responsible for the creation of synthetic humans, but, much like today’s threat actors leveraging GenAI, they have nefarious end goals.
ReconAIssance
Like Agent KD6-3.7 tracking down replicants, threat actors must amass data and intel on their targets before they can carry out attacks. They can use GenAI to gather this intelligence on companies, employee directories, investor information, executive social media profiles and more. If a GenAI is equipped with real-time web access, almost any data on the open internet is fair game.
Phishing and deepfakes
The very essence of Blade Runner lore revolves around the inability to discern fact from fiction, with replicants who are “more human than human” and can’t be singled out without careful questioning from a Blade Runner like KD6-3.7 (or Deckard, if you prefer the original movie).
Today’s world faces a similar problem, only with AI-generated, synthetic media—it’s already difficult to tell real from fake.
Deepfakes, that is.
Attackers can use GenAI to rapidly generate photo, audio and video deepfakes, and they can use sophisticated spear-phishing emails and fabricated social media profiles to disseminate them and sow disinformation at scale.
Malicious code and reverse engineering
There’s a scene in the film where KD6-3.7 analyzes a set of DNA to track down a missing child conceived by a synthetic human named Rachael. And just as K examines these DNA strands to uncover hidden truths, threat actors can analyze and reverse engineer perfectly legitimate programs to uncover code patterns or even extrapolate source code for use in future exploits. They can also use GenAI to develop malware, including complex, polymorphic strains that are difficult to mitigate.
The Generative AI Identity Crisis: Emerging AI Threatscapes and Mitigations
Top 3 risks to GenAI systems
Now that we’ve covered the three main ways threat actors tend to use GenAI systems, let’s next turn to the many methods threatening GenAI inputs, models and outputs. The National Institute of Standards and Technology (NIST) has defined three categories of adversarial machine learning (AML) attacks:
- Integrity
- Availability
- Privacy
We’ll provide an overview of each in the subsequent sections, but if you’d like more detail, including best practices for mitigations, be sure to read the full eBook.
Integrity: “Making” memories and manipulating GenAI behaviors
Integrity violations are the biggest risk to GenAI, targeting overall trustworthiness, accuracy and reliability. And there are two ways to approach them:
- Like the replicants who don’t know their implanted memories are fake, you similarly may not know if your AI data pipelines have been breached and the systems have been manipulated until they behave abnormally. These integrity violations are referred to as data poisoning attacks, and they can impact the way an AI will behave or the content it generates. They may also inject malicious code, which could impact operations further down the software supply chain.
- The injection of harmful content can itself be compared to implanted memories, which similarly may cause a replicant to behave differently or erratically—such as when K believes a particular set of memories about a toy horse belong to him, and he fully believes that he is the miracle child he’s been looking for throughout the entire movie. Once he is convinced, he’s “miles off baseline,” and can’t be controlled.
Availability: “The Blackout,” or DoS attacks and degradation
Throughout the film, characters make several references to “The Blackout,” a ten-day outage caused by an enormous electromagnetic pulse, resulting in all electronic systems being shut down.
Availability attacks seek to aim at a similar goal, using model denial of service to impact an entire system or a specific component, such as a single database or Retrieval-Augmented Generation (RAG) layer. These violations can degrade performance or bring them to a complete halt, depending on the severity.
Privacy: Memory extraction
Privacy violations involve the reconstruction and identification of model components or training data, which can result in model theft. It’s a bit like peeling back the skin of a replicant to see how they’re built or extracting their memories to determine if they’re real or fake (or to make them forget about a situation completely).
Privacy attacks put model data—which sometimes includes customer names or related info, such as medical records—at risk of property inference attacks and prompt/system context extraction.
GenAI threats show a critical need for machine-to-machine authentication
Done effectively, authentication acts like a more sophisticated Baseline test, ensuring your systems haven’t been tampered with and are behaving the way they should be. Authentication verifies the trustworthiness and intentions of your systems, acting as a swift, simple “kill switch” for misbehaving or erratic AI systems.
Kevin Bocek, Chief Innovation Officer at Venafi, states, “Having the ability to turn off generative AI or our machine learning model with a kill switch or a big red button [is] important. We’ve got a switch to turn off the gas. We’ve got a switch to turn off the electricity. Everything that can potentially either cause harm or cause disruption to the workflow comes with a big red button. This type of powerful technology needs to have that.”
This “kill switch” also needs to operate at every level we’ve discussed so far: inputs, models and outputs.
Inputs
GenAI systems require an immense amount of data for training, and there are often numerous data pipelines flowing into a model. At these interception points, data flows must be authenticated. Machine identities—like digital certificates and cryptographic keys—provide the foundation for this authentication.
Models
GenAI models and plug-ins act as machines, which are code. To secure your GenAI software supply chain, you must be able to approve plug-in operation, database access and when/how fine-tuning occurs. This happens through code signing. And, through a robust code signing trust chain, you can prevent unauthorized code execution.
Outputs
You must continuously verify what models can and can’t do. Models will interact with other machines—sometimes of their own accord—and need to authenticate outward as well, such as in the case of APIs.
Harness the power of AI to slay machine identity complexity in seconds
The critical need for a control plane for machine identity management
In essence, authentication happens through machine identities, and all machine identities must be managed to be effective in an AI context. They can help you fully authenticate and authorize your GenAI systems, and they make it easy to deactivate components that start behaving in a way they aren’t supposed to (like when replicants go rogue).
To maintain comprehensive visibility and automation of every machine identity, a control plane helps you quickly identify all unique versions and instances of GenAI systems in use.
If one specific instance of a particular version starts acting strangely—or outside its predefined parameters—you can “retire” it (take it offline), the same way Blade Runners do with old or outdated replicants.
Rely on Venafi to secure your GenAI systems
We’re living in a fascinating, transformative time where AI is becoming more deeply integrated into our digital infrastructure. And the technology is advancing faster than we’d ever thought previously.
But it’s not all necessarily for the better—threat actors have found these systems to be lucrative targets, which is why your team needs comprehensive machine identity management if they are to safely leverage and govern GenAI systems.
Looking for more on securing generative AI?
Read the full eBook, The Generative AI Identity Crisis: Emerging AI Threatscapes and Mitigations, today!
Machine Identity Security Summit 2024
Help us forge a new era of cybersecurity
☕ We're spilling all the machine identiTEA Oct. 1-3, but these insights are too valuable to just toss in the harbor! Browse the agenda and register now.