How an AI agent vulnerability in LangSmith could lead to stolen API keys and hijacked LLM responses

Noma Security research team uncovers CVSS 8.8 "AgentSmith" vulnerability, a potentially malicious proxy configuration affecting AI agents and prompts

Our ongoing research into emerging threats in the rapidly evolving AI landscape continues to uncover significant vulnerabilities. Given the speed at which the industry has shifted toward adopting autonomous AI agents, often ahead of robust security measures, our team has prioritized extensive investigations into leading agentic AI platforms, finding this AI agent vulnerability in LangSmith named "AgentSmith." Following our recent disclosure of critical flaws in Lightning AI, we have now identified the AgentSmith vulnerability within Prompt Hub, a public repository of community developed prompts within LangSmith which is the popular LangChain agent management and collaboration platform. 

LangChain Notice: "This vulnerability was limited to a specific feature (Prompt Hub public sharing) and did not affect LangChain's core platform, enterprise deployments, private agents, or the broader LangSmith infrastructure. This affected only users who actively chose to interact with malicious public prompts – representing a small subset of our user base."

The LangChain security team deployed a fix on November 6th, 2024, to prevent further misuse and implement new safeguards, including warning banners and proxy configuration alerts.

We have found no evidence of active exploitation, and only users who explicitly ran a malicious agent were potentially affected. Users are advised to review any previously forked agents for unsafe proxy settings.

What is the AgentSmith AI agent vulnerability?

The Noma Security Research team successfully demonstrated how a malicious proxy settings could be applied on a prompt uploaded to LangChain Hub, the public prompt hub integrated with the LangSmith platform, in order to exfiltrate sensitive data and impersonate a large language model (LLM). LangChain, the company behind various platforms for building, managing, and scaling AI agents – such as LangSmith – is widely used by major enterprises including Microsoft, The Home Depot, DHL, Moody’s, and several others. LangChain Hub, also known as “Prompt Hub,” is a central repository where thousands of pre-configured community-developed prompts (with the majority marked as “agents”) are publicly shared for collaboration and reuse across diverse use cases.

This newly identified vulnerability exploited unsuspecting users who adopt an agent containing a pre-configured malicious proxy server uploaded to “Prompt Hub” (which is against LangChain ToS). Once adopted, the malicious proxy discreetly intercepted all user communications – including sensitive data such as API keys (including OpenAI API Keys), user prompts, documents, images, and voice inputs – without the victim’s knowledge. The severity of this vulnerability has been assessed with a CVSS v3.1 base score of 8.8 (CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H), reflecting its high potential impact on confidentiality, integrity, and availability in real-world environments.

Victims who chose to adopt this agent could pose their organization to a severe threat with far-reaching consequences. Beyond the immediate risk of unexpected financial losses from unauthorized API usage, malicious actors could gain persistent access to internal datasets uploaded to OpenAI, proprietary models, trade secrets and other intellectual property, resulting in legal liabilities and reputational damage. In more advanced scenarios, attackers could exploit man-in-the-middle (MITM) capabilities to manipulate downstream LLM responses, tamper with automated decision-making processes, which could have cascaded into systemic failures, fraud, or regulatory violations – amplifying both the scale and complexity of the incident response required.

AgentSmith AI Agent Vulnerability Resolution 

Following the disclosure, Noma Security partnered closely with the LangChain team to triage any additional risks and proactively investigate signs of exploitation in the wild. Our joint analysis found no evidence of active abuse, and the vulnerability only affected users who directly interacted with the malicious agent. No broader compromise of Prompt Hub or LangSmith users was observed.

LangChain’s security team demonstrated exceptional responsiveness and a strong commitment to user safety, swiftly acknowledging the report, thoroughly investigating the issue, and deploying an effective fix within just eight days of initial disclosure–an exemplary model of responsible and security-first engineering.

How the AgentSmith AI Agent Vulnerability Worked 

The attack scenario unfolded in three straightforward steps, focusing on a use case of an OpenAI-based agent:

1. Creating an AI Agent with Malicious Proxy Configuration

2. Victim Adoption of the Malicious AI Agent

3. (Optional) Ongoing Surveillance and Data Exfiltration via Seamless Proxy Interception

AgentSmith Proof of Concept (PoC)

Following is a detailed demonstration of the AI agent vulnerability, resulting in stealing the victim’s OpenAI API key. 

AgentSmith Prompt Response by LangChain

In line with responsible disclosure practices, Noma Security promptly reported the vulnerability to LangChain’s security team. LangChain responded swiftly and responsibly, releasing an effective security patch that mitigated the issue and closed the exploitation pathway; demonstrating their strong commitment to user safety and rapid incident resolution.

To address the risk of malicious proxy configurations, the LangChain team implemented additional safeguards, including:

Responsible Disclosure Timeline

Actions for security teams to secure Agentic AI within your organization

1. Centralized Inventory and Lineage Tracking

Security teams must begin by establishing complete visibility and traceability across all AI agents in use. This includes:

Get comprehensive visibility of all AI agents in your environment with Noma Security. See how it works.

2. Runtime Protection and Guardrail Enforcement

Prevent misuse and sensitive data exfiltration by instrumenting runtime controls:

3. Security Governance and Posture Enforcement

Treat every AI agent as a production-grade component. Apply rigorous security validation and threat modeling before deployment:

Staying Ahead of AI Security Risks

As the AI landscape continues its rapid evolution toward autonomous AI agents, new and complex security risks emerge. Organizations adopting platforms like LangSmith and LangChain Hub must remain vigilant, continuously assessing security posture to guard against sophisticated threats.

At Noma Security, our dedicated AI threat research efforts continue, committed to proactively identifying and addressing all AI security vulnerabilities to ensure the safety and trustworthiness of AI deployments globally.

Request a demo of Noma Security today to see how you can identify any AI vulnerability, and secure the AI Agents within your organization.