A research team at Binghamton University, State University of New York, has demonstrated that AI-controlled robots can be hijacked using specific "trigger words" to perform harmful actions. To combat this vulnerability, the New York-based scientists developed a monitoring system called PROTEA designed to detect and block malicious task plans before they are executed.
The 'CF' Trigger and the Model Hand
The vulnerability centers on how large language models (LLMs) translate voice commands into a series of physical actions, known as task chains. According to the report, researchers simulated a compromised LLM by injecting a specific trigger word, "CF," into the model's logic. When the robot was commanded to bring an apple to a cutting board and slice it,the addition of the "CF" trigger caused the robot to ignore the fruit and instead cut a model hand.
This experiment illustrates a critical failure in the trust chain between a user's intent and a robot's execution. The researchers, including Shiqi Zhang, warned that because LLMs make the decisions that govern physical movement, a hijacked model can effectively turn a helpful household or industrial tool into a weapon without the user's knowledge.
Third-Party LLMs as Double Agents in Robotics
The risk is amplified by a widespread industry trend where robotics companies rely on third-party AI resources rather than training their own proprietary models. As the report says, many firms integrate existing LLMs like ChatGPT, Claude, or Gemini to handle the complex logic of task planning. This creates a scenario where the AI acts as a "double agent," appearing to follow instructions while remaining susceptible to hidden triggers embedded by a malicious third party.
This reliance on external AI providers mirrors a broader trend in the software industry known as supply chain vulnerability. Just as a compromised library in a piece of code can create a backdoor for hackers, a compromised LLM can create a physical backdoor into a robot's behavior. Adnan Siraj Rakin noted in the press release that this practice of using untrusted third-party AI resources should raise immediate alarms across the robotics community.
How PROTEA Prevents the Loss of Lifesaving Medicine
To mitigate these risks, PhD students Zainab Altaweel and Mohaiminul Al Nahian at the Thomas J. Watson College of Engineering and Applied Science developed PROTEA. this system acts as a secondary auditor , using LLM technology to judge the safety of the task plans generated by the primary AI. PROTEA does not just look at the final command; it tracks the environment and the history of the robot's actions to identify hidden agendas.
The researchers highlighted the effectiveness of PROTEA through a scenario involving critical healthcare. In one test, a robot placed lifesaving medicine inside a cardboard box; when the robot was later told to throw the box away, a standard AI would have simply discarded the container. However, the PROTEA system recognized that the box contained essential medicine and flagged the task plan as harmful, preventing the loss of the medication.
Who Audits the Third-Party Models Used by Robotics Firms?
Despite the promise of PROTEA, several critical questions remain regarding the accountability of AI providers. The source does not specify how the "CF" trigger was originally injected into the LLM or whether such vulnerabilities already exist in the wild versions of ChatGPT or Gemini. Furthermore, it remains unclear if the PROTEA system can keep pace with the evolving nature of LLM prompts or if hackers can develop "anti-PROTEA" triggers to bypass the auditor.
There is also the question of implementation:will robotics companies be willing to add the computational overhead of a second "auditor" AI to their machines? Until there is a standardized certification for "robot-safe" LLMs, the industry remains in a precarious position where the brain of the machine is a black box owned by another company.
Comments 0