TAPDreamer: Transferable Adversarial Patches for World Action Models

Problem

World action models rely on camera inputs, which can be manipulated to corrupt visual representations. This paper addresses the vulnerability of these models to adversarial attacks, specifically focusing on the lack of defenses against such perturbations. The work is presented as a preprint and has not undergone peer review.

Method

The authors propose TAPDreamer, an attack method that constructs fixed local perturbations capable of transferring across various tasks and action architectures. The method requires only a public encoder and does not necessitate target-policy queries. The core insight is that the interaction between the changes in attention weights induced by the patches and the value vectors leads to a representation shift that extends beyond the immediate footprint of the patch. The optimization goal is to maximize the global L1 distance between the clean and patched encoder representations, utilizing six frames from a single source task for this purpose.

Results

The results demonstrate the effectiveness of TAPDreamer in degrading the performance of world action models significantly. The success rates against various benchmarks are as follows:

• FastWAM: 0.0% success rate compared to 97.7% for the original model.
• LIBERO Tasks: 0.0% success rate compared to 90.8% for the original model.
• RoboTwin Tasks: 0.0% success rate compared to 90.8% for the original model.
• Matched Random Patches (LIBERO): 81.5% success rate.
• Matched Random Patches (RoboTwin): 79.2% success rate.
• DreamWAM Configuration 1: 2.1% success rate compared to the original.
• DreamWAM Configuration 2: 0.8% success rate compared to the original.
• Motus: 10.0% success rate compared to the original. The available text does not report quantitative results for other potential benchmarks or configurations.

Limitations

The authors note that defenses for world action models must be developed to secure shared visual encoders against persistent local perturbations. An obvious limitation not explicitly mentioned is the potential for TAPDreamer to be countered by future advancements in adversarial training or robust encoding techniques.

Why it matters

The implications of this work are significant for the field of adversarial machine learning, particularly in the context of world action models. By demonstrating the vulnerability of these models to transferable adversarial patches, the research highlights the need for robust defenses and encourages further exploration into the resilience of visual encoders against such attacks. This could lead to advancements in the design of more secure AI systems that rely on visual inputs.