Vision-Language-Action (VLA) models have shown strong generalization to generic instructions, but they struggle with personalized commands such as "bring my cup," where the robot must act on a specific instance among visually similar objects. Researchers from the field of robotics present a new approach called Visual Attentive Prompting (VAP) that addresses this challenge without any additional training.
The Personalization Problem in Robotics
According to the research, VLA models fail to distinguish between a user's personal object and other identical-looking items when given instance-specific instructions. This limitation prevents robots from being deployed in environments where they must handle personalized tasks, such as retrieving a particular cup from a shelf of identical cups.
Visual Attentive Prompting (VAP) Approach
VAP is a simple yet effective training-free perceptual adapter that equips frozen VLA models with top-down selective attention. The method works in three steps:
- It treats reference images of the user's object as a non-parametric visual memory.
- It grounds the personal object in the current scene through open-vocabulary detection and embedding-based matching.
- It injects this grounding as a visual prompt by highlighting the object and rewriting the instruction.
Because VAP requires no retraining, it can be applied directly to existing VLA models, making it practical for real-world deployment where retraining is costly or impossible.
Benchmarks and Experimental Results
To evaluate personalized manipulation, the team constructed two simulation benchmarks—Personalized-SIMPLER and Personalized-VLABench—as well as a real-world tabletop benchmark. These benchmarks test multiple robots across various tasks involving personal objects.
Experiments show that VAP consistently outperforms generic policies and token-learning baselines in both success rate and correct-object manipulation. The method bridges the gap between semantic understanding and instance-level control, delivering reliable performance without any fine-tuning.
Implications for Enterprise Automation
For enterprise technology leaders, this research has direct relevance to warehouse and logistics automation, where robots must pick specific items from bins containing similar-looking products. The training-free nature of VAP means it can be integrated into existing robotics stacks without disrupting operations. While the benchmarks are tabletop-scale, the approach is designed to scale to more complex environments. As VLA models become standard in industrial robots, methods like VAP will be essential for enabling personalized, instance-aware manipulation without retraining costs.