Visual Grounding
also called Referring expression grounding, Visual localization
Turning a description like "the red mug on the left" into a location in the image.
The capability of locating a specific object or region in an image or scene from a natural-language description — turning words into coordinates, boxes or masks. It is the perception step that lets a model point at what it is talking about, rather than just describe the scene.
Grounding is what makes computer use and robotics workable: an agent that cannot locate the button it was told to click, or the handle it was told to grasp, cannot act on its plan. GroundingPI, released November 4, 2026, takes the perception-first route — a 4-billion-parameter model trained only on perception data, with no robot action labels, covering object detection, referring expressions, OCR, document layout, UI elements and spatial localization.
The benchmark lead is the team’s own evaluation and the manipulation results are simulation-based, so treat the headline as a claim, not a settled result. GroundingPI scored 73.68 against GPT-6 Astra’s 71.54 across 34 public perception benchmarks, and the team reports higher task success rates using 50% of the demonstration data where comparison models needed 75% — but the project is described as a scientific prototype, not a product.
The perception-first bet is the interesting part: GroundingPI was trained entirely on perception data, with no robot action labels at all, on the argument that accurately perceiving spatial positions is the prerequisite for embodied intelligence and that action data is expensive where perception data is cheap. The team reports higher task success rates using 50% of the demonstration data where comparison models needed 75% — but that result is from simulation, not a factory floor.
Do not read the benchmark lead as a general capability lead. The 73.68-to-71.54 margin is on a 34-benchmark perception suite the team assembled, weighted equally across 11 capability categories; it says nothing about planning, reasoning or acting, and GPT-6 Astra remains the general model. Xpeng is explicit that the work is a scientific prototype with no commitment to put it in a product.
The click that keeps missing
A computer-use agent is told to click the "Export" button in a dense toolbar and keeps hitting "Share" instead.
The planner chose the right action and the wrong coordinates. A grounding step that resolves "Export" to a bounding box before the click is issued turns a planning failure into a perception check you can log, test and improve separately — and a 4-billion-parameter perception model can run that step without calling the frontier model at all.
Confusing it with grounding an answer in sources. Same word, different job: visual grounding locates things in a scene, while grounding a response ties claims to evidence you can check.
- blogXPENG Researchers and Academic Partners Introduce GroundingPI, an Open-Source Visual Perception Model for Physical AI ↗
The launch announcement: a 4B-parameter perception model scoring 73.68 against GPT-6 Astra’s 71.54 across 34 benchmarks, trained on perception data only. The lead is the team’s own evaluation and the manipulation results are simulation-based.
Computer Use · Screen control
An agent that drives a real UI with clicks, typing and screenshots.
Multimodal
A model that handles more than text — images, audio, video, screen pixels.
Grounding
Tying outputs to evidence the user can check.
Evaluation · Evals
How you know the system is actually working.