NVIDIA LocateAnything-3B

1. Model Overview

2. Key Schematic

Capability overview vs. PBD

! [[17-TEMPORARY ATTACHMENT/LocateAnything-3B/hf-teaser.jpg]]

This picture is most suitable for pre-sales opening: the upper part shows LocateAnything covering multi-object positioning, click positioning, layout positioning, GUI grounding, detection and OCR; The second half shows the difference between it and traditional coordinate token decoding. Traditional methods require token-by-token generation of coordinates, and PBD generates a box as an atomic unit in parallel.

Model Architecture and Structured Block Output

! [[17-TEMPORARY ATTACHMENT/LocateAnything-3B/nvidia-method-architecture.png]]

Architecture diagram description: input is image text query; The visual encoder is Moon-ViT, the language side is Qwen2.5, and the middle is connected by two-layer MLP projector. The output is not ordinary natural language, but a structured positioning sequence containing semantic block, box block, negative block and end block.

Parallel Box Decoding method diagram

! [[17-TEMPORARY ATTACHMENT/LocateAnything-3B/nvidia-method-pbd.png]]

This diagram is suitable for explaining the core technological innovation: treating a set of box coordinates as coupled geometric structures rather than independent tokens. For localization tasks, this helps reduce delays and geometric inconsistencies associated with coordinate order generation.

Data Overwrite

! [[17-TEMPORARY ATTACHMENT/LocateAnything-3B/nvidia-data.png]]

The official data shows that the training data covers natural scenes, robots, driving, GUI, documents, OCR, open world detection and other domains. The model card mentions that the training set contains 12M unique images, about 140M natural language queries, and 785M bounding boxes.

GUI / OCR / Detection benchmark Chart

! [[17-TEMPORARY ATTACHMENT/LocateAnything-3B/hf-sspro-gui.png]]

! [[17-TEMPORARY ATTACHMENT/LocateAnything-3B/hf-layout-ocr.png]]

! [[17-TEMPORARY ATTACHMENT/LocateAnything-3B/hf-coco-lvis.png]]

These diagrams correspond to GUI grounding, layout/OCR, general-purpose target detection, etc., and are suitable for placement in customer materials to illustrate that it is not a single detector, but a general-purpose grounding model for multiple visual positioning tasks.

3 What does it do?

The core task of the LocateAnything-3B can be summarized as: give it a picture and a natural language instruction, and it outputs the position of the target in the image, usually bounding box or point.

4. Core Technology Highlights

4.1 Parallel Box Decoding

Traditional VLM grounding usually serialize 2D box coordinates into multiple 1D tokens, and then generate token-by-token self-regression. There are two problems: first, the coordinates are naturally coupled geometric structures, but token-by-token generation will weaken geometric consistency; Second, strict serial generation will become the bottleneck of reasoning.

LocateAnything PBDs decode boxes or points in parallel as atomic geometric units. According to the official paper abstract, PBD simultaneously improves decoding throughput and positioning accuracy, and promotes speed-accuracy frontier.

4.2 three reasoning modes

The model card recommends using max_new_tokens = 8192 'and generation_mode = "hybrid"' to avoid output truncation and balance speed and accuracy.

4.3 large-scale multi-domain data

The model card and project pages highlight the large-scale, multi-domain training data that the LocateAnything-Data contains:

4.4 High-throughput batch inference tool

The model repository not only gives the model weights, but also provides 'batch_infer.py ', 'batch_utils,' and 'kernel_utils '. Where the la_flash backend is used to FlashAttention the varlen sparse range plan, the goal is to avoid constructing the mask attention dense '[B,H,Q,K].

Example of A100 4K image probe given by model card:

Pre-sales explanation: It is not only a model weight, but also contains reasoning engineering optimization ideas for high-resolution, multi-target batch detection.

5. Applicable Scenario

5.1 GUI Agent / Computer Use

LocateAnything can do grounding element the GUI: given a screenshot and natural language instructions, locate a button, menu, icon, or area. It can be used as a perception module of visual GUI Agent, helping Agent to move from "understanding screen shots" to "knowing where to click".

Suitable scenarios:

5.2 Document Understanding and OCR/Layout

It supports layout grounding and OCR localization and can be used to locate headings, paragraphs, formulas, tables, text blocks, specific fields, and more.

Suitable scenarios:

5.3 industrial vision and quality inspection

Open category detection and referential expression grounding are suitable for industrial vision PoC, especially when there are many types of customer objects, many long tail defects, and high maintenance costs of traditional detectors.

Note: Commercial landing can not use the model license directly, it needs to be used as a technical verification, selection reference or communication with NVIDIA business license.

5.4 Robot/Body Intelligence/Autopilot

Both the project page and the model card mention robotics, driving, Physical, and AI. LocateAnything can be used as a bridge module for "verbal instructions to visual locations", such as "grab the left cup", "locate the red button" and "point to the intersection signal".

5.5 automatic labeling and data production

It can be used to generate grounding / detection / pointing candidate annotations, which can then be reviewed manually or by rules. Suitable for training data preparation, long tail object labeling, GUI dataset construction.

6. Not quite the scene

How to use #7.

7.1 installation dependencies

The model card gives the base dependency:

pip install opencv-python-headless==4.11.0.86 transformers==4.57.1 numpy==1.25.0 Pillow==11.1.0 peft torchvision decord==0.6.0 lmdb==1.7.5

PyTorch need to be installed separately by CUDA version. Hopper / Blackwell GPU optional installation MagiAttention for faster MTP inference; fallback to PyTorch SDPA when not installed.

7.2 Python Call Method

The model card provides a 'LocateAnythingWorker' mode: load tokenizer, processor, model at startup, and then service requests for detection, grounding, OCR, GUI grounding, etc. through the' predict()'and task methods.

Simplified example:

from PIL import Image

worker = LocateAnythingWorker("nvidia/LocateAnything-3B")
img = Image.open("example.jpg").convert("RGB")

result = worker.detect(img, ["person", "car", "bicycle"])
print(result["answer"])

result = worker.ground_gui(img, "the search button", output_type="point")
print(result["answer"])

The box/point coordinates in the output format are tokens normalized to '[0, 1000]' and need to be converted to pixel coordinates.

7.3 Supported Prompt Templates

7.4 Batch inference

python batch_infer.py \
  --model nvidia/LocateAnything-3B \
  --attn la_flash \
  --scheduler pipeline \
  --batch-size 4 \
  --image /path/to/image.jpg \
  --query "personcar"

This mode is suitable for offline batch detection, automatic labeling and evaluation, but not for the training path.

8. What can I say before sales

Business-oriented

Technology-oriented

9. PoC Recommendations

9.1 recommended PoC direction

9.2 PoC Design Recommendations

Example of 9.3 Acceptance Index

10. Risks and Considerations

11. Relationship with related technologies

12. My Pre-Sales Judgment

LocateAnything-3B is a good model for "visual agent/Physical AI / GUI grounding" directional communication. It connects many customer concerns: AI not only need to understand the diagram, but also tell the system "where the target is"; Not only can it recognize common objects, but it can also locate GUI controls, document areas, OCR text and dense targets through natural language.

Its pre-sales value lies in providing a strong demo: the customer enters a natural language, and the model locates the target directly in the complex image. For GUI automation, document intelligence, industrial quality inspection, robotics and autonomous driving awareness, these capabilities are very intuitive.

However, it is not currently suitable as an open source model for direct commercial delivery because license restrictions are critical. More suitable positioning is: for research evaluation, PoC verification, solution prototype, technical route selection, or as an entry point for ecological/licensing cooperation with NVIDIA. Formal business scenarios need to address licensing, model deployment, hardware costs, privacy compliance, and stability validation in advance.

13. Common Customer Q & A

14. REFERENCE

-Hugging Face model page: nvidia/LocateAnything-3B

-NVIDIA Project Page: LocateAnything

-Hugging Face Demo Space

-GitHub code: NVlabs/Eagle/Embodied

-Thesis: LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

-Model License

-NVIDIA Nemotron

-NVIDIA Cosmos