Robbyant, the embodied-AI company within Ant Group, has open-sourced LingBot-Vision, a family of self-supervised Vision Transformers built for dense spatial perception. The weights ship under Apache-2.0 on Hugging Face in four sizes — ViT-giant, ViT-large, ViT-base, and ViT-small — together with a technical report and inference code.
Most vision foundation models are trained for semantic invariance: they learn to answer what is in an image while discarding exactly the fine-grained spatial structure — object boundaries, contours, depth discontinuities — that robots and other physically embodied systems depend on. LingBot-Vision inverts that priority. It treats boundaries as a native pretraining signal rather than a downstream output, and the payoff is a 1B-parameter backbone that matches or surpasses models up to 7× larger on dense spatial tasks, including the 7B DINOv3.
What is LingBot-Vision?
LingBot-Vision is a self-supervised pretrained encoder for spatially structured downstream tasks. The flagship ViT-g/16 has roughly 1.1B parameters and is trained with a new objective called masked boundary modeling on a curated corpus of about 161M images — selected from a 2B web pool — with no human labels, no external edge detectors, and no pretrained backbone to bootstrap from. The training is also notably economical: the corpus is an order of magnitude smaller than DINOv3’s LVD-1689M, and the model consumes less than a third of DINOv3’s training samples.
The encoder outputs dense patch-token features intended for frozen readouts. For deployment at smaller budgets, the flagship is distilled into ViT-L (300M), ViT-B (86M), and ViT-S students that lead dense prediction within their size classes.
How Masked Boundary Modeling Works
The method builds on the DINO/iBOT self-distillation paradigm: a teacher — an EMA copy of the student — generates online targets, and the student recovers them from masked views.
Standard masked image modeling hides patches at random, ignoring what each patch depicts. A flat interior patch is cheap to recover from its neighbors; a patch straddling an object boundary carries structure that context alone cannot supply. Boundaries are the least redundant, most informative regions of an image — and random masking treats them like everything else.
LingBot-Vision closes that gap with two ideas.
Boundary-forcing. The teacher predicts a dense boundary field online and identifies the boundary-bearing tokens B. These are forced into the student’s masked set on top of the random mask M, giving the combined mask M⁺ = M ∪ B. Masked tokens are then routed by geometry: boundary tokens receive an explicit geometric target in addition to the semantic self-distillation target, while interior masked tokens keep the standard semantic objective alone. This routing matters because a semantic target is inherently ambiguous exactly where two regions meet — the geometric target is well-posed precisely where conventional masked modeling is weakest, which is what lets semantic and geometric representations co-emerge rather than compete.
Categorical boundary field. Boundaries are modeled as line segments lifted into a dense field: every nearby pixel stores an attribute vector a(p) = (d, θ, φ¹, φ²) recording its distance to the nearest segment and three angles that locate it. Directly regressing this field in a teacher–student loop collapses. The fix is to discretize each channel into K = 32 bins, recasting boundary prediction as per-pixel classification — which lets the boundary branch inherit the same centering and sharpening machinery that stabilizes modern self-distillation.
The categorical form has an elegant side effect. Under the classical a-contrario null hypothesis of “no structure,” boundary orientations are uniformly distributed — and that null is now literally the uniform distribution over bins. Deviation from uniformity is evidence of a real boundary, so a parameter-free Number-of-False-Alarms (NFA) test validates every decoded segment at no extra cost. The teacher exploits this at each iteration: it decodes candidate segments from its own field prediction, keeps only the NFA-validated survivors, and re-renders them into the target field — so unsupported structure never becomes a teaching signal.
The full objective sums four terms:
L = L_DINO + λᵢ · L_iBOT + λᵦ · L_bnd + λₖ · L_KoLeo
Benchmarks and Performance
All dense results below use frozen features with a single linear layer, so performance is attributable to the representation rather than a decoder.
On NYU-Depth v2, LingBot-Vision posts the best RMSE of the entire comparison (0.296), ahead of the 7B DINOv3 (0.309) with roughly 7× fewer parameters, and ahead of the 2B V-JEPA 2.1 (0.307). On KITTI it is the best model below 2B parameters. On semantic segmentation it is on par with the distilled DINOv3 ViT-H+ — 1.3 mIoU behind on ADE20K, matching on Cityscapes, ahead on VOC12 — while improving over the same-size DINOv2 by 4+ mIoU on all three benchmarks; the only remaining gap is to the DINOv3 family itself (2.4 mIoU on ADE20K to the 7B model), whose dense strength comes from distillation and dedicated dense-feature objectives.
Video object segmentation uses training-free label propagation over frozen features. LingBot-Vision reaches 70.0 J&F on DAVIS-2017 and 73.5 on YouTube-VOS — on par with DINOv3 ViT-H+ (71.1 / 74.0) and the 7B DINOv3 (71.1 / 74.1), and the best among all remaining models at any scale. The boundary tokens themselves are stable enough to be tracked through video by plain cosine similarity of frozen features, with no temporal supervision.
The trade-off is image-level recognition: ImageNet-1K linear probing reaches 86.32 and k-NN 83.39, trailing DINOv3-7B, which spends its capacity on image-level invariance. The advantages also survive distillation — the 0.3B ViT-L student matches the 7B DINOv3 on NYUv2 depth (0.310 vs. 0.309) with about 23× fewer parameters.
Use Cases and How to Load It
The frozen patch tokens serve several dense workloads directly: depth estimation reads geometry straight from the features, semantic segmentation benefits from feature transitions that land exactly on object contours, and video object segmentation works through cosine-similarity token matching. The encoder also serves as the initialization for downstream depth-completion training.
Loading a backbone follows the official repository:
cd lingbot-vision
conda create -n lingbot-vision python=3.10 -y
conda activate lingbot-vision
python -m pip install -r requirements.txt
python -m pip install -e .
from lingbot_vision import load_pretrained_backbone, extract_patch_tokens, load_image
device = “cuda” if torch.cuda.is_available() else “cpu”
dtype = torch.bfloat16 if device == “cuda” else torch.float32
# Downloads model.pt from Hugging Face on first use.
backbone, embed_dim = load_pretrained_backbone(
variant=”small”, # giant | large | base | small; defaults to large
device=device,
dtype=dtype,
)
img_norm, _, _ = load_image(
“examples/example.png”,
size=512,
patch_size=backbone.patch_size,
mode=”square”,
)
patch_tokens, patch_grid = extract_patch_tokens(backbone, img_norm, device, dtype)
print(patch_tokens.shape, patch_grid, embed_dim)
# torch.Size([1, 1024, 384]) (32, 32) 384
The variant argument selects the size and defaults to large. Output patch_tokens has shape [B, H*W, C]. Requirements are Python ≥ 3.10 and PyTorch ≥ 2.0, with a GPU recommended for the larger backbones.
LingBot-Depth 2.0: The Downstream Payoff
To show what a spatial-perception-native encoder buys downstream, the team upgraded its depth-completion system to LingBot-Depth 2.0. The masked-depth-modeling recipe is unchanged from version 1.0; exactly two ingredients moved: the encoder initialization switched from DINOv2 to LingBot-Vision (in ViT-L and ViT-g variants), and the curated training data grew from the publicly released 3M samples to 150M.
Those two changes set leading results across 14 depth-completion benchmarks spanning block-mask, sparse, and real-sensor regimes. On block-masked DIODE-Indoor, RMSE is halved from 0.132 to 0.062. The system is strongest on the transparent-object ClearGrasp captures (0.010 / 0.012 RMSE) — the classic failure case of active depth sensing.
Notably, the two changes compound rather than cancel: as training data grows from 3M to 150M, the DINOv2-initialized curve saturates beyond 20M samples while the LingBot-Vision curve keeps improving. More data amplifies, rather than washes out, the advantage of the better starting point.
Key Takeaways
- LingBot-Vision makes boundaries a native pretraining signal, bootstrapped from raw images with no labels, edge detectors, or pretrained backbones.
- Boundary-forcing plus a categorical boundary field lets geometry and semantics co-emerge — and yields a parameter-free NFA validation test for free.
- The 1B backbone posts the best NYU-Depth v2 RMSE in its comparison, ahead of the 7B DINOv3, while training on an order-of-magnitude smaller corpus.
- The advantages survive distillation: the 0.3B ViT-L matches the 7B DINOv3 on NYUv2 with ~23× fewer parameters.
- Swapping only the encoder and scaling data took LingBot-Depth 2.0 to leading results on 14 depth-completion benchmarks, and the encoder’s edge widens with more data.
- Weights ship under Apache-2.0 in ViT-g/L/B/S sizes for every deployment budget.
Interactive Dynamic Explainer
Check out the Paper and Model Weights. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
Note:Thanks to the Ant Research team for the thought leadership/ Resources for this article. Ant Research team has supported this content/article for promotion.
