open-robot-skills
What your robot can do, one directory at a time.
A curated, contributable library of manipulation skills and model-backed
tool bundles for GaP — graph as policy, in the
Anthropic Agent Skills format.
The LLM pipeline composes these bundles into executable robot graphs;
every bundle is one directory, one SKILL.md, one PR.
Clone this repo next to the graph-as-policy checkout (or set GAP_SKILLS_PATH) and
every GaP command — gap run, gap generate, gap benchmark — discovers
it automatically; no flags, no registration.
New to GaP? Start with the libero_quickstart example or the 15-minute tour, then come back here when you want to add a capability.
Contents
- Skills — what the robot can do
- Tools — what the robot can compute
- Install
- Verify a checkout
- Contributing a bundle
- Use with Claude Code
Skills — what the robot can do
Manipulation strategies. A skill owns subgraphs in generated workflows: its
SKILL.md body is the guidance the subgraph agent reads, its
scripts/ are the canonical recipes the graph executes, and its declared
exit_conditions are what downstream routing keys on. Closed-loop VLA
policies (pi05-libero, molmoact-libero) now live under policies/
with kind=policy — they share the bundle format but are surfaced to
graphs through the policy plane, not the skill registry.
| Bundle | Description | Tools | Extra |
|---|---|---|---|
| grasping-direct-ik | Direct IK align-then-descend grasping. | — | open-robot-skills[grasping-direct-ik] |
| grasping-short-axis | Deterministic short-axis-aligned grasp with CuRobo. | — | open-robot-skills[grasping-short-axis] |
| grasping-with-planner | Top-down grasping via a fast axis-locked linear descend with a collision-aware cuRobo fallback. | — | open-robot-skills[grasping-with-planner] |
| perceiving-next-item | Loop-head perception for pack-all / clean-all-items tasks. | — | open-robot-skills[perceiving-next-item] |
| perceiving-object-parts | Hierarchical perception for subpart targeting. | — | open-robot-skills[perceiving-object-parts] |
| perceiving-objects | Fast single-path 3D object perception. | — | open-robot-skills[perceiving-objects] |
| perceiving-objects-oneshot | Lightweight one-shot 3D object perception. | — | open-robot-skills[perceiving-objects-oneshot] |
| tracking-objects | Long-running skill that drives the SAM3 tracker from the graph-scoped observation stream. | tracking-objects.track |
open-robot-skills[tracking-objects] |
| transporting-objects | Move the currently-held object to a destination and release. | — | open-robot-skills[transporting-objects] |
Tools — what the robot can compute
Model-backed typed callables — one bundle per model, no task strategy.
Tool bundles never own subgraphs; they appear in the flat tool catalog every
subgraph agent (and every skill script) calls through ctx.tool(...). Each
bundle's SKILL.md documents its setup: dependencies (declared in the
bundle's own pyproject.toml), environment variables, weights, and quirks.
| Bundle | Description | Tools | Extra |
|---|---|---|---|
| curobo | NVIDIA cuRobo motion planning — collision-free trajectories to grasp goalsets, transport with an attached object, constrained linear moves, single-pose planning, geometric IK, batch grasp feasibility, and joint-trajectory collision validation. | curobo.batch_grasp_feasibility, curobo.plan_directed_linear, curobo.plan_grasp_motion, curobo.plan_linear, curobo.plan_to_grasp_poses, curobo.plan_to_pose, curobo.plan_with_grasped_object, curobo.solve_ik, curobo.validate_joint_trajectory_grasped, curobo.validate_joint_trajectory_robot |
— |
| gemini-er | Open-vocabulary 2D object detection via the Gemini Robotics-ER API — one call returns pixel-space bounding boxes with labels and scores for a text query. | gemini-er.detect |
— |
| geometry | Pure-math 3D geometry toolbox — back-project masks and depth to point clouds, DBSCAN-filter noise, fit oriented bounding boxes, derive top-down/front grasp poses, and reconstruct collision worlds from RGB-D frames. | geometry.build_world_config, geometry.compute_drop_position, geometry.compute_obb, geometry.compute_xy_distance, geometry.depth_to_point_cloud, geometry.exclude_robot_points, geometry.filter_and_compute_obb, geometry.filter_noise, geometry.front_grasp_from_obb, geometry.iou, geometry.mask_to_world_points, geometry.pixel_to_world_point, geometry.pose_distance, geometry.rotate_quat_z90, geometry.select_top_down_grasp, geometry.top_down_grasp_candidates, geometry.top_down_grasp_from_obb, geometry.transform_points |
— |
| grounding-dino | Grounding DINO zero-shot object detection — natural-language queries to labeled 2D bounding boxes with confidence scores. | grounding-dino.detect |
— |
| molmo | Visual pointing and Q&A via the Molmo VLM served from a self-hosted vLLM endpoint (OpenAI-compatible API). | molmo.point_prompt, molmo.query, molmo.query_yes_no |
— |
| sam3 | Segment Anything 3 — text-, point-, and box-prompted instance segmentation, plus a stateful streaming video tracker that carries object identity through SAM3's memory bank. | sam3.segment_box, sam3.segment_point, sam3.segment_text, sam3.tracker_close, sam3.tracker_init, sam3.tracker_update |
— |
| vlm | Free-form and yes/no visual question answering against a hosted vision-language model (OpenRouter API by default; Vertex AI Gemini selectable by config). | vlm.query, vlm.query_yes_no |
— |
Both tables are generated — regenerate after adding a bundle:
uv run gap skills table --format markdown --kind skill # and --kind tool
Install
Tool and policy bundles now own their own pyproject.toml; install them
per-bundle with uv run gap skills install <bundle> (or --all), not via
top-level extras. From the GaP checkout:
cd ../graph-as-policy
uv sync # engine + sim baseline (LIBERO included)
uv run gap skills install --all # every bundle in this registry
uv run gap skills install sam3 grounding-dino geometry # …or pick a subset
CUDA_HOME=/usr/local/cuda uv run gap skills install curobo # CuRobo compiles CUDA at install
uv run gap skills check --download # verify bundles + prefetch model weights
uv.lock pins the exact environment of the acceptance benchmark run. Two
deliberate knobs keep it solvable (documented in pyproject.toml): sam3's
over-strict numpy==1.26 metadata pin is relaxed via
override-dependencies, and nvidia-curobo builds unisolated against the
environment's torch.
Verify a checkout
uv run gap skills check # per-bundle PASS/WARN/FAIL; non-zero exit on FAIL
uv run gap skills check --download # + prefetch model weights (HF_TOKEN for gated repos)
uv run gap skills table # the catalog above (--format markdown|json)
uv run gap check # capability report: which bundles can run HERE
uv run gap tools list # the flat tool catalog with live schemas
gap skills check runs the engine-side format validation (frontmatter
shape per bundle kind, referenced resource paths, allowed_tools
resolution, declared type names, one extra per bundle) plus an import
probe that maps missing dependencies to their install line — the same
install-verification step the GaP quickstart runs. Missing weights are a
WARN, not a FAIL — nothing installs behind your back.
gap check answers the operational question instead: per bundle, are the
deps importable, the declared gap.requires: met (GPU, env vars), the
weights cached — and which skills are therefore runnable right now, with
a fix hint per failure.
This repo is one skill registry — the canonical example. GaP merges
any number of them by precedence (your lab's fork can shadow individual
bundles here): gap registry init scaffolds a new one, gap registry add <name> <path> layers it on top, gap registry list shows the active
set. See the engine's
docs/skills.md.
Contributing a bundle
One bundle = one directory = one PR. Scaffold it:
uv run gap skills new my-skill --kind skill # or --kind tool
That creates the layout (SKILL.md + scripts/ or tools.py) and a
unit-test skeleton (tests/test_my_skill.py); then:
-
Write the
SKILL.md. Spec frontmatter (name== dirname, third-persondescriptionending in a "Use when…" sentence — it is the coordinator's entire view of your bundle), GaP extensions under thegap:key (allowed_tools,exit_conditions,canonical_scripts, …). -
Declare operational requirements — a
gap.requires:block ({gpu: true, env: [MY_API_KEY], env_any: [...], weights: true};requires: {}when it needs nothing — mandatory for tool bundles, the test suite enforces it).gap checkderives runnability from this. Bundles that download weights may add a filesystem-onlyweights_cached() -> bool | Nonenext toprefetch()intools.py. -
Declare dependencies once — add one extra named after your bundle in
pyproject.toml(empty list if it has none) and runuv lock. -
Lazy-load models. Importing your
tools.pymust not import torch/transformers; load weights on first call (the test suite enforces this). -
Test it without hardware using
gap.testing(FakeContext,make_test_observation) — flesh out the scaffolded test;uv run gap skills test my-skill(or plainuv run pytest tests -q) must stay green on a workstation without a robot or accelerator; model-touching smokes go behind thegpumarker. -
Check yourself before the PR:
uv run gap skills check && uv run gap skills test my-skill && uv run pytest tests -q
The full authoring guide — bundle anatomy, the skill-facing ctx API,
exit-condition design, streaming skills — is in
gap/docs/skills.md.
Use with Claude Code
.claude-plugin/marketplace.json indexes
every bundle, so this checkout doubles as a Claude Code plugin marketplace:
the same SKILL.md files that drive GaP's graph generation are loadable as
agent skills.
To drive GaP itself from Claude Code — search these registries, check capabilities, run/generate graphs, author new tested bundles — install the engine's agent skill from the GaP repo (it also re-exports this registry's bundles, so one marketplace covers both):
claude plugin marketplace add graph-robots/graph-as-policy
claude plugin install gap@gap
License
Apache-2.0 for this repo's code and docs; model weights and the pinned upstream packages (SAM3, Grounding DINO, cuRobo, …) keep their own licenses — see gap/NOTICE.md for the attribution table.
Respect the original skill developers' licenses. Skills in this library may wrap, port, or build on code published by third-party skill developers. The original developer's license always governs that skill's code: keep the upstream license and attribution in the skill's folder, preserve them when redistributing, and check them before commercial use. Contributions that derive from existing work must declare their provenance and include the original license alongside the skill.
No comments yet
Be the first to share your take.