Simulation Environments

Sapien

Sapien simulation platform and its official tutorials.

Isaac Sim

Isaac Sim Documentation version 5.1.

Simulation Asset Formats

URDF Files

Articulated Robotics blog URDF section and ROS wiki resources, including ROS 2 tutorials.

Architecture Overview

URDF employs a tree structure using links (components) and joints (articulations) to describe articulated objects. Components can be categorized as:

• Movable individual parts
• Static but distinctive elements (e.g., cameras)

Common joint types include:

1. Revolute: Rotation-limited joint with minimum/maximum angle constraints
2. Continuous: Unlimited rotation joint (e.g., wheels)
3. Prismatic: Linear sliding joint with position limits
4. Fixed: Rigid connection between parent and child components

Basic Syntax Structure

URDF is based on XML (eXtensible Markup Language) with strict nesting relationships focusing on logical structure.

The XML declaration specifies version, with all content encapsulated within robot tags:

Only the name attribute is mandatory for links. Optional components include:

• Visual Properties
  • Geometry: Shape parameters or external mesh models
  • Origin: Adjusts default center position for joint alignment
  • Material
    • Color: RGBA format (0-1 range, A=alpha/transparency)
    • Texture: Image mapping (e.g., .jpg files)
• Collision Properties
  • Geometry and Origin: Typically mirror visual properties, but may use simplified shapes for computational efficiency
• Inertial Properties
  • Mass: Component weight
  • Origin: Center of mass location
  • Inertia: Inertia tensor matrix

Multiple visual and collision elements are supported. In Sapien, multiple collision shapes automatically form compound collision bodies.

Joint Elements

• Name: Joint identifier (optional but recommended)
• Type: Joint classification (fixed, prismatic, revolute, continuous)
• Parent/Child Links: Specifies connected components
• Origin: Position offset between connected links

Additional parameters for non-fixed joints:

• Axis: Motion direection vector
• Limits:
  • Position boundaries (meters or radians)
  • Velocity constraints (m/s or rad/s)
  • Effort limits (Newtons or Newton-meters)

Additional Elements

Sapien-compatible extensions:

Material: Defines colors or textures

Mimic: Joint synchronization mechanism within joint elements:

<mimic joint="reference_joint" multiplier="1.0" offset="0.0" />

Sapien parses this element. When driving the reference joint, the mimicking joint automatically follows using: θ_mimic = mult × θ_ref + offset

Transmission: Actuation system specification, where Sapien distinguishes active and passive joints.

Naming conventions typically append "_link" and "_joint" suffixes respectively.

Xacro Extensions

URDF Modularization

Xacro enables file organization through inclusion mechanisms, effectively embedding sub-file content into main files:

<xacro:include filename="component.xacro" />

File hierarchy:

• Main file: Requires complete robot tag with name and xmlns:xacro namespace
• Included files: Need robot tag and namespace but omit name specification

Common file suffix conventions:

• Top-level assembly: .urdf.xacro
• Sub-modules: .xacro, .urdf.xacro, or custom suffixes

Variable Definitions and Global Properties:

<xacro:property name="component_size" value="0.5" />
...
<cylinder radius="${component_size}" length="7" />

Macro Definitions and Component Reuse: Create reusable part templates with parameterization:

<xacro:macro name="box_inertia" params="weight a b c *position">
    <inertial>
        <xacro:insert_block name="position"/>
        <mass value="${weight}" />
        <inertia ixx="${(1/12) * weight * (b*b+c*c)}" ixy="0.0" ixz="0.0"
                iyy="${(1/12) * weight * (a*a+c*c)}" iyz="0.0"
                izz="${(1/12) * weight * (a*a+b*b)}" />
    </inertial>
</xacro:macro>

... Usage ...

<xacro:box_inertia weight="12" a="2" b="3" c="4">
    <origin xyz="0 2 4" rpy="0 0 0"/>
</xacro:box_inertia>

Mathematical Expressions: Xacro supports calculations within ${} brackets:

<cylinder length="${4*component_size + pi}">

USD Format

USD (Universal Scene Description) is an open-source 3D scene format developed by Pixar.

Common File Extensions:

• .usd: Universal format (binary or text)
• .usda: ASCII text format (human-readable)
• .usdc: Binary format (optimized for large scenes)
• .usdz: Mobile-optimized compressed format for AR applications

Existing High-Quality Datasets

Partnet Mobility

Partnet Mobility visualization and data download resources.

Contains 2,346 models across 46 indoor object categories, sourced from 3D Warehouse and organized following ShapeNet and PartNet methodologies.

Mesh Semantic Segmentation Strategy: Interactive manual annotation

PartNet Annotation Tool Features:

• Model Loading: Import triangular meshes from ShapeNet
• Selection and Grouping: Manual triangle face selection based on semantic templates
• Instance Creation: Distinct instance IDs for semantically similar parts

Automated Segmentation Approaches in Partnet:

Four point cloud segmentation algorithms (PointNet, PointNet++, SpiderCNN, PointCNN) were tested with per-category hierarchical training.

Processing Pipeline:

After removing unlabeled ground truth points, compute IoU between predicted and ground truth point sets for each semantic part category. Calculate category mIoU averages across test shapes.

Limitations:

• Performance degradation from coarse to fine granularity levels
• Poor performance on small parts and visually similar components
• Imperfect mesh boundary generation with edge noise

Partnet Mobility Automated Segmentation:

1. Data Preprocessing: Mesh sampling to point clouds (10,000 points per object with coordinates and optional normals)
2. Feature Extraction: PointNet++ backbone with Set Abstraction and Feature Propagation layers
3. Point-to-Mesh Mapping: Nearest neighbor voting with smoothing via CRF or Graph Cut algorithms
4. Physical Segmentation: Group connected faces by semantic labels and export as separate files
5. Axis Estimation: Regression branch predicting part rotation axis direction

Semi-Automated Segmentation Insights: Existing segmentation boundaries can be migrated to similar new models with manual refinement.

ArtVIP

ArtVIP research paper and implementation details.

Advantages:

1. Visual Realism:
   • Mesh: Manifold preservation with vertex optimization
   • Texture: UV coordinate mapping for surface details
   • Material: PBR rendering with RTX support in Isaac Sim
2. Modular Interaction: Class-specific Isaac Sim Python scripts implementing part interactions including magnetic doors, damped drawers, and triggered interactions.
3. Physical Fidelity:
   • Collision Mesh: Convex hull simplification and decomposition
   • Joint: Nonlinear modeling with angle/velocity-dependent stiffness and damping functions

Mesh Segmentation Strategy: Manual semantic segmentation with algorithmic geometric optimization