ch-To-Skill Sketch-To-Skill

SKETCH-TO-SKILL: Bootstrapping Robot Learning with human-drawn Trajectory Sketches

Anonymous
ICLR 2025
*Indicates Equal Contribution
Paper Supplementary Code

Abstract

Training robotic manipulation policies traditionally requires numerous demonstrations and/or environmental rollouts. While recent Imitation Learning (IL) and Reinforcement Learning (RL) methods have reduced the number of required demonstrations, they still rely on expert knowledge to collect high-quality data, limiting scalability and accessibility. We propose SKETCH-TO-SKILL, a novel framework that leverages human-drawn 2D sketch trajectories to bootstrap and guide RL for robotic manipulation. Our approach extends beyond previous sketch-based methods, which were primarily focused on imitation learning or policy conditioning, limited to specific trained tasks. SKETCH-TO-SKILL employs a Sketch-to-3D Trajectory Generator that translates 2D sketches into 3D trajectories, which are then used to autonomously collect initial demonstrations. We utilize these sketch-generated demonstrations in two ways: to pre-train an initial policy through behavior cloning and to refine this policy through RL with guided exploration. Experimental results demonstrate that SKETCH-TO-SKILL achieves ~96% of the performance of the baseline model that leverages teleoperated demonstration data, while exceeding the performance of a pure reinforcement learning policy by ~170%, only from sketch inputs. This makes robotic manipulation learning more accessible and potentially broadens its applications across various domains.

Overview

Descriptive text

Fig: Learning a new skill in the Sketch-to-Skill framework. Step 1: Capture the task scenario from two views and collect human-drawn sketches. Step 2: Convert 2D sketches to 3D trajectories using a pretrained generator. Step 3: Execute generated trajectories to collect experience data. Step 4: Learn manipulation policy using reinforcement learning bootstrapping from behavior cloning and using guidance for experience data.

Policy Learning

Descriptive text

Fig: Overview of SKETCH-TO-SKILL integrating sketch-generated demonstrations with reinforcement learning. Sketch-generated experiences train an IL policy, which bootstraps the RL process. A discriminator guides exploration by rewarding similarity to sketch-generated trajectories. The final action, combining IL and RL policy outputs, further enhances the exploration guidance.

Hardware Setup

Descriptive text

Fig: Complete setup for the ButtonPress task in a real-world experiment. The configuration includes a UR3e robot arm equipped with a Robot Hand gripper, and a RealSense D435i camera mounted on the wrist.

Overview

Descriptive text

Fig: Environment cameras - corner and corner2, for human-drawn sketches and generating demonstration

Steps Vs Train Score Plot

Descriptive text

Fig: The evaluation success rate of the BC policy of ButtonPress task trained on sketch-generated demonstrations

Videos

BC Policy Evaluation (5x)

The video (4x) demonstrates 5 successful task completions by RL policy with randomized button position for each trial.

BibTeX


        title={SKETCH-TO-SKILL: Bootstrapping Robot Learning with human-drawn Trajectory Sketches},
        year={2025}