lekiwi录制训练推理流程实践

录制

设备端

先确定一下相机编号

cd ~/lerobot/

lerobot-find-cameras

生成路径：outputs/captured_images

然后修改uart的权限

sudo chmod 666 /dev/ttyACM0

启动等待连接

python -m lerobot.robots.lekiwi.lekiwi_host \
  --robot.id=R1225280 \
  --robot.cameras='{
    "front": {"type":"opencv","index_or_path":"/dev/video2","width":640,"height":480,"fps":30},
    "wrist": {"type":"opencv","index_or_path":"/dev/video0","width":480,"height":640,"fps":30,"rotation":"ROTATE_90"}
  }' \
  --host.connection_time_s=300

如果分辨率跑不上去，就让相机用motion-JPEG传输帧，很多UVC相机默认YUYV在640×480只能到25fps。

python -m lerobot.robots.lekiwi.lekiwi_host \
  --robot.id=R1225280 \
  --robot.cameras='{
    "front": {"type":"opencv","index_or_path":"/dev/video2","width":640,"height":480,"fps":30,"fourcc":"MJPG"},
    "wrist": {"type":"opencv","index_or_path":"/dev/video0","width":480,"height":640,"fps":30,"rotation":"ROTATE_90","fourcc":"MJPG"}
  }' \
  --host.connection_time_s=300

遥控端

启动前，需要配置一下，主要修改lekiwi/record.py。

• 主臂型号SO101：将SO100修改为SO101型号。
• 网络配置remote_ip：配置连接设备的ip地址，remote_ip:LeKiwi机器人主机的IP地址。id:机器人实例标识符。
• 机械臂配置port：配置领航臂的串口和标定参数文件，port=”/dev/ttyACM0″, id=”R07252801″。
• 数据集路径HF_REPO_ID：配置录制数据集上传到的Hugging Face仓库，格式为用户名/仓库名，会存储到~/.cache/xxx路径下。
• 任务配置：配置采集回合数NUM_EPISODES，每个回合的时间EPISODE_TIME_SEC（单位秒），重置环境时间RESET_TIME_SEC（秒），任务描述信息TASK_DESCRIPTION。录制帧率FPS。

diff --git a/examples/lekiwi/record.py b/examples/lekiwi/record.py
--- a/examples/lekiwi/record.py
+++ b/examples/lekiwi/record.py
-from lerobot.teleoperators.so100_leader import SO100Leader, SO100LeaderConfig
+from lerobot.teleoperators.so101_leader import SO101Leader, SO101LeaderConfig

 EPISODE_TIME_SEC = 30
 RESET_TIME_SEC = 10
 TASK_DESCRIPTION = "My task description"
-HF_REPO_ID = "<hf_username>/<dataset_repo_id>"
+HF_REPO_ID = "laumy/lekiwi"

-robot_config = LeKiwiClientConfig(remote_ip="172.18.134.136", id="lekiwi")
-leader_arm_config = SO100LeaderConfig(port="/dev/tty.usbmodem585A0077581", id="my_awesome_leader_arm")

+robot_config = LeKiwiClientConfig(remote_ip="10.0.90.31", id="lekiwi")
+leader_arm_config = SO101LeaderConfig(port="/dev/ttyACM0", id="R07252801")
 keyboard_config = KeyboardTeleopConfig()

-leader_arm = SO100Leader(leader_arm_config)
+leader_arm = SO101Leader(leader_arm_config)

配置好后启动脚本进行采集数据，采集完成后数据存储在~/.cache/huggingface/lerobot/laumy/lekiwi/

python examples/lekiwi/record.py

训练

lerobot-train   \
    --dataset.repo_id=laumy/lekiwi   \
    --policy.type=act   \
    --output_dir=outputs/train/act_lekiwi_test   \
    --job_name=act_lekiwi_test   \
    --policy.device=cpu   \
    --policy.push_to_hub=false   \
    --wandb.enable=false \
    --batch_size=8 --steps=10000

• –policy.push_to_hub=false：训练的模型是否上传hugging face。
• –policy.repo_id=your_username/your_model_name：指定推送的路径名称。
• –wandb.enable=false：是否启用Weights & Biases (W&B) 实验跟踪。

推理

设备端

python -m lerobot.robots.lekiwi.lekiwi_host \
  --robot.id=R1225280 \
  --robot.cameras='{
    "front": {"type":"opencv","index_or_path":"/dev/video2","width":640,"height":480,"fps":30,"fourcc":"MJPG"},
    "wrist": {"type":"opencv","index_or_path":"/dev/video0","width":480,"height":640,"fps":30,"rotation":"ROTATE_90","fourcc":"MJPG"}
  }' \
  --host.connection_time_s=300

PC端

启动前先修改配置，这里跟record.py修改差不多。

• 模型路径HF_MODEL_ID：如果不是本地目录则尝试从hugging face上下载。
• 数据集路径HF_DATASET_ID：推理还需要数据集主要是用于输入数据的归一化和策略输出反归一化。
• 设备remote_ip：配置设备端的ip地址。

index 4501008d..f508d448 100644
--- a/examples/lekiwi/evaluate.py
+++ b/examples/lekiwi/evaluate.py
@@ -29,12 +29,12 @@ from lerobot.utils.visualization_utils import init_rerun
 NUM_EPISODES = 2
 FPS = 30
 EPISODE_TIME_SEC = 60
-TASK_DESCRIPTION = "My task description"
-HF_MODEL_ID = "<hf_username>/<model_repo_id>"
-HF_DATASET_ID = "<hf_username>/<eval_dataset_repo_id>"
+TASK_DESCRIPTION = "Move to grab block"
+HF_MODEL_ID = "output/act"
+HF_DATASET_ID = "laumy/lekiwi"

 # Create the robot configuration & robot
-robot_config = LeKiwiClientConfig(remote_ip="172.18.134.136", id="lekiwi")
+robot_config = LeKiwiClientConfig(remote_ip="10.0.90.31", id="lekiwi")

改完之后启动运行

python examples/lekiwi/evaluate.py