Marta Skreta Replan Robotic Replanning 2024

[TOC]

1. Title: RePlan: Robotic Replanning with Perception and Language Models
2. Author: Marta Skreta et. al.
3. Publish Year: 8 Jan 2024
4. Review Date: Thu, Jan 25, 2024
5. url: arXiv:2401.04157v1

Summary of paper

Motivation

• However, the challenge remains that even with syntac- tically correct plans, robots can still fail to achieve their intended goals. This failure can be attributed to imperfect plans proposed by LLMs or to unforeseeable environmental circumstances that hinder the execution of planned subtasks due to erroneous assumptions about the state of objects.

Contribution

• Robotic Replanning with Perception and Language Models that enables real-time replanning capabilities for long-horizon tasks.

Some key terms

Address the challenge of multi-stage long-horizon tasks

• inheriting key ideas from recent progress in foundation models
• introduce REPLAN, an innovative zero-shot approach that harnesses LLMs at multiple levels by iterative re-prompting to serve as a reward generator for robot MPC.

Structure

1. Perceiver Models for High-Level Replanning: We utilize perceiver models to facilitate high-level replanning, enabling the system to adapt and adjust strategies as needed during task execution.
2. Creation of Hierarchical Plans with Language Models: We employ language models to generate hierarchical plans, allowing for structured and organized task execution strategies.
3. Verification of Outputs from Language Models: We implement mechanisms to verify the outputs generated by language models, ensuring the reliability and accuracy of the planned actions.
4. Robot Behavior through Reward Generation: We influence robot behavior through reward generation mechanisms, incentivizing desirable actions and outcomes during task execution.

Results

• similar to Eureka, the low level LLM planner will generate python reward function to give reward to the motion controller