Project / 02 · 2026开源项目 / 02 · 2026
RealMan AgentEmbodied agent
Embodied ReAct & Dual-Layer Memory for Long-Horizon Mobile Manipulation
An embodied ReAct agent framework with Working Memory and SQLite Episodic Memory for long-horizon mobile manipulation, connecting navigation, YOLO-seg + FoundationPose 6D pose estimation, and grasping on RealMan hardware.

Research question核心研究问题
How can a mobile manipulator orchestrate multi-step navigation, perception, and grasping from natural language while turning past failures into reusable planning constraints?
This page treats the project as a system and a research record, not as a list of technologies.将工程项目视为完整的实验记录与系统架构,而非孤立技术的堆叠。
System系统流水线
Method & contribution核心方法与贡献
Built the ReAct closed-loop planner paired with Working Memory (live mission/robot state) and SQLite Episodic Memory (failure reflection and cross-task lesson retrieval via Qwen), integrating chassis navigation, YOLO-seg segmentation, FoundationPose 6D pose estimation, and Verifier feedback.
- 设计面向长时序移动操作的 ReAct 闭环决策器与 Tool Calling 技能编排
- 构建 Working Memory 实时状态维护与 SQLite Episodic Memory 失败归因反思机制
- 集成底盘导航、RGB-D + YOLO-seg 分割、FoundationPose 6D 位姿估计与 RealMan 机械臂抓取
- Ground each ReAct decision in real-time robot feedback rather than open-loop scripts
- Keep preconditions explicit in Working Memory to prevent blind repeated retries
- Consolidate failure trajectories into conditional planning constraints stored in SQLite
- Separate high-level agent orchestration from low-level vision and ROS driver adapters
Evidence实验证据与交付物
- Validated A→B→A object retrieval pipeline with offline replay and Qwen API reflection benchmarks
- Documented ROS packages, FoundationPose adaptation, and hardware validation interfaces
- End-to-end A→B→A eyeglasses-case retrieval scenario and reflection benchmark
- Hardware validation guide and backend API documentation
- Reproducible offline memory and reflection demonstration scripts
Long-horizon reliability depends on accurate 6D pose transforms, precondition verification fidelity, and Qwen reflection prompt quality under novel physical disturbances.