The Intelligence Layer: EmbodiedAgents Roadmap

Q1 & Q3 2026 (Happening Now): Data Collection & State Aggregation

  • Universal Data Collection: Add standardized “Collect Data” hooks to all components to facilitate dataset generation for future model fine-tuning.

  • Extend Compute Architecture Support: Add deployment harness for auto-compilation for NPUs for components that utilize local models (Vision, STT).

  • Natural Language Based Task Composition: Enhance the LLM component to handle composite tasks specified in natural language through Action lookup and chaining.

  • State Aggregation and Memory: Add a specialized component that leverage LLMs, graph structures and vector DBs to synthesize, store and recall the Global Robot State (from Sugarcoat’s Roadmap) into semantically meaningful representations for use by the agent.

  • New Memory Primitive: Add new long term memory primitive which goes beyond storing semantic vectors (inefficient) and leverage learned graph structures for heirarchical spatio-temporal organization. There is currently no such abstraction out there and this would become significant for long running tasks in more generalized environments.

Q3 & Q4 2026: Simulation Extension, Multi Agent Orchestration & Memory Architecture

  • Isaac Sim Extension: Launching a dedicated extension for NVIDIA Isaac Sim to allow developers to build, test, and validate EmbodiedAgents logic in high-fidelity virtual worlds.

  • Structured Decomposition and Verification: Add structured decomposition features to LLM/VLM output post-processing for verifiable reasoning traces and formal verification guarentees.

  • Collaborative Multi-Agent Discourse: Enabling multiple robots to share mission goals, exchange semantic memories, and coordinate complex multi-agent tasks through new communication protocols.