Embodied intelligence refers to agents that sense,reason,act,and learn through a tight “body-environment-task” feedback loop. It is widely regarded as the most promising route to general-purpose artificial intelligence (AI). Paradoxically,although model parameters,training data,and compute budgets have grown by four orders of magnitude in only five years,the evaluation ecosystem remains fragmented and largely irreproducible. The same functional capability (e.g.,“pick-and-place”) is formalized as object-detection mean average precision in one paper,as 3D intersection over union in a second,as task success rate in a third,and as human preference score in a fourth,with absolute gaps that exceed 10%—15% among “state-of-the-art” results. Environments,random seeds,physics parameters,sensor noise models,and success criteria are rarely open-sourced,and thus,the community cannot tell whether an apparent improvement originates from algorithmic innovation,data scale,evaluation cherry picking,or simple benchmark over-fitting. This uncertainty has become a critical bottleneck for scientific progress and industrial adoption. To address the gap,we conduct a systematic survey of 100+ papers published in IEEE/CVF Conference on Computer Vision and Pattern Recognition,IEEE International Conference on Robotics and Automation,Conference on Neural Information Processing Systems,Robotics: Science and Systems,International Conference on Learning Representations,International Journal of Robotic Research,and Journal of Machine Learning Research between 2020 and 2025. Then,we present the first holistic analysis of embodied AI evaluation. We organize the landscape into a three-tier pyramid: static datasets,simulation benchmarks,and real-robot trials. Each tier trades off cost,controllability,and ecological validity. Across all tiers,we decompose evaluation space into four tightly coupled stages: perception,cognition,decision,and execution. Then,we identify fine-grained capability dimensions along with 20+ quantitative metrics. For every dimension,we analyze the following: 1) the underlying methodological assumptions,2) the boundary conditions under which the metric is valid,3) the intrinsic limitations (e.g.,sim-to-real physics mismatch,dataset bias,human-label variance),and 4) the cost-credibility curve that determines when a researcher should ascend from static to simulation,and finally to the real world.All evaluations in this study are based on a theoretical “perception-cognition-decision-execution loop” framework. In embodied AI,perception is the process by which an agent converts raw,asynchronous,and often noisy signals from heterogeneous physical sensors,such as RGB-D cameras,LiDAR,tactile skins,joint encoders,force/torque,and inertial measurement unit units,into a registered,time-aligned digital twin that supplies metric 3D geometry and semantic labels. Cognition is then built on this torrent to distill structured,memory-enabled,and generalizable world models that encode object affordances,physical common sense,and relational and causal links,grounding free-form human language into the resulting latent space. Decision-making combines these world models with an explicit task goal and searches across symbolic,kinematic,and dynamic levels to output a coherent,risk-aware,and resource-efficient plan that is expressed as a hierarchy of subgoals,trajectories,and low-level set points. Finally,execution transforms this plan into real-world mechanical energy through high-rate feedback controllers that track desired motions or forces while compensating for actuator nonlinearities,shock,thermal limits,and human safety constraints,closing the perception-cognition-decision-execution loop that defines embodied AI. In contrast with humans,whose perception-cognition-decision-execution loop is seamlessly integrated,currently available embodied agents suffer structural flaws in every stage. Consequently,evaluation must decouple the four blocks and score accuracy,error,and latency separately to pinpoint bottlenecks. Static tests,i.e.,“written exams”,run on a GPU with a few megabytes of images or point clouds in minutes,but erase dynamics and physics. Simulation,i.e.,the “job interview”,adds interactive physics in tens of seconds but still drifts away from reality. Real robot trials,i.e.,“practical tests”,cost thousands,take days to deploy and minutes per run,but expose every sensing drift,prediction error,or control delay,offering the ultimate ground truth before field deployment. Therefore,by integrating the three-tier pyramid of static test-simulation-real robot evaluation into the closed perception-cognition-decision-execution loop,this work delivers the first systematic quantification of coverage,cost,and accuracy trade-offs across more than 100 embodied AI benchmarks,establishing a reproducible,traceable,and extensible evaluation baseline that not only exposes the exact stage at which state-of-the-art systems fail,but also provides a principled road map for allocating research budgets,standardizing protocols,and certifying safety before real-world deployment,marking a pivotal transition from empirical competition to scientific measurement,and accelerating the field’s development toward trustworthy,deployable,and general-purpose embodied intelligence. The detailed static-dynamic-real embodied intelligence evaluation can be accessed at https://opencompass.org.cn/embodied-intelligence.
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |