Multimodal large language models (MLLMs) have made great strides in visual understanding, yet precise 3D spatial reasoning that respects physical constraints remains challenging. Furniture assembly demands not only extracting step‑level actions from diagrammatic manuals, but also converting semantic attachment relations into 6‑DOF pose updates so that parts can physically interact with the environment and previously assembled components.
We introduce AssemState, a zero‑shot framework guided by both manuals and physical state. It first employs anchor‑guided boundary assembly states to split manual pages into single‑part operations and reconstruct an assembly tree. Then, an iterative after‑state feedback refinement loop drives successive SE(3) pose updates and corrections, while simulation‑based release tests validate physical plausibility.
Compared with the strongest prior baseline, AssemState raises assembly‑tree recovery F1 from 38.58% to 62.80% and Tree Exact Match from 28.24% to 53.92%. On 243 independently evaluated part‑level actions, the iterative refinement boosts judge‑accepted operations from 0 to 5.3% and reduces mean Chamfer distance from 5.4111 to 1.7744.
Nevertheless, visually plausible candidate poses may still suffer from collisions, floating, mirror‑orientation errors, incomplete seating, and wrong‑side attachment, indicating that MLLMs are still limited in spatial relationship reasoning.
Review: AssemState demonstrates substantial gains in operation‑structure recovery and local pose metrics by leveraging manual cues and physical feedback, yet achieving fully reliable 3D assembly will require further advances in the model’s spatial reasoning capabilities.