A framework for robotic excavation and dry stone construction using on-site materials

Automated building processes that enable efficient in situ resource utilization can facilitate construction in remote locations while simultaneously offering a carbon-reducing alternative to commonplace building practices. Toward these ends, we present a robotic construction pipeline that is capable of planning and building freeform stone walls and landscapes from highly heterogeneous local materials using a robotic excavator equipped with a shovel and gripper. Our system learns from real and simulated data to facilitate the online detection and segmentation of stone instances in spatial maps, enabling robotic grasping and textured 3D scanning of individual stones and rubble elements. Given a limited inventory of these digitized stones, our geometric planning algorithm uses a combination of constrained registration and signed-distance-field classification to determine how these should be positioned toward the formation of stable and explicitly shaped structures. We present a holistic approach for the robotic manipulation of complex objects toward dry stone construction and use the same hardware and mapping to facilitate autonomous terrain-shaping on a single construction site. Our process is demonstrated with the construction of a freestanding stone wall (10 meters by 1.7 meters by 4 meters) and a permanent retaining wall (65.5 meters by 1.8 meters by 6 meters) that is integrated with robotically contoured terraces (665 square meters). The work illustrates the potential of autonomous heavy construction vehicles to build adaptively with highly irregular, abundant, and sustainable materials that require little to no transportation and preprocessing.