Researchers from Baidu Research Robotics and Auto-Driving Lab (RAL) and the University of Maryland, College Park, have introduced an autonomous excavator system (AES) that can perform material loading tasks for a long duration without any human intervention while offering performance closely equivalent to that of an experienced human operator.

AES is among the world's first uncrewed excavation systems to have been deployed in real-world scenarios and continuously operating for over 24 hours, bringing about industry-leading benefits in terms of enhanced safety and productivity.

The researchers described their methodology in a research paper published on June 30, 2021, in Science Robotics.

"This work presents an efficient, robust, and general autonomous system architecture that enables excavators of various sizes to perform material loading tasks in the real world autonomously," said Dr. Liangjun Zhang, corresponding author and the Head of Baidu Research Robotics and Auto-Driving Lab.

Excavators are vital for infrastructure construction, mining, and rescue applications. The global market size for excavators was $44.12 billion in 2018 and is expected to grow to $63.14 billion by 2026.

Given this projected market increase, construction companies worldwide are facing hiring shortages for skilled heavy machinery operators, particularly excavators. Additionally, COVID-19 continues to exacerbate the labor shortage crisis. Another contributing factor is the hazardous and toxic work environments that can impact the health and safety of on-site human operators, including cave-ins, ground collapses, or other excavation accidents that cause approximately 200 casualties per year in the U.S. alone.

The industry is therefore taking a scientific approach and looking to create excavator robots that can provide groundbreaking solutions to meet these needs, making the development of systems such as AES a growing trend alongside the implementation of other robots in manufacturing, warehouses, and autonomous vehicles.

While most industry robots are comparatively smaller and function in more predictable environments, excavator robots are required to operate in an extensive range of hazardous environmental conditions. They must be able to identify target materials, avoid obstacles, handle uncontrollable environments, and continue running under difficult weather conditions.

AES uses accurate and real-time algorithms for perception, planning, and control alongside a new architecture to incorporate these capabilities for autonomous operation. Multiple sensors – including LiDAR, cameras, and proprioceptive sensors – are integrated for the perception module to perceive the 3D environment and identify target materials, along with advanced algorithms such as a dedusting neural network to generate clean images.

With this modular design, the AES architecture can be effectively utilized by excavators of all sizes – including 6.5 and 7.5 metric ton compact excavators, 33.5 metric ton standard excavators, and 49 metric ton large excavators – and is suitable for diverse applications.

To evaluate the efficiency and robustness of AES, researchers teamed up with a leading equipment manufacturing company to deploy the system at a waste disposal site, a toxic and harmful real-world scenario where automation is in strong demand. Despite the challenging assignment, AES was able to continuously operate for more than 24 hours without any human intervention. AES has also been tested in winter weather conditions, where vaporization can pose a threat towards the sensing performance of LiDAR. The amount of materials excavated, in both wet and dry form, was 67.1 cubic meters per hour for a compact excavator, which is in line with the performance of a traditional human operator. "AES performs consistently and reliably over a long time, while the performance of human operators can be uncertain," said Dr. Zhang.

Researchers also set up ten different scenarios at a closed testing field to see how the system performed in numerous real-world tasks. After testing a variety of large, medium-sized, and compact excavators, AES was ultimately proven to match the average efficiency of a human operator in terms of the amount of materials excavated per hour.

"This represents a key step moving towards deploying robots with long operating periods, even in uncontrolled indoor and outdoor environments," said Dr. Dinesh Manocha, Distinguished University Professor of Computer Science and Electrical and Computer Engineering at the University of Maryland, College Park.

Going forward, Baidu Research RAL will continue to refine core modules of AES and further explore scenarios where extreme weather or environmental conditions may be present.

Baidu has been collaborating with several of the world's leading construction machinery companies to automate traditional heavy construction machinery with AES. "We aim to leverage our robust and secure platform, infused with our powerful AI and cloud capabilities to transform the construction industry," said Dr. Haifeng Wang, CTO of Baidu.

The research paper, "An autonomous excavator system for material loading tasks,” L. Zhang, J. Zhao, P. Long, L. Wang, L. Qian, F. Lu, X. Song, D. Manocha, was published in the journal Science Robotics on June 30, 2021. Watch the demonstration video of AES on YouTube.

About Baidu Research Robotics and Auto-Driving Lab (RAL)

Baidu Research RAL focuses on robotics and computer vision research, applying innovative research to autonomous driving, industry, and service robots. By combining expertise in robotics, computer vision, machine learning, simulation and systems, and studying holistic robotics solutions, RAL strives to be the first to bring research lab results, either internally or externally, to market.

Media Contact

Tony Peng, 573-999-5521, juntaopeng@baidu.com