Watching skilled excavator operators work is almost hypnotic. They make it look so easy that the machine seems to move like an extension of the operator's own arm. That intuition usually takes years to build, but a new controller from MIT engineers compresses that learning curve to a single session.
Excavators are among the most common machines on any job site, used to dig foundations, clear debris, and level ground. Yet mastering one is anything but simple. Standard joystick controls force an operator to translate every wrist flick into cabin rotation, arm extension, or bucket movement, an unnatural mapping that engineers call a mental model. Building that mental model typically takes years of practice.
A team led by Hermano Krebs, a principal research scientist in MIT's Department of Mechanical Engineering, took a different approach: instead of training people to think like a machine, they built a machine that understands how the human body already moves.
"This is a more intuitive way to command the machine," Krebs says. "With this new interface, we can eliminate a lot of the mental maps that an operator would need to build in order to operate an excavator." The team's findings appear in the Journal of Computing and Civil Engineering.
The device, called the World-Space Interface (WSI), is a miniature excavator arm and bucket that the operator physically grips and moves the way they'd move their own arm while digging by hand: extending it, rotating it, closing a simulated fist over the dirt. A wraparound six-panel display projects a virtual excavator that mirrors every gesture instantly.
"‘World-space’ refers to everything in the world that is outside of yourself, or in this case, outside of the excavator’s cab," Krebs explains. "Normally, operators have to build a mental map of how to manipulate things in the world-space. But now, we can just mime picking up rocks or dirt, and the computer will do that translation to the world-space for us."
The researchers don't see this staying confined to training simulators. Krebs envisions the same controller mounted directly in a real cabin, functioning almost like a prosthetic, an oversized extension of the operator's arm.
"Instead of having joysticks, you might have this miniature arm on the side, where the operator would place their own arm, kind of like an exoskeleton, which would allow them to operate the excavator in the cab," he says. "If work has to be done in a difficult or unsafe environment, you could have an operator sitting off-site in a trailer and using this arm to remotely tele-operate the excavator."
To test whether the concept held up outside the lab, the team ran training experiments with both expert and novice volunteers, who used the WSI as well as a conventional joystick-based simulator. The researchers built 15 virtual scenarios, covering construction sites, highways, forest roads, riverbanks, mining areas, and urban and rural settings. Tasks included scooping and dumping sand or gravel, digging and grading trenches, clearing debris from roads, removing tree branches from water edges, and breaking up rocks. Volunteers trained one hour a day for seven days, mirroring a typical weeklong excavator driving course.
With joysticks, novices performed consistently worse than experts, though they did improve over the training period. With the World-Space Interface, that gap vanished entirely: novices matched expert performance from the very first session. "In this case, joysticks are a non-intuitive way to control and coordinate the machine," says study co-author and MIT postdoc Joao Buzzatto. "This is the first interface that does not require me to command the excavator with joysticks."
The system currently gives the operator no physical feedback, so the team is now working to add haptics (force feedback that simulates the sense of touch) to the mechanical arm. The idea is that as an operator mimes picking up a pile of rocks, the arm would generate a force in response, as if the operator could feel the heaviness of the rocks, confirming that the excavator is indeed picking them up. "Haptics would make this an even more intuitive system," says co-author and visiting engineer Solmon Jeong.
MIT isn't alone in this race. Caterpillar, Hyundai, and Komatsu are all developing their own simulators for training and eventual remote operation, but these remain largely based on traditional joystick controllers that still take time to learn. Whether this approach eventually leads toward mecha-robots like Mazinger Z (exoskeleton-controlled machines) or fully autonomous excavators with no cabin occupant at all remains an open question, but the gap between human gesture and machine motion is clearly starting to close.
Source: MIT
