1. The validation problem
Heavy mobile equipment operates with high inertia, long stopping distances and limited room for experimental failure. Field testing is essential, but it is a poor place to discover basic software defects. Simulation provides a deterministic environment where the same edge case can be run repeatedly against every software revision.
2. Scenario model
A scenario should include map state, road geometry, machine type, payload state, weather or visibility assumptions, dynamic actors and injected faults. Each run should produce machine-state traces and explicit pass/fail criteria rather than relying only on visual inspection.
3. Software-in-the-loop
Software-in-the-loop (SIL) runs the production autonomy code against simulated sensors and machine dynamics. It is useful for regression testing mission logic, planning, fallback behaviour and state transitions. Deterministic replay lets engineers compare two software versions against identical inputs.
4. Hardware-in-the-loop
Hardware-in-the-loop (HIL) adds real compute, network interfaces or vehicle-control hardware while the machine remains simulated. This exposes timing, protocol and integration defects that pure software simulation may miss.
5. Fault injection
Validation should deliberately introduce GNSS degradation, stale perception data, sensor dropouts, blocked routes, communications loss and inconsistent machine feedback. The desired result is not that the mission always continues; it is that the system enters the correct degraded or fallback state.
6. Coverage by operational design domain
Tests should be tagged against the ODD they cover. A system validated for dry daylight haul roads should not quietly inherit claims for dust, heavy rain, underground operation or mixed public traffic. Explicit ODD coverage keeps engineering claims tied to evidence.
7. Practical metrics
- mission completion and abort rate;
- minimum separation distance;
- localisation confidence and error;
- planner replan frequency;
- control tracking error;
- fallback-state correctness;
- communications-loss response;
- scenario regression count.