Aiming at the problems of poor scheduling scheme, low utilization rate and high cost of unmanned trucks in large open-pit coal mines, an optimization method based on digital twin simulation was proposed, and a digital twin system framework for unmanned trucks was constructed. The digital twin simulation model was built by AnyLogic software to realize the real-time mapping of physical and virtual space. A mathematical model aiming at minimizing transportation cost, minimizing total transportation time and maximizing truck utilization was established, and a Quantum Enhanced Particle Swarm Optimization (QEPSO) algorithm was designed to solve the model. The verification results of an open-pit coal mine in Xinjiang show that compared with the traditional particle swarm optimization algorithm, the improved particle swarm optimization algorithm and the genetic algorithm, the maximum completion time of the QEPSO algorithm is reduced by 5.84%-11.34%, the total transportation cost is reduced by 5.12%-10.49%, and the truck utilization rate is increased by 4.32%-9.14%, with a faster convergence speed and better stability. The digital twin scheduling scheme shortens the task completion time by 73 min compared with the traditional scheme, effectively improves the transportation efficiency. The study provides a feasible technical path and method support for the intelligent scheduling of unmanned trucks in open-pit coal mines.