We developed an autoregressive model based on MotionLM, which can be used for prediction, planning, and agent simulation in autonomous vehicles. It is noteworthy that all agent trajectories in the demo are scene-consistent trajectories obtained from a single inference of the model, and the fundamental actions constituting these trajectories are fully differentiable. This characteristic enables our model to also function as a vehicle motion simulator.
Traditional reinforcement learning often suffers from inefficiencies when interacting with environments. However, in our simulator, the interaction between agents (i.e., interaction with the environment) is transformed into inference and sampling from the model, which can be executed on GPUs. This approach significantly enhances the sampling efficiency.