Enhancing multi-scenario applicability of freeway variable speed limit control strategies using continual learning.

Variable speed limit (VSL) control benefits freeway operations through dynamic speed limit adjustment strategies for specific operation scenarios, such as traffic jams, secondary crash prevention, etc. To develop optimal strategies, deep reinforcement learning (DRL) has been employed to map the traffic operation status to speed limits with the corresponding control effects. Then, VSL control strategies were obtained based upon memories of these complex mapping relationships. However, under multi-scenario conditions, DRL trained VSL faces the challenge of performance decay, where the control strategy effects drop sharply for early trained "old scenarios". This so-called scenario forgetting problem is attributed to the fact that DRL would forget the learned old scenario mapping memories after new scenario trainings. To tackle this issue, a continual learning approach has been introduced in this study to enhance the multi-scenario applicability of VSL control strategies. Specifically, a gradient projection memory (GPM) based neural network parameter updating method was proposed to keep the mapping memories of old scenarios during new scenario trainings by imposing constraints on the direction of gradient updates for new tasks. The proposed method was evaluated using three typical freeway operation scenarios developed in the simulation platform SUMO. Experimental results showed that the continual learning approach has substantially reduced the performance decay in old scenarios by 17.76% (valued using backward transfer metrics). Furthermore, the multi-scenario VSL control strategies successfully reduced the speed standard deviation and average travel time by 28.77% and 7.25% respectively. Moreover, the generalization of the proposed continual learning based VSL approach were evaluated and discussed.