Software

Autonomous driving has attracted great attention from both academics and industries. To realise autonomous driving, Deep Imitation Learning (DIL) is treated as one of the most promising solutions, because it improves autonomous driving systems by automatically learning a complex mapping from human driving data, compared to manually designing the driving policy. However, existing DIL methods cannot generalise well across domains, that is, a network trained on the data of source domain gives rise to poor generalisation on the data of target domain. In the present study, we propose a novel brain-inspired deep imitation method that builds on the evidence from human brain functions, to improve the generalisation ability of DNN so that autonomous driving systems can perform well in various scenarios. Specifically, humans have a strong generalisation ability which is beneficial from the structural and functional asymmetry of the two sides of the brain. Here, we design dual Neural Circuit Policy (NCP) architectures in DNN based on the asymmetry of human neural networks. Experimental results demonstrate that our brain-inspired method outperforms existing methods regarding generalisation when dealing with unseen data. Our source codes and pretrained models are available at https://github.com/Intenzo21/Brain-Inspired-Deep-Imitation-Learning-for-Autonomous-Driving-Systems.


Introduction
Autonomous vehicles have received the attention of various industries and academic institutions. Due to their potential to improve the safety and efficiency of the driving experience, they are expected to have a massive economic impact within the next decade [1]. A Machine Learning (ML) approach, known as end-to-end learning, has been used The code (and data) in this article has been certified as Reproducible by Code Ocean: (https://codeocean.com/). More information on the Reproducibility Badge Initiative is available at https://www.elsevier.com/physical-sciences-and-engineering/computer-science/journals. * Corresponding author. to achieve such an autonomous driving system [2]. This approach refers to the use of a single, self-contained system that automatically translates a sensory input, such as a captured image, into a set of instructions for driving. This type of learning is called Deep Imitation Learning (DIL) [3,4].  Deep Learning (DL) models. For example, a data-driven, off-policy Imitation Learning (IL) called Behavioural Cloning (BC) has been proposed to simulate human driving [5], which replaces deterministic behaviour with a human-driven paradigm that is sophisticated and flexible in its response to different visual environments [6]. This method also allows a final system to be built by improving its ability to generalise, that is, enabling the model to operate in a variety of circumstances [7].
Successful Deep Neural Networks (DNNs) with a single, task-specific algorithm should be able to generalise well across domains. Such an algorithm can enable simultaneous expression of generalisation ability by learning coherent representations and interpretable dynamics explanations of the world [8]. To some extent, DNNs, e.g., Convolutional Neural Network (CNN), have shown generalisation ability on image classification, recognition [9] and autonomous driving/collision avoidance [10,11] problems. However, this ability still needs to be further improved, especially in the field of autonomous driving, which requires reflecting a spatiotemporal understanding of dynamic driving settings. From this point onwards, the primary aim of the present study is to explore whether a certain architecture (built on evidence in the human brain) generalises better than other architectures in autonomous driving models, by examining a range of methodologies, including CNNs, single Neural Circuit Policy (NCP) networks, and Dual Neural Circuit Policy (DNCP) networks. We evaluate generalisation performance on diverse datasets, such as Udacity simulator and comma.ai.
In the present study, we employ brain-inspired architectures into DNNs, where we build various end-to-end DIL model architectures and put them into competition with the reference neural network mentioned in [2]. These network architectures are compared in terms of the training and generalisation Mean Squared Error (MSE) result between the model predictions and the actual steering angles provided by a human driver. The present brain-inspired models are built on the evidence of the asymmetry of neural connectivity in the human brain, because converging evidence from structural and functional neuroimaging studies has shown that the neural basis of human cognitive functions is extensively distributed in different regions across the two sides of the brain, which indicates the complexity of how humans respond to the environment via different levels of processing [12]. From an evolutionary perspective, the lateralized functions of human two brains increases with evolution [13]. Specifically, the present study adopts DIL in accomplishing dynamic vehicle control (e.g. steering angle, speed). A Udacity simulator [14] has been employed as a starting point which has been later replaced by the real-world comma.ai dataset [15]. Attempting to solve the task has required the utilisation of the NCP model, a designed sparse Recurrent Neural Network (RNN) based on the Liquid Time Constant (LTC) neuron with synapse model, which is inspired by C. elegans organism nervous system [16]. Therefore, the key contribution of our work is summarised as follows.
• We propose a brain-inspired DIL method to enhance the generalisation ability of autonomous driving models, where the design of DNCP architectures replicates the human brain left and right hemispheres. • Various modifications of the newly developed composite networks are explored for fully leveraging the strengths of the braininspired network architectures. • Our method is evaluated on the Udacity simulator and comma.ai datasets to demonstrate its performance against contemporary architectures.

Methodology
Various brain-inspired DNNs are created to imitate human driving behaviour and resolve the sophistication of autonomous lane-keeping by comparing them and finding the best performing model. Initially, a CNN, demonstrated as an effective approach in the field of autonomous vehicles, is built reckon with the Nvidia's framework (Fig. 1) as described in the paper of Bojarski et al. [2] In our work, this network is used as a feature extractor located at front-end of the stacked models. We add both single and dual NCP-based RNN structures to the CNN head to achieve the CNN-NCP and CNN-DNCP brain-inspired models. With these hybrid frameworks we aim to boost the sequential information extraction of the CNN and perform improved vehicle dynamics control (Fig. 2).
All of the developed models are trained by end-to-end learning, which directly links raw pixels from a single dash cam to steering instructions. In this paper, we use the autonomous lane following as the main criteria for assessing the performance of the developed models. We train the weights of our networks by minimising the MSE between the model-predicted steering angles and the ground truth of steering angles. In addition, we employ the Udacity simulator environment to better visualise our models' generalisation findings. This is accomplished by running the software to autonomously drive the automobile on both of the specified tracks using the pretrained architectures.

Experimental evaluation
Two kinds of performance -training and generalisation -are evaluated in this paper.
• Training Performance: In Fig. 3 we observe the training results of models on Udacity and Comma.ai datasets. Regarding the performance on Udacity dataset, the CNN model achieves an MSE result that is 2.64% lower than the best CNN-NCP hybrid model (Fig. 3a). In terms of loss performance on the comma.ai dataset, CNN once again outperforms the newly created brain-inspired models (Fig. 3b). In other words, the CNN model delivers an MSE that is 1.6% better than the CNN-DNCP v3 framework outcome. Please notice that these are training results. Having satisfactory model performance on data from the same source domain does not guarantee an equivalent generalisation efficiency of the model on data from another domain. Therefore, the generalisation ability of the models is also evaluated in the following section. • Generalisation Performance: With the training performance, the brain-inspired (CNN-DNCP v2) model can reduce 5.21% error when comparing with CNN model on Udacity dataset, which can be found in Fig. 4a. The models are trained on the Udacity lakeside track data and evaluated on the Udacity jungle map data to assess their generalisation performance (Udacity simulator evaluation recordings are available at our channel). In Figs. 4b and 4c, we evaluate the generalisation performance by using cloudy weather and night time data from the comma.ai dataset, where the models are trained on one kind of weather condition and evaluated on another kind of weather condition. The braininspired architecture (CNN-DNCP v4) is 5.82% and 34.05% more effective than the CNN model with regard to MSE on the cloudy and night time comma.ai recordings correspondingly. According to the experimental results, we observe that the brain-inspired hybrid models perform significantly better than the CNN reference architecture in terms of MSE on generalisation performance.

Impact
The direct beneficiaries of the software we present in this study are researchers in the autonomous driving area and automotive companies. The proposed method is modular, hence it can be easily customised into the applications of other researchers. More specifically, our paper introduces a brain-inspired DIL method to enhance generalisation ability in autonomous driving. The design of the proposed network is inspired by neuroscientific evidence on the functional asymmetry of the two sides of the human brain. In particular, we develop 5 variants of brain-inspired networks, which combine visual features extracted from CNN with temporal recursion networks (NCP) for imitating human driving behaviours. The developed networks are evaluated on two big autonomous driving datasets, including the Udacity simulation  dataset and the real-world comma.ai dataset, as a means of assessing their generalisation ability. Our extensive experiments indicate that hybrid brain-inspired network architectures can demonstrate promising performance on end-to-end autonomous driving tasks. Moreover, by using stacked composite models (which communicate time-related and sensory characteristics in addition to constant-length input representations), e.g., CNN-DNCP, an increased generalisation performance is achievable against conventional autonomous driving methods such as CNNs.
Our software provides a novel brain-inspired model architecture with considering the functional asymmetry of the two sides of the brain. Although the developed software is only evaluated on autonomous driving tasks for the moment, we believe it will attract more interest from researchers in various fields due to its promising performance as demonstrated in this paper. Furthermore, diverse robotic applications can also benefit from our work, i.e., robotic arm manipulation, teleoperation, etc.

Future improvements
There is still room for future work since the functions of layers of DL networks are insufficiently understood and the design of networks lacks theory support from psychology and neuroscience. Hence, forthcoming research might encompass linking a well-characterised neural architecture of social learning in humans to DL networks. Subsequently, examining whether the application of this psychological theory can provide novel insights for improving the efficiency of DL networks by reducing redundant layers of learning.