Spiking neural networks have gained traction as both a tool for neuroscience research and a new frontier in machine learning. A plethora of neuroscience literature exists exploring the realistic simulation of neurons, with complex models re- quiring the formulation and integration of ordinary differential equations. Overcoming this challenge has led to the exploration of various numerical integration techniques with the goal of highly stable and accurate simulations. In contrast, training spiking neural networks is often done with simple leaky integrate-and-fire models and rudimentary integration methods such as the forward-Euler method. In this research we explore how more complex numerical integration methods, borrowed from neuroscience research, affect the training of networks based on current-based leaky integrate- and-fire neurons. We derive equations required for the integration process and suggest the use of spike time interpolation. Furthermore, we pro- vide insights into applying backpropagation on numerically integrated networks and highlight possible pitfalls of the process. We conclude that numerically integrated networks can achieve training accuracies close to their theoretical limits, with good convergence and training time characteristics. Specifically, high order integrations achieve robust and computationally viable training. Additionally, we explore the effects of spike time interpolation on network accuracy and use our findings to pro- vide insights into the role of different integration parameters on the effective training of spiking neural networks.

The increasing computational costs of training deep learning models have drawn more and more attention towards more power-efficient alternatives such as spiking neural networks (SNNs). SNNs are an artificial neural network that mimics the brain’s way of processing information. These models can be even more power-efficient when run on specialized hardware like digital neuromorphic chips. These chips are designed to handle the unique processing needs of SNNs like sparse and event-driven computations. A lot of the state-of-the-art performance of SNNs in recent research has been achieved through supervised learning models that leverage intricate error backpropagation techniques. These models impose specific constraints on the network and rely on continuous time to facilitate the backpropagation process. However, this fix imposes a new challenge when converting these mechanisms on a neuromorphic chip. Because time is discrete on hardware numerical errors can be introduced as we can not calculate the infinitely precise value of variables depending on time. This work proposes a time discretization technique that allows for fast and stable backpropagation and analyses the effects of it on the efficiency of the SNN. Specifically, we look at sparsity, latency, and accuracy as the main factors that explain the power efficiency of the model. The experiments show that choosing a suitable time step size can improve sparsity while maintaining a high level of accuracy. However, too large values affect the network’s ability to learn.

Efficient traffic forecasting is an important component of modern traffic management systems, enabling real-time route guidance and traffic control. Graph Neural Networks (GNN) have demonstrated state-of-the-art performance in this domain due to their ability to capture spatial and temporal dependencies in complex traffic data. However, GNNs typically require extensive historical data and are highly dependent on the specific road structure of the training region, posing challenges for their application in areas lacking such data. This study explores the transferability of GNN models in traffic forecasting, specifically how a GNN, trained in the region with long-horizon historical data, performs when applied to structurally different regional scenarios without historical data. The research investigates the impact of spatial differences between regions on the model's performance. The paper examines multiple metrics for regional similarity between training and transfer regions and shows their correlation with the transferred model's performance.

Spiking Neural Networks (SNN) represent a distinct class of neural network models that incorporate an additional temporal dimension. Neurons within SNN operate according to the Leaky Integrate-and-Fire principle, governed by ordinary differential equations. Inter-layer neuronal communication occurs through spike propagation when membrane potentials reach a specified threshold. This unique mechanism renders conventional Artificial Neural Network (ANN) design principles and learning rules inapplicable to SNN. This study presents a theoretical investigation into the impact of time discretization on SNN performance in real-world tasks. We present a network architecture based on the Error Backpropagation Through Spikes model. This approach allows for the transformation of the original system of differential equations into a system of difference equations across multiple time steps. We evaluate our model's performance on the MNIST dataset and identify several phenomena that influence accuracy. Our analysis primarily focuses on gradient dynamics and spectral properties of the voltage signals. These findings contribute to a deeper understanding of SNN behavior and potential optimization strategies.

Spiking neural networks (SNNs) aim to utilize mechanisms from biological neurons to bridge the computational and efficiency gaps between the human brain and machine learning systems. The widely used Leaky-Integrate-and-Fire (LIF) neuron model accumulates input spikes into an exponentially decaying membrane potential and generates a spike when this potential exceeds a set threshold. A LIF neuron is characterized by learnable input weights and a manually selected membrane time constant 𝜏, which determines the decay rate of a neuron's membrane potential. Previous work introduced the Parametric LIF (PLIF) neuron model with a learnable 𝜏. However, the published experiments only featured a single 𝜏 per spiking layer. This leaves space for exploration of the effect of having a learnable 𝜏 for each neuron. The importance of 𝜏 is given by the trade-off it inherently introduces, prioritizing the neuron's capability to convey information about spatial or temporal features. This work examines the effect of introducing a learnable 𝜏 per neuron with a new initialization method and a new regularization term which incentivizes low variance in each PLIF layer. The experiments are done using the DVS128 Gesture dataset and compared to a baseline model from the original paper introducing the PLIF neuron model. Results are inconclusive but suggest that introducing 𝜏 per neuron does not have a significant effect on the accuracy of a spiking neural network. Moreover, the evolution of 𝜏 during training exhibits interesting behavior and leads to two new hypotheses.

This paper explores the scalability of Graph Neural Networks (GNNs) in the context of traffic forecasting, a critical area for improving urban mobility and reducing congestion. Despite GNNs’ demonstrated effectiveness in handling complex spatiotemporal dependencies in traffic data, scaling them to large road networks remains challenging due to increased computational requirements. This study aims to evaluate how the accuracy and computational cost of a state-of-the-art traffic forecasting GNN, the Decoupled Dynamic Spatio-Temporal Graph Neural Network, change with varying road network sizes and complexities (i.e., sensor density). Using two real-world datasets, three experiments are conducted: scaling map area, scaling graph complexity, and testing the geographic location effect. Findings show that larger graphs generally improve accuracy and GPU efficiency. Moreover, geographic location affects accuracy, whereas sensor density has minimal impact.

With the rapid increase in popularity of graph neural networks (GNNs) for the task of traffic forecasting, understanding the inner workings of these complex models becomes more important. This experiment aims to deepen our understanding of the importance that the training data has in regards to the ability of GNNs to accurately predict traffic. By repeatedly training the same GNN model with different training datasets spanning over various time frames and comparing standard performance metrics computed based on the predictions performed by the model, this paper concludes that while using less training data leads to a slight decrease in performance, this is heavily dependent on the quality of the dataset. If the data gathering process is short and the sensors are not properly maintained, GNNs are not able to accurately predict traffic. On the other hand, if the data gathering process goes well and there are few missing values, GNNs perform well even when trained with smaller amounts of historical data.

Traffic forecasting is key to improving urban transport and reducing congestion and pollution. While advanced models like Graph Neural Networks (GNNs), can capture complex patterns in traffic flow, they are resource-intensive and do not scale well. This problem can be mitigated by using simpler models that are less influenced by the size of the road network, making them more practical for real-world applications. This study investigates whether simpler network-based models, particularly Long Short-Term Memory (LSTM) networks, can match or surpass the performance of GNNs, such as the Diffusion Convolutional Recurrent neural network (DCRNN), in specific scenarios. Using popular benchmark datasets, we compared the performance of the LSTM and DCRNN models under different conditions, including different sensor distributions and prediction horizons. The results indicate that while DCRNN highly outperforms LSTM with numerous sensors and longer prediction horizons, LSTM gives promising results with fewer sensors and shorter horizons. In this scenario, the difference in performance is minimal regardless of the location of sensors, also offering significant computational efficiency. These findings suggest that LSTM models may be a practical alternative for traffic forecasting in resource-constrained scenarios, providing a path to more efficient urban traffic management.

Traffic forecasting is a branch of spatiotemporal forecasting that involves predicting future traffic speed or volume based on real-world data. It has a significant impact on urban mobility and quality of life, as it directly contributes to improving traffic management and trip planning. This study evaluates the performance of Graph Neural Networks (GNNs) in handling long-term forecasting, defined as predictions made up to 10 hours ahead. It addresses the evolution of performance and factors that may impact accuracy, such as fluctuations in traffic speed and road network configurations. The experiments are done using subsets of a benchmark dataset for traffic forecasting and a state-of-the-art GNN model. The findings showcase a logarithmic growth in prediction errors and the presence of two types of traffic jams—sudden and regular—along with their impact on prediction accuracy. Furthermore, the results highlight the complexity of quantifying the influence a factor has on forecasting performance, such as road network configuration or missing values.