This work focuses on three time-series related tasks. The first task involves parameter inference, where the goal is to determine key characteristics of a sinusoidal wave, such as amplitude and frequency, directly from a set of observed data points. The second task is time-series forecasting, where the objective is to predict the future evolution of a sinusoidal wave based on historical data. The third task extends the application of time-series forecasting to stock market data.
This work focuses on three timeseries related tasks. The first task involves parameter inference, where the goal is to determine key characteristics of a sinusoidal wave, such as amplitude and frequency, directly from a set of observed data points. The second task is timeseries forecasting, where the objective is to predict the future evolution of a sinusoidal wave based on historical data. The third task extends the application of timeseries forecasting to stock market data.
To address these tasks, we evaluated the performance of the minGRU architecture \cite{feng2024rnnsneeded}, a simplification of GRU architecture \cite{chung2014empiricalevaluationgatedrecurrent}. We compare minGRU with other established architectures, assessing their strengths and limitations in these two tasks. Additionally, in the second task, training time versus dataset size was evaluated for minGRU, GRU, and LSTM \cite{10.1162/neco.1997.9.8.1735} to compare their computational efficiency.
...
...
@@ -38,8 +38,8 @@ The datasets used in this exploration are tailored to the specific requirements
\subsection{Parameter Inference}
For the parameter inference task, a custom synthetic dataset was generated, where each sample consists of 100 data points representing a sinusoidal wave with varying amplitude and frequency.
For the time-series forecasting of sinusoidal waves, another custom synthetic dataset was generated. Each sample in this dataset represents a sinusoidal wave, consisting of 100 data points as input and one additional target data point for prediction.
For the timeseries forecasting of sinusoidal waves, another custom synthetic dataset was generated. Each sample in this dataset represents a sinusoidal wave, consisting of 100 data points as input and one additional target data point for prediction.
\subsection{Stock Market Forecasting}
For the stock market forecasting task, real-world stock data for IBM was sourced using the \texttt{yfinance} library. The dataset spans from \texttt{2014-01-01} to \texttt{2024-04-01} and includes daily stock prices. Preprocessing focused on the closing prices.
...
...
@@ -59,17 +59,17 @@ For the parameter inference task, the models are evaluated using the Mean Absolu
\end{itemize}
The GRU, LSTM, and Transformer-encoder models were implemented using the PyTorch \texttt{nn} library, while the minGRU architecture was implemented from scratch based on its original paper.
For time-series forecasting of sinusoidal waves, models are evaluated using the normalized Root Mean Squared Error (nRMSE). The comparison includes the following models:
For timeseries forecasting of sinusoidal waves, models are evaluated using the normalized Root Mean Squared Error (nRMSE). The comparison includes the following models:
All models except minGRU are implemented and evaluated using the DARTS time-series forecasting library. This task focuses on comparing the ability of the models to predict future values of sinusoidal waves based on historical data. For minGRU, autoregressive prediction was used, where each model used its previous predictions as inputs to forecast future values. During training, a sliding window approach was employed, where the model used the previous 100 data points to predict the next data point.
All models except minGRU are implemented and evaluated using the DARTS timeseries forecasting library. This task focuses on comparing the ability of the models to predict future values of sinusoidal waves based on historical data. For minGRU, autoregressive prediction was used, where each model used its previous predictions as inputs to forecast future values. During training, a sliding window approach was employed, where the model used the previous 100 data points to predict the next data point.
During evaluation of training time versus dataset size in time-series forecasting of sinusoidal waves, a different implementation of LSTM and GRU was used to account for the high optimization of PyTorch's default implementation \cite{yiasemis2023rnnfromscratch}.
During evaluation of training time versus dataset size in timeseries forecasting of sinusoidal waves, a different implementation of LSTM and GRU was used to account for the high optimization of PyTorch's default implementation \cite{yiasemis2023rnnfromscratch}.
\subsection{Stock Market Forecasting}
For forecasting stock market data, the models are also evaluated using nRMSE. The following models are compared:
The GRU model achieved the lowest error rates for both amplitude and frequency estimation. MinGRU and LSTM show comparable performance, while the Transformer had the highest errors in both categories.
The dataset used for this task consisted of 5000 samples, where each sample had 100 data points plus one target point. The maximum amplitude and frequency were both set to 10. The models were trained on 4000 samples for 100 epochs and their performance was evaluated on test set, comprised of the remaining 1000 samples. The averaged results of the 1000 samples are presented in Table~\ref{tab:sinusoidal_forecasting}.
The evaluation of time-series forecasting models for sinusoidal waves includes a comparison of training times across different dataset sizes.
The evaluation of timeseries forecasting models for sinusoidal waves includes a comparison of training times across different dataset sizes.
\begin{figure}[h]
\centering
...
...
@@ -147,7 +147,7 @@ The evaluation of time-series forecasting models for sinusoidal waves includes a
\label{fig:training_time_comparison}
\end{figure}
As shown in Figure~\ref{fig:training_time_comparison}, the minGRU consistently demonstrates faster training times compared to the LSTM and GRU, particularly as the dataset size increases. This efficiency makes minGRU a suitable choice for large-scale time-series datasets.
As shown in Figure~\ref{fig:training_time_comparison}, the minGRU consistently demonstrates faster training times compared to the LSTM and GRU, particularly as the dataset size increases. This efficiency makes minGRU a suitable choice for large-scale timeseries datasets.
\subsection{Stock Market Forecasting}
For stock market data forecasting, the models were evaluated using nRMSE. The results on the test set for IBM stock data are shown in Figure~\ref{fig:stock_market_pred}.
...
...
@@ -167,15 +167,15 @@ However, GRU surprisingly outperforms minGRU by a somewhat large margin in estim
In the stock market forecasting task, minGRU achieved competitive performance compared to established models, although specialized models like Prophet still exhibited superior accuracy.
Overall, these findings suggest that minGRU is a viable and efficient alternative for time-series analysis, particularly in scenarios where computational resources and training speed are critical.
Overall, these findings suggest that minGRU is a viable and efficient alternative for timeseries analysis, particularly in scenarios where computational resources and training speed are critical.
The minGRU architecture offers a significantly more efficient and scalable alternative to traditional GRU models, demonstrating competitive performance in time-series forecasting tasks.
The minGRU architecture offers a significantly more efficient and scalable alternative to traditional GRU models, demonstrating competitive performance in timeseries forecasting tasks.
This analysis supports the adoption of minGRU in various time-series forecasting scenarios, where it consistently matches or outperforms other architectures, such as Transformer and N-BEATS, especially in environments with limited computing resources.
This analysis supports the adoption of minGRU in various timeseries forecasting scenarios, where it consistently matches or outperforms other architectures, such as Transformer and N-BEATS, especially in environments with limited computing resources.
Further analysis could explore the application of minGRU in diverse domains such as weather prediction and anomaly detection, as well as evaluate its scalability and performance in natural language processing tasks to determine whether it can compete effectively with Transformer-based architectures.
