Stationarity:

• In time series analysis, stationarity is an essential concept. A time series that exhibits constant statistical attributes over a given period of time is referred to as stationary. The modelling process is made simpler by the lack of seasonality or trends. Two varieties of stationarity exist:
  • Strict Stationarity: The entire probability distribution of the data is time-invariant.
  • Weak Stationarity: The mean, variance, and autocorrelation structure remain constant over time.
• Transformations like differencing or logarithmic transformations are frequently needed to achieve stationarity in order to stabilise statistical properties.
• Let’s check the stationarity of the ‘logan_intl_flights’ time series. Is the average number of international flights constant over time?
• 
• A visual inspection of the plot will tell you it’s not. But let’s try performing an ADF test.
• 
• The Augmented Dickey-Fuller (ADF) test is a prominent solution to this problem. Using a rigorous examination, this statistical tool determines whether a unit root is present, indicating non-stationarity. The null hypothesis of a unit root is rejected if it is less than the traditional 0.05 threshold, confirming stationarity. The integration of domain expertise and statistical rigour in this comprehensive approach improves our comprehension of the dataset’s temporal dynamics.
• The ADF test starts with a standard autoregressive (AR) model: Y(t) = pY(t-1) + α + E(t);  where Y(t)​ is the value of the time series at time t, p is the autoregressive coefficient, $\alpha$ is a constant, and E(t)​ is a white noise error term. The presence of a unit root is indicated by p=1.
• The ADF test extends this by including lagged differences of the time series:ΔY(t)=β(Y(t−1)​−Y(t−2​))+γY(t−1)​+α+E(t​)
  • ΔYt​: Differenced series at time $t$.
  • Yt-1​: Lagged value at time $t-1$.
  • $\beta$: Coefficient of the lagged difference term that is Y(t−1)​−Y(t−2​) representing the change in the time series from the previous to the second previous time step
  • $\gamma$: Coefficient of the lagged level term that is Y(t-1) representing the value of the time series at the previous time step.
  • $\alpha$: Constant term.
  • Et​: White noise error term at time $t$.
• The difference between a time series variable’s current value and its value at a prior time point (referred to as the “lagged value”) is known as the “lagged difference“.
• The ADF test typically involves choosing the number of lags ($p$) to include in the model to account for autocorrelation in the time series. The corrected equation would be:
  • ΔYt​= β1​(ΔYt−1)​ + β2(​ΔYt−2)​ +….+ βp​(ΔYt−p)​ + γYt−1​+α+εt
  • $\beta 1,\beta 2,\dots ,\beta p$ are the coefficients for the lagged differences up to lag $p$. The parameter $p$ is determined based on statistical criteria or domain knowledge.​
• You might wonder what exactly this p does in relation to the our ADF test. It is nothing but the number of lags of the differenced series that are included in the model. These lags are added to the time series in order to account for autocorrelation. The correlation between a time series and its own historical values is known as autocorrelation. Any residual autocorrelation in the differenced series is better captured by the model’s inclusion of lag differences.
• The goal is to find an appropriate value for $p$ such that the differenced series ΔYt​ becomes stationary.
• With a Unit Root (p$=1$):
  • If p$=1$, the autoregressive process has a unit root, leading to non-stationarity.
  • With p = 1, Yt​ is solely dependent on the immediately preceding valueYt−1​ plus a random noise term E.
  • In the ADF equation with ΔY(t), the differenced series is equal to the lagged difference plus a term involving Y(t-1), a constant α, and a white noise error term E​. The presence of Y(t−1)​ in this equation, along with the unit root in the autoregressive process, contributes to the non-stationarity of the time series.
• Without a Unit Root ($\rho <1$):
  • If $\rho <1$, the autoregressive process does not have a unit root, indicating stationarity.
  • The series lacks a stochastic trend, and its statistical properties are more likely to be constant over time.
  • This would imply that the model only incorporates a small number of lag differences.
  • If the p-value is smaller, it could indicate that the autocorrelation structure is modelled with fewer historical observations.
  • The results of the hypothesis test and the corresponding p-value are used to determine whether there is stationarity or non-stationarity; a lower p does not necessarily indicate non-stationarity.

Differencing:

• We have already seen how differencing works, it is nothing but a method for achieving stationarity in a time series is differencing. It entails calculating the variations between subsequent observations. 
• By removing seasonality and trends, this procedure improves the time series’ analytical accessibility. The first-order difference is calculated as Y (t) – Y (t−1) 
• How does the first-order differenced series look compared to the original series?
• 
• Has differencing stabilized the statistical properties of the time series? A visual assessment tells me that the differencing has been successful in making the series more stationary, as I observe a stabilisation of the mean, a removal of trends. But determining stationarity may not always be possible through visual inspection.
• NO!
• Let’s try a higher-order differencing method, more precisely, the second-order differencing and then run the ADF test once more.
• YES!

Autocorrelation Function (ACF):

• A statistical tool called the Autocorrelation Function (ACF) calculates the correlation between a time series and its own lagged values. It facilitates the discovery of trends, patterns, and seasonality in the data. Correlation coefficients for various lags are shown in the ACF plot, making it possible to identify important lags and possible autocorrelation structures in the time series.
• The ACF at lag k is the correlation between the time series and itself at lag k.
• Mathematically, ACF(k)=Cov(Yt​,Yt−k​)/ root [Var(Yt​).Var(Yt−k​)]
• Positive ACF: High values at one time point may be related to high values at another time point if the ACF is positive, indicating a positive correlation.
  
• Negative ACF: An inverse relationship between values at different times is suggested by a negative ACF, which denotes a negative correlation.
  
• ACF values in the neighbourhood of zero signify little to no linear correlation.​
• Lag Structure: Plotting the ACF against various lags usually reveals the correlation structure over time.
  Correlation between adjacent observations is represented by the first lag (lag 1), correlation between observations two time units apart by the second lag (lag 2), and so on.
• Significance Analysis: ACF values are frequently subjected to significance testing in order to ascertain their statistical significance. Each lag is plotted with confidence intervals and an ACF value may be deemed statistically significant if it is outside of these intervals.
• Seasons and Recurring Trends: Seasonality or recurring patterns in the time series may be indicated by peaks or valleys in the ACF plot at particular lags. A notable peak at lag 12 in a monthly time series, for instance, can indicate annual seasonality
• How to Interpret Noisy ACF:  Autocorrelation values may indicate that the time series is less significant and more random if they oscillate around zero without any discernible pattern.
• Identification of the Model: In time series models, the autoregressive (AR) and moving average (MA) terms can have potential parameters that can be found using the ACF plot. An AR term may be required if there is positive autocorrelation at lag 1, whereas an MA term may be required if there is negative autocorrelation at lag 1.
• To give you a condensed explanation:
  • The presence of a significant positive peak at lag 1 implies a correlation between the value at time t and the value at time t−1, suggesting the possibility of autoregressive behaviour.
  • The presence of a noteworthy negative peak at lag 1 implies a negative correlation between the value at time t and the value at time t−1, suggesting the possibility of moving average behaviour.
  • In upcoming blogs, I’ll try to go into much more detail about these behaviours. Stay tuned!

• Let’s analyse the Economic Indicators to see if we can answer these questions:
  • Are there significant autocorrelations at certain lags?
  • Does the ACF reveal any seasonality or repeating patterns?
• 
• This provides insights into into the temporal dependencies within the ‘logan_intl_flights’ time series. The ACF plot indicates a strong positive correlation with the past one month (lag 1), meaning that there is a tendency for the number of international flights in a given month to positively correlate with the number in the month before. This discovery can direct additional research and modelling, particularly when taking into account techniques like autoregressive models that capture these temporal dependencies.
• Peaks or spikes signify significant autocorrelation at particular lags. The time series exhibits a strong correlation with its historical values at these spikes.
• The autocorrelation’s direction and strength are shown on the y-axis. Perfect positive correlation is represented by a value of 1, perfect negative correlation by a value of -1, and no correlation is represented by a value of 0.
• Generally speaking, autocorrelation values get lower as you get farther away from lag 0. A quick decay indicates that the impact of previous observations fades quickly, whereas a slow decay could point to longer-term dependencies.
• 
• Values of Autocorrelation: lag 0: The series’ correlation with itself is represented by the autocorrelation, which is always 1.
  lag 1 through lag 10: As the lag grows, the autocorrelation values progressively drop. Significantly, there is a strong positive correlation between consecutive observations at lag 1, as indicated by the high autocorrelation (0.87) at this point.
• Confidence Intervals of 95%: The range that the true population autocorrelation values are most likely to fall inside is provided by the confidence intervals. The autocorrelation values are regarded as statistically significant if they are outside of these ranges. The absence of zero in the lag 1 confidence interval ([0.6569, 1.0846]) indicates that the autocorrelation at lag 1 is statistically significant. This is consistent with the ACF plot’s clearly visible strong positive correlation.
• The significant positive correlation between the number of international flights in consecutive months is indicated by the high autocorrelation at lag 1 (0.87). This might suggest a level of momentum or forward motion in the sequence.
  As the lag lengthens, the autocorrelation values may show a diminishing impact from earlier observations. The statistical significance of the autocorrelation values is evaluated with the aid of the confidence intervals. Values that fall outside of these ranges are probably not random fluctuations but rather real correlations.
  • The ACF values gradually decrease as the lag increases, indicating a declining influence of past observations.

Forecasting a Random Walk:

• We have already seen what a Random Walk is in the previous blog.
• The underlying premise of the concept is that any deviation from the most recent observed value in a time series is essentially random and that future values in the series are dependent on this value.
• The random walk, in spite of its simplicity, is a standard by which forecasting models are measured, particularly when predicting intricate patterns is difficult. 
• Forecasting Process: Initialization: The last observed value in the historical data is frequently needed as an initial value to begin the forecasting process.Iterative Prediction: The forecast for the following observation is just the most recent observed value for each successive time period. Any deviations or modifications are presumed by the model to be random and unpredictable.Evaluation: By comparing the predicted values to the actual observations, metrics such as Mean Absolute Error (MAE), Mean Squared Error (MSE), or Root Mean Squared Error (RMSE) are commonly used to evaluate the performance of the random walk model.
• Use cases and Limitations:Baseline Comparison: Random walk forecasting is frequently used as a benchmark model to assess how well more complex forecasting methods perform. If a more sophisticated model is unable to beat the random walk, it indicates that it may be difficult to capture the inherent randomness in the data.Short-Term Predictions: These models work well for short-term forecasts when the persistence assumption(refer last blog post) is somewhat true.Limitations: The random walk model’s simplicity is both a strength and a weakness. Although it offers a baseline, it might miss intricate patterns or modifications to the time series’ underlying structure.
• Time to answer some questions.
  • How well does the random walk model predict future international flights?
  • Does the model capture the inherent randomness in the time series?
• 
• What’s interesting to me is that, by its very nature, stationarity is not required for the random walk model. Actually, non-stationary time series are frequently subjected to the random walk method, and the effectiveness of this method is assessed by its capacity to capture short-term dynamics as opposed to long-term trends.
• I went on to evaluate the model using the MAE metric and found that, on average, our random walk model’s predictions deviate by approximately 280.16 units from the actual international flight counts.
• The interpretation of the MAE is dependent on the scale of our data. In this case, the MAE value of 280.16 should be interpreted in the context of the total international flight counts.
• The performance of more complex forecasting models is measured against this MAE value. When a more complex model attains a lower mean absolute error (MAE), it signifies an enhancement in forecast precision in contrast to the naive random walk.
• The evaluation metrics assist you in determining how well the random walk captures the underlying patterns in the ‘logan_intl_flights’ time series, even though it offers a baseline.
• 
• When considering the scale of the data, the Random Walk model’s average prediction error is approximately 5.33% of the maximum flight count.
• On average, the international flights at Logan Airport amount to around 3940.51.
• The MAE for the Mean Benchmark is approximately 578.06. That is, if a simple benchmark model that predicts the mean were used, the average prediction error would be approximately 578.06 units.
• The Random Walk model, with an MAE of 280.16, outperforms a simple benchmark model that predicts the mean (MAE of 578.06). This indicates that the Random Walk captures more information than a naive model that predicts the mean for every observation.
• The scale-adjusted MAE of 5.33% provides a relative measure, suggesting that, on average, the Random Walk’s prediction errors are modest in comparison to the maximum flight count.
• When interpreting MAE, it’s essential to consider the specific context of your data and the requirements of your forecasting task.

Moving Average Model: MA(q):

• The Moving Average Model, also known as MA(q), is a time series model that takes into account how previous random or “white noise” terms may have affected the current observation. The moving average model’s order, or the number of previous white noise terms that are taken into account, is indicated by the notation MA(q). For instance, MA(1) takes the most recent white noise term’s impact into account.
  
• What is the optimal order (q) for the moving average model?
• How does the MA(q) model compare to simpler models in predicting international flights?
• We will be answering these questions next time.