The Moving Average Model: or MA(q), is a time series model that predicts the current observation by taking into account the influence of random or “white noise” terms from the past. It is a part of the larger category of models referred to as ARIMA (Autoregressive Integrated Moving Average) models. Let’s examine the specifics:

• Important Features of MA(q):

1. 1. Order (q): The moving average model’s order is indicated by the term “q” in MA(q). It represents the quantity of historical white noise terms taken into account by the model. In MA(1), for instance, the most recent white noise term is taken into account.
   2. White Noise: The current observation is the result of a linear combination of the most recent q white noise terms and the current white noise term. White noise is a sequence of independent and identically distributed random variables with a mean of zero and constant variance.
   3. Mathematical Equation: An MA(q) model’s general form is expressed as follows:
      Y t = μ + Et + (θ(1) ​E(t−1) )​+( θ(2) E(t-2) ​)+…+(θq​Et−q)
      Y t is the current observation.
      The time series mean is represented by μ.
      At time t, the white noise term is represented by Et.
      The weights allocated to previous white noise terms are represented by the model’s parameters, which are θ 1​, θ 2​,…, θ q​.

• Key Concepts and Considerations:

1. 1. Constant Mean (μ): The moving average model is predicated on the time series having a constant mean (μ). 
   2. Stationarity: The time series must be stationary in order for MA(q) to be applied meaningfully. Differencing can be used to stabilise the statistical characteristics of the series in the event that stationarity cannot be attained. 
   3. Model Identification: The order q is a crucial aspect of model identification. It is ascertained using techniques such as statistical criteria or autocorrelation function (ACF) plots.

• Application to Time Series Analysis:

1. 1. Estimation of Parameters: Using statistical techniques like maximum likelihood estimation, the parameters θ 1, θ 2, …, θ q, are estimated from the data.
   2. Model Validation: Diagnostic checks, such as residual analysis and model comparison metrics, are used to assess the MA(q) model’s performance.
   3. Forecasting: Following validation, future values can be predicted using the model. Based on the observed values and historical white noise terms up to time t−q, the forecast at time t is made.

• Use Cases:

1. 1. Capturing Short-Term Dependencies: When recent random shocks have an impact on the current observation, MA(q) models are useful for detecting short-term dependencies in time series data. 
   2. Complementing ARIMA Models: To create ARIMA models, which are strong and adaptable in capturing a variety of time series patterns, autoregressive (AR) and differencing components are frequently added to MA(q) models.

Let’s try to fit an MA(1) model to the ‘logan_intl_flights’ time series from Analyze Boston. But before that it’s important to assess whether the ‘logan_intl_flights’ time series is appropriate for this type of model. The ACF and PACF plots illustrate the relationship between the time series and its lag values, which aids in determining the possible order of the moving average component (q).

Okay, now what?

Perplexing, isn’t it? In a time series model, understanding the Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots can help determine the Moving Average (MA) component’s order.

• Reading the ACF (Autocorrelation Function) plot:
  • Lags on the x-axis: The x-axis represents the number of lags, indicating the time span between the current observation and its lagged values.
  • Correlation values on the y-axis: The y-axis displays the correlation coefficient between the time series and its lagged values.
  • Interpretation: A significant spike at a particular lag indicates a strong correlation with the observations at that lag. In the ACF plot, we’re looking for significant spikes that decay as lags increase.
• Reading the PACF (Partial Autocorrelation Function) plot:
  • Lags on the x-axis: Similar to the ACF plot, the x-axis represents the number of lags. 
  • Partial correlation values on the y-axis: The y-axis displays the partial correlation coefficient, which measures the unique correlation between the current observation and its lagged values, removing the influence of intermediate lags. 
  • Interpretation: Similar to the ACF plot, significant spikes in the PACF plot indicate a strong partial correlation with the observations at those lags. PACF helps identify the direct influence of each lag on the current observation.

In essence, ACF is “How each lag influences subsequent times” while PACF  tells you “The direct influence of each lag on the current time” .

To interpret our plots we look for ACF Decay, that is if the ACF plot shows a rapid decay after a few lags, it suggests that the series might not require a large MA order. And PACF Cutoff, whether or not the PACF plot has a significant spike at lag 1 and then cutoffs for subsequent lags, it may indicate an MA(q) model where q is the last significant lag in the PACF plot. A potential direct relationship between the current observation and its immediate past value is indicated by a cutoff in the Partial Autocorrelation Function (PACF) at lag 1.

Let’s go through each component of the SARIMAX results:

• Coefficients Table:

1. 1. const (Constant):
      • Coefficient (coef): -0.6513
      • Standard Error (std err): 1.579
      • z-value: -0.412
      • P-value (P>|z|): 0.680
      • 95% Confidence Intervals: [-3.746, 2.444]
      • Interpretation: The constant term represents the intercept of the model. In this case, the estimated intercept is approximately -0.6513. The p-value (0.680) suggests that the constant is not statistically significant.
   2. ma.L1 (Moving Average Order 1):
      • Coefficient (coef): -0.9989
      • Standard Error (std err): 2.897
      • z-value: -0.345
      • P-value (P>|z|): 0.730
      • 95% Confidence Intervals: [-6.678, 4.680]
      • Interpretation: The ma.L1 coefficient represents the strength of the moving average effect at lag 1. In this case, it is estimated as approximately -0.9989. The p-value (0.730) suggests that this coefficient is not statistically significant.
   3. sigma2 (Residual Variance):
      • Coefficient (coef): 1.142e+05
      • Standard Error (std err): 3.4e+05
      • z-value: 0.335
      • P-value (P>|z|): 0.737
      • 95% Confidence Intervals: [-5.53e+05, 7.81e+05]
      • Interpretation: Sigma2 represents the estimated variance of the model residuals. The large standard error and p-value (0.737) suggest uncertainty about the accuracy of this estimate.

• Diagnostic Tests:

1. 4. Ljung-Box (Q) Statistic:
      • Value: 0.86
      • P-value (Prob(Q)): 0.35
      • Interpretation: The Ljung-Box test assesses the autocorrelation of residuals at lag 1. A non-significant p-value (0.35) suggests that there is no significant autocorrelation in the residuals.
   5. Jarque-Bera (JB) Statistic:
      • Value: 1.45
      • P-value (Prob(JB)): 0.48
      • Interpretation: The Jarque-Bera test assesses the normality of residuals. A non-significant p-value (0.48) suggests that the residuals do not significantly deviate from a normal distribution.

• Other Statistics:

1. 6. Heteroskedasticity (H):
      • Value: 1.18
      • P-value (Prob(H)): 0.68
      • Interpretation: The test for heteroskedasticity assesses whether the variance of the residuals is constant. A non-significant p-value (0.68) suggests that there is no significant evidence of heteroskedasticity.
   7. Skewness:
      • Value: 0.26
      • Interpretation: Skewness measures the asymmetry of the residuals. A value around 0 suggests a symmetric distribution.
   8. Kurtosis:
      • Value: 2.61
      • Interpretation: Kurtosis measures the “tailedness” of the residuals. A value of 2.61 suggests moderate tailedness.

• Summary:
  • The coefficients table provides estimates for the model parameters, including the constant, moving average effect, and residual variance.
  • Diagnostic tests indicate that the residuals do not exhibit significant autocorrelation, deviate significantly from a normal distribution, or show evidence of heteroskedasticity.
  • Overall, while the model provides parameter estimates, the lack of statistical significance for some coefficients and the uncertainty in the residual variance estimation may indicate that the model might need further refinement or exploration. 

 

• Refinement? In the field of time series modelling, choosing the right values for an ARIMA (AutoRegressive Integrated Moving Average) model—often represented by the letters p, d, and q—necessitates a careful process of trial and error. The model’s moving average, differencing, and autoregressive components are determined, respectively, by these parameters.
  • Step 1: An examination of the autocorrelation function (ACF) plot sets the stage for the journey. The correlation between a time series and its lagged values can be understood by looking at this graphical representation. Potential values for p, the autoregression order, are represented by peaks in the ACF plot, and a good value for q, the moving average order, is indicated by the first notable dip.
  • Step 2: Iterations and comparisons against statistical metrics are part of the process. AIC (Akaike Information Criterion) and BIC (Bayesian Information Criterion) are two of the most notable of these. Lower values of these information criteria indicate a better model. They assess the trade-off between model fit and complexity. 
  • Step 3: Equally important is the coefficients’ statistical significance. Each model coefficient denotes the impact of values that are lags on the current observation. We are looking for a middle ground, avoiding an unduly complicated model and choosing statistically significant coefficients (p-values usually ≤ 0.05). For the model to be both statistically reliable and comprehensible, this equilibrium is essential.

To help you understand it much better, allow me to walk you through the procedure once more using our plot. Take another look at the ACF. Do you see how significant the spikes are up until the third lag before the graph disappears into the confidence interval? Indeed, it rises once more after 10 lags. Thus, there are roughly six options for “p”. Now look at the PCF plot, you can see that 1, 7, 10, 13, and so forth are the spikes that are outside of the confidence interval. These are the potential values for “q”. We know that “d” should be 2 because our data isn’t stationary and that second-order differencing fixes it from the previous blog.

A common strategy is to start with a simple model, gradually introducing complexity until reaching a point where further additions do not substantially improve fit or interpretability. This careful calibration ensures that the model captures essential patterns without succumbing to overfitting.

In essence, the journey to find the right combination of p, d, and q in ARIMA modeling is a dance between visual exploration, statistical metrics, and the art of striking the delicate balance between model complexity and effectiveness. 

If you’re still reading, you probably already understand how to interpret this.

• ARIMA Order: The model is specified as ARIMA(0, 2, 7), indicating a non-seasonal differencing of order 2 and a seasonal differencing of order 7.
• MA Coefficients (ma.L1 to ma.L7)
  • Interpretation: A subset of the coefficients (p-value < 0.05) are not statistically significant, but some are. This implies that different lags have different relative importance in explaining the current observation.
• Sigma 2 Residual Variance: sigma2 (p-value < 0.05) = 6.606e+04
  • Interpretation: The statistical significance of the estimated residual variance indicates the presence of variability that cannot be explained by the model.
• Diagnostic Tests
  • Q-statistic for Ljung-Box (L1): 0.04 (p-value > 0.05)
  • JB test result: 2.50 (p-value greater than 0.05).
  • 1.10 is the heteroskedasticity (H) (p-value > 0.05).
  • Interpretation: The Jarque-Bera test and the Heteroskedasticity test indicate that there are no significant problems with normality or heteroskedasticity, respectively, and the Ljung-Box test reveals no significant autocorrelation at lag 1.

We’re probably at our best here. Please let me know if you are aware of a more effective way to achieve the desired combination without repeatedly iterating. Our next goal is to try and forecast the future values of flights at Logan International Airport using the SARIMA(0, 2, 7) model.

I tested the model’s performance on unseen data (10% of the dataset) after training it on historical data, which accounted for 90% of the dataset. 386.20 was determined to be the Mean Absolute Error (MAE), a metric used to assess the accuracy of predictions.

In order to forecast future flights on the test set, the code first divided the data into two sets: a training set and a test set. Then, it fitted the SARIMA(0, 2, 7) model to the training set. Next, a comparison was made between the outcomes and the real test set values.

The mean absolute error (MAE) of 386.20 indicates that our predictions were off by about 386 flights on average from the real values. Although the MAE is a helpful metric, it’s crucial to understand its interpretation in relation to the particular dataset and the flight domain at Logan International Airport. Additionally, the Mean Absolute Percentage Error (MAPE) of 8.96% suggests that, on average, our predictions deviated by about 8.96% from the actual values.

MAPE of 8.96% suggests that the model is providing reasonably accurate predictions on the test set.

It’s critical to remember that the particular context and demands of your forecasting task will determine how MAPE should be interpreted. A MAPE of less than 5% is usually desired but  however the acceptable degree of error can change depending on the application and industry.