Reliability of macroeconomic forecasts by the Ministry of Finance for the years 1976-2016 - Fiscal policy evaluation

 Summary of the results

As part of its fiscal policy evaluation, the National Audit Office (NAOF) assesses the reliability of forecasts by the Ministry of Finance (FM). This report focuses on forecasts for the current year and one‐year ahead fiscal year on the growth of the Finnish gross domestic product (GDP), the unemployment rate and inflation in 1976–2016.

The autumn economic forecasts by the FM were selected for the review, because these are the forecasts that are used when planning the state budget for the year ahead. The ministry’s economic forecasts were compared with forecasts produced by the Research Institute of the Finnish Economy (Etla), the Organisation for Economic Co‐operation and Development (OECD), the Labour Institute for Economic Research (PT) and Pellervo Economic Research (PTT).

No characteristics (bias) that would require corrective action based on the Government Decree on the General Government Fiscal Plan were detected in the GDP, unemployment rate and inflation forecasts by the Ministry of Finance.

Reliability of the forecasts was studied using the following questions:

1. How accurately have the FM’s forecasts predicted future economic development when compared to forecasts by other key forecasters (accuracy)?

2. Have errors in the FM’s forecasts been temporally independent and non‐persistent?

3. Have the FM’s forecasts been unbiased?

4. Have the FM’s forecasts encompassed the naïve forecast?

5. Have the FM’s forecasts covered all the pertinent information at the time of forecasting?

Based on the results, reliability of the FM’s forecasts on GDP growth, unemployment rate and inflation for the year ahead (t+1) and the current year (t+0) was, together with the forecasts by Etla, among the best of the compared forecasts.

The accuracy was studied by comparing the mean absolute error in the FM’s forecast with those of the other forecasters. The FM’s mean was, as a general rule, within the 95% confidence intervals of the two best forecasters. Only Etla was statistically slightly better than the Ministry of Finance in the current year’s unemployment rate forecasts. In accuracy of the current year’s GDP growth and inflation as well as in all of the year ahead variables, the FM was statistically among the most accurate forecasters.

In the case of forecasts for the budget year (t+1), the accuracy of the FM in 1976–2016 was statistically as good as that of the other forecasters. The FM, PTT and Etla were the most accurate in forecasting the GDP and unemployment rate during the sample period. In inflation forecasts, the FM was – together with the OECD – the least accurate, but the differences were minor. Statistically speaking, the FM has been as good as the best Finnish forecasters in the reference group in the forecasting accuracy of the current year’s (t+0) GDP growth and inflation. In the sample between 1976 and 2016, Etla was the most accurate in forecasting the unemployment rate, followed by PTT. Also in the case of GDP growth, Etla was, together with PTT, the most accurate, while the FM was the third in both categories. PTT was the most accurate in the current year’s inflation forecasts, closely followed by the FM and Etla.

Questions 2–5 were answered by studying the reliability of the forecasts by the FM and the reference group with statistical tests corresponding to the tests used by the European Commission in assessing its own forecasts. The FM’s budget forecasts on the year ahead (t+1) passed, together with the forecasts by Etla, the most reliability tests: both institutions passed 13 out of 15 tests. The FM’s forecasts proved to be statistically unbiased in the case of all the three variables, and there was no temporal persistence in the forecast errors. Forecasts by the Ministry of Finance and Etla most often passed the test on encompassing the naïve forecast and the test on coverage of all pertinent information at the time of the forecast.

The results on the current year’s forecasts (t+0) were almost the same: the FM and Etla passed 14 out of 15 reliability tests, and PTT and PT were not far behind, passing 13 out of 15 tests. The FM’s forecasts were unbiased and there was no temporal persistence of forecast errors. Comprehensiveness of the information included in the forecasts on the current year’s GDP, unemployment rate and inflation by the FM and Etla proved to be good. Results of the statistical tests have been compiled into a summary table, which is available in the Summary and conclusions section of this report.

Reliability of forecasts and independence of the preparation of forecasts used in state budget planning should be continuously monitored in Finland as well. According to the European Commission, the arrangement used by Finland in the preparation of official forecasts is exceptional in the EU: the forecasts used as the basis for the fiscal policy are prepared by the Ministry of Finance, and they are not confirmed by an independent third party. Elsewhere in the euro area, macroeconomic forecasts are either produced by independent institutions or the forecasts prepared by a country’s finance ministry are approved by an independent body. The arrangement in Finland is, however, in accordance with the EU regulations.