Predicting Economic Activity through Richmond Fed Surveys

In addition, both the service sector survey and the manufacturing survey report both current activity and the level of activity anticipated by respondents at their establishments during the next six months (compared with the current month). If the diffusion index is positive, then that is generally interpreted as an expansion in activity while negative values are interpreted as a contraction.

The Richmond Fed is not unique in developing measures of service and manufacturing sector activity; some other Reserve Banks and other institutions also do so, as discussed in a March 2014 Richmond Fed Economic Brief by David Price and Aileen Watson. The methodology the Richmond Fed uses to create the diffusion indices is consistent with that used by other Federal Reserve Banks in their sector surveys as well as that of many other surveys, such as the Institute for Supply Management Index and the Michigan Survey of Consumers Index of Consumer Sentiment.

The Explanatory Power of the Surveys

Most of the survey indices are unique in the information that they provide. For example, there is no source of data on manufacturing new orders or service firm revenues at the state level, which makes it impossible to aggregate that information to the District level. Therefore, it is generally difficult to evaluate the accuracy of the survey indices for new orders or service firm revenues. To the extent that data are available, they are not as frequent and often lag considerably; for example, the annual Census Bureau data on state-level manufacturing shipments and capital expenditures for the year 2014 were not released until December 2015. Thus, it could be valuable to be able to use the sector surveys as leading indicators of these measures of economic activity. But how reliable are the surveys for this purpose?

There are two indicators for which externally provided data exist in a monthly or quarterly series: employment and wages by state and industry. The Quarterly Census of Employment and Wages (QCEW) provides considerable information on employment by state (and sub-state) and industry across the United States. (See "State Labor Markets: What Can Data Tell (or Not Tell) Us?" Econ Focus, First Quarter 2015.) We can use the QCEW data to understand how our survey measures perform. But to do that, we need to be able to better interpret what the survey index is measuring and how that corresponds to the measures of employment and wages provided through the QCEW.

As already discussed, the Richmond Fed diffusion index for employment in the manufacturing and service sectors captures the share of respondents who said employment increased compared with the previous month minus the share of respondents who said that employment decreased (see chart below). An analogous measure can be developed using the QCEW data, which are derived from the quarterly tax reports submitted to state workforce agencies by employers subject to state unemployment insurance laws. The QCEW data represent about 97 percent of all wage and salary civilian employment in the country and are available down to the county level by industry as granular as the six-digit NAICS code. (An exception is that the data are suppressed if the number of establishments in the county/industry or state/industry combination is small enough to potentially compromise the confidentiality of the reporting firms.) 

This aggregate employment growth measured by the QCEW data masks underlying details. For example, a moderately high aggregate employment growth in a particular area may result from a few sectors growing rapidly with other sectors growing more slowly or declining — or it may result from every sector growing at a moderately high rate. In other words, the aggregate growth rate can be interpreted as (approximately) arising from changes in an intensive margin (the difference between the intensity with which expanding sectors grew and with which contracting sectors declined) and changes in an extensive margin (the difference between the fraction of sectors that expanded and the fraction of sectors that contracted).

Since the Richmond Fed diffusion indices give the share of sectors whose employment increased (taking the respondent firms as representing the sector) versus the share of sectors whose employment fell, they are, in effect, providing the extensive margin. In other words, diffusion indices, appropriately scaled, capture the contribution of the extensive margin to changes in the aggregate series of interest, as discussed in a 2015 working paper by Santiago Pinto, Pierre-Daniel Sarte, and Robert Sharp.

To compare the employment or wage diffusion indices to the QCEW aggregate employment data, we need to mathematically decompose the aggregate employment change into an intensive and extensive margin. Doing so shows that variations in employment growth are greatly influenced by changes in the extensive margin (see chart below). One exception was during the Great Recession, when changes in both the intensive and extensive margins seemed to play an equal role. Since 2009, the expansion in aggregate employment in the Fifth District has relied heavily on the extensive margin — that is, on an increase in the share of sectors that increased employment.

The importance of the extensive margin outlined above suggests that diffusion indices may serve as a close indication of aggregate growth. Can we use the actual employment data to validate the Richmond Fed sector indices as early proxies for direct measures of employment changes? In particular, if we use this extensive margin to develop a "synthetic" diffusion index from the employment data, how do the Richmond Fed sector indices correlate with the synthetic diffusion index? As it turns out, the survey-based diffusion index produced in real time by the Richmond Fed lines up remarkably well with the synthetic diffusion index produced from the later employment data (see chart below). The correlation for the entire time period is 0.68, and from June 2002 through December 2014 it is 0.77.

This is a particularly useful finding since the surveys conducted by the Richmond Fed are much timelier than the aggregate employment data. The Richmond surveys are available almost in real time: The survey period ends on the third Wednesday of every month, and the survey results are released on the fourth Tuesday of every month. On the other hand, the QCEW data are released with a six-month lag so that the data through the end of 2015, for example, will not be available until June 2016. This analysis shows that given the role of the extensive margin in understanding aggregate employment growth, the real-time diffusion index developed by the Richmond Fed is a reasonable measure of employment activity and can be relied on to understand employment changes in the Fifth District in a timely fashion.

Wages and Other Measures

The reliability of a diffusion index in understanding "true" economic activity hinges on the relative contribution of the extensive margin of activity to overall growth. For employment, that contribution was significant. For other measures of activity, however, it might not be as significant. For example, another indicator that is available both in aggregate growth terms and through the Richmond Fed survey questionnaire is a measure of wage changes. When following the same exercise for employment as for wages, however, we find that the wage index fails to effectively track aggregate wage growth in the District. This is because changes in wages over time are driven to a greater extent by the intensive margin — the percent change in wages in sectors whose wages are changing in a given month — rather than the extensive margin, or the number of sectors whose wages are either increasing or decreasing in a given month.

Therefore, even if a survey-based index were to exactly mimic its "true" synthetic counterpart constructed with data observed ex post, it may perform poorly in tracking the aggregate series of interest. Therefore, the validity of each individual question survey index in predicting overall economic activity in that area will vary — an important factor to consider if the measure is to be used as either a leading indicator or a sole indicator of economic activity.

What's Next: State-Level Indices?

Although the survey-based diffusion index for the Fifth District aids in understanding economic activity at the District level, it is even more useful to understand economic activity at the state level, especially given the role that state boundaries play in economic activity and policymaking. In light of the dearth of state-level data, the manufacturing and service sector surveys have the potential to serve as a useful source of information that is not otherwise available at the state level (such as manufacturing new orders, retail shopper traffic, and projections of future activity) in a timely fashion. How reliable are the survey-based diffusion indices of employment as state-level indicators for the Fifth Federal Reserve District states? And how well do the Fifth District indices perform in capturing economic activity at the state level in each Fifth District state?

The level of representation of states in the Fifth District survey and in the QCEW data is generally consistent with economic activity in that state's manufacturing or service sector. The bulk of economic activity in our measures is occurring in North Carolina and Virginia and then, to a slightly lesser extent, Maryland and South Carolina (see table below). The contributions of the District of Columbia and West Virginia are considerably lower.

The number of responses at the state level is not enough to support state-level diffusion indices. We might be able to rely on the Fifth District survey index to accurately track the performance of each individual state in the District, however. To better understand this performance, we decomposed state employment growth in the QCEW data into extensive and intensive margins and then constructed state-level employment diffusion indices. The analysis shows notable differences in the relative importance of the intensive and extensive margin across states. The extensive margin explains the bulk of variations in state employment growth in North Carolina, South Carolina, and Virginia, but it seems less relevant to employment growth in the District of Columbia, Maryland, and West Virginia.

To understand how the Fifth District index might help us understand state economic activity, then, we developed synthetic indices for each state and compared them to the Fifth District synthetic index, as well as each state index to each other. These correlations differ notably across states (see table below). Again, the diffusion indices for Virginia and North Carolina most closely follow the performance of the Fifth District index, with a correlation of approximately 0.9. The correlation for South Carolina and Maryland was also quite high (0.85 and 0.82, respectively). The correlations between the state and Fifth District indices are much lower for the District of Columbia and West Virginia. In the same way that we should be careful in creating diffusion indices for indicators where the extensive margin does not explain the bulk of the change in aggregate growth, we should be careful in relying too heavily on diffusion indices for regions or states where the extensive margin does not account for most of the change in the economic indicator.

Conclusion

The Richmond Fed relies on diffusion indices developed through its surveys of manufacturing and service sector growth to understand economic activity in the Fifth Federal Reserve District. These indices are reliable only to the extent that change in the series of interest is driven by changes in the extensive margin; to the extent that changes are the result of a few firms or sectors driving aggregate growth, the indices will be less effective at measuring "true" change. Research indicates that for a series such as employment and for particular states, the survey-developed diffusion index is a good measure of economic activity in a much timelier fashion than other available data. Further analysis is required to better understand the efficacy of other survey measures or of these survey measures for different areas of the Fifth District.

For a more in-depth discussion of this topic, see Santiago Pinto, Sonya Ravindranath Waddell, and Pierre-Daniel Sarte, "Monitoring Economic Activity in Real Time Using Diffusion Indices: Evidence from the Fifth District," Federal Reserve Bank of Richmond Economic Quarterly (forthcoming).