The recorded history of modern financial markets and macroeconomic developments is limited. Hence, statistical analysis of macro trading factors often relies on panels, sets of time series across different currency areas. However, country experiences are not independent and subject to common factors. Simply stacking data can lead to “pseudo-replication” and overestimated significance of correlation. A better method is to check significance through panel regression models with period-specific random effects. This technique adjusts targets and features of the predictive regression for common (global) influences. The stronger these global effects, the greater the weight of deviations from the period-mean in the regression. In the presence of dominant global effects, the test for the significance of a macro factor would rely mainly upon its ability to explain cross-sectional target differences. Conveniently, the method automatically accounts for the similarity of experiences across markets when assessing the significance and, hence, can be applied to a wide variety of target returns and features. Examples show that the random effects method can deliver a quite different and more plausible assessment of macro factor significance than simplistic statistics based on pooled data.
The below post is based on proprietary research of Macrosynergy Ltd. The post contains a few formulas, but they have been kept as simple as possible and should be helpful for illustrating the principles behind the macro factor test.
This post ties in with this site’s summary on quantitative methods for macro information efficiency.
The importance of panels for macro trading research
We define macro factors as real-time series of macroeconomic information that plausibly could predict financial market returns, volatility, correlation, or similar characteristics. If financial markets are not fully information efficient with respect to economic developments, macroeconomic information should help predict macro asset classes, such as rates, currencies, credit, and equity.
However, ascertaining the significance of macro factors is challenging. Meaningful economic events, such as business cycles, are not frequent, requiring months or even years to unfold. Also, as macroeconomic data mostly report developments at a monthly or quarterly frequency the number of observations is limited. Most inconveniently, the recorded macroeconomic history concurrent to modern financial markets is rather short, limited to a few decades.
This scarcity of history calls for the use of panels, i.e. time series across different and diverse currency areas. The typical hypothesis is that a specific economic state reliably predicts a financial return. For example, we may wish to verify that a high inflation trend is regularly followed by low or negative fixed-income returns. This hypothesis can be tested across many currency areas that operate standard macroeconomic policy regimes and that have a liquid bond or interest rate swap markets. Such analysis is the domain of panel regression.
The danger of simply pooling data across countries or markets
The simplest approach to testing predictive power is to pool the observations of targets (explained variables, typically returns) and features (explanatory variables) for all available cross-sections (typically currency areas) and time periods into one-dimensional data sets. Pooling largely disregards the two-dimensional structure of the dataset. It just stacks the experience of various currency areas or markets and, therefore, results in a larger data set. Running linear “pooled” regression on this data set seems to make it much easier to diagnose the significance of macro predictors of financial returns.
Alas, very often the behavior of returns and macro-quantamental predictors across countries or markets is not independent. Indeed, empirical evidence shows that time series of directional returns and some explanatory quantamental indicators are often correlated, and sometimes strongly so. This reflects that many financial returns and economic developments are driven by global factors such as the 2008 global financial crisis or the 2020-21 pandemic. Thus, the assumption of the pooled regression that all observations are independent and that a longer data set adds proportionately more information can be badly misleading. Specifically, assuming time series within a panel to be generally independent leads to pseudo-replication of experiences and incorrect inference, overstating confidence in results.
The case for considering random effects
The random effects model is an effective econometric technique to tackle pseudo-replication in panels. In general, random effects are commonalities of measurements that are connected, for example by belonging to the same cross-section or period, and that are not explained by the model features. Affected observations are not independent. Importantly, the term ‘random’ does not mean that the effects are erratic or not measurable. They are merely influences that are not explained by the features of the model at hand and are specific to a level, i.e. a period or a cross-section. In a random-effects model, we consider level-specific commonalities as draws of a random variable from some (unknown) probability distribution. This implies that the individual instances of this random influence are of no particular interest themselves, but their distribution is. This reflects that the random effects model principally tackles dependence of observations, as opposed to the fixed effects model, which also cares for the actual parameter estimates of cross-section- or period-specific effects and principally tackles the omitted variable bias.
Typically, the most important level-specific effect in panel analysis of financial returns is a period-specific communal return component that is not explained by the feature. This is different from most other applications of panel regression, which place greater emphasis on cross-section-specific effects. We must account for the importance and distribution of that component, to immunize panel regression against pseudoreplication.
Panel predictor assessment with period-specific random effects
This section and the following use the general format and explanation of a post on random effects by Sachin-Sate but apply it to period-specific rather than cross-section effects.
The random effects regression model can be used to account for the effect of period-specific characteristics of a panel. Here random effects are target variations across periods that apply to all cross-sections and that are not accounted for by the feature (predictor) of the model. Even though their causes are, by definition, not observed and cannot be used for prediction, the effects themselves must be considered, lest they leak into the error term in an uncontrolled way and compromise related statistics, such as significance tests for the predictor.
Considering period-specific random effects in a return prediction panel model means that the target value of cross-section i for period t (yi,t) is a linear function of the previous period’s feature of the cross-section (xi,t-1) with a common panel coefficient (β), a linear function of an unobserved “global” feature (zt) with a period-specific coefficient (γt) and an idiosyncratic, i.e. cross-section and period-specific, error term (εi,t):
Here we only consider a single predictor, but the subsequent logic applies to multiple features similarly. All unpredicted period-specific influences are assumed to be captured by the linear impact of the global feature. These are purely theoretical terms. The global factor cannot be observed.
In general, level-specific effects can be modeled as fixed or random. Both address the pseudo-replication issue that arises from correlated cross-sectional time series. However, fixed effects models are often an excessively “expensive” method insofar as they fully discard level-specific variation of the target, whereas random effects models only discard part of level-specific variation:
- The fixed effect model estimates the level-specific coefficients of dummy variables that identify each instance of the level. Here this would require estimating one coefficient per period and, thus, be very costly in terms of degrees of freedom. A key benefit of fixed effects is the mitigation of any omitted variable bias that may arise if the unobserved global factor was correlated with the observed and tested feature. The bias affects the estimated coefficients of the observed features and is – under standard assumptions – proportional to the covariance between the observed and unobserved features. However, omitted variables biases are not typically a concern for financial market prediction. As long as the correlation between observed and unobserved features is stable a biased coefficient is perfectly suitable for prediction.
- The random effects model, unlike the fixed effects model, does not assume period-specific influences of unobserved variables. Instead, it assumes that all variations in the target are appropriately explained by the features and an error term. However, within the error term there are period-specific are realizations of a random term that is distributed around a common mean that equal across all cross-sections.
Thus, the distinguishing characteristic of the random effects model is that, for given observed features, unobserved influences are expected to be equal across periods and any actual divergences in influence are random, not replicable, or explainable:
Consequently, the random effects panel model features a special common intercept (α) and special period-specific random variable (ut):
Considering the period-specific random effects thus enhances a simple pooled regression by two realistic features:
- A constant that captures the mean of all period-specific effects. An example would be a long-term risk premium that is common to all markets of a similar type and that is completely unrelated to the feature.
- An error term that is common across sections, e.g. currency areas, and whose magnitude is proportional to the variance of period-specific effects. An example would be the common response of a range of similar markets to changes in sentiment or global political risk.
Estimation of a period-specific random effects model
The combination of the idiosyncratic error term and the period-specific error term is a defining characteristic of the random effects regression model. The assumptions for the distribution of its parts shape the model estimation:
- The idiosyncratic error term (εi,t) is a random variable that, conditional on the feature, is assumed to be distributed with constant variance (σε2)around a mean of zero. It is also assumed to be uncorrelated across cross-sections and time periods.
- The period-specific error term (ut): is also assumed to be distributed around a zero-mean with constant variance (σu2).
- The two error terms are uncorrelated.
The objects of estimation of this basic period-specific random effects model are the regression coefficients and the two error terms. To understand what this estimation produces we should consider the following steps in sequence.
Step 1: Estimation of pooled and fixed-effects regression models
This is a preparatory step for estimating the idiosyncratic and period-specific error term variance, which, at a subsequent stage, is required for coefficient estimates.
- We train a pooled OLS model on the full panel data set, which is equivalent to an OLS regression on a stacked (or flattened) version of the panel data set. From this estimation, we obtain an estimate of the pooled model’s error term variance (σε,pool2).
- Similarly, we train a period-specific fixed effects model. From this estimation, we obtain an estimate of the fixed-effects model’s error term variance (σε,fixed2)
Step 2: Estimation of the random effects model variance components
It can be shown that the variance of the idiosyncratic error term in the random-effects model is equal to the variance of the error term of the fixed-effects model:
Further, it can be shown that the variance of the random effects model’s period-specific error term is the difference between the pooled model’s error variance and the fixed effect model’s error variance:
The intuition behind this relation is simple: if period-specific effects explain a large share of the pooled model’s variance the difference between the pooled error variance, which does not account for the effects, and the fixed effect model’s variance, which fully accounts for it, must be large. This would be the case, for example, for a panel of directional country equity index future returns, which are highly correlated and difficult to predict at higher frequencies.
Step 3: Calculate re-scaled panel data sets
We first calculate the period-specific means for the target and feature panels by averaging over all cross-sections:
Then we partially de-mean the targets and features for each period, based on the period-specific means. However, instead of using simple means, we subtract scaled versions of the period-specific means, where the scaling factor theta (θ) depends positively on the contribution of the period-specific random effect to overall variation. The stronger the unexplained period-specific variation, the higher the scaling factor:
Thus, features and targets are adjusted for period-specific means to the extent that period-specific unexplained variations of the target dominate the overall error variance of a simple pooled model:
The scaling factor theta is between 0 and 1 and determines to what extent targets and features are adjusted for period-specific means. If unexplained period-specific target variation is very high the scaling factor will be close to one and we would effectively regress on fully de-meaned data, making the regression equivalent to the period-specific fixed effects model. And if theta is close to 1 the model converges on a pooled regression This would imply that variations of average returns (target means) have no bearing on coefficient estimation. The only variations that matters are the relative target and feature values versus the period-specific means. For example, if a large panel of country equity index returns is dominated by unexplained periodical fluctuations, the coefficient that characterizes the predictive power of the feature will mainly depend on the relationship between relative targets and relative features.
Step 4: Estimate random-effects coefficients using the re-scaled panels
For coefficient estimation and statistical inference, we apply OLS regression to a panel of rescaled targets and features:
This model is the basis for estimating the coefficient and significance of the quantamental predictor.
Implications of using period-specific random effects for predictor tests
The above panel model is equivalent to a weighted application of pooled and fixed effects regression. If period-specific average errors are small the pooled estimation will dominate. This could be the case if we analyzed a panel of relative returns and relative features (against a basket of returns or features). If period-specific average errors are large the fixed effects regression will dominate. This could be the case if we analyzed directional equity index returns across a broad set of similar markets.
Thus, the period-specific random effects model chooses the appropriate weight of pooled versus fixed-effect regression in accordance with the properties of the panel data. For the purpose of panel predictions of returns based on macro-quantamental indicators, this is a one-method-fits-all solution. It provides inference free from distortions that come with pseudo-replication. And unlike fixed-effects models, it does not discard all period-specific target fluctuations but only sacrifices degrees of freedom to the extent that is necessary. Thus, the model and related significance tests can be applied to all types of returns, including relative returns across markets, returns of hedged positions, and returns of specific portfolios.
Testing in practice: Three examples
For testing predictive power, time series of macro information must at any date reflect what the market could have known on that date. For the below analysis, we use proper point-in-time macro quantamental indicators, such as those provided by the J.P. Morgan Macrosynergy Quantamental System (“JPMaQS”). Quantamental indicators of this service are real-time information states of the market with respect to an economic concept and, hence, are suitable for testing relations with subsequent returns and backtesting related trading strategies.
We can test the significance of a macro factor by using it as a regressor for a target return for a panel of markets or currency areas. In the examples below we use various types of quantamental macroeconomic information at a monthly frequency (month-end status) and relate it to the next month’s 5-year interest rate swap fixed receiver returns for the period 2000 to 2022 and for 6 major developed markets: the U.S. (USD), the euro area (EUR), Japan (JPY), the UK (GBP), Canada (CAD), and Australia (AUD).
The simplistic way of checking significance would be to simply stack up all country data and run a pooled regression. Alternatively, one can check the significance of a correlation coefficient for the stacked data set. In both cases, the implicit assumption is that the experiences of the different markets are independent and unrelated. By contrast, the method suggested above requires us to run a panel regression with period-specific random effects.
We run our analysis by using the plm package in R on the quantamental data set.
Example 1: Deflating significance
First, we check the significance of core CPI trends, measured as percent change over the last 6 months versus the previous 6 months, seasonally and jump-adjusted. The JPMaQS ticker is CPIC_SJA_P6M6ML6AR and documentation can be found here (requires login).
The pooled regression diagnoses high significance of the 1-month lagged core CPI trend for duration returns. As should be expected the relation between inflation and subsequent duration returns has been negative and the probability of the relationship not being accidental according to the pooled model has been over 99% (indicated by the p-value below being below 0.01.
However, economists would point out that core inflation trends are particularly important in the U.S. and that both inflation and duration returns are globally correlated. Hence, this is a classic example, in which treating countries’ experiences as independent leads to pseudoreplication. The results are plausibly misleading with respect to the general importance of core inflation rates across countries.
Indeed, the period-specific random effects regression gives a very different result. The negative relation between the month-end core CPI trend and next month’s return still shows as negative, but the statistical probability of this relation is only at 87-88%, below the level that many researchers would judge as significant. Also, the theta (degree of effective cross-sectional de-meaning of the data) has been near 2/3, indicating the importance of period-specific effects explaining the returns in the panel and testifying to the risk of pseudo-replication in pools.
Example 2: Confirming significance
Next, we check the significance of real 5-year swap yields, measured as the difference between IRS yields and formulaic inflation expectations based on recent CPI trends and effective inflation targets. The JPMaQS ticker is RYLDIRS02Y_NSA and documentation can be found here (requires login).
The pooled regression shows a very high significance of month-end real yields for subsequent returns. This relation has naturally been positive, with high yields predicting high returns, and with a statistical probability of more than 99.99%.
In this case, the period-specific random effects regression confirms the high significance of the predictor. The strength of the relation is shown to be a little weaker than for the pool. But statistical probability remains above 99.99%.
Example 3: Enhancing significance
Sometimes, the significance test based on random effects regression even reinforces evidence of significance provided by simplistic pooled analysis. This plausibly would be the case if relative (de-meaned) values of the features and targets have a close relation.
To illustrate that point, we look at the predictive power of estimated GDP growth trends for duration returns. The chosen quantamental indicator is “intuitive growth trends”. These are real-time estimated recent GDP growth rates (% over a year ago, 3-month averages) based on regressions that use the latest available national accounts data and monthly-frequency activity data. The estimation relies on GLS regression with autocorrelated errors. Unlike standard academic models, the intention is to mimic the intuitive methods of market economists. The JPMaQS ticker is INTRGDP_NSA_P1M1ML12_3MMAand documentation can be found here (requires login).
The pooled regression shows the lagged growth trends failing to take the significance hurdle at the 5% level. The statistical probability of a (negative) relation to next month’s duration returns is estimated at just above 93%
Growth trends are often better indicators of relative country performance, because they are comparable and fairly exact, as opposed to survey data and early indicators, which are timelier and better suited to track changes to the absolute growth performance of a particular country. Hence, the consideration of period-specific effects, which partly look at relative country performance and returns, could plausibly reinforce their importance.
Indeed, period-specific random effects regression here shows a slightly higher significance of the predictor. Statistical probability exceeds 95%.
