Univariate Portfolio Sorts

Note

You are reading the work-in-progress edition of Tidy Finance with Python. Code chunks and text might change over the next couple of months. We are always looking for feedback via contact@tidy-finance.org. Meanwhile, you can find the complete R version here.

In this chapter, we dive into portfolio sorts, one of the most widely used statistical methodologies in empirical asset pricing (e.g., Bali, Engle, and Murray 2016). The key application of portfolio sorts is to examine whether one or more variables can predict future excess returns. In general, the idea is to sort individual stocks into portfolios, where the stocks within each portfolio are similar with respect to a sorting variable, such as firm size. The different portfolios then represent well-diversified investments that differ in the level of the sorting variable. You can then attribute the differences in the return distribution to the impact of the sorting variable. We start by introducing univariate portfolio sorts (which sort based on only one characteristic) and tackle bivariate sorting in Chapter 9.

A univariate portfolio sort considers only one sorting variable \(x_{t-1,i}\). Here, \(i\) denotes the stock and \(t-1\) indicates that the characteristic is observable by investors at time \(t\).
The objective is to assess the cross-sectional relation between \(x_{t-1,i}\) and, typically, stock excess returns \(r_{t,i}\) at time \(t\) as the outcome variable. To illustrate how portfolio sorts work, we use estimates for market betas from the previous chapter as our sorting variable.

The current chapter relies on the following set of packages.

import pandas as pd
import numpy as np
import sqlite3
from plotnine import *

import statsmodels.api as sm
from mizani.formatters import percent_format

Data Preparation

We start with loading the required data from our SQLite-database introduced in Chapters 2-4. In particular, we use the monthly CRSP sample as our asset universe. Once we form our portfolios, we use the Fama-French market factor returns to compute the risk-adjusted performance (i.e., alpha). beta is the tibble with market betas computed in the previous chapter.

tidy_finance = sqlite3.connect("data/tidy_finance.sqlite")

crsp_monthly = (pd.read_sql_query(
    sql="SELECT permno, month, ret_excess, mktcap_lag FROM crsp_monthly",
    con=tidy_finance,
    dtype={"permno": np.int32},
    parse_dates={"month": {"unit": "D","origin": "unix"}})
  .dropna()
)

factors_ff_monthly = (pd.read_sql_query(
    sql="SELECT month, mkt_excess FROM factors_ff_monthly",
    con=tidy_finance,
    parse_dates={"month": {"unit": "D","origin": "unix"}})
  .dropna()
)

beta = (pd.read_sql_query(
    sql="SELECT permno, month, beta_monthly FROM beta",
    con=tidy_finance,
    dtype={"permno": np.int32},
    parse_dates={"month": {"unit": "D","origin": "unix"}})
  .dropna()
)

Sorting by Market Beta

Next, we merge our sorting variable with the return data. We use the one-month lagged betas as a sorting variable to ensure that the sorts rely only on information available when we create the portfolios. To lag stock beta by one month, we add one month to the current date and join the resulting information with our return data. This procedure ensures that month \(t\) information is available in month \(t+1\). You may be tempted to simply use a call such as crsp_monthly['beta_lag'] = crsp_monthly.groupby('permno')['beta'].shift(1) instead. This procedure, however, does not work correctly if there are non-explicit missing values in the time series.

beta_lag = (beta
  .assign(month = lambda x: x["month"] + pd.DateOffset(months=1))
  .get(["permno", "month", "beta_monthly"])
  .rename(columns={"beta_monthly": "beta_lag"})
  .dropna()
)

data_for_sorts = (crsp_monthly
  .merge(beta_lag, 
         how="inner", 
         on=["permno", "month"])
)

The first step to conduct portfolio sorts is to calculate periodic breakpoints that you can use to group the stocks into portfolios. For simplicity, we start with the median lagged market beta as the single breakpoint. We then compute the value-weighted returns for each of the two resulting portfolios, which means that the lagged market capitalization determines the weight in np.average().

beta_portfolios = (data_for_sorts
    .groupby("month", group_keys=False)
    .apply(lambda x: x.assign(portfolio=pd.qcut(x["beta_lag"], 
                                                q=[0, 0.5, 1], 
                                                labels=["low", "high"])))
    .groupby(["portfolio","month"], group_keys=False)
    .apply(lambda x: np.average(x["ret_excess"], weights=x["mktcap_lag"]))
    .reset_index(name="ret")
    )

Performance Evaluation

We can construct a long-short strategy based on the two portfolios: buy the high-beta portfolio and, at the same time, short the low-beta portfolio. Thereby, the overall position in the market is net-zero, i.e., you do not need to invest money to realize this strategy in the absence of frictions.

beta_longshort = (beta_portfolios
  .pivot_table(index="month", 
               columns="portfolio", 
               values="ret")
  .reset_index()
  .assign(long_short = lambda x: x["high"] - x["low"])
)

We compute the average return and the corresponding standard error to test whether the long-short portfolio yields on average positive or negative excess returns. In the asset pricing literature, one typically adjusts for autocorrelation by using Newey and West (1987) \(t\)-statistics to test the null hypothesis that average portfolio excess returns are equal to zero. One necessary input for Newey-West standard errors is a chosen bandwidth based on the number of lags employed for the estimation. Researchers often default on choosing a pre-specified lag length of 6 months.

model_fit = (sm.OLS
  .from_formula("long_short ~ 1", data=beta_longshort)
  .fit(cov_type="HAC", cov_kwds={"maxlags": 6})
  .summary(slim=True)
)
model_fit_coefs = model_fit.tables[1]
model_fit_coefs

	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	0.0003	0.001	0.222	0.824	-0.002	0.003

The results indicate that we cannot reject the null hypothesis of average returns being equal to zero. Our portfolio strategy using the median as a breakpoint hence does not yield any abnormal returns. Is this finding surprising if you reconsider the CAPM? It certainly is. The CAPM yields that the high beta stocks should yield higher expected returns. Our portfolio sort implicitly mimics an investment strategy that finances high beta stocks by shorting low beta stocks. Therefore, one should expect that the average excess returns yield a return that is above the risk-free rate.

Functional Programming for Portfolio Sorts

Now we take portfolio sorts to the next level. We want to be able to sort stocks into an arbitrary number of portfolios. For this case, functional programming is very handy: we define a function that give us flexibility concerning which variable to use for the sorting, denoted by sorting_variable. We use np.quantile() to compute breakpoints for n_portfolios. Then, we assign portfolios to stocks using the pd.cut() function. The output of the following function is a new column that contains the number of the portfolio to which a stock belongs.

In some applications, the variable used for the sorting might be clustered (e.g., at a lower bound of 0). Then, multiple breakpoints may be identical, leading to empty portfolios. Similarly, some portfolios might have a very small number of stocks at the beginning of the sample. Cases, where the number of portfolio constituents differs substantially due to the distribution of the characteristics, require careful consideration and, depending on the application, might require customized sorting approaches.

def assign_portfolio(data, sorting_variable, n_portfolios):
    breakpoints = np.quantile(data[sorting_variable].dropna(), 
                              np.linspace(0, 1, n_portfolios + 1), 
                              method="linear")
    assigned_portfolios = pd.cut(data[sorting_variable],
                                 bins=breakpoints,
                                 labels=range(1, breakpoints.size),
                                 include_lowest=True)
    return assigned_portfolios

We can use the above function to sort stocks into ten portfolios each month using lagged betas and compute value-weighted returns for each portfolio. Note that we transform the portfolio column to a factor variable because it provides more convenience for the figure construction below.

beta_portfolios = (data_for_sorts
  .groupby("month", group_keys=False)
  .apply(lambda x: x.assign(portfolio = assign_portfolio(x, 'beta_lag', 10)))
  .groupby(["portfolio", "month"], group_keys=False)
  .apply(lambda x: x.assign(ret=np.average(x["ret_excess"], 
                                           weights=x["mktcap_lag"])))
  .merge(factors_ff_monthly, 
         how="left", 
         on="month")
)

More Performance Evaluation

In the next step, we compute summary statistics for each beta portfolio. Namely, we compute CAPM-adjusted alphas, the beta of each beta portfolio, and average returns.

beta_portfolios_summary = (beta_portfolios
  .groupby("portfolio", group_keys=False)
  .apply(lambda x: x.assign(alpha=sm.OLS.from_formula(
    formula="ret ~ 1 + mkt_excess", data=x).fit().params[0],
                                      beta=sm.OLS.from_formula(
                                      formula="ret ~ 1 + mkt_excess", data=x)
                                      .fit().params[1],
                                      ret=x["ret"].mean()).tail(1))
  .get(["portfolio", "alpha", "beta", "ret"])
  .reset_index(drop=True)
)

Figure 1 illustrates the CAPM alphas of beta-sorted portfolios. It shows that low beta portfolios tend to exhibit positive alphas, while high beta portfolios exhibit negative alphas.

plot_beta_portfolios_summary = (ggplot(beta_portfolios_summary, 
    aes(x="portfolio", y="alpha", fill="portfolio"))
  + geom_bar(stat="identity")
  + labs(title="CAPM alphas of beta-sorted portfolios",
         x="Portfolio", y="CAPM alpha", fill="Portfolio")
  + scale_y_continuous(labels=percent_format())
  + theme(legend_position="none"))
plot_beta_portfolios_summary.draw()

Title: CAPM alphas of beta-sorted portfolios. The figure shows bar charts of alphas of beta-sorted portfolios with the decile portfolio on the horizontal axis and the corresponding CAPM alpha on the vertical axis. Alphas for low beta portfolios are positive, while high beta portfolios show negative alphas. — Figure 1: Portfolios are sorted into deciles each month based on their estimated CAPM beta. The bar charts indicate the CAPM alpha of the resulting portfolio returns during the entire CRSP period.

These results suggest a negative relation between beta and future stock returns, which contradicts the predictions of the CAPM. According to the CAPM, returns should increase with beta across the portfolios and risk-adjusted returns should be statistically indistinguishable from zero.

The Security Market Line and Beta Portfolios

The CAPM predicts that our portfolios should lie on the security market line (SML). The slope of the SML is equal to the market risk premium and reflects the risk-return trade-off at any given time. Figure 2 illustrates the security market line: We see that (not surprisingly) the high beta portfolio returns have a high correlation with the market returns. However, it seems like the average excess returns for high beta stocks are lower than what the security market line implies would be an “appropriate” compensation for the high market risk.

sml_capm = (sm.OLS.from_formula("ret ~ 1 + beta", data=beta_portfolios_summary)
            .fit().params)

plot_sml_capm = (ggplot(beta_portfolios_summary,
    aes(x="beta", y="ret", color="portfolio"))
  + geom_point()
  + geom_abline(intercept=0,
                slope=factors_ff_monthly["mkt_excess"].mean(),
                linetype="solid")
  + geom_abline(intercept=sml_capm["Intercept"],
                slope=sml_capm["beta"],
                linetype="dashed")
  + scale_y_continuous(labels=percent_format(),
                       limits=(0, factors_ff_monthly["mkt_excess"].mean()*2))
  + scale_x_continuous(limits=(0, 2))
  + labs(x="Beta", y="Excess return", color="Portfolio",
         title="Average portfolio excess returns and average beta estimates")
)
plot_sml_capm.draw()

Title: Average portfolio excess returns and average beta estimates. The figure shows a scatter plot of the average excess returns per beta portfolio with average beta estimates per portfolio on the horizontal axis and average excess returns on the vertical axis. An increasing solid line indicates the security market line. A dashed increasing line with lower slope than the security market line indicates that the CAPM prediction is not valid for CRSP data. — Figure 2: Excess returns are computed as CAPM alphas of the beta-sorted portfolios. The horizontal axis indicates the CAPM beta of the resulting beta-sorted portfolio return time series. The dashed line indicates the slope coefficient of a linear regression of excess returns on portfolio betas.

To provide more evidence against the CAPM predictions, we again form a long-short strategy that buys the high-beta portfolio and shorts the low-beta portfolio.

beta_longshort = (beta_portfolios
  .assign(portfolio=lambda x: x["portfolio"]
          .apply(lambda y: "high" if y == x["portfolio"].max()
                                 else ("low" if y == x["portfolio"].min()
                                       else y)))
  .query("portfolio in ['low', 'high']")
  .pivot_table(index="month", columns="portfolio", values="ret")
  .assign(long_short=lambda x: x["high"] - x["low"])
  .merge(factors_ff_monthly, how="left", on="month")
)

Again, the resulting long-short strategy does not exhibit statistically significant returns.

model_fit = (sm.OLS.from_formula("long_short ~ 1", data=beta_longshort)
  .fit(cov_type="HAC", cov_kwds={"maxlags": 1})
  .summary(slim=True)
)
model_fit_coefs = model_fit.tables[1]
model_fit_coefs

	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	0.0023	0.003	0.750	0.453	-0.004	0.008

However, the long-short portfolio yields a statistically significant negative CAPM-adjusted alpha, although, controlling for the effect of beta, the average excess stock returns should be zero according to the CAPM. The results thus provide no evidence in support of the CAPM. The negative value has been documented as the so-called betting against beta factor (Frazzini and Pedersen 2014). Betting against beta corresponds to a strategy that shorts high beta stocks and takes a (levered) long position in low beta stocks. If borrowing constraints prevent investors from taking positions on the SML they are instead incentivized to buy high beta stocks, which leads to a relatively higher price (and therefore lower expected returns than implied by the CAPM) for such high beta stocks. As a result, the betting-against-beta strategy earns from providing liquidity to capital constraint investors with lower risk aversion.

model_fit = (sm.OLS.from_formula("long_short ~ 1 + mkt_excess", data=beta_longshort)
  .fit(cov_type="HAC", cov_kwds={"maxlags": 1})
  .summary(slim=True)
)
model_fit_coefs = model_fit.tables[1]
model_fit_coefs

	coef	std err	z	P>\|z\|	[0.025	0.975]
Intercept	-0.0045	0.002	-2.014	0.044	-0.009	-0.000
mkt_excess	1.1654	0.072	16.212	0.000	1.024	1.306

Figure 3 shows the annual returns of the extreme beta portfolios we are mainly interested in. The figure illustrates no consistent striking patterns over the last years - each portfolio exhibits periods with positive and negative annual returns.

beta_longshort_year = (beta_longshort
  .assign(year=lambda x: x["month"].dt.year)
  .groupby("year")
  .aggregate(low = ("low", lambda x: 1-(1 + x).prod()),
             high = ("high", lambda x: 1-(1 + x).prod()),
             long_short = ("long_short", lambda x: 1-(1 + x).prod()))
  .reset_index()
  .melt(id_vars="year", var_name="name", value_name="value")
)

plot_beta_longshort_year = (ggplot(
  beta_longshort_year, aes(x="year", y="value", fill="name"))
  + geom_col(position='dodge')
  + facet_wrap("~name", ncol=1)
  + theme(legend_position="none")
  + scale_color_discrete(guide=False)
  + scale_y_continuous(labels=percent_format())
  + labs(x="", y="", title="Annual returns of beta portfolios")
)
plot_beta_longshort_year.draw()

Figure 3: We construct portfolios by sorting stocks into high and low based on their estimated CAPM beta. Long short indicates a strategy that goes long into high beta stocks and short low beta stocks.

Overall, this chapter shows how functional programming can be leveraged to form an arbitrary number of portfolios using any sorting variable and how to evaluate the performance of the resulting portfolios. In the next chapter, we dive deeper into the many degrees of freedom that arise in the context of portfolio analysis.

Exercises

Take the two long-short beta strategies based on different numbers of portfolios and compare the returns. Is there a significant difference in returns? How do the Sharpe ratios compare between the strategies? Find one additional portfolio evaluation statistic and compute it.
We plotted the alphas of the ten beta portfolios above. Write a function that tests these estimates for significance. Which portfolios have significant alphas?
The analysis here is based on betas from monthly returns. However, we also computed betas from daily returns. Re-run the analysis and point out differences in the results.
Given the results in this chapter, can you define a long-short strategy that yields positive abnormal returns (i.e., alphas)? Plot the cumulative excess return of your strategy and the market excess return for comparison.

References

Bali, Turan G, Robert F Engle, and Scott Murray. 2016. Empirical asset pricing: The cross section of stock returns. John Wiley & Sons. https://doi.org/10.1002/9781118445112.stat07954.

Frazzini, Andrea, and Lasse Heje Pedersen. 2014. “Betting against beta.” Journal of Financial Economics 111 (1): 1–25. https://doi.org/10.1016/j.jfineco.2013.10.005.

Newey, Whitney K., and Kenneth D. West. 1987. “A simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance Matrix.” Econometrica 55 (3): 703–8. http://www.jstor.org/stable/1913610.