In this chapter, we conduct portfolio backtesting in a realistic setting by including transaction costs and investment constraints such as no-short-selling rules. We start with standard mean-variance efficient portfolios and introduce constraints in a step-by-step manner. To do so, we rely on numerical optimization procedures in R. We conclude the chapter by providing an out-of-sample backtesting procedure for the different strategies that we introduce in this chapter.

Throughout this chapter, we use the following packages:

library(tidyverse)
library(RSQLite)
library(scales)
library(alabama)

Compared to previous chapters, we introduce the quadprog package to perform numerical constrained optimization for quadratic objective functions and alabama for more general non-linear objective functions and constraints.

## Data Preparation

We start by loading the required data from our SQLite-database introduced in Chapters 2-4. For simplicity, we restrict our investment universe to the monthly Fama-French industry portfolio returns in the following application.

tidy_finance <- dbConnect(
SQLite(),
"data/tidy_finance.sqlite",
extended_types = TRUE
)

industry_returns <- tbl(tidy_finance, "industries_ff_monthly") |>
collect()

industry_returns <- industry_returns |>
select(-month)

## Recap of Portfolio Choice

A common objective for portfolio optimization is to find mean-variance efficient portfolio weights, i.e., the allocation which delivers the lowest possible return variance for a given minimum level of expected returns. In the most extreme case, where the investor is only concerned about portfolio variance, she may choose to implement the minimum variance portfolio (MVP) weights which are given by the solution to $w_\text{mvp} = \arg\min w'\Sigma w \text{ s.t. } w'\iota = 1$ where $$\Sigma$$ is the $$(N \times N)$$ covariance matrix of the returns. The optimal weights $$\omega_\text{mvp}$$ can be found analytically and are $$\omega_\text{mvp} = \frac{\Sigma^{-1}\iota}{\iota'\Sigma^{-1}\iota}$$. In terms of code, the math is equivalent to the following chunk.

Sigma <- cov(industry_returns)
w_mvp <- solve(Sigma) %*% rep(1, ncol(Sigma))
w_mvp <- as.vector(w_mvp / sum(w_mvp))

Next, consider an investor who aims to achieve minimum variance given a required expected portfolio return $$\bar{\mu}$$ such that she chooses $w_\text{eff}({\bar{\mu}}) =\arg\min w'\Sigma w \text{ s.t. } w'\iota = 1 \text{ and } \omega'\mu \geq \bar{\mu}.$ We leave it as an exercise below to show that the portfolio choice problem can equivalently be formulated for an investor with mean-variance preferences and risk aversion factor $$\gamma$$. That means the investor aims to choose portfolio weights as the solution to $w^*_\gamma = \arg\max w' \mu - \frac{\gamma}{2}w'\Sigma w\quad \text{ s.t. } w'\iota = 1.$ The solution to the optimal portfolio choice problem is: $\omega^*_{\gamma} = \frac{1}{\gamma}\left(\Sigma^{-1} - \frac{1}{\iota' \Sigma^{-1}\iota }\Sigma^{-1}\iota\iota' \Sigma^{-1} \right) \mu + \frac{1}{\iota' \Sigma^{-1} \iota }\Sigma^{-1} \iota.$ Empirically, this classical solution imposes many problems. In particular, the estimates of $$\mu$$ are noisy over short horizons, the ($$N \times N$$) matrix $$\Sigma$$ contains $$N(N-1)/2$$ distinct elements and thus, estimation error is huge. Seminal papers on the effect of ignoring estimation uncertainty, among others, are Brown (1976), Jobson and Korkie (1980), Jorion (1986), and Chopra and Ziemba (1993).

Even worse, if the asset universe contains more assets than available time periods $$(N > T)$$, the sample covariance matrix is no longer positive definite such that the inverse $$\Sigma^{-1}$$ does not exist anymore. To address estimation issues for vast-dimensional covariance matrices, regularization techniques are a popular tool (see, e.g., Ledoit and Wolf 2003, 2004, 2012; Fan, Fan, and Lv 2008).

While the uncertainty associated with estimated parameters is challenging, the data-generating process is also unknown to the investor. In other words, model uncertainty reflects that it is ex-ante not even clear which parameters require estimation (for instance, if returns are driven by a factor model, selecting the universe of relevant factors imposes model uncertainty). Wang (2005) and Garlappi, Uppal, and Wang (2007) provide theoretical analysis on optimal portfolio choice under model and estimation uncertainty. In the most extreme case, Pflug, Pichler, and Wozabal (2012) shows that the naive portfolio which allocates equal wealth to all assets is the optimal choice for an investor averse to model uncertainty.

On top of the estimation uncertainty, transaction costs are a major concern. Rebalancing portfolios is costly, and, therefore, the optimal choice should depend on the investor’s current holdings. In the presence of transaction costs, the benefits of reallocating wealth may be smaller than the costs associated with turnover. This aspect has been investigated theoretically, among others, for one risky asset by Magill and Constantinides (1976) and Davis and Norman (1990). Subsequent extensions to the case with multiple assets have been proposed by Balduzzi and Lynch (1999) and Balduzzi and Lynch (2000). More recent papers on empirical approaches which explicitly account for transaction costs include Gârleanu and Pedersen (2013), and DeMiguel, Nogales, and Uppal (2014), and DeMiguel, Martín-Utrera, and Nogales (2015).

## Estimation Uncertainty and Transaction Costs

The empirical evidence regarding the performance of a mean-variance optimization procedure in which you simply plug in some sample estimates $$\hat \mu$$ and $$\hat \Sigma$$ can be summarized rather briefly: mean-variance optimization performs poorly! The literature discusses many proposals to overcome these empirical issues. For instance, one may impose some form of regularization of $$\Sigma$$, rely on Bayesian priors inspired by theoretical asset pricing models or use high-frequency data to improve forecasting . One unifying framework that works easily, effectively (even for large dimensions), and is purely inspired by economic arguments is an ex-ante adjustment for transaction costs .

Assume that returns are from a multivariate normal distribution with mean $$\mu$$ and variance-covariance matrix $$\Sigma$$, $$N(\mu,\Sigma)$$. Additionally, we assume quadratic transaction costs which penalize rebalancing such that \begin{aligned} \nu\left(\omega_{t+1},\omega_{t^+}, \beta\right) = \frac{\beta}{2} \left(\omega_{t+1} - \omega_{t^+}\right)'\left(\omega_{t+1}- \omega_{t^+}\right),\end{aligned} with cost parameter $$\beta>0$$ and $$\omega_{t^+} = {\omega_t \circ (1 +r_{t})}/{\iota' (\omega_t \circ (1 + r_{t}))}$$. $$\omega_{t^+}$$ denotes the portfolio weights just before rebalancing. Note that $$\omega_{t^+}$$ differs mechanically from $$\omega_t$$ due to the returns in the past period. Intuitively, transaction costs penalize portfolio performance when the portfolio is shifted from the current holdings $$\omega_{t^+}$$ to a new allocation $$\omega_{t+1}$$. In this setup, transaction costs do not increase linearly. Instead, larger rebalancing is penalized more heavily than small adjustments. Then, the optimal portfolio choice for an investor with mean variance preferences is \begin{aligned}\omega_{t+1} ^* &= \arg\max \omega'\mu - \nu_t (\omega,\omega_{t^+}, \beta) - \frac{\gamma}{2}\omega'\Sigma\omega \text{ s.t. } \iota'\omega = 1\\ &=\arg\max \omega'\mu^* - \frac{\gamma}{2}\omega'\Sigma^* \omega \text{ s.t.} \iota'\omega=1,\end{aligned} where $\mu^*=\mu+\beta \omega_{t^+} \quad \text{and} \quad \Sigma^*=\Sigma + \frac{\beta}{\gamma} I_N.$ As a result, adjusting for transaction costs implies a standard mean-variance optimal portfolio choice with adjusted return parameters $$\Sigma^*$$ and $$\mu^*$$: $\omega^*_{t+1} = \frac{1}{\gamma}\left(\Sigma^{*-1} - \frac{1}{\iota' \Sigma^{*-1}\iota }\Sigma^{*-1}\iota\iota' \Sigma^{*-1} \right) \mu^* + \frac{1}{\iota' \Sigma^{*-1} \iota }\Sigma^{*-1} \iota.$

An alternative formulation of the optimal portfolio can be derived as follows: $\omega_{t+1} ^*=\arg\max \omega'\left(\mu+\beta\left(\omega_{t^+} - \frac{1}{N}\iota\right)\right) - \frac{\gamma}{2}\omega'\Sigma^* \omega \text{ s.t. } \iota'\omega=1.$ The optimal weights correspond to a mean-variance portfolio, where the vector of expected returns is such that assets that currently exhibit a higher weight are considered as delivering a higher expected return.

## Optimal Portfolio Choice

The function below implements the efficient portfolio weight in its general form, allowing for transaction costs (conditional on the holdings before reallocation). For $$\beta=0$$, the computation resembles the standard mean-variance efficient framework. gamma denotes the coefficient of risk aversion $$\gamma$$, beta is the transaction cost parameter $$\beta$$ and w_prev are the weights before rebalancing $$\omega_{t^+}$$.

compute_efficient_weight <- function(Sigma,
mu,
gamma = 2,
beta = 0, # transaction costs
w_prev = rep(
1 / ncol(Sigma),
ncol(Sigma)
)) {
iota <- rep(1, ncol(Sigma))
Sigma_processed <- Sigma + beta / gamma * diag(ncol(Sigma))
mu_processed <- mu + beta * w_prev

Sigma_inverse <- solve(Sigma_processed)

w_mvp <- Sigma_inverse %*% iota
w_mvp <- as.vector(w_mvp / sum(w_mvp))
w_opt <- w_mvp + 1 / gamma *
(Sigma_inverse - w_mvp %*% t(iota) %*% Sigma_inverse) %*%
mu_processed
return(as.vector(w_opt))
}

mu <- colMeans(industry_returns)
compute_efficient_weight(Sigma, mu)
 [1]  1.395  0.293 -1.390  0.477  0.363 -0.320  0.545  0.446 -0.132
[10] -0.675

The portfolio weights above indicate the efficient portfolio for an investor with risk aversion coefficient $$\gamma=2$$ in absence of transaction costs. Some of the positions are negative which implies short-selling, most of the positions are rather extreme. For instance, a position of $$-1$$ implies that the investor takes a short position worth her entire wealth to lever long positions in other assets. What is the effect of transaction costs or different levels of risk aversion on the optimal portfolio choice? The following few lines of code analyze the distance between the minimum variance portfolio and the portfolio implemented by the investor for different values of the transaction cost parameter $$\beta$$ and risk aversion $$\gamma$$.

transaction_costs <- expand_grid(
gamma = c(2, 4, 8, 20),
beta = 20 * qexp((1:99) / 100)
) |>
mutate(
weights = map2(
.x = gamma,
.y = beta,
~ compute_efficient_weight(Sigma,
mu,
gamma = .x,
beta = .y / 10000,
w_prev = w_mvp
)
),
concentration = map_dbl(weights, ~ sum(abs(. - w_mvp)))
)

The code chunk above computes the optimal weight in presence of transaction cost for different values of $$\beta$$ and $$\gamma$$ but with the same initial allocation, the theoretical optimal minimum variance portfolio. Starting from the initial allocation, the investor chooses her optimal allocation along the efficient frontier to reflect her own risk preferences. If transaction costs would be absent, the investor would simply implement the mean-variance efficient allocation. If transaction costs make it costly to rebalance, her optimal portfolio choice reflects a shift toward the efficient portfolio, whereas her current portfolio anchors her investment.

transaction_costs |>
mutate(risk_aversion = as_factor(gamma)) |>
ggplot(aes(
x = beta,
y = concentration,
color = risk_aversion,
linetype = risk_aversion
)) +
geom_line() +
guides(linetype = "none") +
labs(
x = "Transaction cost parameter",
y = "Distance from MVP",
color = "Risk aversion",
title = "Portfolio weights for different risk aversion and transaction cost"
)

Figure 1 shows rebalancing from the initial portfolio (which we always set to the minimum variance portfolio weights in this example). The higher the transaction costs parameter $$\beta$$, the smaller is the rebalancing from the initial portfolio. In addition, if risk aversion $$\gamma$$ increases, the efficient portfolio is closer to the minimum variance portfolio weights such that the investor desires less rebalancing from the initial holdings.

## Constrained Optimization

Next, we introduce constraints to the above optimization procedure. Very often, typical constraints such as short-selling restrictions prevent analytical solutions for optimal portfolio weights (short-selling restrictions simply imply that negative weights are not allowed such that we require that $$w_i \geq 0 \forall i$$). However, numerical optimization allows computing the solutions to such constrained problems. For the purpose of mean-variance optimization, we rely on the solve.QP() function from the package quadprog.

The function solve.QP() delivers numerical solutions to quadratic programming problems of the form $\min(-\mu \omega + 1/2 \omega' \Sigma \omega) \text{ s.t. } A' \omega >= b_0.$ The function takes one argument (meq) for the number of equality constraints. Therefore, the above matrix $$A$$ is simply a vector of ones to ensure that the weights sum up to one. In the case of short-selling constraints, the matrix $$A$$ is of the form $A' = \begin{pmatrix}1 & 1& \ldots&1 \\1 & 0 &\ldots&0\\0 & 1 &\ldots&0\\\vdots&&\ddots&\vdots\\0&0&\ldots&1\end{pmatrix}'\qquad b_0 = \begin{pmatrix}1\\0\\\vdots\\0\end{pmatrix}.$

Before we dive into constrained optimization, we revisit the unconstrained problem and replicate the analytical solutions for the minimum variance and efficient portfolio weights from above. We verify that the output is equal to the above solution. Note that near() is a safe way to compare two vectors for pairwise equality. The alternative == is sensitive to small differences that may occur due to the representation of floating points on a computer, while near() has a built-in tolerance. As just discussed, we set Amat to a matrix with a column of ones and bvec to 1 to enforce the constraint that weights must sum up to one. meq=1 means that one (out of one) constraints must be satisfied with equality.

n_industries <- ncol(industry_returns)

w_mvp_numerical <- solve.QP(
Dmat = Sigma,
dvec = rep(0, n_industries),
Amat = cbind(rep(1, n_industries)),
bvec = 1,
meq = 1
)

all(near(w_mvp, w_mvp_numerical$solution)) [1] TRUE w_efficient_numerical <- solve.QP( Dmat = 2 * Sigma, dvec = mu, Amat = cbind(rep(1, n_industries)), bvec = 1, meq = 1 ) all(near(compute_efficient_weight(Sigma, mu), w_efficient_numerical$solution))
[1] TRUE

The result above shows that indeed the numerical procedure recovered the optimal weights for a scenario, where we already know the analytic solution. For more complex optimization routines, R’s optimization task view provides an overview of the vast optimization landscape.

Next, we approach problems where no analytical solutions exist. First, we additionally impose short-sale constraints, which implies $$N$$ inequality constraints of the form $$w_i >=0$$.

w_no_short_sale <- solve.QP(
Dmat = 2 * Sigma,
dvec = mu,
Amat = cbind(1, diag(n_industries)),
bvec = c(1, rep(0, n_industries)),
meq = 1
)
w_no_short_sale$solution  [1] 5.17e-01 3.06e-18 4.27e-16 7.90e-02 3.47e-18 2.65e-17 1.48e-01 [8] 2.56e-01 8.74e-18 0.00e+00 As expected, the resulting portfolio weights are all positive (up to numerical precision). Typically, the holdings in the presence of short-sale constraints are concentrated among way fewer assets than for the unrestricted case. You can verify that sum(w_no_short_sale$solution) returns 1. In other words: solve.QP() provides the numerical solution to a portfolio choice problem for a mean-variance investor with risk aversion gamma = 2, where negative holdings are forbidden.

solve.QP() is fast because it benefits from a very clear problem structure with a quadratic objective and linear constraints. However, optimization often requires more flexibility. As an example, we show how to compute optimal weights, subject to the so-called Regulation T-constraint, which requires that the sum of all absolute portfolio weights is smaller than 1.5, that is $$\sum_{i=1}^N |w_i| \leq 1.5$$. The constraint enforces that a maximum of 50 percent of the allocated wealth can be allocated to short positions, thus implying an initial margin requirement of 50 percent. Imposing such a margin requirement reduces portfolio risks because extreme portfolio weights are not attainable anymore. The implementation of Regulation-T rules is numerically interesting because the margin constraints imply a non-linear constraint on the portfolio weights. Thus, we can no longer rely on solve.QP(), which is defined as solving quadratic programming problems with linear constraints. Instead, we rely on the package alabama, which requires a separate definition of objective and constraint functions.

initial_weights <- rep(
1 / n_industries,
n_industries
)

objective <- function(w, gamma = 2) {
-t(w) %*% (1 + mu) +
gamma / 2 * t(w) %*% Sigma %*% w
}

inequality_constraints <- function(w, reg_t = 1.5) {
reg_t - sum(abs(w))
}

equality_constraints <- function(w) {
sum(w) - 1
}

w_reg_t <- constrOptim.nl(
par = initial_weights,
hin = inequality_constraints,
fn = objective,
heq = equality_constraints,
control.outer = list(trace = FALSE)
)
w_reg_t$par  [1] 3.41e-01 -1.81e-06 -1.08e-01 1.40e-01 7.30e-02 -1.08e-02 [7] 2.46e-01 3.14e-01 1.35e-01 -1.30e-01 Note that the function constrOptim.nl() requires a starting vector of parameter values, i.e., an initial portfolio. Under the hood, alamaba performs numerical optimization by searching for a local minimum of the function objective() (subject to the equality constraints equality_constraints() and the inequality constraints inequality_constraints()). Note that the starting point should not matter if the algorithm identifies a global minimum. Figure 2 shows the optimal allocation weights across all 10 industries for the four different strategies considered so far: minimum variance, efficient portfolio with $$\gamma$$ = 2, efficient portfolio with short-sale constraints, and the Regulation-T constrained portfolio. tibble( No short-sale = w_no_short_sale$solution,
Minimum Variance = w_mvp,
Efficient portfolio = compute_efficient_weight(Sigma, mu),
Regulation-T = w_reg_t$par, Industry = colnames(industry_returns) ) |> pivot_longer(-Industry, names_to = "Strategy", values_to = "weights" ) |> ggplot(aes( fill = Strategy, y = weights, x = Industry )) + geom_bar(position = "dodge", stat = "identity") + coord_flip() + labs( y = "Allocation weight", fill = NULL, title = "Optimal allocations for different strategies" ) + scale_y_continuous(labels = percent) The results clearly indicate the effect of imposing additional constraints: the extreme holdings the investor implements if she follows the (theoretically optimal) efficient portfolio vanish under, e.g., the Regulation-T constraint. You may wonder why an investor would deviate from what is theoretically the optimal portfolio by imposing potentially arbitrary constraints. The short answer is: the efficient portfolio is only efficient if the true parameters of the data generating process correspond to the estimated parameters $$\hat\Sigma$$ and $$\hat\mu$$. Estimation uncertainty may thus lead to inefficient allocations. By imposing restrictions, we implicitly shrink the set of possible weights and prevent extreme allocations, which could result from error-maximization due to estimation uncertainty . Before we move on, we want to propose a final allocation strategy, which reflects a somewhat more realistic structure of transaction costs instead of the quadratic specification used above. The function below computes efficient portfolio weights while adjusting for transaction costs of the form $$\beta\sum_{i=1}^N |(w_{i, t+1} - w_{i, t^+})|$$. No closed-form solution exists, and we rely on non-linear optimization procedures. compute_efficient_weight_L1_TC <- function(mu, Sigma, gamma = 2, beta = 0, initial_weights = rep( 1 / ncol(Sigma), ncol(Sigma) )) { objective <- function(w) { -t(w) %*% mu + gamma / 2 * t(w) %*% Sigma %*% w + (beta / 10000) / 2 * sum(abs(w - initial_weights)) } w_optimal <- constrOptim.nl( par = initial_weights, fn = objective, heq = function(w) { sum(w) - 1 }, control.outer = list(trace = FALSE) ) return(w_optimal$par)
}

## Out-of-Sample Backtesting

For the sake of simplicity, we committed one fundamental error in computing portfolio weights above: we used the full sample of the data to determine the optimal allocation . To implement this strategy at the beginning of the 2000s, you will need to know how the returns will evolve until 2021. While interesting from a methodological point of view, we cannot evaluate the performance of the portfolios in a reasonable out-of-sample fashion. We do so next in a backtesting application for three strategies. For the backtest, we recompute optimal weights just based on past available data.

The few lines below define the general setup. We consider 120 periods from the past to update the parameter estimates before recomputing portfolio weights. Then, we update portfolio weights which is costly and affects the performance. The portfolio weights determine the portfolio return. A period later, the current portfolio weights have changed and form the foundation for transaction costs incurred in the next period. We consider three different competing strategies: the mean-variance efficient portfolio, the mean-variance efficient portfolio with ex-ante adjustment for transaction costs, and the naive portfolio, which allocates wealth equally across the different assets.

window_length <- 120
periods <- nrow(industry_returns) - window_length

beta <- 50
gamma <- 2

performance_values <- matrix(NA,
nrow = periods,
ncol = 3
)
colnames(performance_values) <- c("raw_return", "turnover", "net_return")

performance_values <- list(
"MV (TC)" = performance_values,
"Naive" = performance_values,
"MV" = performance_values
)

w_prev_1 <- w_prev_2 <- w_prev_3 <- rep(
1 / n_industries,
n_industries
)

We also define two helper functions: one to adjust the weights due to returns and one for performance evaluation, where we compute realized returns net of transaction costs.

adjust_weights <- function(w, next_return) {
w_prev <- 1 + w * next_return
as.numeric(w_prev / sum(as.vector(w_prev)))
}

evaluate_performance <- function(w, w_previous, next_return, beta = 50) {
raw_return <- as.matrix(next_return) %*% w
turnover <- sum(abs(w - w_previous))
net_return <- raw_return - beta / 10000 * turnover
c(raw_return, turnover, net_return)
}

The following code chunk performs a rolling-window estimation, which we implement in a loop. In each period, the estimation window contains the returns available up to the current period. Note that we use the sample variance-covariance matrix and ignore the estimation of $$\hat\mu$$ entirely, but you might use more advanced estimators in practice.

for (p in 1:periods) {
returns_window <- industry_returns[p:(p + window_length - 1), ]
next_return <- industry_returns[p + window_length, ] |> as.matrix()

Sigma <- cov(returns_window)
mu <- 0 * colMeans(returns_window)

w_1 <- compute_efficient_weight_L1_TC(
mu = mu,
Sigma = Sigma,
beta = beta,
gamma = gamma,
initial_weights = w_prev_1
)

performance_values[[1]][p, ] <- evaluate_performance(w_1,
w_prev_1,
next_return,
beta = beta
)

# Naive portfolio
w_2 <- rep(1 / n_industries, n_industries)

performance_values[[2]][p, ] <- evaluate_performance(
w_2,
w_prev_2,
next_return
)

# Mean-variance efficient portfolio (w/o transaction costs)
w_3 <- compute_efficient_weight(
Sigma = Sigma,
mu = mu,
gamma = gamma
)

performance_values[[3]][p, ] <- evaluate_performance(
w_3,
w_prev_3,
next_return
)

}

Finally, we get to the evaluation of the portfolio strategies net-of-transaction costs. Note that we compute annualized returns and standard deviations.

performance <- lapply(
performance_values,
as_tibble
) |>
bind_rows(.id = "strategy")

performance |>
group_by(strategy) |>
summarize(
Mean = 12 * mean(100 * net_return),
SD = sqrt(12) * sd(100 * net_return),
Sharpe ratio = if_else(Mean > 0,
Mean / SD,
NA_real_
),
Turnover = 100 * mean(turnover)
)
# A tibble: 3 × 5
strategy   Mean    SD Sharpe ratio Turnover
<chr>     <dbl> <dbl>          <dbl>    <dbl>
1 MV       -0.635  12.5         NA     213.
2 MV (TC)  12.3    15.0          0.820   0.0298
3 Naive    12.3    15.0          0.818   0.230 

The results clearly speak against mean-variance optimization. Turnover is huge when the investor only considers her portfolio’s expected return and variance. Effectively, the mean-variance portfolio generates a negative annualized return after adjusting for transaction costs. At the same time, the naive portfolio turns out to perform very well. In fact, the performance gains of the transaction-cost adjusted mean-variance portfolio are small. The out-of-sample Sharpe ratio is slightly higher than for the naive portfolio. Note the extreme effect of turnover penalization on turnover: MV (TC) effectively resembles a buy-and-hold strategy which only updates the portfolio once the estimated parameters $$\hat\mu_t$$ and $$\hat\Sigma_t$$indicate that the current allocation is too far away from the optimal theoretical portfolio.

## Exercises

1. Consider the portfolio choice problem for transaction-cost adjusted certainty equivalent maximization with risk aversion parameter $$\gamma$$ $\omega_{t+1} ^* = \arg\max_{\omega \in \mathbb{R}^N, \iota'\omega = 1} \omega'\mu - \nu_t (\omega, \beta) - \frac{\gamma}{2}\omega'\Sigma\omega$ where $$\Sigma$$ and $$\mu$$ are (estimators of) the variance-covariance matrix of the returns and the vector of expected returns. Assume for now that transaction costs are quadratic in rebalancing and proportional to stock illiquidity such that $\nu_t\left(\omega, B\right) = \frac{\beta}{2} \left(\omega - \omega_{t^+}\right)'B\left(\omega - \omega_{t^+}\right)$ where $$B = \text{diag}(ill_1, \ldots, ill_N)$$ is a diagonal matrix, where $$ill_1, \ldots, ill_N$$. Derive a closed-form solution for the mean-variance efficient portfolio $$\omega_{t+1} ^*$$ based on the transaction cost specification above. Discuss the effect of illiquidity $$ill_i$$ on the individual portfolio weights relative to an investor that myopically ignores transaction costs in her decision.
2. Use the solution from the previous exercise to update the function compute_efficient_weight() such that you can compute optimal weights conditional on a matrix $$B$$ with illiquidity measures.
3. Illustrate the evolution of the optimal weights from the naive portfolio to the efficient portfolio in the mean-standard deviation diagram.
4. Is it always optimal to choose the same $$\beta$$ in the optimization problem than the value used in evaluating the portfolio performance? In other words: can it be optimal to choose theoretically sub-optimal portfolios based on transaction cost considerations that do not reflect the actual incurred costs? Evaluate the out-of-sample Sharpe ratio after transaction costs for a range of different values of imposed $$\beta$$ values.

## References

Arnott, Rob, Campbell R. Harvey, and Harry Markowitz. 2019. A backtesting protocol in the era of machine learning.” The Journal of Financial Data Science 1 (1): 64–74. https://doi.org/10.3905/jfds.2019.1.064.
Balduzzi, Pierluigi, and Anthony W Lynch. 1999. Transaction costs and predictability: Some utility cost calculations.” Journal of Financial Economics 52 (1): 47–78. https://doi.org/10.1016/S0304-405X(99)00004-5.
———. 2000. Predictability and transaction costs: The impact on rebalancing rules and behavior.” The Journal of Finance 55 (5): 2285–2309. https://www.jstor.org/stable/222490.
Brown, Stephen J. 1976. Optimal portfolio choice under uncertainty: A Bayesian approach.” PhD Thesis, University of Chicago.
Chopra, Vijay Kumar, and William T. Ziemba. 1993. The effect of errors in means, variances, and covariances on optimal portfolio choice.” Journal of Portfolio Management 19 (2): 6–11. https://doi.org/10.3905/jpm.1993.409440.
Davis, Mark H. A., and Andrew R. Norman. 1990. Portfolio selection with transaction costs.” Mathematics of Operations Research 15 (4): 676–713. https://doi.org/10.1287/moor.15.4.676.
DeMiguel, Victor, Alberto Martín-Utrera, and Francisco J. Nogales. 2015. Parameter uncertainty in multiperiod portfolio optimization with transaction costs.” Journal of Financial and Quantitative Analysis 50 (6): 1443–71. https://doi.org/10.1017/S002210901500054X.
DeMiguel, Victor, Francisco J. Nogales, and Raman Uppal. 2014. Stock return serial dependence and out-of-sample portfolio performance.” Review of Financial Studies 27 (4): 1031–73. https://doi.org/10.1093/rfs/hhu002.
Fan, Jianqing, Yingying Fan, and Jinchi Lv. 2008. High dimensional covariance matrix estimation using a factor model.” Journal of Econometrics 147 (1): 186–97. https://doi.org/10.1016/j.jeconom.2008.09.017.
Garlappi, Lorenzo, Raman Uppal, and Tan Wang. 2007. Portfolio selection with parameter and model uncertainty: A multi-prior approach.” Review of Financial Studies 20 (1): 41–81. https://doi.org/10.1093/rfs/hhl003.
Gârleanu, Nicolae, and Lasse Heje Pedersen. 2013. Dynamic trading with predictable returns and transaction costs.” The Journal of Finance 68 (6): 2309–40. https://doi.org/10.1111/jofi.12080.
Hautsch, Nikolaus, Lada M. Kyj, and Peter Malec. 2015. Do high-frequency data improve high-dimensional portfolio allocations? Journal of Applied Econometrics 30 (2): 263–90. https://doi.org/10.1002/jae.2361.
Hautsch, Nikolaus, and Stefan Voigt. 2019. Large-scale portfolio allocation under transaction costs and model uncertainty.” Journal of Econometrics 212 (1): 221–40. https://doi.org/10.1016/j.jeconom.2019.04.028.
Jagannathan, Ravi, and Tongshu Ma. 2003. Risk reduction in large portfolios: Why imposing the wrong constraints helps.” The Journal of Finance 58 (4): 1651–84. https://doi.org/10.1111/1540-6261.00580.
Jobson, David J., and Bob Korkie. 1980. Estimation for Markowitz efficient portfolios.” Journal of the American Statistical Association 75 (371): 544–54. https://doi.org/10.1080/01621459.1980.10477507.
Jorion, Philippe. 1986. Bayes-Stein estimation for portfolio analysis.” Journal of Financial and Quantitative Analysis 21 (03): 279–92. https://doi.org/10.2307/2331042.
Kan, Raymond, and Guofu Zhou. 2007. Optimal portfolio choice with parameter uncertainty.” Journal of Financial and Quantitative Analysis 42 (3): 621–56. https://doi.org/10.1017/s0022109000004129.
Ledoit, Olivier, and Michael Wolf. 2003. Improved estimation of the covariance matrix of stock returns with an application to portfolio selection.” Journal of Empirical Finance 10 (5): 603–21. https://doi.org/10.1016/S0927-5398(03)00007-0.
———. 2004. Honey, I shrunk the sample covariance matrix.” The Journal of Portfolio Management 30 (4): 110–19. https://doi.org/10.3905/jpm.2004.110.
———. 2012. Nonlinear shrinkage estimation of large-dimensional covariance matrices.” The Annals of Statistics 40 (2): 1024–60. https://doi.org/10.1214/12-AOS989.
Magill, Michael J P, and George M. Constantinides. 1976. Portfolio selection with transactions costs.” Journal of Economic Theory 13 (2): 245–63. https://doi.org/10.1016/0022-0531(76)90018-1.
Pflug, Georg, Alois Pichler, and David Wozabal. 2012. The 1/N investment strategy is optimal under high model ambiguity.” Journal of Banking & Finance 36 (2): 410–17. https://doi.org/10.1016/j.jbankfin.2011.07.018.
Turlach, Berwin A., Andreas Weingessel, and Cleve Moler. 2019. quadprog: Functions to solve quadratic programming problems. https://CRAN.R-project.org/package=quadprog.
Varadhan, Ravi. 2022. alabama: Constrained nonlinear optimization. https://CRAN.R-project.org/package=alabama.
Wang, Zhenyu. 2005. A shrinkage approach to model uncertainty and asset allocation.” Review of Financial Studies 18 (2): 673–705. https://www.jstor.org/stable/3598049.