Mondo Visione Worldwide Financial Markets Intelligence

FTSE Mondo Visione Exchanges Index:

Three Decades of Indexing: What Have We Learned About Indexing and Index Construction

Date 07/06/2004

Burton G. Malkiel
Princeton University

I have believed in indexing even before index funds existed.  I noted in the first edition of my book, A Random Walk Down Wall Street, more than 30 years ago that our stock markets are remarkably efficient and that index funds were likely to outperform all but a handful of active managers.  The evidence from the past 30 years makes me more convinced than ever.

Evidence of Market Efficiency

For me, the most direct and most convincing evidence of market efficiency are tests of the ability of professional fund managers to outperform the market as a whole.  A remarkably large body of evidence over the past 30 years suggests that professional investment managers are not able to outperform index funds that buy and hold the broad stock market portfolio.  Exhibit 1 indicates that the Standard and Poor’s 500 Stock Index regularly beats about two thirds of active managers.  Over the past 20 years, more than 80 percent of active managers have been outperformed by the index and an S&P 500 Index Fund has beaten the average equity mutual fund by over 200 basis points per year.  The 200 basis point difference can be explained entirely by differences in expenses.

Exhibit 1

Comparison of Returns-Average Equity Fund vs. Indexes to 12/31/0
  1 Yr 3 Yrs 5 Yrs 10 Yrs 20 Yrs
S&P vs. Large Cap Equity Funds 73% 72% 63% 86% 90%
Percentage of Large Capitalization Equity Funds Outperformed by Index Ending December 31, 2003

        10 Yrs 20 Yrs
S&P 500       10.99% 12.78%
Average Equity Fund*       8.47% 10.54%
S&P 500 Advantage

(percentage points)

      2.52 2.24

Source: Lipper, Wilshire, & the Vanguard Group.

*Consists of ALL Lipper equity categories.  Fund Returns are show after expenses.

Exhibit 2 shows that over the more than the thirty-year period from the 1970 through the end of 2003, one can literally count on the fingers of one’s hand the number of equity funds that have significantly beaten the market.  Interestingly, we note that there were 355 general equity funds in 1970, but almost 200 of them did not survive.  The 156 that are plotted in the graph are the survivors and we know that these are the funds with the best records.  Poorly performing funds tend not to survive.  Exhibit 3 shows a similar record for actively managed global equity funds.

I believe that the advantages of indexing are likely to be even greater in the future.  During the 1982-March 2000 period the United States market as a whole produced 18 percent returns in large part because of changes in the valuation of stocks.  The P/E multiple for the market as a whole rose from about 8 in 1982 to almost 30 in early 2000.  In such an environment, investors were not too unhappy paying 200 basis points of extra expenses because net returns of 16 percent seemed quite generous.  But one can’t expect P/E multiples to rise from their levels in 2004.  If P/E multiples remain unchanged, the gross return from the stock market will most likely be in the high single digit range for the period ahead.  In a low return environment, the low cost of indexing will be more important than ever.

Exhibit 2

The Odds of Success

Returns of Surviving Mutual Funds* 1970 - 12/2003
Number of Equity Funds 1970: 355
Number of Equity Funds 2003: 139
Non-survivors:  216

*Compared with the S&P 500 Stock Index

Exhibit 3

The Odds of Success

Returns of Surviving Global Active Equity Funds*

*Compared with the MSCI World Index

A simple illustration will illuminate the point.  If the stock market delivers an 8 percent annual return over the next 35 years, an investment of $1,000 today will grow to $16,000 at the end of the period (quarterly compounding—dividends reinvested).  But the investor who buys an actively-managed fund with 200 basis points of extra expenses will realize only a 6 percent annual return.  At 6 percent, a $1,000 investment grows to about $8,000 in 35 years—only half the amount realized by the index investor.

Problems in Index Construction

We have also learned that problems can arise from decisions that are made concerning how indexes are constructed and how they are altered over time.  Experience has shown that three problems deserve particular attention.

1. The Weighting Scheme

Most indexes are capitalization weighted which is the appropriate weighting for investment managers who replicate the index as an investment portfolio.  The advantage is that as particular stocks rise in value, their increased share in the capitalization of the index is automatically reflected in the manager’s portfolio without the need for any portfolio trades.  Capitalization weighting can create problems, however, when governments, other businesses, or insiders own a significant share of the total capitalization that they do not intend to sell.  As a result, these companies get a significantly greater representation in the index than they have in the open market of tradable shares.  In certain countries, such as Japan, where significant business cross-ownership of shares is common, a world capitalization weighted index will tend to overweight Japan.  When Company A owns half of Company B and vice versa, one in effect double counts the weight of the two companies in the index by simply summing their total capitalizations.  This was one reason why Japan tended to be overweighted in the MSCI EAFE index during the decade of the 1990s and why active managers tended to outperform EAFE.  All one had to do to outperform EAFE was to weight Japan in terms of its GDP or the capitalization of its float rather than in terms of its market capitalization.  The same issue can arise in United States domestic indexes as well when companies such as Lockheed Martin and UPS have significant numbers of untradeable shares.

2. Index Turnover

When stocks are added or removed from an index, the transaction takes place without friction that is assuming that no transactions costs are incurred.  But managers of index funds do incur transactions costs when they buy and sell.  Moreover, changes to popular indexes such as the S&P 500 usually lead to large—even if temporary—price changes that complicate the ability of managers to keep their portfolios in line with the index.  Some indexes such as the Russell 2000 experienced a turnover of almost 50 percent in the year 2000 when the index was reconstituted on June 30, forcing index managers to make massive trades.  Exhibit 4 shows the turnover of selected indexes since 1998.  Moreover, the Russell procedure of rebalancing on a single day creates substantial performance distortions around the rebalancing date.  It has been estimated that the speculative trading activity that surrounds rebalancing depresses the returns of the Russell 2000 Index by approximately 200 basis points per year.  The problem arises because speculators can easily predict in advance those stocks likely to be added to the index and their buying can result in those stocks entering the index at artificially inflated prices.  Similarly, a stock that will have to leave the index because its market cap has fallen will be sold prior to the reconstitution date depressing its price and the index return.  Indeed, part of the success of some so-called “enhanced indexers” undoubtedly results from their exploitation of some of the anomalies involved in the construction of the index.  Also, some of the apparent underperformance of the S&P 600 index as well as the performance of active small cap managers may be explained by the same phenomenon.  The problem can be particularly acute for bond indexes.  A bond removed after a rating decrease may be removed at its month-end bid price but the liquidity of the issue may be severely compromised and index bond managers are unlikely to find buyers at that price.

Exhibit 4 - Estimates of Index Fund Turnover

S&P 500 S&P 500/Barra Value S&P 500/Barra Growth
1998 6.9% 31.9% 24.3%
1999 5.4% 33.8% 31.6%
2000 7.1% 28.6% 31.9%
2001 2.5% 30.6% 27.4%
2002 2.9% 17.1% 16.7%
2003 2.8% 14.3% 15.5%
Average 4.6% 26.1% 24.6%
  Russell 2000 Russell 2000
Value
Russell 2000
Growth
1998 32.0% 41.7% 40.6%
1999 35.4% 30.1% 56.2%
2000 47.6% 41.7% 69.6%
2001 37.3% 50.0% 41.3%
2002 31.3% 50.3% 37.7%
2003 21.9% 36.3% 39.2%
Average 34.3% 41.7% 47.4%

Sources:  Frank Russell Co. and The Vanguard Group of Investment Companies

3. Problems with Sector Indexes

Special problems may arise for sector indexes.  Suppose, for example, that an investor wished to purchase a large-capitalization value index fund.  The first issue is how to divide the large-cap index (the S&P 500, for example) into its constituent growth and value segments.  One method, the one employed by Standard and Poor’s, is to divide stocks according to their price to book value ratios (P/BV).  Stock with low P/BV are considered “value stocks.”  Stocks with high P/BV ratios are considered “growth” stocks.  The Russell indexes add earnings growth as another criterion and MSCI adds more parameters.  The different methodologies can produce very different investment results and an index fund manager may be subjected to transactions charges as individual companies shift from growth to value (and vice versa) as their ratios tend to shift over time.

The Future of Indexing and Index Construction

With this discussion as background, let me now turn to the future and present my own expectations concerning the future of indexing and improvements in the construction of indexes.

1. Growth

First, I am convinced that the use of indexing will continue to grow.  Today, only about 10 percent of investment funds are indexed in the United States and an even lower percentage is indexed abroad.  I am convinced that that the proportion of indexed funds will increase over the years ahead.  There is no doubt that indexing works.  Moreover, the importance of minimizing costs will increase in the years ahead if, as I believe, we are living in a low return environment for the foreseeable future.  In a low return environment, both individual and institutional investors will, I believe, come to better appreciate the enormous advantages of low cost indexing. 

2. Indexing the Core of the Portfolio

At the very least, I anticipate that more investors will choose to index the core of their portfolios and use active managers for particular sectors of the market.  I am not convinced that active management works any better for small capitalization stocks or for non-U.S. stocks than it does for large capitalization U.S. stocks.  While it is true that the market maybe somewhat less efficient in these sectors, this relative lack of efficiency may actually make indexing particularly useful.  In many foreign markets and in the market for small capitalization stocks, bid-asked spreads tend to be very much wider and, therefore, the cost of trading is increased.  Moreover, in many foreign stock markets, a variety of transfer taxes increases the cost of trading even more.  Nevertheless, I am well aware that investors will still want to seek enhanced returns and, by indexing the core of their portfolios, they can do so with very much less risk than would be the case if the entire portfolio was actively managed.

3. Emphasis on Broader Indexes

I believe that in the future there will be more emphasis on broader rather than narrower indexes.  In my judgment, the intellectual case for indexing is strongest if one holds the entire market portfolio.  In such a case, one holds both small and large capitalization stocks, both “growth” and “value” stocks.  When a small capitalization stock becomes successful, one should not be required to add it to the portfolio, the investor should already own it.  When value stocks outperform growth stocks, the broad base indexer will know that her portfolio will be well represented in the better performing sector.  Transactions costs will be minimized and the investor will be assured of earning the total market return.  To the extent that there is a perfect index, it should include all the stocks in a market that are available for purchase.  I am persuaded that attempting to move back and forth between styles and categories is simply impossible.  No one is likely to be consistently successful betting against the market weighting.

4. Indexes will be Better Constructed

I believe that in the future we will learn from some of our past errors with respect to index definition and that indexes will evolve to be more optimally constructed.  Indexes will tend to be float weighted rather than capitalization weighted.  The definitions of “value” and “growth” as well as “small” and “large” will more likely be in terms of overlapping bands rather than definitive cut off points.  Moreover, the migration of stocks between sub indexes should be a gradual transition over time rather than a 100 percent move on a single day.  The best indexes will be those where turnover is minimized and index fund managers will be able to adjust their portfolios slowly overtime.  A major problem with many current indexes is that turnover is too high and adjustments are too sudden.

5. ETFs will Continue to Grow

Exchange traded funds have certain tax advantages over regular mutual funds because of their ability to affect redemptions in kind that are not considered taxable events for the fund itself.  Moreover, to the extent that the ETF represents a style, sector, or industry, the ETF form is ideally suited for investors who want to gain exposure to that segment of the market.  While I am highly skeptical of investors’ ability to make shifts between various styles, sectors, or industries, there is no question that many investors are convinced that they can add value by doing so.  If such switches are made within the mutual fund framework, transactions costs are created for buy and hold investors.  Everyone can gain if switchers and market timers can be migrated from mutual funds to exchange traded funds.

Conclusion

In summary, I believe that indexing has proved itself as an optimal investment strategy over the past three decades.  In the words of Jack Bogle, the founder of the Vanguard Group, “Indexing wins.”  And it wins by a substantial margin.

We have learned, however, that there have been problems that arise for index fund managers depending upon the particular methods used in index construction.  I have made reference to the most important of them and suggested that indexes in the future will tend to be better constructed so that turnover is minimized.  In my judgment, these changes will tend to increase the effectiveness of index fund management.  And I believe that the best indexes are those that are the most broadly based.

I recognize that there will continue to be a role for active management and investors will continue an attempt to gain excess returns by investing in illiquid asset classes such as venture capital, real estate and so forth.  But they will be able to do so with greater confidence if the core of their portfolios is indexed.  In that connection, I expect that so-called enhanced indexing will play a role as well.  With enhanced indexing, risk is minimized and expense ratios tend to be only slightly higher then are available from pure indexing.  Investors will increasingly come to realize, however, that in many cases they are paying active management fees for funds that, in effect, behave little differently from index funds themselves.