Operational Issues

This page will explore many of the operational issues that are critical to the proper function of a performance management department. For example, these include:

The monthly processing cycle;
Using spreadsheets to do performance analysis;
Relationships with the marketing department and front office;
Data suppliers;
Software suppliers; and
How to staff a performance measurement & attribution team.

Front Office, Middle Office, Back Office

For decades, there has been a distinction between "front office" activities (such as portfolio management and dealing), and "back office" activities (such as portfolio accounting, unit pricing, client reporting, compliance, and marketing). In recent years, there has been increased talk of the "middle office". Middle office activities include performance measurement, attribution, and risk management. One acronym that has been used to describe middle-office activities is RAP (Risk And Performance).

These distinctions are not absolutely clear-cut, and they evolve over time. For example, compliance is also a duty that might very well be handled in the middle office.

There are sensible reasons for keeping RAP separate from both the front office and the back office. On one hand, there is moral hazard in allowing the front office to analyze risk and performance. There is a natural temptation for the front office to over-estimate portfolio performance, and to under-estimate risk. Notoriously, portfolio managers can always find good reasons why the attribution analysis should show them as being more successful. It is therefore a good practice to have a clear separation between the front office and the RAP team. On the other hand, the modern concept of RAP is inconsistent with the traditional idea of a back office which can operate independently of any day-to-day knowledge of how the portfolio is being managed. The RAP team needs to be intimately familiar with the tactics that are being used in each portfolio. Ideally, they should be able to offer the portfolio managers daily feedback on what's happening to the portfolio.

If RAP is treated as a back office function, the typical outcome is "one size fits all" performance reports, which are not properly customized for each different portfolio. So, for example, a particular portfolio might be managed using a bottom-up investment process, based on selecting individual stocks, regardless of factors such as their industry classification. A "one size fits all" attribution report might do sector attribution by industry, whereas in fact the attribution model that is more congruent with a bottom-up management process is the one shown on the Stock Level Attribution page.

In recent years, the role of the middle office team has typically increased in its scope. It is now more common to see middle office team members whose qualifications include postgraduate degrees in finance, or other specialized qualifications in the investment field.

Portfolio Valuation

One of the most fundamental tasks that has to be done regularly is valuing the portfolio. This entails finding the most up-to-date prices for all the assets and liabilities in a portfolio. These prices are then combined with accounting information about the number held for each instrument, in order to determine the value of each holding in the portfolio. By summing the value of a portfolio's holdings, one obtains the value of the entire portfolio. Some unexpectedly difficult aspects of the portfolio valuation process are as follows:

For bank accounts, one needs to calculate the interest which has accrued since the previous valuation. It may be the case that the bank does not provide daily information about accrued interest. In this case, you will need to develop your own ability to estimate the rate at which interest accrues in each different bank account.
Fixed income income instruments accrue coupon payments which are then paid from time to time. From one day to the next, the value of a fixed income instrument will change not just due to factors such as the term structure of the bond market, but also due to the non-stop process of coupon payments accruing. These accruals are an important part of the value of a fixed income security. For this reason, it is common to revalue portfolios when the month-end falls on a week-end.
Dividends on stocks can be tricky to deal with. Consider a stock called XYZ. It becomes clear that the price of XYZ includes an amount for the next dividend payment when the company declares the size of the dividend that will be paid. Later on, there is the ex date, where the share price for XYZ no longer includes the value of the dividend in it. However, at this stage, the dividend has not been received, so somebody needs to hold an accrual for it. Ideally, the accrual should be linked with the portfolio's holding in XYZ, rather than with the cash sector, since it would give a very wrong impression to see that the market value of XYZ had diminished purely due to the stock going ex-dividend. A few weeks later, the pay date may take place. On this date, the portfolio will receive the cash dividend for XYZ. This will appear as a cashflow, and to compensate for this, the dividend accrual should be written down to zero at the same time.
Finding the correct closing price for a security can be difficult. There are a number of different electronic information services. Each of them provides data which is mostly correct, but none of them provides data which is 100% correct. Furthermore, the assumptions or rules that index providers use can vary. Hence, the closing price of the same security can be slightly different according to two different index providers who include it in their indices. It can be different again according to the stock exchange or an electronic data provider. This can produce spurious differences between portfolio and benchmark performance.
Not all securities have liquid markets. Indeed, there are many bonds and derivatives that trade very infrequently. For these securities, the most recent market price may not be a very good estimate of the current market value. Hence the need may arise to use a valuation model (for example, one might use the Black-Scholes model for options).
It can be a problem for a performance measurement system when the portfolio accounting system records cashflows (e.g. dividend payments, purchases, or sales) for dates for which there is no portfolio valuation. One might wonder how these cashflows could arise, if the portfolio accounting system has valued all the securities on every business day. Sometimes the answer is simply that the back office has been working overtime, and the purchases or sales were mistakenly entered for a date on which they did not happen.

There can be tension on the portfolio valuation process, because valuations are needed for various different purposes. For example, on unit-priced funds, there is a strong imperative to carry out valuations within a reasonably short period of time, so that the unit prices can be determined. Yet, on the other hand, for the sake of performance analysis, it is more important for the portfolio valuations to be consistent with the benchmark data, even if this incurs more delay (sometimes benchmark data is delayed considerably due to processing problems, so the unit pricing valuations need to be done before the benchmark data has arrived). One way of resolving this tension, if it ever arises, is to revalue the portfolio using the benchmark data, in order to create the valuations that will be used for the sake of performance analysis. This can be a good trade-off in some cases.

Dependencies on the Portfolio Accounting Process

As the preceding discussion shows, performance analysis can depend heavily on information that flows out of the portfolio accounts. In fact, good practice in performance analysis depends on the performance department, but it also depends heavily on the back office.

This dependency can be a particularly acute issue when back-office activities such as portfolio accounting are outsourced. Problems with the portfolio accounts can impede performance analysis. It is therefore vital to have a good commercial relationship with the back-office (whether or not it is outsourced), so that problems can be fixed -- and more importantly, so that problems do not occur again in the future.

One particularly important issue is the use of trade-date accounting. This simply means that, if an event such as a purchase or sale, or a dividend payment, takes place on a particular date, the portfolio accounts should reflect it as having taken place on that date. If you're accustomed to this convention, it can seem a bit mystifying that it is necessary to explicitly spell-out the requirement for trade-date accounting. However, a number of old-fashioned portfolio accounting systems had no ability to correctly record the dates on which cashflows took place (they merely assumed that a cashflow took place at the same time it was recorded in the portfolio accounting system). The use of trade-date accounting is necessary for producing accurate performance reports. Even when the portfolio accounting system uses trade-date accounting, it can be useful to reconcile cashflows against market values: inconsistencies between the two can lead to errors in performance analysis.

Spreadsheets and Operational Risk

One of the biggest fallacies in performance analysis is the idea that spreadsheets are the best solution, because they seem to come at essentially no cost.

In actual fact, the cost savings that arise from using spreadsheets need to be weighed against the risk of errors.

One public source of information about spreadsheet errors is at the European Spreadsheet Risks Interest Group http://www.eusprig.org/stories.htm. They illustrate common problems and errors that occur with the uncontrolled use of spreadsheets, with comments on the risk and possible avoidance action.

Another good source of information about spreadsheet errors is Ray Panko, from the College of Business Administration at the University of Hawaii:

http://panko.cba.hawaii.edu/ssr/Mypapers/whatknow.htm

One of Panko's most penetrating observations concerns the cognitive phenomenon of overconfidence.

"In experiments, Panko and Halverson (Panko & Halverson, 1997) and Panko and Sprague (Panko & Sprague, 1998) found high confidence among subjects despite the fact that these subjects made many errors. In both studies, furthermore, confidence in correctness was unrelated to the actual number of errors. More recently, Panko and Halverson (2001) had subjects develop a spreadsheet individually or in triads (groups of three). When asked to estimated the probability that their spreadsheet contained an error, individuals had a mean of 18%, while members of triads had a mean of 13%. The actual figures percentage of incorrect spreadsheets 86% and 27% for individuals and triads."

In summary, spreadsheets appear to be cheap and easy. However, they almost lead to errors when they are used for non-trivial tasks. As Panko states, "All in all, the research done to date in spreadsheet development presents a very disturbing picture. Every study that has attempted to measure errors, without exception, has found them at rates that would be unacceptable in any organization."

The high error rates in spreadsheets, and the systematic overconfidence in regard to error rates, combines to make them a very high source of operational risk. However, due to overconfidence, and the appearance that spreadsheets are almost free, it is very difficult to institute reform in this area.

Possible remedies for the operational risk arising from spreadsheets fall into two separate categories:

Firstly, it makes sense to use software that has been professionally developed and tested. This would especially apply to complicated and data-intensive activities.

Secondly, in cases where spreadsheets are used, it would be wise to invest more effort in training, documentation, and in verification that the spreadsheets are correct.

Software for Performance Measurement and Attribution

Given what we know about the limitations of spreadsheets, it is very clear that a proper software system for investment performance measurement and attribution is essential for any professional investment management firm. Some key decisions are:

How to ensure that the portfolio data will be adequate for use in the system;
Where to obtain index data;
How to integrate the system with the firm's other business processes;
Considering the "buy or build" issue (see below);
Choosing a system:
- Evaluating the technology;
- Evaluating the calculation methods;
- Negotiating commercial terms with the vendor;
- Designing reports.
Planning the implementation (this is a very high-risk area);
Carrying out the implementation;
Planning the day-to-day operations, and carrying them out.

There are a number of commercially-available software systems for investment performance measurement and attribution. The dispersion of prices between different systems is very wide. Similarly, the dispersion of capabilities between different systems is very wide. To put it bluntly, some of the higher-priced systems are not very good.

Like all major capital expenditures, choosing a system requires specialist skills. Part of the decision is whether to "buy or build". Building a system is only likely to be a realistic option for a firm that has a large, well-resourced team of specialists. However, most performance measurement and attribution systems allow for some degree of customization (for example, in how the data is loaded, or in how the reports are designed). In this way, the "buy or build" decision is not a simple either/or choice.

Typically, when an investment management firm licenses a system for performance measurement and attribution, they will continue to use the same system for many years, if not decades. Accordingly, it is very worthwhile to make a careful decision. Having said this, one should remember that "making a careful decision" is quite different from "not getting around to making a decision". Many firms spend years deciding what to do about a performance measurement and attribution system. Taking this long to make a decision is a very bad idea, because the opportunity set of products that is available changes considerably from one year to the next.

More To Come...

More information will be added to this page on a later date.