The Misuse of Correlation Part 2 – the results

In this post, I want to talk about an insidious error that creeps in with the usage of correlation in finance.

FT lexicon supports the idea that:

a correlation is said to be positive if movements between the two variables are in the same direction and negative if it moves in the opposite direction.”

This definition is not unusual, commonly seen in finance textbooks.

Occasionally the formula may be presented:

But caveats in using the formula will likely be absent or at the least hidden from view.
By that, I mean the terms and are critically important but this importance is rarely appreciated. [1]

In the previous post, I asked you about the correlation of the changes of the two assets in the chart below:

a. Positive correlation

b. Negative correlation

c. They are uncorrelated

d. Not sure (be honest!)


The most obvious answer is of course b)
One line goes up and the other goes down so this means they have a negative correlation. This is unfortunately strictly incorrect if you paid attention to the instruction to consider the “changes in the two assets”.

A good answer is d)
Given the amount of information I had supplied, it’s a perfectly reasonable one.

Because another answer is a)

The correlation of the changes in the variables is +1, PERFECT POSITIVE correlation
and the lines are going in the OPPOSITE direction!!

(If you doubt this result please look at the data and calculations in the sheet attached (download) and use the CORREL function in excel.)

b) is an intuitive answer but a) is the answer that a financial analyst would calculate. If you imagine of situations where you are being given financial advice, it is clear there could be an immediate conflict!


First insidious confusion – the importance of the mean

If you have never seen this before, you may think I am lying or this is a convoluted trick. But it rests upon one key part of the calculation of correlation that is missing from virtually every definition I see, and is certainly missing from the vast bulk of work done by analysts in the finance industry.

The key is that correlation is calculated by looking at the relationship in deviations from the means (the terms and in the complicated mathematical equation).

In our example, the changes in the two variables in the chart have equal and opposite means, and so trend in different directions. However, the day to day volatility (deviation from the mean of the changes) is identical for both variables, and it is this term that drives the correlation whilst having no impact on the trend.

Here is a scatterplot of the % changes for each variable. Observe all the dots are distributed along the line – a perfect POSITIVE correlation.

This has a clear relationship to the way we think about the change in market prices of any asset:

In financial markets, the daily noise is usually much greater than the daily trend, and so forms the focus of most market commentary.

The key result is that if the noise term correlates for two assets, then they will correlate irrespective of their underlying trend, given the way correlation is calculated.
i.e. they could end up in very different places even if they are positively correlated!

Second insidious confusion – levels vs changes

The second insidious confusion can arise from a reference to correlation of the CHANGES or a correlation in the LEVELS of the two variables.

In financial markets, the method invariably used is to look at the changes in variables. In our example, we get the answer of positive 1 i.e. perfect positive correlation.

If we calculate the correlation using the levels or prices, we get an answer of -0.97
i.e. strong negative correlation

The intuitive result is the opposite of the result most likely to be calcuated by financial analysts.

Why does finance prefer the use the correlation of changes?

It is done for good reason. When you are looking at data with strong trends, as a lot of asset prices do, the correlation of levels can yield very strange results. Let’s take an example.

Let’s look at the US equity market (S&P 500 price – white line) and its PE ratio (orange line) over the last 30 years.

If we first look at the correlation of levels, we get a correlation of virtually zero.
This suggests a rather unintuitive result that there is no meaningful correlation between PE ratio and equity prices!

If we instead look at the correlation of changes, we get that there is a meaningful positive correlation of 0.78 which makes a lot more sense.

Conclusion

If these differences in the correlation results is were just some statistical fluke, from a couple of silly examples, then it would not matter.
But it is not an unusual result and it occurs when looking at the biggest and most commonly traded financial markets. It is therefore critical to avoid confusions such as these when thinking about what type of correlation to use or, more often, what someone else has used in the analysis you are reading.


[1] I very much enjoyed this paper by Francois-Serge Lhabitant which explains this issue very well. http://www.edhec-risk.com/edhec_publications/all_publications/RISKReview.2011-09-07.3757/attachments/EDHEC_Working_Paper_Correlation_vs_Trends_F.pdf

The misuse of Correlation Part 1 – Quick Refresher and Quiz

First, let’s refresh our memories of what correlation means.
This may seem very basic right now, but I would like to make sure the meaning is clear before we move on to its use.

I have included a question at the end, once you have read and thought about the definition:

  • A definition from the FT Lexicon:
    “a correlation is said to be positive if movements between the two variables are in the same direction and negative if it moves in the opposite direction.”
  • You can read examples in a number of sources such as

https://www.mathsisfun.com/data/correlation.html

and
http://www.bbc.co.uk/schools/gcsebitesize/maths/statistics/scatterdiagramsrev2.shtml

Here is a range of correlations, shown via a scatterplot:

Some important concepts

  • A positive correlation is “when the values increase together”
    An example would be temperature and ice cream sales as “warmer weather and higher sales go together”.
  • A negative correlation is “when one value increases and the other decreases
    Note this is sometimes called an “inverse correlation”.
    An example would be weight of a car and its fuel efficiency as “cars that are heavier tend to get less miles per gallon.”
  • No correlation is when “there is no connection”. An example would be IQ and house number.”
  • For those of you with a more formal approach the mathematical formula for correlation is:
  • In practice, most of us find it much easier to use the function CORREL() in Excel!

Question time

Here is an example with two asset prices A and B. When we represent the data in a chart it can often be done in one of two ways.

This chart has two lines, showing how both the prices of asset A and B moved over time.

The other way to chart this is to put the prices of A and B on the two axes instead. It looks like this.

To make sure you have understood the basic concept of correlation, I would appreciate it if you could vote on an answer to the following question. (all anonymous of course!)

Career Tips

I was asked recently to speak at an undergraduate event. Part of it was to give some career advice in the form of 3 tips. Here is what I came up with:

Many people after leaving university find adjusting to the world of work difficult and become very unhappy. Focusing on a lack of “meaning” in their job while searching for a “mentor” to guide them, they can quickly come to resent their firm and co-workers.

It does not have to be this way.

The most important thing to realise is that the workplace is not going to feel like an extension of education – it is completely and fundamentally different. For at least the first two decades of your life, focusing on your knowledge and your skills is the key and the whole environment around you is geared to helping you develop. However, the ability of a student to successfully transition into a happy and productive career has remarkably little to do with the knowledge and skills they start with.

What really matters is how well they can change their mindset.

Here are 3 things to focus on:

  1. It’s not about you any more

This is the piece of advice students generally find the most upsetting. A big change in mindset is required to succeed in a work environment compared to the one needed for education.

In education, the student is the product. The ultimate aim for a student, with the help of teachers, is to gain the skills and knowledge required to pass exams. This does not mean that students have complete free rein to do what they want. There will be various restrictions on behaviour, such as a requirement to go to lectures, prepare for tutorials, do reading, problem sets and essays – however these are all designed with the success of the student in mind. The best attitude for the student is to be focused on themselves and their own needs.

In the workplace, the business is the product. The ultimate aim for a new employee is to become useful. Many graduates find this transition to the workplace a shock. Senior members of staff may not think that a key part of their role is to educate you and make you more productive or happy. In a few years’ time, you will also be more senior too and it will be obvious to you that this is not a priority either. You will want to be productive at work, impress your boss, get promoted, get a bonus etc.

Adjusting to this new reality, the best attitude is to be focused, not on yourself, but on the needs of the people around you and of the firm – Be useful! You will then find good things will start to happen to you. Given reciprocity (see “Influence: The Psychology of Persuasion” by Robert Cialdini), people you help will also help you. Senior people will start to spend time helping you learn and improve. You will have signalled to the firm that you have the right mentality to succeed and so will be promoted more quickly, paid more and given more training.

Having a real job is extremely helpful in preparing you for work and choosing a career path. I spent my Gap year working full time as an economist, but working at McDonalds may have perhaps been even better. You need to understand what it is like to be the other side of the counter.

  1. Be flexible.

In education, a targeted focus and narrow determination are extremely helpful for excelling with high results. The world of academia is fragmented and siloed, with status derived from expertise in ever more specialised areas.

The world of work is very different. A modern and successful career will come with many parallel and some orthogonal leaps into new areas, combined with an ability to master a broad range of cross-disciplinary problems.

I could easily have become a consultant or economist and I think I would have really enjoyed it and been successful. In banking and hedge funds, my career could have gone in lots of different directions. The only way to take opportunities is by being open minded.

  1. Work with people you would like to become.

This piece of advice was given to me as an undergraduate, and it has repeatedly proven itself true as my career developed.

Don’t think that you can join an Investment bank for the money and not become like them. Either you will change to fit in, or you will not and you will hate it and leave.

You must judge it from meeting real employees, not from impressions from TV shows. Being a lawyer is not the way it is on Suits just as being a Hedge Fund manager is not like Billions (well mostly anyway). That is why internships are so useful.

Conclusion

The world of work is can be a stimulating and fulfilling experience. For that to happen you need to be able to have the right mindset to take advantage of the opportunities on offer.

Money 4 – Why does it matter?

The elimination of money from economics theory and teaching leads to major practical problems.

  1. Why did we have the financial crisis and the prolonged recession?

The Queen famously asked why economists failed to see the crisis and ensuing recession coming. What is less talked about is how they subsequently also failed to understand a) what was happening as it was occurring and b) the nature of the recovery. Once you appreciate that money and credit are central to a modern economy, and academic macroeconomists were using models without money or credit, this failure is much easier to understand.

Some policy makers did a better job of learning and adapting to the crisis. Ben Bernanke, at the US Fed, with his study of the 1930s depression years, was well placed to support the economy once the crisis was underway. The Bank of England was not so well led. Mervyn King appeared to believe in a banking model in which the lender of last resort need not exist. When this model failed to have any correspondence to reality, he acted as though reality was at fault, not his personal model.

The financial crisis and its aftermath was predicted and understood by some people however.
But they were likely to be eclectic economists, on the fringes of the mainstream, who did not exclude the views of Keynes and Minsky for their lack of “microfoundations”.

  1. Why did the enormous monetary stimulus not lead to a stronger recovery?

The answer is that the monetary stimulus was not so enormous. The numbers were large, but the transmission mechanism was very weak, and therefore the recovery has been slower than most predicted.

Another misunderstanding follows, since the recovery has been slower than expected, new ideas have been sought to explain it away, such as secular stagnation. But once you accept the idea that QE is eye-catching, but not very powerful for the economy (it may be more powerful for asset prices but that is a different matter) then the slow recovery is not so surprising.

  1. Why do we ever have unemployment at all?

The academic models we have been looking at, theoretically make the existence of unemployment impossible. Given that this is evidently not the case, the models must be augmented with ad-hoc frictions, to make them have some connection to observed reality.

If money is allowed in the model at the start then you do not run into such issues, and surely this is evidence that the theories with don’t include it, don’t make much sense.

Why do economists believe these myths?

If an economist is typical pressed on this, responses vary from claiming that the representation is broadly accurate (it is not!) or more likely that it does not matter (it does!). If the assumption does not matter, why choose such a strange one?

A more recent defence has been that the latest batch of sophisticated new Keynesian models incorporate money and credit and a banking sector. But if that is the case why not change all the teaching? Why is money tacked onto the end of a model rather than incorporated as a critical building block?

I think that they attempt to tack money onto the end of their model building because it is not possible to incorporate it at the start. The assumptions which exclude money are critically important to the complex mathematical models that the current breed of academic economists revel in building. The worry for me is that armed with them, they go on to lead to key policy and market implications. It would also be fair to say that pretty much everything I do in studying the macroeconomy would not be classed as macroeconomics by a current mainstream academic.

Modern academic economists believe that conversations about macroeconomics should be based upon General Equilibrium (GE) and rational expectations and have “microfoundations”. The most recent iteration is the Dynamic Stochastic General Equilibrium (DSGE) model. GE is a truly majestic piece of mathematics which describes an economic system based upon essentially perfect barter.

The concept of money is added as purely commodity money. Any asset can be arbitrarily chosen as the denominator in which to price all others, it is just the numeraire. This helps with the solution as it reduces the number of independent variables by one when solving a set of simultaneous equations.

The advantage of building models in this way is that you can translate many concepts used in micro economics and apply them to macroeconomic questions. This is known as “microfoundations” and many Noble Prizes have been won, tying the neat General Equilibrium theory up with clever mathematics.

After the financial crisis, it is obvious that money and credit had to be included, and so the most recent batch of Neo-Keynesian models attempt to do so. But this is an ad hoc tacking on of a couple of new variables that do not connect to the central mechanism of the model. I see these models as sophisticated in the same vein as the geocentric models used to argue against Galileo.

If we use Kuhn’s model of paradigms, then this looks like economists trying to bury “anomalies” during a period of “model drift” when their models are increasingly unable to answer the questions people think matter. The next stage is “model crisis”. Or perhaps we are already there.

Relationship to Politics and Free-market thinking

This model creates the illusion of a perfect economy in which everything works, with the practicalities of reality being termed “imperfections” such as imperfect competition or sticky wages. This links strongly to the ideology of free markets being the answer to all questions i.e. the idea is to make reality behave more like the model.

Economists of a more interventionist or left-wing persuasion can exist within this paradigm. But ad hoc elements such as asymmetric information have to be added, combined with some pretty inventive and tortuous modelling, eventually producing models which suggest intervention is the correct policy response.

Conclusion

Recent mathematical models cannot be held responsible for the birth of the myths of money and banking. In Classical economics the concept of value is separate from money and logically prior to it and so JS Mill told us that “There cannot, in short, be intrinsically a more insignificant thing, in the economy of society, than money”.

We have recently seen stirrings from eminent economists that all is not well with the profession, https://piie.com/system/files/documents/pb16-11.pdf, but it is not yet filtering through to how the subject is being taught at grass roots.

Where we are left is a deeply divided set of disciplines. Practitioners, both in financial markets and many Central Bankers have a different approach to pure academics. But even academia is split between macroeconomists who study an economy without money and Finance professors who study a monetary system without an economy.

Both can be seen, to borrow a phrase from Keynes, as “an extraordinary example of how, starting with a mistake, a remorseless logician can end in bedlam”.

Money 3 Banking – a money creation myth

I will again use the bestselling undergraduate textbook “Macroeconomics” by Greg Mankiw as the source for this story.


How does money get created?

In the beginning, there were bank reserves……

 

  1. The Central Bank determine Bank Reserves – this is the money that the banks have on deposit with the Central Bank
  2. Banks, holding these reserves, then lend them out to customers, who then either put that money on deposit themselves or transfer the money to someone else who does. The mountain of customer deposits is generally many times larger than bank reserves. This is known as the “reserve-deposit ratio, rr”.
    It is an “exogenous variable” in other words “the model takes as given”.
  3. There is also the currency in circulation which the central bank also determines.
    This is known as the “currency-deposit ratio, cr”. It is also an “exogenous variable”.
  4. Combining these via “money multiplier”, gives you the Money Supply.

In this way, the Central Bank determines the level of reserves, and thus controls the money supply in a predictable way.


From bank reserves to money supply to inflation……

The next stage in this story is the “Quantity Theory of Money” which “remains the leading explanation for how money affects the economy in the long run.”
This starts with a key identity or equation:




Add a few assumptions….

P * Y is also nominal GDP, so if we assume that V (the income velocity of money) is constant (or “exogenous”), then a change in M leads to a change in nominal GDP.

Via a separate assumption, the level of output Y is determined by a production function which does not include money, therefore a change in M leads to a change in P i.e. changing the money supply causes inflation.

Economists are taught this key conclusion at university:
Thus, … the central bank, which controls the money supply, has ultimate control over the rate of inflation.”

Unfortunately, like many Creation Myths none of this is true.
As Mankiw says, this model “is simplified. That is not necessarily a problem.”
I agree, “all models are simplified”. But when causation is the wrong way around, this is a massive problem.

Banking – a personal perspective

When I left university, I became a trader at a bank, spending many years on a money markets desk. It is at the least glamorous end of trading, but I found it fascinating being at the centre of the banking system, funding the bank’s activities, and forming the link between the Central Bank and the markets. What was immediately striking was that bank operations were nothing like the models I had been taught at university.

In the story above, the driving force is the Central Bank adding reserves, causing banks to lend money. The mechanisms described are correct, just in the exact opposite order. The actual sequence goes something like this:

  1. Customer decides to buy something and uses credit card for the purchase.
  2. Transaction goes through. i.e. bank lends the customer money for the purchase.
  3. The money shows up as a credit entry on the shop’s bank account and a debit entry on the credit card.
  4. The banking system now has a debit and a credit. Banks move the money between them to square their accounts.
  5. It is important to note that the central bank wasn’t required to do anything in this process.

What if the customer takes out cash? Then the banking system is short of reserves.
This is not a problem as the central bank just adds or takes out reserves on a regular basis to make sure the banking system has exactly as much as it demands.

It is not the case that the Central Bank tells the bank funding desk it has more reserves, who then calls round the rest of the bank to tell them to do some more lending. In simple terms the central bank sets the rate of interest (Fed Funds in the US and the Base Rate in the UK) and then supplies money as demanded. The supply of money is determined completely by the demand for money.

Evidence

Correlation of money supply with inflation

Milton Friedman and Anna Schwartz “wrote two treatises on monetary history that documented the sources and effects of changes in the quantity of money over the past century.” What they did was document that money supply and inflation are positively correlated. This is a most obvious prediction from either story of money, and so I have never seen an argument that it supports one over the other. As inflation rises, then more money will be demanded in the economy to facilitate transactions. The central bank accommodates this so we see a direct relationship between money and inflation. This tells us nothing about causation. Friedman’s attempts to show causation by econometric tricks with “long and variable lags” are completely bogus.

Stability of velocity of money

An argument for why one can assume V is a constant, is that historically, over short periods, it has been. Unfortunately, this again is a direct prediction from both stories. If the central bank always supplies as much money as is demanded then there is no reason for velocity to change.

Why did QE not lead to hyperinflation?

According to this monetary theory in the textbooks, the vast increase in reserves caused by Quantitative Easing should have led to an explosion in bank lending and a rapid rise in inflation.

This clearly hasn’t been the case, and in fact, the taught theories really struggle with reality here. They are forced to rely on ad hoc and non-quantitative explanations such as “animal spirits” or a reduction in confidence. Since this “confidence” is not directly predictable or even observable, it requires a leap of faith, equivalent to “magic”.

It’s a wonderful coincidence that a model which predicts a MASSIVE stimulus finds, in reality, an unseen counterbalancing force which is of EXACTLY the same magnitude. But still, I read that MV=PY holds and the miraculous drop in V to exactly offset the rise in M, was a bizarre coincidence and that once the velocity of money rises back to “normal”, inflation will come.

There is a much simpler explanation. The amount of loans created by banks was never constrained by reserves and so increasing reserves has no effect on the behaviour of banks or their clients.

What do central bankers say they are doing?

Central bankers involved in monetary policy and the oversight of the banking system must understand how banking works. What do they say is going on?
They agree with my model and say that “the reality of how money is created today differs from the description found in some economics textbooks” and describe the model that is taught as “some popular misconceptions”. http://www.bankofengland.co.uk/publications/Documents/quarterlybulletin/2014/qb14q1prereleasemoneycreation.pdf

Conclusion

Money is simply not exogenous and does not cause inflation.

Amongst practitioners, including bankers and central bankers, this is obviously well understood. Bagehot famously described it perfectly in 1873. What is striking is the contrast to academic economists who persist with a very different mythical version of banking and continue to educate our bright, young minds with a story of pure fantasy. So why do they do it? I will speculate on that in the next post.

Brexit – Deal or No Deal?

Before the election

Before the general election, whether we reached a deal with the EU or not, did not seem to matter much. We seemed determined to head for a Hard Brexit, which looked similar to No Deal.

With the UK outside the Single Market and Customs Union, we would face a massive negative economic shock and potentially a resulting financial crisis. I did not discount the idea that we might not leave the EU or get a Soft Brexit, but did not see a path to get there. (https://appliedmacro.com/2017/05/04/what-game-are-brexiteers-playing/)

This all changed with the election…..

The election result

After the election, there is no longer a mandate, either for Hard Brexit or for exiting without a deal. I loved the article in the FT last week which laid out 6 options (https://www.ft.com/content/7be243f2-51ae-11e7-a1f2-db19572361bb) It is very funny and I highly recommend a read. I will leave the comedy to Robert Shrimsley, and think about what paths are likely now.

What is striking about any possible path is that none have a mandate or parliamentary majority.
Let’s start by separating them into:

  1. Deal
  2. No Deal

What does a Deal look like?

A deal will have to cover an extremely broad range of areas:

  • Rights of UK citizens in the EU and EU citizens in the UK
  • Settling the accounts (“Brexit Bill”)
  • What to do with Northern Ireland
  • Single Market
  • Customs Union
  • Trade agreement
  • Services agreement
  • Role of ECJ
  • Regulation
  • Tax
  • Etc ……
    the mind boggles at how complex this is.

It is very hard for me to see the current UK government managing to cobble together a common negotiating position on all of these in its tenuous state, never mind go through all the details with the EU.

Where next?

Although the tide of public opinion seems to have turned against her mindless slogans, Theresa May seems likely to go on, for as long as possible, using silly phrases like “Brexit means Brexit” and “Strong and Stable” and “Red, White and Blue Brexit”. Her problem is that any attempt to lay out a strategy with sufficient clarity that the EU can understand it, is likely to bring down her government.

There is no parliamentary majority FOR any Brexit option, but I think there is a parliamentary majority AGAINST all of them.

The fall of the government would lead to another election. In this, I am not too optimistic that the main parties will lay out clear and distinct sets of ideas on Brexit. May previously tried this and completely failed. The “success” of Corbyn was perhaps to be so incoherent on Brexit that no one understood it well enough to object to it.

So again, how can a government without a clear UK negotiating position make a deal?

Maybe they just can’t.


What does no Deal look like?

Which means we get left with No Deal.

This has two clear and distinct paths:

i) Stay in the EU

ii) Leave without a Deal

Now I really do not know which is more likely.

A new Referendum posing this much clearer question?
My money would be that the vote will go against whatever the government in power proposes.
If May wants to Remain then we will Leave, if she wants Hard Brexit we will stay.

Brexit has gone from being an odds-on fiasco to utter chaos.
I am not sure yet if this is better or worse.

What have we learned about the parties from the election?

I think worth laying out a few ideas on what I have learned after this election.

Things that seem clear – but we probably knew already

  • May is a terrible leader
  • May is a terrible campaigner
  • The Conservative Party is deeply divided
    and may break out into open civil war at any moment

Things that seem new

  • The country is dominated by division on age lines not class lines
    I found this chart to be shockingly good as a model for voting behaviour.
    I am at exactly the crossover age, which perhaps explains why I dislike both so much!

  • Your income says nothing about who you will vote for
    The next chart is also stunning

Things that seem incorrect

· For Labour, Corbyn is an electoral asset whose positive campaign resonated with the electorate and he is poised to win next time.

Corbyn set expectations so low this feels like a landslide victory for him. But it was not.

1979 Callaghan 269 seats, he admitted defeat & resigned.

1992 Kinnock 271 seats, he admitted defeat & resigned.

2017 Corbyn 262 seats, claims victory and orders the winner to resign.

This was yet another vote AGAINST the elites.
Labour’s campaign was much better than the Tories but the idea of a Progressive Alliance doesn’t really exist. The Progressive Alliance is defined by its OPPOSITION to the Tories.
In that way, it reminds me of the Lib Dem party. A group of disparate protest votes united through negative cohesion as they all hate the same things. But once in power, they fall to pieces losing large portions of their supporters feel betrayed by almost any action they take.

However if the seemingly inevitable Tory civil war is bitter enough, maybe even Corbyn can win.

  • Hard Brexit is unpopular

A mandate for a Hard Brexit was the central theme of May’s campaign. But the Brexiteers have perhaps, done far too good a job of convincing the electorate that Brexit has already happened. The continual media suggestions that the economy is doing much better than predicted post the Brexit vote, helps to support the myth that Brexit has already happened.  If Brexit has already happened then it is hard to argue that it is still the most important issue.

If this is the case then it was not just the negative campaign against Corbyn that backfired, it was that the central “positive” element just didn’t make any sense and Labour’s focus on more traditional election issues such as Health and Education resonated far more strongly.

Hard Brexit is very unpopular with me and I think is the dominant issue we face. But I do not think my views are commonly shared by the electorate.

  • Opinion polls are worthless

I think the problem is that people do not understand what a poll is, and especially what uncertainties are inherent in them. This means that they get over-interpreted, and then the source is later blamed for not producing for which was impossible in the first place.

What I learned from the opinion polls during the election period was, not that the Tories had a big lead, but that the polls were massively volatile. Minor changes in methodology between polling companies seemed to be part of the outsized swings day to day – this is extremely useful information. The conclusion being that the uncertainty band around the poll should be seen to be extremely high, and this result fell easily within my uncertainty band.