Applied Macro

The misuse of Significance

Definition

What does the word “significant” mean?

Dictionaries most often suggest a range of closely related definitions.
In a more everyday sense:

Importance e.g. this new discovery is a significant development
Meaningful e.g. the significance of the message was not lost on John

In mathematics, you get the example of:

Significant figures – e.g. 1.524658 is 1.5 to 2 sig fig

This use of the word is mathematical jargon with a precise meaning, but it also tallies with our general use of the word. We only want to look at the digits which are important and mean something.

In statistics:

“significant” means probably true (not due to chance)

Some issues arise from this

Something statistically significant may not be important

A result may be true and therefore significant when backed up by statistics, it doesn’t however mean it is important in the more standard English usage sense. I think this statistical interpretation can easily come into conflict with the everyday meaning and is fraught with danger.

When you jump out of a plane without a parachute it is likely that holding up an umbrella has a “significant” effect on your speed. I doubt you would think that this effect was important when you hit the ground.

I’m sure you can think of many things that are probably true but not important!

Statistical relationships are not transitive

An example from medicine, drugs for the most part are tested against a placebo rather than against each other. Drug A may perform better in tests against a placebo than Drug B. (ie has more significant results) However, that does not mean you know that Drug A will perform better in tests against Drug B. Unfortunately, current medical practice makes this implicit assumption when approving drugs.

This is a common misconception that you can use simple logic to infer other relationships. Unfortunately, this is not true. There is a similarly confused relationship with correlation. Statistical relationships like this are not transitive. https://iase-web.org/documents/papers/isi56/CPM80_CastroSotos.pdf

The 5% threshold for statistical significance is arbitrary

When you say that one result is significant and another is not because one has a 4.9% chance of being random and the other has 5.1%. This is the correct usage of the technical term but people ascribe more meaning to the word than that. One of the ideas is held to be “true” and the other is discarded.

A significant result may have happened by random chance

Saying that a certain outcome would only occur 1 time in 20 if it were random sounds good. But what if you ran 20 sets of analysis? By random chance you should expect one of them to pass the “significance” test.

Was the test constructed properly?

This relates to a supremely important point that often statistics are quoted in situations they are not supposed to be used or have been not properly applied

How many relationships did you test?
In finance, all analysts look at lots of different data sets, over different time periods in search of something “significant”.

Did you look at any of the data before choosing what test to run?
I cannot imagine how someone could not fall into this trap. We only run tests on things we think might work. But the reason we think they might work is that we have done some rough statistical work already e.g. looked at a picture or perhaps just subconsciously noted some signs of a relationship. This means that the data has been mined and your choice of test is not independent.
How many people are trying to find these relationships?
Let’s say that you are extremely careful in how you do your statistics. Let’s imagine that everyone else in the firm you work at is similarly careful. Then when you produce a “significant” result you may reasonably think it is meaningful. After all you only ran one test and it worked! You then show your boss. Should she be impressed? Maybe not.

How many failed tests are not shown?
In my experience, analysts do not show me large quantities of research they have done which they think is completely meaningless. Highly trained with great degrees, they want to show me “good” work with “good” results. This means that the 19 analysts that did not find anything today do not show me anything. From the perspective of the individual the result appears to be strongly non-random. From my perspective, it looks entirely consistent with being random.

Is it meaningless?

No. it just means exactly what the equation says it means. You should remain aware of the context if you want to use it. My interaction with professionals of all types is that they are enormously well trained in the complexity of statistical methods and woefully under trained in the limitations of them. In fact, their high proficiency with manipulating the data and the methods makes them even more prone to methodological error of this type as they have essentially been trained in the art of data-mining.

Conclusion

I am yet to read a research piece from a bank which presents data demonstrating that their hypothesis is has no statistical significance. We should remember that this is significant.

Framework for valuing equities Part V – Relationships between Components

In Framework for Equity Valuation Part II I laid out this approach.

Breakdown of the Equity Price

Using some very simple algebra, I split the equity price into components:

Price = (Price/ Earnings) * Earnings

Price = (Price/ Earnings) * ( Earnings / Nominal GDP) * Nominal GDP

In this work, there was an assumption that the components are independent. I will now examine if this is sensible.

Are P/E and E/GDP independent?

I can find no consistent relationship between the two.
There appears to be a mild negative correlation overall, but at times there can be extended periods of both components falling, such as 1967-74 and 2000-2003, or both rising such as 1994-2000.

An intuitive relationship occurs when there is an expectation of a large rise or fall in earnings and the equity price rises or falls in anticipation. This means the PE ratio would rise in anticipation of earnings rising, and then fall back down as earnings expectations are realised. In this situation, I see earnings as the driver and the PE ratio as a passive variable.

A situation where PE was the independent driver was in the 1980s, when a broad fall in yields meant the PE ratio rose without any need for an expectation of a change in earnings. This supports the approach that we can look at the two factors as independent drivers.

Are Growth and PE ratio related?

This is a relationship that is often assumed to exist as we think periods of low growth or recession are associated with low confidence and high awareness of risk. This high “risk premium” means low PE ratio.

But the evidence to support this idea is not so clear. Of the past 9 recessions, the PE ratio only fell twice. There is some evidence to support the idea that the PE ratio falls in the year before the recession in anticipation of an earnings drop, then recovers quickly as those expectations are realised. This happened in 5 of the last 9 recessions so it is still a fairly mild effect.

Are Growth and earnings related?

I find the chart below intuitive and compelling. The reason that recessions drive equity markets down is because recessions drive corporate earnings down. If we look at earnings as a share of GDP from 1 year before the recession to the low during the recession they fell each time. The average fall was 21% with the smallest still a 9% fall and the largest (2008) down a massive 42%.

The rationale for this comes from thinking about the breakdown of national income in the NIPA data (Framework for Equity Valuation Part III Earnings Outlook). If there is downward pressure on nominal GDP whilst wages remain sticky, then the impact is felt in a magnified way in corporate earnings.

The magnitude of changes in earnings are very large during recessions and early recovery, so it is during these periods we should be especially alert when forming an equity outlook. The impact of whether growth is 2.5% or 2.8% is imperceptible by comparison. Lots of work by economists, strategists and asset managers is done to fine tune these types of economic forecast but a) it is not possible for them to be that accurate b) even if you could, the relationship to market prices is so loose as to make it useless information.

Conclusion

There is one important interrelationship we need to be very aware of. In previous recessions, earnings as a share of GDP have fallen rapidly and normally bottomed at around 7%. If that were repeated in the next recession, earnings would need to fall by 40% from current levels.

What is our education system for?

Education plays a big part in every election campaign. But it seems that the only debate is over funding. It is implicitly assumed that the only way to improve our education system is by higher funding levels and that if only more young people could do A levels and get a degree then our economic productivity would rise and they would all be better off.

What I would like to see is some examination of what our education system actually does and reconsider if that really should be its goal.

Learn to write

Education can be seen as the way that students are taught to write and communicate “properly”. In particular, they are taught Standard Written English (SWE). David Foster Wallace’s essay “Authority and American Usage” examines how “rules” in English usage and grammar can be better understood as “norms” more similar to “ethics” than to “scientific laws”. Importantly “a dialect of English is learned and used either because it’s your native vernacular or because it’s the dialect of a Group by which you wish to be accepted”. Therefore, students from less privileged backgrounds have to learn SWE because it is the dialect of “power and prestige.”

You may find this form of education objectionable. My view is that given the structure of our society, it is useful for the individual. Highly paid professions require fluency in this dialect and if we want any social mobility this has to be taught.

If I look at what is taught at a UK university, I quickly conclude that they go much further than this in the enforcement of dialect. Peter Elbow in “Everyone Can Write” discusses the teaching of “academic discourse” which is the language academics use to write to each other. This is the form of writing that is taught and highly valued at university. But as he points out, there is not a single form of “academic discourse”. Historians do not write in the same style as Biologists. Even within subjects, there can be wildly different forms of acceptable dialects.

The purpose of these dialects is to provide a barrier to entry to the discipline. It is how academics can signal to each other that they are part of the same sub-group, and by enforcement of their dialect exclude outsiders. Much of academic writing appears to be deliberately obstructive to the lay reader. Within academia this is irritating, but when it forms a central part of the education of a population it is a lot worse than useless.

Learning to write in this dialect does not prepare the student for the tasks they will face after university. The language of business is very different from the language of English Professors. Hence the common complaint that not only do graduates have to be taught so much, they actually have to “unlearn” what they have been taught.

Learn a subject

In UK universities, it appears that the purpose is to train future academics. The subject matter is very narrow, the syllabus relates to one discipline and the student is encouraged to go deeply into a specific area within that subject. Ask a history student about anything and they will say “not my period”. Talk to an economist and they will refuse to have a conversation without a mathematical model. Lawyers learn the intricacies of Roman Law.

Unfortunately, it is not obvious that excelling at a narrow and specialised area has many transferrable benefits. It does not produce a well-rounded graduate with a range of interests and perspectives. It provides a very highly refined ability to do something they will never be asked to do again, unless they become an academic of course or maybe a macro manager.

It is a signal

This is a compelling driver for getting an education. It can be used to give very valuable signals which are important in your life. For example

Getting into a selective university signals that I am intelligent and hard-working
High grades signal that I am intelligent and hard-working
Going to certain universities signals that I have been socialised into a specific culture and am motivated to belong to it. This is why investment banks interview Harvard students.

These signals have numerous costs, not just the financial and time cost of a university education.

Getting good grades at school to gain entry into a top university has become a growing driver of school education. We are obsessed with league tables, and education up to the age of 18 appears increasingly to be a competition. There are many other things that could be the focus of our children’s attention. Teaching to pass an exam necessarily leads to focus on a defined syllabus and the subordination of creativity and imagination to regurgitation of the approved answers.

The desire by students for high grades creates a strong demand for courses which can be graded and for the students to be ranked. To rank students effectively, a syllabus is required which leads to a test with a decent dispersion of results. This leads to a particular kind of subject matter being preferred. In subjects such as economics and finance, what I observe is a lot of time focused on the teaching of complex and rather arcane methods of mathematical and statistical modelling. This creates an exam which even some of the very smart students cannot do well and so it is possible to differentiate between them. But not on a basis which is necessarily meaningful.

If I taught a course, I have lots of things I would like to include. But I have no idea how I would examine it. What I would want to teach are concepts which are not very difficult. Everyone in the class could understand them and get an A. But just because there are not difficult does not mean that they are obvious or commonly understood. Perhaps the most important lesson would be that they should never use any of the sophisticated mathematical techniques they are learning in their other classes. Care and attention in assessing what the characteristics of the data are and what can reasonably be done with it are far more important.

For an individual, this signalling can produce compelling and powerful success stories. In many ways, my personal story can be expressed in this way. Teresa May wants to bring back Grammar schools because she went to one and ended up being Prime Minister. But this type of anecdotal reasoning leads to poor policy and worse outcomes for most children. http://blogs.ft.com/ftdata/2013/01/28/grammar-school-myths/

What should the purpose of education be?

We could add a focus on developing knowledge and skills which are useful for people to have

In the workplace
In their lives
As a member of society

The opportunity is there. I meet many students who enjoy their time at university. I meet very few who enjoy their academic work and even fewer who think their academic work is useful.

My recent experience

I have to admit a tendency to get overexcited in the adoption of new things. I spent a week touring US colleges recently and was overwhelmingly surprised and impressed. I met students who described doing courses that were “not too hard, but really useful”. I never did courses like that.

I do not want to single out any one place as there were so many positive impressions. But this was the best video.

https://www.youtube.com/watch?v=tGn3-RW8Ajk

Framework for valuing equities Part IV – PE Outlook

In previous post (Framework for Equity Valuation Part II – Equity Drivers), we have seen that the PE ratio can be an important medium term driver of equity prices. Given the debatable outlook for aggregate corporate earnings, this makes the outlook for the PE ratio a critical factor in forming a view on equities.

Simple PE ratio

It should be very clear from the normalised chart below that the powerful driver of the equity market performance since 2012 has been an expansion of the PE ratio. The S+P has risen by 72% over that period, and the majority of that is explained by PE ratio which has risen from 14 to over 21, an increase of almost 50%.

Can PE ratios go higher from here?

If we look over the long run, it is very rare for the PE ratio to move higher from where we currently are. In fact, it has only happened 3 times; 1991, 1999 and 2009.

In 2 of these 3 examples, high PE ratios were observed during a recession and ensuing bear market with earnings falling even more than prices . For example, in 2009, the high PE ratio was driven by the collapse in earnings not the soaring of equity prices to record highs. These are not helpful precedents for equity bulls right now.

The only previous period where the PE ratio drove the market higher from this level was the dot com bubble. The name given to this period gives a big clue as to what we now think of what happened. If that were to be repeated, then there would be another 40% left in this rally due to PE expansion. This is not impossible but relying on a repeat of the biggest valuation bubble in a century is not reassuring to me.

“Fed Model”

The post I wrote about the Fed model implied that equities represent good value compared with bonds. This generates a counter-argument to my scepticism of a repeat of the dot-com bubble. We have never seen bond yields this low before, so why should we not also see unprecedented low yields in equities (high PE ratio)?

As I explained in my previous post (Framework for valuing equities Part 1- Compared to bonds), I do not think that bonds are good value and so simply beating their performance may not be a high enough benchmark. Most importantly, if QE-driven low yields are pushing up PE ratios, then the termination of QE and rising bond yields should be very harmful for equities.

The other problem is that, even if it is true that holding equities for the next 10 years may work out, the volatility and drawdown you experience may be hard to handle.

For example,
In May 2007 from my equity model (Framework for valuing equities Part 1- Compared to bonds), the expected 10 year return for equities was 8.1% (annualised)

It actually turned out to be 7.1% annualised – which resulted in a total return of almost 100% over 10 years.

That sounds pretty good.
But I bet it would not have felt so good less than 2 years later in March 2009 after a 53% drawdown.

With hindsight waiting for a better moment to enter the long equity trade would have been phenomenally better. If you had waited to buy in March 2009 instead (I know, ludicrous cherry picking, but just about any time around then was great) then your returns would have been a total of over 300%.

Conclusion

The outlook for equities from the perspective of high nominal GDP or high earnings growth look rather limited. Earnings is near record highs as a share of GDP and we are at the stage of the cycle where wages are rising instead.

If we rely on a PE expansion to make us optimistic, we need to be comfortable buying at levels which previously have been associated with a “bubble”.

We can perhaps consider equities being good value compared to bonds, but we must then remember that yields are too low given fundamentals and the termination of QE.

If you are happy to hold them for a decade and do not worry too much about drawdowns, then I come up with an expected annual return of 6.5%. This is higher than bonds right now but perhaps waiting for a better entry level will turn out to be a better strategy.