Written by MarketPsych CEO Dr Richard Peterson, our free newsletter provides commentary on current events from a behavioral perspective. Additional topics include behavioral theories and our latest research results.Subscribe to our mailing list here
We also offer a premium version of the newsletter to our research subscribers, which features additional commentary and trading recommendations. Find out more here >>
<< Newer articles | Older articles >>
In general, statistical analysis of price patterns demonstrates the existence of two opposite sentiment-based patterns in financial prices: overreaction (e.g., panic bottoms and blow-off tops) and underreaction (e.g., trends). The concepts of underreaction and overreaction refer not only to patterns of prices but also to the collective investor reactions to information that fuel such patterns.
Daniel Plainview (played by Daniel Day Lewis): Are you envious? Do you get envious?
Henry Brands: I don't think so. No.
Daniel Plainview: I have a competition in me. I want no one else to succeed. I hate most people.
Henry Brands: That part of me is gone... working and not succeeding- all my failures has left me... I just don't... care.
~ There Will be Blood (2007)
As most of our readers are aware, in partnership with Thomson Reuters MarketPsych derives and distributes the Thomson Reuters MarketPsych Indices (TRMI), representing a real-time quantification of emotional and macroeconomic references in the media to 8,000 companies, 52 stock indexes, 32 currencies, 35 commodities, and 130 countries. When the sentiments extracted from such text are statistically analyzed alongside historical price data, insights emerge into how such information moves crowds of investors and prices themselves. Oil is one of the more sentiment-driven asset, based on our testing.
Today's newsletter explores social frustration, affluenza, underreaction in the S&P 500, and more to the point of behavioral economics, science-based evidence on how to change society for the better. As usual, it's a long newsletter, and we recommend skipping ahead to sections of interest.
Terrorist attacks and rising homicides fray social trust. The decline is trust is not restricted to one ethnic or religious group. Since 2001 among the 16 people arrested for terrorism in Southern California are Irish, English, Spanish, Arab, and Vietnamese surnames. Obviously, terrorist behavior crosses ethnic and religious lines, so we need to look deeper for explanations. First, we 'll look at the social forces and emotions that fuel terrorist thinking, starting with anger.
Fear is the path to the dark side. Fear leads to anger. Anger leads to hate. Hate leads to suffering.
When all media anger is added together, we see the following plot of anger expressed in relation to the United States as a whole (superimposed on a chart of the S&P 500). To generate this chart, all specific references to anger in relation to a physical location within the United States were collected and quantified from 1998 to the present. Note that anger as a percentage of overall U.S.-focused chatter is now much higher than 1998 through 2001, and it is also higher than it was in 2007-2008 (the two anger averages plotted are 200-day and 500-day averages, so they are very lagged).
First following the Sep 11 attacks on the World Trade Center, and again after the financial crisis, American anger has risen and remains high. However, the United States has a fairly middle-range level of anger on a global basis, as seen in the below heatmap of anger for the month of November, 2015 (El Salvador, Zimbabwe and Uruguay stand out as particularly high in anger):
In a past newsletter we described the role of anger in driving prices. Across cross-sections of stocks trust has an inverse relationship with future price action (low trust stocks tend to rise). Investors ought to buy the angriest and least trusted stocks. Such stocks' prices are more likely to rise going forward.
Having access to more information, more quickly, is fundamental to optimal forecasting. Yet we are overwhelmed with such information. IBM estimates that global data volume doubles every 18 months. For example, as of 2013, over 100 billion business emails were sent daily. If computers could autonomously make sense of this data deluge, then the volume would not be problematic. But computers do not understand meaning, and they need human assistance to develop better forecasts from the data.
Most of the empirical research in finance, whether published in academic journals or put into production as an active trading strategy by an investment manager, is likely false.
~ Campbell Harvey and Yan Liu, “Evaluating Trading Strategies,” 2014.
For the humans who create forecasting models on this data, making predictions is an exercise fraught with risk. The potential problems range across humans biases to data errors to inappropriate application of statistical tools. Perhaps the most common problem is overfitting - a problem made worse by the availability of increasingly complex statistical tools. MIT professor Andrew Lo summed up the problem of statistical overfitting neatly with this statement, "I’ve often said that, you know, whatever back test you’d like to see, I can certainly produce it for you. If you torture the data long enough, it will basically tell you anything you want."
Given the expertise needed to find predictive meaning in the massive volume of global information, and the predictably poor models and forecasts of experts, Phillip Tetlock set out to identify the traits of what he calls Superforecasters - people who are excellent at synthesizing information and making predictions of the future.
All right! We did not die today, I call that an unqualified success.
~Fear, Inside Out (Disney, 2015)
On a long flight somewhere over Asia last week I watched the new Disney-Pixar movie "Inside Out." As usual for a Pixar film, the movie is a brilliant piece of entertainment. What surprised me was that it is also a path-breaking exploration into the nature of the unconscious mind. Perhaps amazingly, this children's movie contains lessons for investors, and those are the topic of today's newsletter.
Inside Out is set in the mind of a girl named Riley from Minnesota. We see the interactions of five manifestations of her emotions — Joy (yellow in the image below), Sadness (blue), Disgust (green), Fear (purple) and Anger (red). The emotions influence Riley's experiences, behavior, and memories via a control console. Joy acts as the dominant emotion to keep Riley in a happy state, but when Riley is 11 years old, her family relocates to San Francisco. Riley has trouble at her new school, and Joy loses control of Riley's experiences. The premise of the movie - that emotions drive most of our behavior - is supported by modern research. Despite this seemingly esoteric topic, the movie is very popular, having grossed over $774.5 million worldwide as of September 27th.
Just as Riley's life was moving along happily until the shock of her move to San Francisco, market prices usually plod along in fairly rational ways - happily on autopilot - until a crisis occurs. While on autopilot, markets are susceptible to underreacting to negative information, giving emotionally-saavy investors an opportunity to act before others understand what is happening. During a crisis, anger or fear dominate investors' collective behavior, creating patterns that investors can identify and take advantage of. Today's newsletter examines how both happily "rational" periods and those of crisis exhibit price patterns. The letter also addresses how investors can maintain optimal emotional function and adapt to changes in the dominant market emotions.
# What Moves Crude?
In "Inside Out" the Train of Thought - is depicted racing through the mind while linking memories, plans, and feelings. The train of thought represents one's conscious experience of the world, and it is the foundation of analytical thought. While emotions always color thought, the use of analytical reasoning about these emotions allows one to gain useful perspective. Using our Thomson Reuters MarketPsych Indices, we can analytically examine the effects of emotions - and loaded topics - on prices. We have begun to identify the predictable aspects of both happily rational and negatively emotional markets.
In 2013 we developed a crude oil forecasting strategy using moving average crossovers (MACDs) of a specific type of sentiment index called PriceDirection. As a quick aside, sentiment is difficult to detect for assets like crude oil, in which media commentary may reflect the sentiment of either producers or consumers, whose feelings are diametrically opposed whether prices rise or fall. As a result, we find that discussions of the price action itself is a more useful sentiment indicator for crude oil. For example, when prices are widely discussed to be rising, momentum tends to set in, and they do subsequently rise. We developed a sentiment-style index called the "PriceDirection" index to capture this phenomenon.
Given the sentence, "Crude prices rose today," the PriceDirection Index would receive a positive score (PriceUp = 1) from that sentence. However, if the text analysis engine analyzes this text string: "News of oversupply drove oil prices lower in late trading," the PriceDirection index would receive a negative score (PriceDown = 1). If both sentences occur in the same hour and no other such commentary is received, then the PriceDirection index would be zero value (PriceDirection = PriceUp - PriceDown = 1 - 1 = 0).
Using both short-term and long-term moving averages of the PriceDirection index, our Chief Data Scientist Aleksander Fafula found that when short term discussion leans towards prices falling, they subsequently tend to fall. When short term conversation about prices rising dominates, they tend to rise. This is a recent image, and depictions of this strategy back to 1998 show significant value. In recent years the overall strategy yields a more than 25% average annualized return with a Sharpe ratio > 1, as described in this prior newsletter.
The PriceDirection index does a good job of predicting price trends when there are no dramatic or rare events affecting the oil markets. Joy and pleasant feelings create a trend that carries forward. Good feelings beget good feelings, and vice versa. However, when a militia threatens to blow up an oil pipeline, we see a sharp rise in emotions such as fear. And as fear comes to dominate perceptions, prices overreact.
The best statistical tool to capture the effects of high emotion levels on price behavior is a decision tree. Decision trees examine the effects of extreme levels of one sentiment versus the subsequent price action in the asset. When the fear TRMI is increasing from week to week (rising emotions), at the most extreme level we see 19 samples with a large jump in weekly fear, and these are followed by an average weekly price increase of 3.2% over the subsequent week for crude oil.
However, fear-based mispricings do not remain in market prices for long. When fear is high for one month, crude oil prices subsequently fall. Using monthly averages, in the 19 months since 1998 with the highest fear (top decile), the following month had an average price decline of -5.4% for crude oil, and prices declined in 100% of subsequent months.
When crude oil fear is neither high nor is changing dramatically, then the crude oil price tends to trend, and the PriceDirection indicator is the best predictor of the price at those times of trending. Overall, using PriceDirection alone is a viable strategy, since rare events are, by definition, rare.
# How Feelings Sway the S&P 500
While crude oil is largely trending on autopilot punctuated by periods of fear, the stock market responds to a diverse range of emotional provocations. All together, the emotional tone of information is summarized as sentiment. As sentiment in the media turns, prices tend to follow.
The following chart of sentiment is similar to that of crude oil, but longer averages were used to improve readability since the chart is a longer duration (back to 1998). This simple strategy returns more than double the S&P 500 by going either long or short based on the prevailing sentiment balance about the S&P 500 (it turned negative this year, contributing to our bearish forecast on the S&P 500 in July 2015). This strategy would have held short through the financial crisis and the dot-com collapse.
Aleksander Fafula found that fear is mildly predictive of next-day reversals in the S&P 500. On a daily basis, short term fear is an overreaction for the S&P 500, as it is on a monthly (and daily) basis for crude oil.
# Managing Your Train of Thought
Crying helps me slow down and obsess over the weight of life's problems.
~ Sadness, Inside Out (Disney)
Crowd behavior (a.k.a. herding) occurs when events exert a strong pull on investors. Investors can use the power of conscious thought to reframe events and replace such compelling feelings, thus avoiding the underreaction or overreaction of the herd.
Personality traits represent ingrained emotional habits, and they color one's thinking about the world. Over 28,000 people have taken our free online financial tests since 2004, and the Investment Personality Test is the most popular test. The personality tests measure the Big 5 personality traits: Extraversion vs Introversion, Agreeableness vs Self-interest, Neuroticism vs Emotional Stability, Openness to New Experiences vs Traditionalism, and Conscientiousness vs Spontanaeity. The Big 5 personality traits are not strongly correlated with overall investing performance (several sub-traits are more interesting), but given our large sample size, we are able see a statistically significant correlation between long-term investing success and two traits: Emotional Stability and Openness to New Experiences (Openness).
If you find yourself low on Emotional Stability, consider whether this is a bad thing. Many people find the full experience of emotions useful, as it helps them to be more sensitive to their environment. At the same time, when emotions become extreme for a long period, poor decision making may result. For investors it is useful to avoid situations that will provoke damaging emotions. Preventive stress management - healthy routines and rhythms to your day - can minimize the likelihood overwhelm.
The second identified performance trait - Openness - is strongly correlated with mental flexibility and adaptability, and it is a trait of top investors such as George Soros (e.g., his Open Society Institute).
As Riley does in the movie "Inside Out," redirecting one's train of thought (cognitive reframing) allows one to dwell on topics or memories that create a productive state of mind. At the same time, as Riley experiences in the movie, one ought not to use cognitive reframing to avoid nagging problems, as they will only grow worse if continually reframed without relieving the emotional core of the issue.
# Housekeeping and Closing
Joy: Come on, group hug! You too, Anger.
Anger: Don't touch me.
~ Inside Out (Disney)
As go the forces that drive Riley's behavior in Inside Out, market prices are either 1) floating along quietly, driven by reasonable expectations while underreacting to new information or 2) dramatic, emotional, and characterized by overreaction. We see predictability in both contexts, but while using very different TRMI indexes. Broad feelings drive markets during #1, while negative emotions like fear are the best predictors during #2.
We love to chat with our readers about their experience with psychology in the markets. Please send us feedback on what you'd like to hear more about in this area.
Please contact us if you'd like to see into the mind of the market using our Thomson Reuters MarketPsych Indices to monitor real-time market psychology and macroeconomic trends for 30 currencies, 50 commodities, 130 countries, 50 equity sectors and indexes, and 8,000 global equities extracted in real-time from millions of social and news media articles daily back to 1998.
Celebrating all of our investing emotions,
The MarketPsych Team
<< Newer articles | Older articles >>