The world is awash in data, so much so that a new field
has evolved called “big data.” The explosion in data can be attributed to three
factors: the increase in computer processing power at lower costs, the start of
the World Wide Web in 1990, and the development of cellular technology. The
explosion of digital data can only be compared with printed material in the
Western world after Gutenberg invented the printing press in 1440.
How much digital data is available? No one knows
precisely, but that doesn’t stop people from making estimates. In digital
terms, everything starts with bits (short for binary digits, 0 or 1, computers
use to store and process data) and bytes (8 bits, which is the basic unit of
computing). It escalates from there:
kB kilobyte = 1,000 bytes PB
petabyte = 1,000 TB
MB megabyte = 1,000 kB EB
Exabyte = 1,000 PB
GB gigabyte = 1,000 MB ZB
zetabyte = 1,000 EB
TB terabyte =1,000 GB YB
yottabyte = 1,000 ZB
Remember that the first personal computer had 56
kilobytes of memory and todays’ usually have a couple of gigabytes. But the
total data in the world is estimated to be several zetabytes with more being
produced every day.
As mentioned previously, most of the data in existence is
noise and not useful for making decisions. But we try to decipher and glean
from all of the data useful information. Our brains are surprisingly powerful
processors of data and information, but selecting information to be used in
decision making is hampered by several psychological biases.
It is only human nature that we get more pleasure from
being right than wrong. So if we have beliefs or have made a decision, we
become selective in collecting and using only confirming information. We filter
out and reject information that is contrary to our beliefs and decisions. It’s
much easier to support
than contradict. Sometimes we even use ambiguous and
perhaps wrong information as supportive.
Investors are especially subject to confirmation bias.
Once we buy a stock or bond, we are much more receptive to supporting
information than contradiction. Some investors have strong opinions about the
direction of the market in general, short term and long term. If you are a
perma (long-term) bull or a perma bear, eventually you may be right, but it’s
those intervening years that hurt.
What makes Warren Buffett such a successful investor is
that he actually seeks out nonconfirming information. He has billions of
dollars of his wealth, almost all, invested in Berkshire-Hathaway stock. At
this year’s annual meeting, for example, he invited one of the most negative
investors concerning Berkshire-Hathaway to address the 20,000 shareholders.
This investor had taken a large short position in the stock, expecting it to
decline in price.
In addition to confirmation bias, investors also have a
tendency to use readily available information that can be easily recalled,
which is called the availability bias. We also have a recency bias by giving
more weight to more recent information and events and less to that more distant
in time.
Investors also generalize with insufficient information,
which means we use a small statistically insignificant sample or anecdotal
evidence as information to make decisions. Most often it represents our own
experience or something we are familiar with. The future looks like something
we know or are familiar with based upon recent events or frequency of events.
This is one of the reasons individual investors have been reluctant to invest
in stocks again after the bear markets of 2000-2002 and 2007-2009, even though
the market has appreciated 150 percent since the S&P 500 lows in March
2009.
It is easy to have opinions about almost everything. It
is much more difficult to have informed opinions. But we never know if we are
fully informed or not. Even though we may believe that we have correct
information, it has a high probability of being biased. That is why uncertainty
always prevails, especially in the financial markets.
I am not sure why this blog post should not be called something around confirmation bias. I am also not sure about whether it is connected with big data.
ReplyDeleteMaybe but playing in the "market" ( defined as financial instruments with large commissions)is bias due to the prize and the randomness of the risk. Whereas playing in the reality market ( defined as making, selling and providing product and service) is less bias because the risk can be defined and the task to mitigate is up to the owner's experience which in itself is a lot of data to sort through.
ReplyDeleteJames, tell that to all the people whose businesses went bankrupt each year -- and to all those people who lost jobs in the recession.
DeleteThis post gave me a headache. I'm an IM for Pete's sake not a statistician.
ReplyDeleteFuzz, Take two aspirins and call me in the morning.
DeleteWhy would I call you "in the morning?" Your name is Larry!
ReplyDeleteDr. Larry, please.
DeleteSorry, sir.
ReplyDelete