Wednesday, July 31, 2013

Biases in Processing Information by Guest Blogger, Buck Klemkosky

The world is awash in data, so much so that a new field has evolved called “big data.” The explosion in data can be attributed to three factors: the increase in computer processing power at lower costs, the start of the World Wide Web in 1990, and the development of cellular technology. The explosion of digital data can only be compared with printed material in the Western world after Gutenberg invented the printing press in 1440.

How much digital data is available? No one knows precisely, but that doesn’t stop people from making estimates. In digital terms, everything starts with bits (short for binary digits, 0 or 1, computers use to store and process data) and bytes (8 bits, which is the basic unit of computing). It escalates from there:

kB kilobyte = 1,000 bytes                               PB petabyte = 1,000 TB
MB megabyte = 1,000 kB                               EB Exabyte = 1,000 PB
GB gigabyte = 1,000 MB                                ZB zetabyte = 1,000 EB
TB terabyte =1,000 GB                                  YB yottabyte = 1,000 ZB

Remember that the first personal computer had 56 kilobytes of memory and todays’ usually have a couple of gigabytes. But the total data in the world is estimated to be several zetabytes with more being produced every day.

As mentioned previously, most of the data in existence is noise and not useful for making decisions. But we try to decipher and glean from all of the data useful information. Our brains are surprisingly powerful processors of data and information, but selecting information to be used in decision making is hampered by several psychological biases.

It is only human nature that we get more pleasure from being right than wrong. So if we have beliefs or have made a decision, we become selective in collecting and using only confirming information. We filter out and reject information that is contrary to our beliefs and decisions. It’s much easier to support
than contradict. Sometimes we even use ambiguous and perhaps wrong information as supportive.

Investors are especially subject to confirmation bias. Once we buy a stock or bond, we are much more receptive to supporting information than contradiction. Some investors have strong opinions about the direction of the market in general, short term and long term. If you are a perma (long-term) bull or a perma bear, eventually you may be right, but it’s those intervening years that hurt.

What makes Warren Buffett such a successful investor is that he actually seeks out nonconfirming information. He has billions of dollars of his wealth, almost all, invested in Berkshire-Hathaway stock. At this year’s annual meeting, for example, he invited one of the most negative investors concerning Berkshire-Hathaway to address the 20,000 shareholders. This investor had taken a large short position in the stock, expecting it to decline in price.

In addition to confirmation bias, investors also have a tendency to use readily available information that can be easily recalled, which is called the availability bias. We also have a recency bias by giving more weight to more recent information and events and less to that more distant in time.

Investors also generalize with insufficient information, which means we use a small statistically insignificant sample or anecdotal evidence as information to make decisions. Most often it represents our own experience or something we are familiar with. The future looks like something we know or are familiar with based upon recent events or frequency of events. This is one of the reasons individual investors have been reluctant to invest in stocks again after the bear markets of 2000-2002 and 2007-2009, even though the market has appreciated 150 percent since the S&P 500 lows in March 2009.


It is easy to have opinions about almost everything. It is much more difficult to have informed opinions. But we never know if we are fully informed or not. Even though we may believe that we have correct information, it has a high probability of being biased. That is why uncertainty always prevails, especially in the financial markets. 

8 comments:

  1. I am not sure why this blog post should not be called something around confirmation bias. I am also not sure about whether it is connected with big data.

    ReplyDelete
  2. Maybe but playing in the "market" ( defined as financial instruments with large commissions)is bias due to the prize and the randomness of the risk. Whereas playing in the reality market ( defined as making, selling and providing product and service) is less bias because the risk can be defined and the task to mitigate is up to the owner's experience which in itself is a lot of data to sort through.

    ReplyDelete
    Replies
    1. James, tell that to all the people whose businesses went bankrupt each year -- and to all those people who lost jobs in the recession.

      Delete
  3. This post gave me a headache. I'm an IM for Pete's sake not a statistician.

    ReplyDelete
    Replies
    1. Fuzz, Take two aspirins and call me in the morning.

      Delete
  4. Why would I call you "in the morning?" Your name is Larry!

    ReplyDelete