When Real Estate Decision Making Goes Bad
How too much information leads to expert overconfidence
T3 Real Estate Leadership Summit Keynote, Las Vegas (2014)
So, I’m not a superstitious person, but when they said they were looking for a 40 year-old digital native that knows a thing or two about Realogy to follow Richard Smith, I got a little nervous.
Thank you so much for those very kind words of introduction. I’m thrilled to be here representing The New York Times, and very grateful for the opportunity to share some of our digital real estate product, search and platform perspectives with you all this morning. As some of you know, I wasn’t able to make T3 last year. So being here, in every sense of the word, is very special for me.
Stories about people and their connection to place are at the heart of what we do at The New York Times, no more so than within the real estate section. So today, I’m going to share with you a series of short stories. And why’s that? Well, it’s because I believe that storytelling is at the heart of communities, families and cultures. It’s also at the heart of memories. It’s the most shareable type of content that has ever existed, on or offline.
And as a passionate advocate of real estate brands, products and building things to make the process more enjoyable, especially in New York, I believe it’s in the DNA of what it feels like to own a home.
So today I’m going to talk a little about how we approach decision making, and consumer confidence.
As the web deepens its embrace of social media, and online behavior becomes increasingly informed by conversations inside of streams, instead of the passive browsing of page-centric destinations, our interactions with others are becoming a much larger part of how we experience being digitally connected, than ever before.
For publishers like us in the product group at The New York Times, this poses a new and unique set of exciting challenges. Ones which force discussions around having a much stronger understanding of how people make decisions based on what their friends and other connections are doing, as opposed to messages they may absorb elsewhere in their lives. Real estate content at The Times receives hundreds of thousands of social referrals each month, and it’s a strong and growing part of our audience ecosystem.
But confidence, trust, and faith in services become essential, especially in the real estate section. Curating stories around those ideas becomes a great place to focus.
And on the product side in particular, a deep, thorough and insightful understanding of how decision making is determined, how personalized priorities are chosen, and ultimately how the emotional brain reacts to different stimuli, is becoming a much sought after skill in creating meaningful, digital real estate search experiences.
So when we think of decision making for real estate products at The Times for example, the idea of confidence is an important one.
Confidence greatly informs how people make the kinds of decisions they do, as well as when, and with whom. It also allows us to empathize with the customer and truly understand where they are in the process. We’ve heard a lot recently about real estate consumer confidence and its role in the economy’s recovery and stability, and the translation of this climate to online search is an interesting problem to work on.
By extension, confidence in local housing markets is something real estate professionals work on every day with their clients. Lack of confidence has actively disintermediated under-performing agents, led to more empowered and informed customers, and established a digital climate of massive search volume, especially in mobile.
But let’s not forget that enormous search activity isn’t business. That there’s an enormous difference at the agent level, especially with sellers, between referral traffic and phone calls.
And while confidence has traditionally been a benchmark of proficiency in decision making, especially within the top tiers of the realtor community, psychological research conducted into what happens when an abundance of confidence causes decision making to go very, very wrong, is important to explore for anyone looking to understand how online search relationships really work.
What happens when customers or agents are so supremely skilled at their positions that they no longer see their own inability to make the right decisions?
Is it possible to be armed with too much data and not enough insight?
What are the consequences of such accidental behavior?
The author Malcolm Gladwell, in a recent lecture at High Point University in North Carolina, explores such an idea . That of mistakes caused by people who are exceptionally proficient, such as a brokerage’s top producers. Mistakes made by people because they were simply too good at what they do.
It’s a symptom he calls ‘expert failure’.
He suggests at length that just as with incompetence, there’s a greater degree of risk caused by accidents happening through an over-abundance of expertise and proficiency, and points to a growing trend whereby these kinds of phenomena will begin to occur more frequently as the world adopts more and more of a digital infrastructure.
Gladwell uses military examples in order to explain the concept of failing through too much technology-driven knowledge, specifically the Battle of Chancellorsville in May 1863, at the height of The Civil War. Here’s how The Times covered the story, which is now available through our Times Machine archive, which is actively digitizing over 160 years of news coverage and making it all searchable. Often seen as one of Robert E. Lee’s greatest strategic victories, Gladwell illustrates what happens when overconfidence takes hold, through the story of Fightin’ Joe Hooker, the commander of the Union forces.
Outnumbering Confederate troops by more than two to one, and surrounding them on three sides, the Union forces were poised to inflict one of the largest and most devastating defeats on the Confederate Army it had ever experienced, with the potential to even prematurely end the war. Specifically through the, at the time highly technologically advanced method of using hot air balloons to gather intelligence about the enemy across the river, Hooker knew more about what Lee’s troops were doing than Lee himself.
The evening before the battle was set to commence, Hooker famously addressed his troops and proudly boasted:
“I’ve got Robert E. Lee right where I want him, and even God himself cannot stop me from destroying him.”
Gladwell characterizes Hooker’s comments as stemming from an over-abundance of confidence, fueled by too much information, something he calls ‘miscalibration’, a term used by psychologists to illustrate when someone’s perception of their ability to achieve a task far outranks their actual ability to do it.
He continues by saying that what actually happens is that more information doesn’t lead to better decision making, it simply leads to more confidence in those decisions you’ve already made. In an era of digital overload, there are obvious parallels when it comes to miscalibration through reliance upon too much information, especially within real estate search experiences.
And this is critically important when we think about the real estate search experience on and offline. Focusing on arming the customer with as much information as possible, as many of the more popular sites have been doing in recent years, has the obvious benefits of more empowered searchers. But it also carries risk of overconfidence in simply making the wrong decisions. This is an important point to remember in streamlining search products, and when we think about this at The Times, very often it’s the difference between searching for something, and finding it. Let’s not forget that buyers and sellers aren’t interested in search. They’re interested in find.
Importantly, confidence instills hope, persistence, ambition and drive, but it doesn’t lead to specifically better judgments or recommendations. In one of history’s greatest strokes of military strategy, Lee famously split his troops into three sections, fooling Hooker into thinking that they were in retreat. What really happened is that the Confederate troops began to circle around the rear of Hooker’s men, and surround them. When Lee’s troops emerged from the nearby forests, they so surprised the relaxing Union troops that they immediately fled in retreat, and were driven back several miles. Hooker was later relieved of his command.
Echoing Gladwell’s story, Woody Allen famously, and now perhaps infamously shared: “Confidence is what you have before you understand the problem.”
So, it poses an interesting real estate question for those in the room here today, where most of us buy into the idea that the more information we give to each other, our customers or our advertisers, the more informed and more astute our collective decision making will be.
What are the consequences in the belief of perfect information? Data accuracy and integrity has been a lightning rod between the major syndication portals in recent years, and it’s the source of many uphill technical challenges for those building search products, especially brokerages.
Many have proposed that as we obtain more information about a topic, our judgment suffers, but our confidence improves. In doing this, studies performed in the 1960's by psychologist Stuart Oskamp found that supplying more information to a person about a topic does very little to improve their capacity to make better decisions.
So what’s happening, especially when it comes to online real estate, is that there’s a tremendous over-estimation in the value of information distribution.
More information increases consumer confidence (as we see from large syndication portals, who trade on comprehensive data sharing — all the listings, market reports, housing estimates, detailed mapping and much more), but it doesn’t lead to a change in consumer decision making — this is perhaps best illustrated by the average time the prospective homebuyer is performing research online, prior to reaching out to an agent, which continues to lengthen each year based on the economic climate. Currently it’s an average of about 18 months.
And as you can imagine, the insight, knowledge and guidance shared through articles published at The Times becomes a valuable commodity within that 18 month window, especially paired with organic search, and other destinations where people are already spending disproportionate amounts of time. A recent video on Manhattan housing affordability was so popular that it was one of the most viewed items across our entire site during the Super Bowl weekend, generating almost 100,000 views.
So what’s happening, through a strategic distribution of as much housing content as we can share as an industry, and under the guise of transparency, is more and more confident, search-centric consumers. Not a bad thing, perhaps, but not necessarily better decision makers. Certainly it generates individuals who are more assured of the decisions they’re making.
However, this notion of confidence in the accuracy of one’s judgment, based on the abundance of available information, leads to the concept of what I referred to earlier as miscalibration. This specifically means how good you think you are, versus how good you actually are. For example, many in the real estate industry confuse these issues around the topics of customer service.
Arnold Kling, in his fantastic online essay ‘The Era of Expert Failure’ argues that societies (like the web) where knowledge is decentralized, where decisions are made faster and more accurately, and where overconfidence exerted by its leaders becomes frequently held in check, is a good thing. And as a result, centralized power and decision-making is becoming increasingly anomalous.
By extension, aggressive, contentious consolidation of attention, audience and ultimately revenue is something we’ve seen a lot of within the real estate industry already this year.
Similarly, expertise in the real estate industry is constrained by market competition and checked by consumer choice. That’s a real positive. We can all choose, just like everybody else, to ignore the expert, and Oskamp’s previous research reinforces that this has little impact on the decisions you’re making anyway.
In many ways people prefer information from a confident source rather than an honest or genuinely more knowledgeable one.
We want faith in our sources of information, even if it’s at the expense of getting the right information. This is part branding, part content, but all trust.
So especially as real estate customers, we want to be reassured and feel good about the information we’ve asked for, perhaps reinforcing something we had previously thought. Once a topic has been mastered, we actually fail to be able to spot signs of what might go wrong any more.
Those that have the belief that they are ‘bigger and stronger’ than others are severely miscalibrated. Too big to fail perhaps.
Finally, in his wonderful piece for The New York Times entitled ‘The Hazards Of Confidence’, Daniel Kahneman takes the financial industry to task on the theme of expert failure, outlining how the strategic placement of investments is little more than a game of chance, with, for example, at least two out of every three mutual funds underperforming the overall market in any given year. He suggests that the financial industry is wrought with inconsistent underperformance, for which is is still not held to account and ironically, consistently continues to reward itself for. Kahneman proposes that the investment industry’s performance should be perceived through the following lens. He says:
“The illusion of skill is not only an individual aberration, it is deeply ingrained in the culture of the (investment) industry.”
This is where problems of experts, especially real estate experts, truly comes into play — they confuse confidence with expertise. The two things are similar, but mutually exclusive terms. It’s true that confidence is essential for mundane activities, to the extent that we now take it for granted, but it can often be described as being fire-like in substance. Useful in controlled amounts, but with the capacity to swiftly burn out of control when left unchecked.
The practice of real estate professionals, their decisions made with their customers, where they choose to focus their dollars and energies, and who they align with, is at the heart of this discussion, especially in a climate where their expertise is in question from online sites preaching greater consumer transparency. The role of authority is one that’s shifting more towards the consumer, at scale, than ever before, and we can all become masters of very niche information at the touch of an app.
Potential home buying consumers are becoming more confident as the window of time prior to contacting an agent increases annually, so much so that the risk of severe consumer miscalibration and damaging decision making is heightened, should the efforts of increased digital transparency go unchecked.
Overconfidence is sometimes described as happening when ‘people are blind to their own blindness’. And while it’s a critical aspect of restoring the housing market, more information does not lead to better decision making. Customers will continue to make the decisions they were going to make anyway, but increasing the flow of information will increase the confidence they have in those same decisions.
But keep in mind that too much information, under the guise of unchecked transparency and disclosure, can often lead to miscalibration, wherein the consumer especially, runs a much higher risk of making the wrong decision.
It’s something to keep in mind the next time you’re working with data.
I hope you enjoyed my stories. Thank you.
