Transparency is Essential to Efficient Markets

My background is in the electronic securities trading industry. In the 90’s, trading was done by voice. Orders were phoned in (or shouted across the trading floor), paper tickets were completed by hand, and orders were manually filled. This system was replete with human error at every stage in the trading lifecycle, and errors were expensive. So in 1992, Fidelity Investments and Salomon Brothers got together and developed a protocol to enable electronic communication of equity trading data between their firms. They called it the Financial Information eXchange or FIX. This eventually evolved into a very robust FIX trading protocol.

Over the next 15 years, FIX was steadily adopted globally in nearly every asset class. It is used by buy and sell-side firms, trading platforms, exchanges and even regulators to facilitate every step in the trading process. This non-proprietary, free and open standard is used by tens of thousands of firms every day to complete millions of transactions.

The protocol has been a key contributor to minimizing trading costs and improving market efficiency and transparency. Today, it’s used not only to transmit trades between counterparties, but also to facilitate integration between in-house systems. In other words, it has become the language of trading.

The FIX protocol includes two layers: transport and application. The transport defines how messages are transmitted between counterparties. The application layer determines the format of the data. It’s a type of data dictionary that connotes meaning to the data. It establishes the format and usage of each field.

This protocol ensures that the messages recording orders and trades are transmitted between firms consistently, so systems on both sides can interpret the data accurately. It enables consistency in electronic communications between counterparties, even when their processing platforms are different.

FIX revolutionized the trading industry. It reduced transaction costs by lowering overall fixed costs. It enabled the markets to increase transparency. Better integration between markets served to increase liquidity, which lowered indirect trading costs by reducing bid-ask spreads. It also spurred tremendous innovation and the introduction of new value-add services from brokers and vendors. It lowered the barrier to entry for new players, furthering innovation. It dramatically lowered the cost of investing. It made possible a much wider scope of services available to both institutional and retail investors.

I believe that the real estate industry needs something like FIX—data dictionary of sorts that enables the entire industry to standardize data.

This could revolutionize the real estate industry, bring a transformative level of transparency, enable better benchmarking and improved analytics, and create a more level playing field.

A standardized approach to data is essential to making the real estate markets transparent like the rest of the capital markets.

Transparency Attracts Investors

Transparency is essential to efficient markets and to attracting investment capital. JLL’s research director Jeremy Kelly writes that “‘Highly Transparent’ markets account for around 75% of all commercial real estate investment globally.” Kelly points out that investors are drawn to locations that have “favourable operating conditions, transparent market practices, [and] readily available data and performance benchmarks.”

Some will argue that the real estate markets in the US and some other English speaking countries are more transparent now than they’ve ever been. And that’s true. The chart below from JLL and LaSalle’s biennial Global Real Estate Transparency Index ranks the most transparent cities.

But even in these highly transparent cities, there is still a huge amount of information asymmetry. For example, leases are not public knowledge and often have complicated terms. Incomplete market data around leases makes benchmarking difficult if not impossible.

Dark Markets Make Real Estate Inefficient

Dark (non-transparent) markets favor informed participants, locking in natural competitive barriers. But it also makes the real estate market shockingly inefficient compared to the global capital markets.

Jeff Jacobson, Global CEO of LaSalle Investment Management points out that real estate can become a “mainstream” asset class only with “greater transparency of information, better measurable returns and benchmarks, stronger legal and tax frameworks, and more efficient methods for investors to acquire, lease, manage and sell their property assets.”

Consider the benefits that the capital markets achieved through greater transparency — improved pricing efficiency and price discovery, fairness, competitiveness and attractiveness of the markets. This dramatically increased liquidity and overall participation in the markets.  

There are also important regulatory and legal drivers for transparency. The public spotlight on beneficial ownership and public debates around corruption, tax evasion and money laundering all force transparency into the public discourse.

Demand for more transparency is also coming from new investors.

Commercial real estate is becoming an increasingly attractive alternative investment strategy for high net worth and institutional investors. According to the Institutional Real Estate Allocations Monitor, institutional allocations grew significantly over the past 12 months. However, institutions remain under-invested relative to their target allocations. So there is substantial new capital waiting to be deployed. A recent study by Cornell University and advisory firm Hodes Weill & Associates found that institutional investors are planning to increase the allocation of their funds into real estate to 10.6% in 2019. That’s up from 8.9% in 2013.

As these investors increase their allocations to real estate, I expect them to bring increased demand for data and performance benchmarking. In addition, increased participation from pension funds, mutual funds, other retirement assets and individuals will eventually draw increased regulatory attention.

Institutions are accustomed to a high degree of market transparency in other asset classes. When these investors direct more capital toward real estate, they tend to bring high expectations for portfolio reporting and benchmarking based on the tools they have access to in those other asset classes. This new demand creates an opportunity for competitive advantage for investment managers that provide more operational and performance transparency.

Why Is Transparency and Data Accessibility Important?

There are dozens of arguments in favor of transparency, and a few against it. I’ll address some of these, including information asymmetry and the unlevel playing field, real estate’s influence on the 2008 financial crisis, challenges with accurate benchmarking and valuation, and issues with data governance.

Large incumbent players in the commercial real estate industry have a very significant competitive advantage over newer entrants and smaller firms. Much of this is created by information asymmetry. Large firms have teams of people with several decades of experience in real estate. They have mountains of internal data and VERY smart, experienced analysts who derive meaning from the data. They also have the advantage of deep industry relationships. These firms would probably argue against more transparency, because it very likely would impact their competitive edge. It’s understandable that these firms would want to defend their business models and advantages. They’ve spent decades building businesses and invested tens of millions of dollars into creating and maintaining their competitive positions.

But while lack of transparency helps incumbents, it also creates significant weaknesses in the market. For example, lack of transparency was a significant contributor to the scale and scope of the 2008 financial crisis. Banks and intermediaries created sophisticated financial products in the pursuit of better returns. They packaged subprime and prime loans together into securitized products that obscured the risk and traded them in non-transparent markets. Executives hid dangerously leveraged positions from shareholders by keeping risky transactions off balance sheets and out of view.

There was a lack of accountability where firms packaged and resold products — from brokers to lenders to securitizers to investor portfolios. As long as the participants could transfer the risk in the next stage, they had no accountability for the quality of the assets. And we all know what happened next. As defaults quickly mounted, the foundation underneath the banking industry’s Jenga game of financial products collapsed on itself.

A lack of transparency enables a lack of accountability. And a lack of accountability creates opportunity for fraud and deceit. But it also creates opportunity for poor decision-making and obfuscates the effect of those decisions until it’s too late to reverse them.

Of course, shining a light into transparent markets doesn’t eliminate greed and artifice, and higher transparency does not necessarily lead to higher welfare. However, transparency and data accessibility can help to identify risk earlier in the cycle.

Data accessibility and market transparency do enhance market fairness. It creates new opportunities for competition, and competition breeds innovation. It improves price discovery. It drives pricing efficiency. It gives new investors an opportunity to participate in this asset class which will attract new money and grow the overall market.

Richard Bloxam, Global Head of Capital Markets at JLL, points out in JLL’s latest market transparency report, “Transparency is critical to the operation of efficient markets. As real estate continues to elevate out of its role as part of Alternatives in global AUM strategies, more private individuals are exposed to real estate whether that’s via their pension funds and insurance policies, through their holdings in REITs, listed property companies and open ended funds, or as pensioners deriving income from sovereign wealth funds. The industry is increasingly in the spotlight and transparency is a significant factor in enabling us to create healthy real estate markets.“

Impediments to Data Accessibility and Market Transparency

Data accessibility is fundamental to creating transparency. And this is where the key impediments lie. Throughout every layer in the commercial real estate industry, data is fragmented, siloed, often updated manually. Much of the data is stale and even unreliable because of how it’s managed. Some is still paper-based.

“Data accessibility is essential for benchmarking and valuation – not just at the market level, but also deep within an asset manager’s portfolio,” explained Amy Millard, Chief Marketing Officer at VTS. “CRE asset managers are incredibly sophisticated in their use of data. What’s currently holding them back is that this data usually isn’t in one place and can be inaccurate, which makes it tedious and time consuming to understand the information at a glance and make strategic decisions quickly.” Consequently, these firms have trouble identifying trends in their own portfolios in order to leverage that information for decision-making.

The problem is not a lack of data. In fact, there is so much data available in this industry that firms have difficulty compiling it and making use of the information. However, the data is disparate, dirty, and non-standard. It lives in silos that are not easily integrated.

Here’s what I mean by that:

Disparate data is heterogeneous in nature. It sits in multiple silos and differs in structure. Two different datasets may use different approaches to describe something that is fundamentally similar. But because the data sources are so different in structure, they can’t be merged in their current state. Consequently, they provide limited insight. This is a fundamental obstacle for many big data initiatives, regardless of industry. It’s particularly vexing in commercial real estate which has a lot of data like lease terms and facility infrastructure that is difficult to shoehorn into a single number.

Dirty data contains errors. Dirty data can be caused by a number of factors including duplicate records, incomplete or outdated data, and the improper parsing of record fields from disparate systems. It can be caused when disparate data is improperly merged. For example, real estate terminology tends to vary in different markets across the US. The terms “modified gross” and “industrial gross” are used to describe similar data, but the calculations are different. There are also differences in conventions. For example, in San Francisco, net effective rent generally does not include tenant improvements while New York does. If these two data sets are simply merged into an integrated database, you can end up with dirty data. To analyze data and derive meaningful information such as benchmarks, you have to have it in a clean, normalized format.

Non-standard data comes in different formats. It can refer to issues with the way data in a specific field is formatted. But more commonly in CRE, data is not digitized at all. For example, lease agreements and subscription papers are kept in paper or PDF files, and the details are not tracked in a database.

Siloed data lives in different systems that are not integrated. This is a huge problem in commercial real estate. We have some large data providers like Moody’s and CoStar that are actively consolidating the market by buying up and/or partnering with various data providers to deliver a more complete data set. But even these sources are limited. There are 10s of thousands of different data sets and data sources controlled by vendors, brokers, property owners and managers, and municipalities. Zillow has accomplished a huge feat by aggregating data from multiple MLS systems for the US residential markets. But so far, no one has achieved anything similar on the commercial side, although some firms are trying.

Public records data around transactions is accessible by all market participants. But even this is a significant problem. For example, in the US residential real estate market, transaction and property data for more than 100 million properties is recorded and made publicly accessible. However, as Estated CEO and Founder Joshua Fraser pointed out, “Property data is recorded in many different ways and stored in thousands of different databases. Some localities have modernized their systems but it’s not uncommon to see data systems that were popularized during World War II still in use. Outdated systems also have a higher potential for creating data inaccuracies, particularly when information is manually keyed into a digital database from paper (sometimes handwritten) records. So even after a firm collects the data from all these disparate sources, they face a monumental hurdle of cleansing the dataset, removing errors and normalizing the output––which is more art than science.”

Fraser points out that “Normalizing public data is a formidable task. It’s not the quantity of data that makes this difficult but the uniqueness of each county, city, and parcel contained in the dataset that creates endless specialized data fields. In order to remove entry errors and draw insights from an aggregate of this data, it must first be normalized and then organized in a way that allows easy access.”

Crowd sourced data has other problems. For example, CompStak is crowdsourcing commercial real estate data from brokers to track leases, sales transactions, and property level information. Michael Mandel, CEO of CompStak says their business model is based on their ability to process dirty data. They capture unstructured data and use their proprietary processes to produce high quality, structured, clean data.  Mandel said, “We get data in emails and scanned PDFs. It can come in totally unstructured formats or in no format at all. We standardize and normalize it, cleaning it up to create a robust data set. We’ve invested tens of millions of dollars to be able to take dirty data and through our proprietary algorithms, analyst intervention, AI and machine learning, provide data to our members that is high quality.”

Another challenge with crowd-sourced data is its reliability. Some firms contact brokers and other market participants by phone to obtain transaction data. Others rely on firms to submit that information. But in both cases, the accuracy is not consistent if the market participants are not incentivized to disclose accurate details. Without meaningful incentives, brokers, landlords and lessees may be reluctant to share at all.

Fortunately, we’re seeing more technology providers emerge that collect the data in a way that makes it reliable. For example, data companies such as CompStak and Estated and property management technology firms like RealPage and VTS collect highly accurate information and are promising to provide insights based on aggregated data. This should help with market projections and accurate estimates of prevailing rents in a specific area. But each of these firms focus on specific market sectors. For example, Estated focuses on residential single family homes, RealPage focuses on multifamily and VTS primarily covers office buildings.

As more platforms emerge, they create their own silos of disparate data, as each firm protects its data as part of its competitive advantage. In doing the research for this article, I spoke with more than 30 different proptech vendors. Each has its own data set, it’s own “special sauce” for aggregating and using that data, and its own data model. This level of vendor proliferation creates its own set of problems. And it’s growing bigger every quarter as new players emerge. The sector is attracting more venture capital than ever before, and growth in proptech shows no signs of slowing down.

Standards Are Essential to Data Accessibility

As Prasan Kale, CEO of Rise Buildings pointed out, “big data is useless, actionable and relevant data is king.” Data has to be standardized or normalized in order to be actionable. Dirty, disparate, and non-standard data is not actionable. So what will it take to make commercial real estate data more actionable? First, we need to invest a lot more work in creating open standards.

Why are standards important? Our ability to share information on websites, our mobile devices, even our cars and homes are all built using industry standards. As Estated points out in this blog, “Whether you use Beats or Bose you use Bluetooth. Whether you drive a Ford or a Toyota you use the same gas pump. The fact that you can read these words on your phone or computer on the internet is due to agreed upon standards from multiple participating organizations.…” Without standards, virtually nothing in our modern lives would be possible. Standards create interoperability. Interoperability allows platforms to work together.

RESO accomplished creating a standard data model for residential real estate.  Now we need something similar for commercial properties.

Standards Enable Open APIs for Data Accessibility

The key to the FIX protocol that I described at the beginning of this article is that it enables a standard approach to integrating systems. In a way, it’s an open API.

An open API is a publicly available application programming interface that provides developers with programmatic access to a proprietary software application or web service. APIs allow distinct applications to communicate and interact with each other.

Many of the original accounting systems in CRE were developed as on-premise systems. They were designed before the concept of an open API existed. However, that is changing. It’s common today for applications to be designed with an API built in.

However, even though most modern PropTech platforms have APIs, integrating them and normalizing data between them is a challenging task. Different systems use different data models, different approaches to calculations, and different workflows. For example, a single asset manager might use systems that calculate net effective rent differently.

Integration can be made much simpler if both systems use a common language – a common data standard, common definitions for fields, and common communication layers. This makes information easier to share between systems.

Interoperability is Key

This ability to share information between systems is called “interoperability.” This means that two systems can be integrated, share common data, and trigger workflows and other actions based on that common set of data.

Real estate operators typically have to access data across multiple disparate systems to manage their back office. These systems are not built to natively connect with each other, so operators are left with either manually pulling data from different systems or relying on vendors to integrate.

Amy Millard said, “For asset managers, their data is not just inaccessible, it’s siloed. So to accomplish an analysis, not only do they have to figure out where the piece of data they need is contained, the next piece of data isn’t necessarily in the same place, so they have to hunt. These original systems weren’t designed for interoperability.”

Fortunately, many modern PropTech vendors do integrate with other platforms in a typical real estate management technology stack.

For example, IMS integrates data from multiple platforms for its clients. Ron Rossi, Vice President of Business Development at IMS said, “Those systems could include corporate accounting, property accounting, tenant and leasing, renovation and construction budgets, investor data, organizational charts, loan information, pipeline reports and more. Managing data in that many applications is terribly inefficient and leads to data integrity issues. IMS has built an open ecosystem that allows for streamlined connectivity to other software platforms through APIs. This allows firms to make better decisions and position for scale by centralizing data and removing manual intervention from workflows.”

VTS was also built with an open API. Amy Millard said, “When VTS was created, we developed an integration layer to become the front end to many of these applications so people could see their accounting data in VTS. We integrate with most major ERP systems and everything from Yardi, MRI, JD Edwards, and Salesforce to niche systems using our API.”

Amy Millard said VTS did 128 integrations this year alone. “We’re used to going into a customer that has a bespoke system and figuring it out. If we didn’t, we wouldn’t be able to grow.” Millard pointed out that “People in property world should expect their tech providers should be doing this for them. This is what software companies were invented to do. This is our job.”

But this is not straightforward. “There are 2 kinds of standards to enable interoperability,” Millard said. “There’s the communication layer, and the standard of what data is exchanged. Like what is a net effective rent in this system versus that system?” So once you integrate the siloed data together, you have to dig deep into the weeds to make decisions on standardization. Otherwise, you get dirty data, and that’s not useful.

How often can real estate agents identify the top five factors homebuyers are looking for in their search? If we are being honest, it is probably a low percentage. But this isn’t necessarily the agent’s fault. People have a hard time predicting what they want, so they might be focusing on the wrong factors.

Standards are Key to Interoperability

Standards are essential to creating truly interoperable systems. But the industry has been slow to consider data standards. For example, OSCRE is a standards body for commercial real estate. OSCRE has been around for 15 years, but very few of the commercial real estate professionals and vendors I’ve spoken with recently have even heard of the organization.

I talked with Lisa Stanley, CEO of OSCRE. “Real estate has been slow in recognizing the power of emerging technologies. Most of the participants are comfortable with spreadsheets. They can manage their businesses with 300-500 columns. But these workbooks are generated by humans, passed by humans. There is a tacit assumption is that no one makes a keystroke error.” Once errors are introduced into spreadsheets, it can be quite difficult to find and correct them.  

Stanley says “Increasingly, corporate and investment managers are beginning to recognize importance of data governance. You can’t put a data governance function in place without standardized data.”

Interoperability Delivers Outsized Advantages Within a Firm

While FIX first focused on interoperability between firms, standardization delivers dramatic benefits inside a single firm. Aggregating, standardizing and normalizing the data can give a completely fresh view of a firm’s performance. This can deliver outsized impact, particularly in tighter economic cycles and with technology savvy investors.

As investors shift larger portions of their portfolios to real estate, they bring higher expectations for data quality, data-informed decision making, and data governance. These investors are very likely to evaluate CRE firms based on the information the firms are reporting.

Imagine that you’re a investor considering two real estate investment firms. One firm has standardized data and can show you benchmarks and performance in a standardized manner across their portfolio. You can see that the information being reported is consistent across the company. The other is a spreadsheet-driven company with different teams managing different spreadsheets with different approaches to the data. Which would you choose? All other things being relatively equal, are you more likely to go with the firm that’s got the standardized approach?

“Real estate is in a state of transformation,” Stanley said. “The companies who are able to build a digital ecosystem will emerge as extremely powerful competitive forces in the marketplace, in a way that could significantly impede or even eliminate competitors.”

Hurdles to Standardization

The commercial property industry has some significant hurdles to get to this point of standardization. The most glaring is the need to establish common entity identifiers. In the trading world, we use CUSIPs – nine-digit, alphanumeric numbers that are used to identify all stocks and registered bonds in the US and Canada. CUSIPs are designed for use by computerized trading and record-keeping systems and facilitates settlement and clearing of trades.

In the real estate world, we have addresses, block and lot identifiers, and GPS coordinates. But these don’t always accurately identify a property. Address resolution is a tricky thing. According to Brett Friedman, head of sales and marketing for Cherre, “you have to make sure the real estate related entities (lot, building, unit, etc.) match across data sets before you can begin to think about appending data fields to a particular address or looking at the bigger picture. In a complex market like New York City, you encounter challenges such as alternate addresses, vanity addresses, entrances on different streets or avenues, or units described inconsistently.” Friedman says that in New York City, these “edge cases” where addresses between two different records might not match are the norm, and you’re in fact making the data lake dirtier by not resolving the entities as a first step in your process.

Asset descriptions also need to be standardized. Friedman provided an example. Let’s say we’re doing a valuation on a multi-family property in New York. We can look at comps, but they might not reflect the entire composition of the building. In a commercial context, you may have a comp for the building as a whole, but includes information about only twenty percent of the units. Or in a residential context you might be comparing units on lower floors with fewer windows and views in one building to another where the units are on top floors with great exposure. The comps won’t be accurate and therefore, the valuations won’t either.

If a firm wants to do data-driven modeling, consistency in the data is essential.  You must have standardized, complete, connected data. If your data is incomplete or inaccurate, you’re better off just trusting your gut instinct.

So Where Do We Start?

After Salomon Brothers and Fidelity launched FIX in 1992, it continued to be developed and refined by working groups with hundreds of volunteers from about 70 financial institutions. In the early days, the global steering committee felt that vendors would skew the protocol. So they excluded vendors from working groups for the first ten years.

But I don’t think this would be wise path for the commercial real estate industry to follow. In fact, I think vendors should lead the attack, joined by large real estate firms. First, vendors need the interoperability that a standard will bring. They’re already working to integrate systems on behalf of clients and stand to benefit from reduced costs and faster deployments. They can take advantage of economies of scale as they deploy to a broad audience of clients.

Second, vendors have the technical resources needed to drive a standards body. I don’t think it’s something we can expect smaller real estate companies to drive, and if the standard is left to larger firms only, they might skew the protocol to favor their unique business needs. But this should not be led by just a small handful of vendors either. Too few, and we’ll not cover a broad enough set of use cases. Then the standard will address only a small segment of the needs.

When FIX began to proliferate, adoption was primarily driven by the largest firms. That can happen in CRE too. Large asset managers can drive adoption by requiring vendors and business partners to conform to the standard.  However, small firms can also jump into the mix, and frankly, they’ll be the ones to leverage the benefits most quickly, because their use cases will be less complex.

Standards Create Interoperability, Interoperability Enables Transparency

FIX revolutionized the capital markets, making them more transparent, more efficient and more accessible. Open standards and interoperability will also revolutionize the commercial real estate industry. These are the foundations to creating transparency both within a firm’s portfolio and across markets. This could revolutionize the real estate industry, bring a transformative level of transparency, enable better benchmarking and improved analytics, and create a more level playing field.

Transparency creates market confidence. It will help to increase liquidity, reduce costs, improve pricing efficiency, and spur even more innovation. It will also help reduce corruption and increase accountability, making the markets safer and more resilient.