Rising to the data challenge in the private markets

By Jennifer Tribush; Tom Vogt, State Street

Published: 30 November 2021

The private markets are in the midst of a dramatic, two-pronged shift. First, investors are pouring money into this asset class as they hunt for alpha and opportunities to diversify. According to Preqin, assets under management in the private markets are expected to grow by nearly 13 percent annually to US$12 trillion by 2025.1 And second, private market assets such as real estate, infrastructure, private equity, and private debt, which were previously available mainly to institutional investors, are increasingly becoming commoditised as they become available to more retail channels.

This is all great news for asset managers in the private markets. But are they ready for it? A crucial factor in the changing nature of private market investing revolves around data. Managers need to become more effective and efficient at managing, analysing, and reporting on the vast quantities of complex and unstructured data related to existing and prospective investments. Improvements in these areas could be a significant driver of alpha, but more pressingly, asset owners are demanding it. They need greater transparency on holdings, the investment process, costs and on sustainability criteria, placing tremendous strain on the existing reporting capabilities of many asset managers.

To find out if they are prepared to handle this data challenge, we surveyed 85 asset managers and 85 asset owners in the private markets in August and September 2021. What we found is that many institutional investors want to do more with their own analytics and modelling, and they need greater levels of detail from asset managers. They have become accustomed to the high bar of transparency that public markets have set in terms of data quality and accessibility. Furthermore, regulators are demanding greater accountability for all stewards of capital, which requires more robust reporting from asset managers. Our survey also found that many asset managers are struggling to keep up with increased demands from their investors. Their data is often siloed and not standardised, requiring manual intervention, which makes producing a unified, overall view of private market portfolios difficult, and a public-private portfolio view a dream.

What asset owners want

Generally, asset owners want to plough more money into private markets investments, but several issues are constraining their ability to do so. Asked how they believed data and technology could boost private markets allocations, asset owners cited improved transparency, product availability and higher quality data. These were particularly important issues for wealth management organisations such as private banks, family offices, discretionary fund managers and independent wealth managers. Many of these client groups are entering private equity for the first time and are likely bringing high expectations set by experience in public markets.

Private market investors need consistent asset-level data across their portfolios for a variety of purposes, such as performing forward-looking scenario analysis, helping with deal origination, calculating valuations, portfolio monitoring and risk management. More than two-thirds of asset owners say there is a significant opportunity cost because of problematic private markets data. Sourcing high quality opportunities, collecting and processing the necessary data, and other related administrative tasks all take up more time in private markets, leaving staff with less time for other tasks and limiting their ability to expand the portfolio. Solving or mitigating these issues takes a significant amount of staff time for asset owners, meaning these allocations are resource intensive. Some asset owners are particularly sensitive to this: public sector pension funds in the US have been criticised in recent years for the high cost of their private equity portfolios relative to public markets funds and holdings.

It is no surprise, therefore, to discover that checking data management and quality formed an important part of asset owners’ due diligence processes when selecting private markets managers. More than half (58%) agreed that managers operating in the private markets sectors needed to upgrade their technology and data management systems to improve their service, including making greater use of application programing interfaces (APIs), for example. This means that having a robust data management architecture will increasingly be seen as a competitive advantage for asset managers.

Breaking down silos

Despite asset owners’ wishes, many asset managers acknowledge their data management systems may lack the capabilities to fully meet client demand. More than two-thirds of asset manager respondents (69%) said they provide ‘siloed’ data reports to investors – i.e., a format that does not allow for the data to be directly integrated into investors’ systems, such as in spreadsheets or PDFs.


A similar proportion said they provide ad-hoc reporting customised to investor requirements. These ways of working are resource intensive and can make obtaining a unified, overall portfolio view difficult for both asset managers and asset owners. Relevant data resides in disparate systems and documents with no cohesive approach or standard format. Outdated processes such as the use of spreadsheets and macros increase the time required to process information and are more errorprone. In addition, it can be harder to scale up strategies if the technology being used cannot keep up with additional demand for information or increasing numbers of inputs.

This manual approach to data management leads to long delays for asset managers to deliver sometimes very basic real-time information to their investors. We asked both groups for their views on how long it takes to generate the net asset value (NAV) of a portfolio, and to put together forward-looking analysis based on a market event such as an interest rate increase. While managers and owners are fairly aligned on how long it takes to strike a fund NAV – almost half of managers said it takes up to a week or more – the difference in perception on time required to run forward-looking analysis was stark. More than half (51%) of asset managers and 85% of asset owners say it takes up to a week or more to run scenarios across their portfolios.


There is little automation around these reports. Four in five (81%) asset managers said forward-looking analysis relied on manual processes to some extent – including 44% that said manual inputs were relied upon “to a very large extent.” More than three quarters (76%) said a fund NAV calculation relied on manual processes to some extent, leaving plenty of room for improvement.

The path forward

To truly capture the next stage of growth in private markets and meet rising investor expectations, asset managers will have to put money to work improving their data management and reporting processes. Integration is a key element of most solutions in private markets. Having one integrated platform allows a manager to confidently venture into new asset classes with scale, flexibility, and adaptability. For institutional investors, access to insights, and performance capabilities designed specifically for private markets, can help generate alpha. An integrated platform allows managers to access data and fund servicing specialists for consolidated  information, reporting, and insights. The right platform can also provide agility, with faster responses to client and reporting demands, and enhance risk management. It should also improve access to flexible cloud-based asset management tools such as deal management, environmental, social, and governance (ESG) reporting, investor relations and portfolio monitoring.

The private markets world is changing. Demand is growing for existing and novel types of investments and data, both from existing investors and from a whole new class of investors – and expectations are high. Investors want data fed directly into their systems as they are accustomed to with public assets. Operating models based on outdated systems and processes cannot be expected to cope with the challenges of today, never mind tomorrow. The opportunity is vast, and those that recognize the importance of modernising their technology will be in good stead to capitalise.

1 Preqin Markets in Focus: Alternative Assets in the Americas