The way in which investment firms acquire, cleanse and store data sets is changing rapidly, with near real-time quality control and delivery and new technologies that facilitate secure data access and sharing. In addition, there has been a rapid increase in both the number and of data vendors and aggregators on offer, as well as the types of data packages they provide.
That proliferation of data and potential insights is providing opportunities as well as challenges for investment data professionals, the 6th Investment Data and Technology Summit has heard.
“Data sourcing is becoming easier and harder at the same time,” Sandra Booth, Director, Data Capability at IFM Investors, told conference attendees. “Accessibility to broader, deeper and historical data in various formats is increasing and we are moving away from sourcing directly from a single originator to accessing data from multiple parties and aggregators that are bringing in multiple third-party data sets together. At the same time, we are self-servicing and self-sourcing via application programming interfaces (APIs) and equivalents,” Booth said.
At the same time, as data availability grows, identifying the information that is right for the business is becoming more challenging, Booth said. “It is almost like there is a data deluge with infinite amounts of data available, but really getting to the right data that is going to offer insights to our investment professionals [is a challenge]. You can have a smorgasbord of data, but it is really about understanding and making sure that it can meet what the business needs,” she said.
Understanding data needs and being clear on the underlying problems that firms are looking to resolve was a key message that was heard throughout the conference.
Stephen Bappert, Vice President, Content and Technology Solutions at FactSet, said firms that manage data successfully are the ones that match data sets to specific business use case and requirements.
“The clients that I have seen doing this successfully are the ones that have answered a couple of questions before they get into it. ‘What is my target operating model?’ ‘What am I trying to accomplish here?’ ‘How is that data going to be stored and how is it going to be used?’ ‘Who is the data servicing?’ ‘What are their needs?’” he said.
Bappert said that different data consumers within a firm could have very different data requirements. “Real time on demand is expensive and that is fine if that is data that might be very valuable, but make sure you are solving it for the right people. If you are trading, real-time information is super important, but if you are managing for a reporting requirement at the end of the month, you’d prefer the data to be correct rather than in real-time, so making sure you are answering the question of who you are trying to service within your firm is important,” Bappert said.
Sunny Feng, Regional Lead APAC at Reformis, said that closer interaction between front office users and data and technology teams is also key to generating better insights and minimising data wastage.
“Data sourcing is becoming a bit more involved as data service and technology teams are starting to ask more questions. Instead of letting the data drive how reporting should be done, it should be the other way around. If you start from the question of ‘what is the desired outcome’ that will then lead to what ‘what data do I need, where do I source my data from, where can I focus on my data quality controls and then how do I facilitate that across the organisation,” he said. “When you see that happening, organisations are getting a competitive advantage because investment and technology teams are working together to solve for the data issue,” Feng said.
At IFM, Booth leads a data capability team that is charged with understanding investment data usage within the firm as well as managing the acquisition, strategy and governance to avoid unnecessary duplication and maximise user insights.
One current project is a cross-firm analysis of all investment data sources to get a solid understanding of data requirements and maximise the efficient use of information across the business. “It’s about understanding our need for market data and how that data is used, because that drives licence requirements. That includes managing data vendors and really challenging the business when a request comes through for a data point, because we might already have this data coming from a different source within our systems. For us it is really about simplifying and optimising,” Booth said.
In addition to the value of the particular insights, other key considerations when selecting vendors are typically the speed at which data is delivered and how easy it is to customise and integrate with the firm’s systems.
Feng said that while large investment managers typically have the resources to manage this process internally, that may not be the case for smaller asset managers. In that case, he said outsourcing data aggregation and quality controls can be a viable alternative. “Larger firms usually have more resources, so they might have a dedicated team that can clean the data in house. When these firms choose a data vendor, they might choose someone that can distribute the data as quickly as possible, in a more raw format. Smaller firms have less resources so they might choose a provider that has the expertise in aggregating and running all these data quality checks before feeding it downstream to their clients, but obviously – with more controls in place – the turnaround time for the data delivery is a bit longer. It really depends on the size of the firm,” he said.
Understanding the problem set is equally important when thinking about data storage and accessibility, which again is an area where there is no shortage of solutions, whether it be a data lake, a data warehouse, a data lakehouse or an outsourced data as a service arrangement. Broadly, the conference heard that a shift is well underway. If a few years ago, the focus was on data storage, these days it is data access and consumption that is the priority. However, that does not mean that traditional data warehouses have become obsolete.
“These solutions all have their place, but it really goes down to what a particular solution does and whether that aligns with the problem you’re trying to resolve,” Bappert said. “You have to understand your workflows and make sure you pick a solution that fits your firm’s specific needs, rather than putting something in place that just sits there, but doesn’t help anyone.”
It’s also important to think ahead. “What you have might be working for you now, but if you are not embracing change and adapting to new technologies then you will behind your competitors, who will be able to get data quicker and make decisions much faster than you,” Feng said.
Booth said that while there will likely be a place for structured relational databases for a long time yet, technology is driving change and what is successful today may not work even twelve or 18 months down the track. “It’s possible we’ll end up with a solution that allows us to pick and choose as we wish, which will have an impact on storage as well. It comes down to how we use the data and what insights do we want. Storing those insights is also becoming important for us.”
