By 
This week we pick up again with Star Director of Product Strategy and Marketing Tim Ward, for more on how data warehouses work in general and Star's design and development process in particular. Read part one of this two-part series here.
Data warehouses have been around for awhile. Why build one now, Tim?
At the enterprise level a firm may have tens, or even hundreds, of thousands of employees, and the STAR system has been collecting all types of employee activity continuously for years. A lot of data is being generated and stored, and we want to increase the value and utility of that data. Further, not only is the amount of data increasing but so is the number of data sources.
We have two conflicts-monitoring products now: Employee Conflicts Of Interest and Compliance Control Room. We also have various reference data sources, covering the markets, news, and political contributions. We have broker feeds. With more data sources and more volume than ever, it made sense for clients to be able to aggregate and crosscheck within a single database instead of having it siloed in various locations and disparate formats. With a data warehouse they can feed all of those data points in, cross-reference within it, and then the consumer—the person generating the report—just runs the query and gets back her results. We're now doing all the heavy lifting in the powerful infrastructure of the data warehouse.
What was the process in building the data warehouse?
We were thoughtful about the design. A lot of time was spent thinking about what the best structure for the data would be and how to arrange it so it can be used for multiple different purposes, such as delivering analytical dashboards or allowing firms to extract it and supply their own data warehouses. And we made sure everything is secure. That it supports all the different types of authentication. That we got the security model properly embedded. So it was a comprehensive design process—thinking about the data people will actually need and want to use. It's all a real evolution in how we process data, and it will be for our clients, as well.
What are the common use cases for a data warehouse?
Once you've got a data warehouse, it can be put to multiple uses. The first one is, clients can use it to extract data for their own particular use cases. They might have their own corporate data warehouse and think, Star's hosting this application, there's a lot of useful data in there, we can retrieve it all, put it in our own data warehouse, and then cross-reference that with our data sources. That could be customer databases, accounting systems, risk-profile systems, sanctions lists, or lists of politically exposed persons, etcetera. So clients can end up with an even bigger, more useful data warehouse when all is said and done, with a much wider scope than even ours.
Clients can also use Star's data warehouse to connect with their own reporting tools—like Tableau or QlikView—business intelligence applications that allow for analysis and data manipulation, some of which can be quite sophisticated. They might have machine-learning algorithms or data-science applications, too, which can call into our data warehouse via a suitable integration method, pull the data, and visualize it within their own applications.
The third use case is us embedding data within our own application. That's what our new Compliance Dashboards are all about: pulling data from the data warehouse and embedding it into the STAR app as dashboard analytics—near-real time, dynamic reports which give clients more immediate and more extensive visibility into what's happening up and down the org chart. This is the capability that will allow compliance teams to push more responsibilities down to the first line of defense—employee supervisors—to lighten the load on compliance. It's important to note that, while all of this data has always been there in the primary-line STAR database, before dashboard analytics it's never been so easy to access or so usefully and intuitively presented.
Why are data warehouses especially good for dashboard functionality?
Our data warehouse serves up data to our dashboards: data that's been aggregated for a particular purpose—to a dedicated subset of users—and pre-calculated. This arrangement is what makes the speed and sophistication of the reports they deliver possible. It means you can splice and dice data on the screen via many different dimensions. Without the data warehouse, every time you put a filtering criteria on the system would have to recalculate with all your new values, which could take minutes, or even hours, depending on the amount of data you’re trying to pull. The functionality would be so clunky you just wouldn't use it. And sometimes you don't know what you're looking for until you see it. You want to explore and navigate. Drill in and sort. Just kind of follow your nose, to get an understanding of why the data looks the way it does.
Will the data warehouse be of particular help with any other Star products?
One particular use case we're seeing is the data warehouse acting as a bridge, or common component, between our Employee Conflicts Of Interest monitoring suite and Compliance Control Room. One of the things Compliance Control Room let's you do now is log a deal or situation that relates to a financial transaction. Another is running a conflicts search to see if any of those situations are in conflict with each other. Are you representing two competitors? Are you dealing with a client who's got bad credit? Whatever the potential conflict might be. The next phase of Compliance Control Room will allow people to run conflict searches not just against the deals themselves but also other sources, including employee activity. But instead of going to the Employee Conflicts Of Interest database, that work will be done in the data warehouse, so the operation of the primary-line application database won't be impacted.
And because we've decoupled these systems from each other, and connected them through an interface, clients will be able to use similar interface tech to connect STAR to their own systems: the advantage being, they can run conflicts searches against their own databases. To be clear, that information will exist in the firm's own data warehouses or applications, and the STAR system will be able to run those conflict searches against it. So rather than us having to pull all of this data physically into the same location, we're aggregating Star data in our own data warehouse and data marts as well as allowing data to sit in other locations so it can be called on-demand.
As a real world example of how this functionality might work, instead of us getting a list of, say, every single customer a large client bank might have, which could number in the millions, and then pull that data to Star every time it changes, which could be very often, we're saying 'keep it where it is. Our system will instead call yours and pose a specific question.' So will the Star data warehouse be of use outside of dashboards? Yes. 100% yes.
