Growth of SaaS: blessing or curse?

The public SaaS model has been growing consistently over the last decade (figure 1). And the predictions are that it will keep growing. Applications are being delivered more and more as a SaaS model, this comes with the benefit that users and companies don’t have to install software, but can easily log in using a webpage and directly use the application. On the other hand, this also means that their data resides in the database of the supplier.

Figure 1: Growth of SaaS: Source: https://www.statista.com/statistics/510333/worldwide-public-cloud-software-as-a-service/

And the last part is exactly what this blog is about “their data resides in the database of the supplier”. Since the need for data-driven decisions is growing as well, read here and here. My view is that this brings a tension field between the two:

• On one side almost all the applications in modern organizations are provided as SaaS, so the organization does not “own” the data.

• On the other side, we have the growing need to use data to make dashboards, reports, analysis and apply advanced analytics, which needs data.. a lot!

Dispersed data landscape

An efficient data team delivering high-quality analysis and reports necessitates a centralised data storage system where data from various source systems is stored. Whether you manage this through a datalake, data warehouse, data lakehouse etc is not what this article is focused on.

But it is a given that the data is dispersed and stored in the suppliers’ database when using SaaS applications. This will cause the data team to spend a lot of time extracting data (manually) from the different systems to make their analysis. And what happens when the analysis needs to refresh every hour? We have seen many cases where our clients spend couple hours daily to make several csv / excel exports.

The solution: build data pipelines and centralize your data!

Our advice is to start investing in a strategy to centralize your data and even better, model your data in a data warehouse. This starts with building data pipelines to extract the data from the source systems.

On the other hand, it should be an awareness in the organization that buying new applications means that the companies data and information is stored in a different location and something needs to happen to bring that information and data to the companies data warehouse:

Data should be seen as a company asset, not a resource.

In most of the cases we see with clients is that centralizing data becomes a need when there’s a need for a certain analysis, model, or report. Then the whole process starts with discussing which data is needed exactly, and where is it stored? Does the application provide an API endpoint? What is the frequency of the data refresh? And to make it more complex, do we need incremental updates? This can take months before the actual product, the report, is built, which in the end will cause frustration in the business. To use a famous quote I’ve seen somewhere on Twitter:

Business: “Hey can you get me this data? I need it asap”

Data engineer: “Yes, let me SELECT * FROM magic_table_which_consists_of_10_different_source_systems_over_multiple_dimensions”

Think in information and data models when you think about data

More and more people and departments are starting to use data, which is a good development. Not only the data analyst in the company needs it, but finance and marketing for example are starting to use PowerBI or other Self Service BI tools to make their analysis.

But in many cases, the organisation thinks of data as spreadsheets where some information is stored. When you zoom out, the picture is actually bigger (for the data engineers, Kimball and Inmon preach this as well):

The company practices a certain business, this needs information and data and thus the companies business can be translated into a general data model

Let me clarify: let’s think of a simple business that rents cars. In terms of data and information we can think of some kind of general model:

• Cars information (current fleet)
• Planning of the cars, which car is rented to who and when?
• Financial transactions, when a car is rented, the invoice has to be sent and the customer pays
• Customers (CRM)

We can probably go and make it bigger and more detailed, but the idea is clear I think.

Source: https://docs.telerik.com/help/openaccess-classic/quickstart-carrental-sample-database.html

So when we translate it to everything I wrote above, when we think of “which information has to be centralised and available for the organisation”, we can start with the general business and data model we described above. So it’s all practically linked.

Conclusion

So to summarise, SaaS can be a blessing when you treat your data as an asset and make sure the information of your organisation is available in a central place where users can access it. This means there has to be a strategy and a team that provides this by building data pipelines that extract data from the source systems and model it in a data warehouse. Otherwise, it can be a curse from the data management perspective, since the data is scattered over different software vendors and when the time comes, it’s not easy to access.