.png)
A data warehouse is useful for automating reporting workflows, and essential for running sophisticated analytics on larger datasets
Your data warehouse should contain a copy of all key data in your various business systems, such as CRM (e.g. Salesforce), ERP (e.g. Netsuite), and WFM (e.g. Kronos). Data is replicated daily, or sometimes twice daily. Your business intelligence tools will point to the data warehouse as their data source to run analytics and generate reports
We recommend having a data warehouse in place before scaling up reporting and dashboarding
The deployment process typically follows six steps and can range from 4 weeks to 3+ months depending on the quantity and complexity of your systems
- We run discovery sessions with the business users to identify the key use cases, goals, and overall requirements
- Concurrent with step 1, we align with the IT team on the technical architecture, technology stack, naming conventions, and data governance
- Working with both the business and IT, we define the list of data sources and objects to replicate, as well as the ETL method for each object
- Our data engineering team spins up the data infrastructure, then develops and tests the replication pipelines
- While most tables will be in the Silver layer (meaning they will exactly match the sources), we also create a set of Gold layer denormalized tables for the frequently-used business concepts like sales
- We provide documentation and training to both the business users and the IT team
We use the Microsoft Azure data stack, as mid-market companies are usually already in the Microsoft ecosystem, making for a seamless user experience. The specific products within the stack will be recommended after we understand your needs. We find that a simple setup with Azure SQL Server and Azure Data Factory is sufficient, cost-effective, and simple to manage for most SMEs, but if complex data transformations are required on extremely large volumes of data, we utilize Databricks and Python
Replication pipelines may occasionally break due to issues in the source systems, such as expired credentials, connectivity issues, or schema changes. We will be available to fix any issue that comes up after project completion, either on a time-and-material basis when an issue is spotted, or with an on-going monitoring and maintenance service
