In a perfect world, marketers wouldn’t have to think about data aggregation.
You wouldn’t have to worry about stitching together the campaign data from Google, Facebook and other sources in a single easy-to-understand report. You could focus on what you do best: creating effective, efficient campaigns that get results.
But aggregation is an important concept to understand because it can ultimately affect how quickly (or even if) you’re able to obtain useful insights from your marketing data.
Not to be too dramatic, but data aggregation is one of the foundations of marketing analytics, something that makes more advanced insights and analysis possible. Without data aggregation, there’s no marketing attribution, no predictive modeling. You’ll be spending all your time just trying to get basic, run-of-the-mill reports finished — and staying late or working on weekends more than you’d like.
In this post, we’ll define data aggregation, explain how it works and share some best practices for using it in your reporting and insights.
What Is Data Aggregation?
Data aggregation is the process of bringing together data from multiple sources and consolidating it in a storage solution for analysis and reporting.
In practice, data aggregation can be challenging for marketers because we use so many different media platforms, ad networks and analytics tools.
There are roughly 8,000 martech platforms on the market, according to ChiefMartec.com’s Marketing Technology Landscape supergraphic for 2020. The average multi-channel campaign will include 7.2 different channels, Gartner notes. That’s a lot of data from a lot of sources!
What’s more, accessing that data can be tricky, too. There are basically two ways to do that: manual or automated data aggregation.
What is Manual Data Aggregation?
This is where you manually download campaign data directly from a media platform or ad network. The platform will export your data into a spreadsheet that you can then edit or combine with data from other sources.
It’s how most marketers used to access their campaign data, and it is, to be frank, kind of miserable.
Now, it’s not too bad if you’re a small business that only uses two or three media platforms, and you only do reporting on a monthly or quarterly basis.
But what if you’re an agency with 20 or 30 clients, each of whom uses 5 to 7 channels in their campaigns and who all want weekly reporting?
Or what if you’re a marketer for a retailer with a regional or national footprint that needs reporting broken out by ZIP code or some other geography, again on a weekly basis?
Or maybe you’re running complex multi-channel campaigns that run offline and online, and you need to match up all the like dimensions and metrics to report on performance.
In cases like that, manual data aggregation can consume almost all of your reporting team’s time. Your team will be so busy downloading data and building reports, they don’t have any time to analyze the data for useful insights. It can get really bad — we know marketers who’ve spent 80 percent of their workweeks just on data management.
What Is Automated Data Aggregation?
The good news is there’s another option: automated data aggregation. You can use an aggregation platform to automatically pull campaign reporting data from a source system like Google Analytics or Facebook Ads.
Most media platforms have APIs that allow users to connect directly to their accounts and access data. Authorize the connection with your aggregation software, and boom, the software will collect all your data for you — on a daily basis in most cases.
Of course, APIs aren’t always available. A lot of marketing analytics software and aggregation platforms brag about their vast libraries of connections, but inevitably, there’s always one data source that you really, absolutely gotta have … and they don’t have it.
(In cases like that, Alight’s ChannelMix platform uses automated FTP and email processes as an alternatives for ingesting data.)
Automated Data Aggregation Still Requires Human Beings
One thing to keep in mind: Automated data aggregation is awesome, but it’s not magic. It will occasionally break.
This usually happens on the data source’s end. Maybe Facebook suddenly adds or deprecates a dimension. Or the API goes down for a day.
Because of cases like that, you’ll need someone on your team who can monitor your aggregation platform and, if necessary, get things back on track by pulling missing data from a source system after the API comes back online.
How Does Automated Data Aggregation Work?
You may have heard the term ETL — Extract, Transform, Load — a computing term that describes how data is moved from one system (or many systems) into another.
Data Extraction
The first step, Extract, is when you pull the data from Facebook, Google Analytics or whatever source system you’re connecting to.
Data Transformation
The second step, Transform, is where your platform applies a set of rules to your data and prepares it for reporting, analysis or storage. That could include data cleansing, i.e. checking a dataset for incomplete or corrupted entries, or data harmonization, which makes data from different sources conform to a unified standard. For example, making all the date entries use the same format.
Data Loading
The last step is Load. Your aggregation platform sends the data to its final destination. It could be something as simple as a spreadsheet, or it could involve a more advanced storage solution, like a cloud-based data warehouse.
ETL is one model used in data aggregation. There’s also ELT, or Extract Load Transform. And depending on your platform and data sources, you might go through multiple rounds of Transform and Load (or Load and Transform). But that’s a little beyond the scope of this post.
How Data Warehouses Can Help With Data Aggregation
A data warehouse is a central repository for the aggregated data from all your source systems. Popular data warehouse providers include Amazon Redshift, BigQuery and Snowflake, among others. For most marketers, a data warehouse is going to be the best choice because …
- It’s centralized. It’s the one place where all your data lives, making it much easier for you to track down what you need. If you’re sharing data across teams, a data warehouse can also ensure that everyone is working from the same data — a single source of truth, if you will.
- It’s scalable. Data warehouses can accommodate vast amounts of structured data, millions or billions of times more than what you could store in spreadsheets. If you plan to produce more advanced forms of analytics, you may need larger amounts of historical data, and a data warehouse can help here.
- It’s queryable, like a database, not like files stored on a hard drive or in the cloud. You can pull back data just about any way you want. Ask complex questions about the data (in SQL) and it will search and quickly build a report for you.
- It’s more reliable. Spreadsheets get deleted or corrupted. Someone will input the wrong numbers or accidentally alter a formula. That’s much less likely to happen with a data warehouse.
Once your data is aggregated, you can use it for reporting or analysis. If you’ve loaded it into a spreadsheet, it’s ready to go. If you want to use it with a visualization tool like Tableau, you can set up a connection to your data warehouse.
BLOG: What’s a Data Warehouse?
Examples of Data Aggregation Platforms and Software
Data aggregation software tends to come in three forms:
Stand-Alone
Sometimes, it’s a stand-alone application like Supermetrics, Fivetran or Funnel, where the focus is strictly on the ETL process. Storage is usually not included with these types of stand-alone tools. A lot of times, they’re part of a larger technology stack that will include a storage solution and a dedicated BI tool like Google Data Studio or Power BI. Stand-alone tools could be a good fit for a marketing team that is reporting on multiple data sources and that possesses the skills to build out their own analytics stack.
End-to-End Software
You’ll also see end-to-end analytics software like Domo or Improvado, where data aggregation is one part of a larger suite that also includes storage and visualization. This type of software tends to be very self-service — the goal is that a nontechnical employee will be able to aggregate data and build reports on their own. But you may find that it requires more support or technical expertise (like a working knowledge of SQL).
End-to-End Solution
At Alight, we take an end-to-end solutions approach to marketing analytics, where the vendor offers platform, services and a roadmap for using them. ChannelMix, our platform, includes data aggregation, storage and harmonization, alongside attribution modeling to power visualizations in Tableau, Power BI and other leading dashboarding tools.
And it’s supported by a team of data engineers, strategists and data scientists who maintain the data warehouse, assist with connections and trouble-shoot problems. Marketers get the data they need without worrying about where it came from.
Marketers Shouldn’t Have to Worry About Data Aggregation
While it’s useful to understand data aggregation, it isn’t something you as a marketer should have to think about on a daily basis.
The key is to choose the right aggregation software for your needs and, when necessary, support it with technical expertise inside or outside your organization.
The results — faster reporting, more valuable insights, better business outcomes — are worth the effort.
Save Hundreds of Hours on Campaign Reporting and Data Prep
Alight’s end-to-end analytics solutions enable marketers to spend less time on manual data aggregation and more time creating valuable insight. Schedule a free solution consultation with our team!