Sales has CRM, accounting has accounting software, HR has HRM. So what does Marketing have?! Marketers typically have… dozens of Chrome tabs, Excel files downloaded from every conceivable data source, and an array of separate tools: from Meta Ads, Google Ads, TikTok Ads to Google Analytics, CRM, Fanpage Insights, e-commerce platforms like Shopee, TikTok Shop, Lazada… and countless more downloaded Excel files.
In general, marketing data spans multiple platforms, partners, and departments, making it both massive in volume and diverse in sources. The pace of change in both data and tools is relentless.
This scattered, fragmented data storage creates significant challenges for marketers:
1. Time wasted centralizing data: Marketers spend too many hours “gathering” data instead of analyzing it.
2. Incomplete view, prone to errors: It’s easy to make mistakes when looking at data from just one channel or isolated report.
3. Wasted potential: Each reporting cycle requires hours of manual data processing, time that could be spent optimizing campaigns, improving efficiency, and creating greater business value.
Data Centralization: The Key to Helping Marketers Master Their Data
The good news is that data centralization has become far more feasible and cost-effective in terms of both budget and technology. Data centralization isn’t just a technical solution. It’s a strategic key that helps you shift from a reactive to a proactive position. By unifying your data, you can build a “single source of truth” that provides a more comprehensive view of your customers and market.
Building a data warehouse and Dashboard from 2 Zeros:
>
ZERO ENGINEERS: I don’t have an IT background. I’m purely from Marketing and Communications. However, I taught myself through Google courses and started centralizing my team’s data sources into BigQuery using free-to-very-cheap tools: Coefficients, Google Sheets, BigQuery.
>
This doesn’t mean I can replace Data Engineers. Rather, I want to share that for the needs of a small, agile department like marketing, you can absolutely start with low-code solutions and basic SQL skills. Data Engineers are still essential for more complex, systematic, multi-stakeholder projects.
>
ZERO COST: In the first year, with basic marketing needs and a “make do with what you have” mindset, I incurred virtually no costs. I even split data into small batches for connector transfers, combined with BigQuery’s free 10TB tier for storage and processing. My team spent nothing, except for the effort of building and processing.
>
In the second year, after the trial period proved the value of data in optimizing results, we decided to invest more seriously: paid connectors for larger data volumes and restructured our data warehouse for bigger, more complex analytical models.
Overall, investing in data is a long-term vision that requires ongoing investment in skills and budget. However, it doesn’t have to be complex or expensive from the start. Here are 5 steps for you and your team to reference:
Step 1: Identify your data sources
This is straightforward. List all the data sources you’re currently using or have available but haven’t utilized yet. Here are some common ones:
- Customer data: CRM, Loyalty programs
- Transaction data: Sales from POS, e-commerce (Shopee, TikTok Shop, Lazada, Tiki…)
- Advertising and media data: Ad platforms (Facebook Ads, Google Ads, TikTok Ads…), social media, email marketing, website analytics like GA4, Microsoft Clarity
- Internal data: ERP, events, surveys…
Step 2: Prioritize your data sources
Not all data is equally important. After listing them, evaluate and prioritize which sources should be centralized first based on three factors:
- Frequency of use: Which data do you or your team use daily/weekly for decision-making?
- Importance: Which data has the biggest impact on campaign effectiveness, ROI, or customer understanding?
- Difficulty: Which data is easy to extract with clear structure, versus needing significant effort to clean and standardize?
You can create a simple scoring table to clearly set priorities.
Step 3: Choose your Data Warehouse & Connectors
To store data centrally, you need a Data Warehouse, a secure and efficient place to aggregate, store, and process large volumes of data from multiple sources.
For marketing data needs, I chose Google BigQuery based on several advantages:
- Free first 10TB and flexible pricing: BigQuery provides 10TB of free data for storage and queries, well-suited for internal department needs.
- User-friendly for non-technical users: Many basic tasks like data connection and table joining can be done directly through BigQuery’s intuitive web UI.
- Great integration with marketing data sources: BigQuery easily connects with Google data sources like Google Ads, Google Analytics, etc.
Next, to move data from various sources into the data warehouse, you can choose between two approaches:
- Code it yourself with Python: low cost, mainly time-intensive if you have many data sources, requires Python coding skills
- Use APIs/Connectors: These are automated “bridges” that extract and load data from sources into the warehouse.
Step 4: Execute data integration and processing
After data is loaded into the Data Warehouse, the next step is processing and transforming it into valuable information. There are two main approaches:
- ETL (Extract, Transform, Load): Data is extracted, then transformed (cleaned, standardized, aggregated) before being loaded into the warehouse.
- Pros: Data is cleaned before entering the warehouse, keeping it “clean” and manageable.
- Cons: Time-consuming intermediate transformation step. If requirements change, you must rerun the entire process.
- ELT (Extract, Load, Transform): Data is extracted from the source, then loaded immediately into the warehouse. Transformation happens inside the warehouse using its processing power.
- Pros: Maximizes BigQuery’s computing power, more flexible since raw data is always available for different transformations.
- Cons: Raw data stored in the warehouse may take more space (though cloud storage costs are quite affordable).
Step 5: Back up and verify data regularly
An often-overlooked but critical aspect of data management is backup and verification. Even with the most advanced automation systems, errors can occur, or source data may change unexpectedly.
- Importance:
- Error prevention: Helps restore data to a previous state if there are errors during syncing or processing.
- Dispute resolution: If there are discrepancies between departments or partners, you can reference the original backed-up data for comparison.
- Implementation:
- Create daily snapshots for critical, frequently-changing data sources.
- Data Integrity Checks: Periodically run simple SQL queries to compare row counts and key metric totals between backed-up data and sources.
- Set up alerts: Consider setting up automatic alerts if there’s a significant discrepancy (e.g., >5%) between Data Warehouse and source data.
Today’s article was quite long and perhaps heavy for beginners, but if you’ve had the patience to read to the end, I believe you’ll have the persistence to learn, experiment, and gradually master your data, ultimately improving the effectiveness of your marketing activities.
Cheers,
marketingintelligence.vn