Building a cloud analytics data warehouse 105: How to activate your data with tips and advice

So, what is Data Activation? It’s the process of getting your data ready to team up with tools beyond your data warehouse. These tools can then use this data to perform better. Data Activation is extremely useful in situations like online marketing or personalizing websites. In these scenarios, special tools can work much more effectively with some extra data to rely on. Imagine you’re sharing specific groups of customers with social media for marketing purposes. Alternatively, you might be orchestrating certain emails to be sent at the perfect time for maximum impact. Even the product recommendations you see on websites – that’s a direct result of activated data. It’s like providing your tools with a supercharged data boost.

To get a better overview of a few use cases possible in the realm of data activation, take a look at our article on orchestrating data in a warehouse environment. 

The Importance of Pre-processing Data

Before we dive deeper, let’s talk about one of the initial steps involved in data activation. To get your data ready, pre-processing is key. Pre-processing gets your data ready by aggregating relevant information from your data warehouse and matching it with necessary identifiers from your first-party data. This is particularly useful when creating, for example, a target group of your best-performing customers.

Key Steps in Data Activation

There are a number of key steps data analysts will want to take for successful data activation:

Data Pre-processing: Prepare your data by aggregating relevant information from your data warehouse and matching it with necessary identifiers from your first-party data.

Identify Target Use Cases: Determine the specific areas where data activation can add value, such as online marketing, personalized recommendations, or targeted email campaigns.

Select Activation Tools: Choose the tools outside your data warehouse that will directly use the activated data to enhance their performance.

Map Data: Connect your pre-processed data with the chosen tools, ensuring seamless data flow and compatibility.

Implement Activation Logic: Configure the tools to use the activated data effectively, enabling them to optimize processes and improve outcomes.

Monitor and Refine: Regularly assess the performance of your activated data and tools, making adjustments as needed to achieve the best results.

Integrating Data with Third-party Tools

At the heart of data activation lies the essential task of ensuring your data finds its way to the right systems efficiently and effectively. This process is typically automated, necessitating a seamless flow that connects and integrates data with external systems. The frequency of this integration varies depending on the specific needs of each use case – it can be periodic, or even near real-time for some scenarios.

Here’s a breakdown of how data activation works in practical scenarios:

Social Media Marketing:

Maintain Audience Relevance: Keep your first-party audience updated on platforms like Facebook.

Periodic Uploads: Regularly upload audience information through the backend API of platforms like Facebook.

Common Identifiers: Link your users with Facebook users using shared identifiers.

Email Campaigns:

Targeted Communications: Trigger specific email campaigns based on your data warehouse information.

Regular Data Uploads: Periodically upload data to your email tool using available connections (SFTP server, API, or direct database integration).

Personalized Website/App Experience:

Tailored Recommendations: Elevate user experience by suggesting products or services based on past behavior.

Accessible API: Expose recommendation data or machine learning models through an API format.

User Identifier Integration: Enable your website or app to call the API with a user identifier already available within the platform.

By adeptly integrating your data with diverse tools, you unlock its full potential, driving actionable insights and achieving optimal outcomes.

Navigating Data Integration for Activation: Build or Buy?

When exploring data integration options for data activation, few landscapes prompt more questions about building versus buying than the realm of data activation. A new kind of tool within the realm of the ‘Customer Data Platform’ (CDP), seems to emerge almost every month, as numerous tools claiming to address specific parts of online marketing seek to offer an all-in-one solution.

Amidst the plethora of choices in the data activation arena, three primary directions emerge:

The CDP Approach:

Use a Customer Data Platform (CDP).

Basic Integration: Let your data warehouse handle the essential integration of valuable data into this CDP.

The Reverse-ETL Tool Path:

Define your use cases and orchestration logic in the data warehouse.

External Integration: Employ a ‘reverse-ETL’ tool to manage the actual data integration with external tools.

The Headless CDP Route:

Construct everything within and around your data warehouse environment.

From Preparation to Integration: Take charge of data preparation, orchestration, and integration.

As you embark on your data activation journey, these three approaches offer distinct strategies to align data integration with your objectives. Below we break down each option so you can make the best decision going forward.

Choosing the CDP Path for Data Activation

A Customer Data Platform (CDP) offers two main functions: easy integration of 1st party data with external marketing tools and a user-friendly way to coordinate marketing efforts across these platforms.

CDP Basics:

Integrating Data: Connect 1st party data from your website/app with 3rd party marketing tools.

Orchestration: Seamlessly manage marketing strategies across various platforms.

The Process:

Data Setup: CDPs gather user behavior data from your website and user profiles from your data warehouse.

Enhancement: Enrich the CDP with your prepared data from the data warehouse.

Integration: Upload data files to an SFTP or link directly to APIs for CDP interaction.

Orchestration: Easily decide which marketing campaigns trigger for specific users in different platforms.

Data Connection: Connect accounts by inputting necessary IDs or credentials for integration.

Website Integration:

Automatically Collect Data: CDPs often integrate directly with your website, capturing behavioral data for targeting.

Personalize Website: Feed audience data back to personalize the website.

AB Test Functionality: Some CDPs enable HTML manipulation through AB test-like features.

CDP Advantages:

User-Friendly: Great for common data activation needs.

Simplified Process: Easily integrate and orchestrate campaigns.

Behaviour Insights: Gather behavioural data for targeted strategies.

CDP Drawbacks:

Cost Considerations: Subscriptions might not align with your data strategy’s scope.

Limited Flexibility: Might not suit advanced data activation projects.

CDP Options: Examples include Bloomreach, Blueconic, Relay42, Oracle CX Marketing, and Tealium AudienceStream, each with pros and cons.

While this series won’t guide your CDP selection, it’ll help you weigh using a CDP against other methods for effective data activation.

The Reverse-ETL Approach

Sitting between building and buying is the Reverse-ETL tool path. The term ‘reverse’ signifies that data flows out of your data warehouse environment, diverging from the typical external-to-internal movement. Essentially, a Reverse-ETL tool simplifies creating API connections with the tools in your marketing stack. By linking your data warehouse environment and marketing tools, you’re set for data activation after some setup.

Key Differences from CDP

Orchestration Responsibility: Unlike CDP, you’re in charge of arranging data activation architecture.

Data Preparation: In your data warehouse, you ready data separately for each use case, formatting it for specific endpoints.

Custom Data Stitching: You manage customer data stitching in your data warehouse, unlike CDP’s automatic handling.

Website Integration: No website integration for personalization or A/B testing.

Advantages of Reverse-ETL

Flexibility: More adaptable and cost-effective.

Cost Efficiency: Lower expenses.

Considerations:

Technical Expertise: Requires technical know-how to set up and maintain.

Examples of Reverse-ETL Tools:

Hightouch

Census

Fivetran

Matillion

The Headless CDP Route: The DIY Build Option

Enter the comprehensive build approach within our build or buy strategy framework: the headless CDP route. Here, you’re essentially constructing all the functionality essential for your data activation needs on your own (imagine creating your own CDP without the interface or ‘head’). This entails preparing data, orchestrating it for specific uses, and integrating it with external tools – all from your data warehouse or cloud environment.

Generally, this path results in an architecture composed of various low-level tools collaborating harmoniously:

Basic Data Prep: Scheduled Queries, Cloud Functions, Cloud Composer, Dataform, or DBT for fundamental data preparation (Vertex AI if machine learning is involved).

External Integration: Cloud Functions, Cloud Composer, or Dataflow for linking with external SFTP servers, APIs, or databases.

Personalization API: Cloud Run, App Engine, or Google Kubernetes Engine to create an API for on-site or in-app personalization, coupled with frontend/app development to manifest personalization on your website/app.

Orchestration: Google Workflows, Cloud Composer, or Mage to seamlessly organize your data operations into pipelines.

Advantages of the Headless CDP Approach:

Flexibility: Offers remarkable adaptability.

Cost Efficiency: Generally more cost-effective.

Considerations of the Headless CDP Approach:

Technical Expertise: Requires specialized technical resources.

Maintenance: Demands capabilities to uphold the architecture.

Knowledge Gap: The intricate setup can result in a gap between technical and marketing teams, potentially hindering value creation from data activation.

The headless CDP option is favored for its flexibility and cost-efficiency. It enables building one use case at a time, as opposed to committing to a subscription upfront. However, it necessitates technical expertise and care for architecture maintenance. Moreover, bridging the knowledge gap between technical and marketing teams is crucial to maximize value creation. When making your decision among these three options, consider the following factors:

Who’s Driving Data Activation: If non-technical marketers are leading data activation, a CDP approach might be suitable, granting them the autonomy to create many use cases.

Number of Use Cases: If you plan to experiment with a few use cases in the coming year, a full CDP subscription might not be cost-effective.

Expected Value: Estimate the value you anticipate from data activation. If a CDP subscription costs around €100,000 annually (excluding setup costs), ensure the potential gains outweigh the investment.

Use Case Complexity: Assess if your use cases align with tool capabilities. Understand the tool’s limits for effective data activation.

Technical Resources: Determine available technical expertise. This influences choosing a headless CDP route.

Ultimately, the choice should align with your team’s expertise, your company’s needs, and the value you aim to achieve through data activation.

Next Steps

As we move forward, our focus shifts towards refining your data warehouse architecture for superior reliability, maintainability, and usability. In our upcoming article, we'll take a deeper dive into the technical intricacies, particularly exploring CI/CD within your cloud data warehouse. Get ready for an in-depth guide on how to optimize and enhance your data infrastructure. Stay tuned for valuable insights that will empower your data journey. Key Takeaways

1) Data Activation Unveiled: Data activation involves pre-processing and integrating your data with external tools, boosting their performance in areas like online marketing and website personalization.

2) Supercharge Your Tools: Picture giving your marketing and analysis tools a performance boost by supplying them with the right data. This enhanced data makes tools more effective in tasks like customer targeting and recommendation generation.

3) Key Steps for Success: Follow the essential steps we outlined for successful data activation.

4) Seamless Integration: Efficient data integration with external tools is at the core of data activation. Whether it's for social media marketing, email campaigns, or personalized website experiences, a well-orchestrated integration process is essential.

5) Pathways to Explore: When considering data integration, consider the three approaches outlined above.

With these key takeaways, you're equipped to enhance your data activation strategies, choose the right integration path, and drive value from your data.