An integration user account provides access to Batch Data Capture, which supports batch processing and enhances the data integration capabilities with REST API. This functionality is optimal to manage large datasets that include customer information and transaction histories. For detailed instructions on batch processing, limitations, and best practices, see API Policy.

Capabilities

The following are the capabilities of an integration user account:

• Create Integration User Account: Create an integration user account through the Integrations module so that you can send data to REST API. With this account, you can manage authentication mechanisms and get access to the DW tracker.

• Bulk Processing: Use this method for one-time, significant upload of historical data, either in a single batch or distributed loads based on your data volume. Larger data volumes may require more processing time. To minimize the impact on system performance, these processes are executed during designated time windows scheduled for off-peak hours.

• Incremental Processing: Use this method for dynamic data environments. With this method, you can update your datasets at regular intervals. This method efficiently incorporates new records and modifications. You can send data in incremental batches that are refreshed every 24 hrs.

• Event-based Near-real time Processing: Use this method to elevate data processing with real-time updates. This method is used for capturing event-driven data changes, such as interactions through Email Service Providers (ESPs), SMS messages, in-app activities, and web activities. This method ensures immediate data relevancy and responsiveness.

• Data Validation: Ensure that new data points are successfully added to CDP from upstream source systems. Use the Interactive Queries application to validate that the dataset is surfacing correctly in CDP.

An integration user account provides access to Batch Data Capture, which supports batch processing and enhances the data integration capabilities with REST API. This functionality is optimal to manage large datasets that include customer information and transaction histories. For detailed instructions on batch processing, limitations, and best practices, see API Policy.

Capabilities

The following are the capabilities of an integration user account:

• Create Integration User Account: Create an integration user account through the Integrations module so that you can send data to REST API. With this account, you can manage authentication mechanisms and get access to the DW tracker.

• Bulk Processing: Use this method for one-time, significant upload of historical data, either in a single batch or distributed loads based on your data volume. Larger data volumes may require more processing time. To minimize the impact on system performance, these processes are executed during designated time windows scheduled for off-peak hours.

• Incremental Processing: Use this method for dynamic data environments. With this method, you can update your datasets at regular intervals. This method efficiently incorporates new records and modifications. You can send data in incremental batches that are refreshed every 24 hrs.

• Event-based Near-real time Processing: Use this method to elevate data processing with real-time updates. This method is used for capturing event-driven data changes, such as interactions through Email Service Providers (ESPs), SMS messages, in-app activities, and web activities. This method ensures immediate data relevancy and responsiveness.

• Data Validation: Ensure that new data points are successfully added to CDP from upstream source systems. Use the Interactive Queries application to validate that the dataset is surfacing correctly in CDP.

Use cases

The following are the use cases of an integration user account for IT leads or data engineers:

• Create integration user accounts tailored to strategic requirements. This preparation is essential to enable CDP to efficiently receive datasets, regardless of the chosen processing methods.

• Select Bulk Processing, especially during the onboarding of CDP with new data sources or while addressing misaligned data that necessitates a bulk data adjustment.

• Select Incremental Processing to collect data on regular basis. Incremental Processing incorporates incremental sets of records every 24 hours, aligning with the orchestration schedule of the platform. It ensures that the daily management system remains up-to-date, eliminating the need for full-scale data refreshes.

• Select Event-based Incremental Processing to capture data near real-time. Event-based Incremental Processing processes data within minutes of a user interaction. As users engage with the services, the system instantly captures the event and updates preferences based on the latest interactions. This functionality enables marketing firms to customize content with accuracy and significantly enhance user engagement.

User flow

The following diagram depicts the user flow:

Important considerations

REST API schemas are standardized and support only simple JSON structures. REST API schemas do not support complex structures. For instance, in a customer entity with multiple nested consumer objects, you cannot nest other entity objects within a main object. In this instance, a customer object cannot contain nested events, transactions, products, or other related entities.