This page provides information about how to add and view the input connectors for the following scenario:
Data engineers and IT professionals want to monitor all the upstream integrations with the CDP through source system APIs or SFTP. They also want to track, observe, and find the root causes of any failures in record processing.
The system displays the Input Connectors page with the Batch and Real-time tabs.
If a connector encounters an error, the tab header displays alert notifications and the number of failed connectors.
Batch input connectors
The Batch tab displays the following information about Batch input connectors:
Connector Name: The name of the Batch input connector.
Data Source: The source from which the data originated. There is a predefined list of sources available that you can select during the connector setup process.
Ingestion Status: The following are the possible statuses of the configured Batch input connector:
Completed: The connector was successfully run in the last attempt.
Failed: The connector was not successfully run in the last attempt due to an error. To resolve the error, contact your CDP customer value manager (CVM).
In Progress: This connector is actively ingesting data and will update to Completed or Failed.
Ready: The connector is ready to run or ingest data.
Unpublished: The setup for this connector is not complete. The connector must be published before it can ingest data.
Frequency: The frequency with which the connector is scheduled to run and bring data into CDP. This can be set to hourly, daily, weekly, monthly, or a custom time period, such as every 6 hours. You can contact Acquia Support to run these manually.
Last Completed: The date and time that the last successful batch was ingested from this connector. This is applicable to Batch connectors only.
Actions: This field contains other actions that you can perform on the connector.
Real-time input connectors
The Real-time tab provides the following information about Real-time input connectors:
Connector Name: The name of the Real-time input connector.
Data Source: The source from which the data originated. There is a predefined list of sources available that you can select during the connector setup process.
Ingestion Status: The following are the possible statuses of the configured Real-time input connector:
Connected: The connector is connected to the source and ingesting data as it is generated.
Failed: The connection has an issue that needs to be resolved before data can be ingested.
Unpublished: The setup for this connector is not complete. It will need to be published before it is ready to ingest data.
Actions: This field contains other actions that you can perform on the connector.
In the Actions column, click the Kebab menu icon and click Ingestion Details.
CDP displays the Ingestion Details dialog box with content based on the type of input connector:
For non-SFTP connectors, the dialog box displays the entities:
For SFTP connectors, the dialog box displays the destination entities and source files:
You can view the following information:
Destination Entities: The destination where the data from the source file is processed, transformed, and stored in CDP. Destination entities represent different aspects of data, such as customer profiles, transactions, and events. Input connectors map and transfer data from source files to these destination entities. You can use and analyze the data for marketing campaigns, analytics, and reporting.
Source Files: The data files from where input connectors access and retrieve data. Input connectors connect to the source file to pull data from various sources, such as databases, APIs, flat files, cloud storage, and other data repositories. This tab is only available for SFTP input connectors.
Entity (For Destination Entities tab): The entity where the source data is ingested.
File (For Source Files tab): The data file that is processed, transformed, or generated as a result of data integration.
Valid Records: The data records that meet the specified criteria and data quality standards for transfer from Source Files to Destination Entities. Valid records adhere to predefined data formats, schemas, and business rules, and are considered accurate and complete.
Invalid Records: The data records that do not meet the specified criteria and data quality standards for transfer from Source Files to Destination Entities.
Adding input connectors
Before adding an input connector, you must create an instance. For more information, see Creating instances. To add an input connector:
In Name, specify a name for the input connector and click Next.
In Instance Type, select an instance from the available options.
The system displays only the instance types that belong to the same category as the selected connector. In addition, CDP displays only the instances that are in the Ready to Use state.
To edit an instance, select the instance and click Edit Instance.
Click Save.
The system saves the input connector in the Unpublished state.
To publish the connector, click Publish.
The system saves the input connector in the Ready state. You can use the connector for further processing.
Deleting input connectors
You can only delete Real-time connectors when the instance type is DWTracker and the status is Connected. To delete the connectors that use other instance types, contact Acquia Support.
To delete a Real-time input connector:
Click Integrations > Input Connectors.
Click the Real-Time tab.
Do one of the following:
Click the connector name.
In the Actions column, click the Kebab menu icon and click View Connector Details.
Click Delete Connector.
Adding and viewing input connectors
This page provides information about how to add and view the input connectors for the following scenario:
Data engineers and IT professionals want to monitor all the upstream integrations with the CDP through source system APIs or SFTP. They also want to track, observe, and find the root causes of any failures in record processing.
The system displays the Input Connectors page with the Batch and Real-time tabs.
If a connector encounters an error, the tab header displays alert notifications and the number of failed connectors.
Batch input connectors
The Batch tab displays the following information about Batch input connectors:
Connector Name: The name of the Batch input connector.
Data Source: The source from which the data originated. There is a predefined list of sources available that you can select during the connector setup process.
Ingestion Status: The following are the possible statuses of the configured Batch input connector:
Completed: The connector was successfully run in the last attempt.
Failed: The connector was not successfully run in the last attempt due to an error. To resolve the error, contact your CDP customer value manager (CVM).
In Progress: This connector is actively ingesting data and will update to Completed or Failed.
Ready: The connector is ready to run or ingest data.
Unpublished: The setup for this connector is not complete. The connector must be published before it can ingest data.
Frequency: The frequency with which the connector is scheduled to run and bring data into CDP. This can be set to hourly, daily, weekly, monthly, or a custom time period, such as every 6 hours. You can contact Acquia Support to run these manually.
Last Completed: The date and time that the last successful batch was ingested from this connector. This is applicable to Batch connectors only.
Actions: This field contains other actions that you can perform on the connector.
Real-time input connectors
The Real-time tab provides the following information about Real-time input connectors:
Connector Name: The name of the Real-time input connector.
Data Source: The source from which the data originated. There is a predefined list of sources available that you can select during the connector setup process.
Ingestion Status: The following are the possible statuses of the configured Real-time input connector:
Connected: The connector is connected to the source and ingesting data as it is generated.
Failed: The connection has an issue that needs to be resolved before data can be ingested.
Unpublished: The setup for this connector is not complete. It will need to be published before it is ready to ingest data.
Actions: This field contains other actions that you can perform on the connector.
In the Actions column, click the Kebab menu icon and click Ingestion Details.
CDP displays the Ingestion Details dialog box with content based on the type of input connector:
For non-SFTP connectors, the dialog box displays the entities:
For SFTP connectors, the dialog box displays the destination entities and source files:
You can view the following information:
Destination Entities: The destination where the data from the source file is processed, transformed, and stored in CDP. Destination entities represent different aspects of data, such as customer profiles, transactions, and events. Input connectors map and transfer data from source files to these destination entities. You can use and analyze the data for marketing campaigns, analytics, and reporting.
Source Files: The data files from where input connectors access and retrieve data. Input connectors connect to the source file to pull data from various sources, such as databases, APIs, flat files, cloud storage, and other data repositories. This tab is only available for SFTP input connectors.
Entity (For Destination Entities tab): The entity where the source data is ingested.
File (For Source Files tab): The data file that is processed, transformed, or generated as a result of data integration.
Valid Records: The data records that meet the specified criteria and data quality standards for transfer from Source Files to Destination Entities. Valid records adhere to predefined data formats, schemas, and business rules, and are considered accurate and complete.
Invalid Records: The data records that do not meet the specified criteria and data quality standards for transfer from Source Files to Destination Entities.
Adding input connectors
Before adding an input connector, you must create an instance. For more information, see Creating instances. To add an input connector:
In Name, specify a name for the input connector and click Next.
In Instance Type, select an instance from the available options.
The system displays only the instance types that belong to the same category as the selected connector. In addition, CDP displays only the instances that are in the Ready to Use state.
To edit an instance, select the instance and click Edit Instance.
Click Save.
The system saves the input connector in the Unpublished state.
To publish the connector, click Publish.
The system saves the input connector in the Ready state. You can use the connector for further processing.
Deleting input connectors
You can only delete Real-time connectors when the instance type is DWTracker and the status is Connected. To delete the connectors that use other instance types, contact Acquia Support.
To delete a Real-time input connector:
Click Integrations > Input Connectors.
Click the Real-Time tab.
Do one of the following:
Click the connector name.
In the Actions column, click the Kebab menu icon and click View Connector Details.
Click Delete Connector.
Did not find what you were looking for?
If this content did not answer your questions, try searching or contacting our support team for further assistance.
Did not find what you were looking for?
If this content did not answer your questions, try searching or contacting our support team for further assistance.
Acquia CDP