This page provides information about how to add and view the input connectors for the following scenario:

Data engineers and IT professionals want to monitor all the upstream integrations with the CDP through source system APIs or SFTP. They also want to track, observe, and find the root causes of any failures in record processing.

Viewing the upstream input connectors¶

1. Sign in to the CDP user interface.

2. Click Integrations > Input Connectors.

   The system displays the Input Connectors page with the Batch and Real-time tabs.

   If a connector encounters an error, the tab header displays alert notifications and the number of failed connectors.

Batch input connectors¶

The Batch tab displays the following information about Batch input connectors:

• Connector Name: The name of the Batch input connector.
  • Data Source: The source from which the data originated. There is a predefined list of sources available that you can select during the connector setup process.
• Ingestion Status: The following are the possible statuses of the configured Batch input connector:
  • Completed: The connector was successfully run in the last attempt.
  • Failed: The connector was not successfully run in the last attempt due to an error. To resolve the error, contact your CDP customer value manager (CVM).
  • In Progress: This connector is actively ingesting data and will update to Completed or Failed.
  • Ready: The connector is ready to run or ingest data.
  • Unpublished: The setup for this connector is not complete. The connector must be published before it can ingest data.

• Frequency: The frequency with which the connector is scheduled to run and bring data into CDP. This can be set to hourly, daily, weekly, monthly, or a custom time period, such as every 6 hours. You can contact Acquia Support to run these manually.

• Last Completed: The date and time that the last successful batch was ingested from this connector. This is applicable to Batch connectors only. 
• Actions: This field contains other actions that you can perform on the connector.

Real-time input connectors¶

The Real-time tab provides the following information about Real-time input connectors:

• Connector Name: The name of the Real-time input connector.
• Data Source: The source from which the data originated. There is a predefined list of sources available that you can select during the connector setup process.
• Ingestion Status: The following are the possible statuses of the configured Real-time input connector:
  • Connected: The connector is connected to the source and ingesting data as it is generated.
  • Failed: The connection has an issue that needs to be resolved before data can be ingested.
  • Unpublished: The setup for this connector is not complete. It will need to be published before it is ready to ingest data.
• Actions: This field contains other actions that you can perform on the connector.

Viewing connector details¶¶

To view connector details for Batch input connectors:

1. Navigate to input connectors. 

2. Do one of the following:

   • Click the connector name.
   • In the Actions column, click the Kebab menu icon and click View Connector Details.

   CDP displays the Connector Details page.

   

You can view the following information:

• Ingestion Started: The date and time of the initiation of the data ingestion process.
• Status: The data ingestion status of the individual batch. It is either Completed or Failed.
• Run Type: The type of data ingestion for a batch. It is either Scheduled or Manual.
• Initial Data Timestamp: The start date that the connector uses for pulling new data files. It usually aligns with the previous batch end date.
• Final Data Timestamp: The end date that the connector uses for pulling new data files. It usually aligns with the next batch start date.
• Batch ID: A unique identifier for all successfully ingested batches.
• Actions: Other actions that you can perform on the connector by using the Kebab menu icon.

Viewing ingestion details¶

To view the amount and type of data ingested during each run only for a Batch input connector:

1. Access the Connector Details page for the Batch input connector.

2. In the Actions column, click the Kebab menu icon and click Ingestion Details.

   CDP displays the Ingestion Details dialog box with content based on the type of input connector:

   • For non-SFTP connectors, the dialog box displays the entities:

     

   • For SFTP connectors, the dialog box displays the destination entities and source files:

     
     

You can view the following information:

• Destination Entities: The destination where the data from the source file is processed, transformed, and stored in CDP. Destination entities represent different aspects of data, such as customer profiles, transactions, and events. Input connectors map and transfer data from source files to these destination entities. You can use and analyze the data for marketing campaigns, analytics, and reporting.
• Source Files: The data files from where input connectors access and retrieve data. Input connectors connect to the source file to pull data from various sources, such as databases, APIs, flat files, cloud storage, and other data repositories. This tab is only available for SFTP input connectors.
• Entity (For Destination Entities tab): The entity where the source data is ingested.
• File (For Source Files tab): The data file that is processed, transformed, or generated as a result of data integration.
• Valid Records: The data records that meet the specified criteria and data quality standards for transfer from Source Files to Destination Entities. Valid records adhere to predefined data formats, schemas, and business rules, and are considered accurate and complete.
• Invalid Records: The data records that do not meet the specified criteria and data quality standards for transfer from Source Files to Destination Entities.

Adding input connectors¶

Before adding an input connector, you must create an instance. For more information, see Creating instances. To add an input connector:

1. Navigate to input connectors.
2. Click Add Connector.
3. Select a connector and click Next.
4. In Name, specify a name for the input connector and click Next.

5. In Instance Type, select an instance from the available options.

   The system displays only the instance types that belong to the same category as the selected connector. In addition, CDP displays only the instances that are in the Ready to Use state.

6. To edit an instance, select the instance and click Edit Instance.

7. Click Save.

   The system saves the input connector in the Unpublished state.

8. To publish the connector, click Publish.

   The system saves the input connector in the Ready state. You can use the connector for further processing.

Deleting input connectors¶

You can only delete Real-time connectors when the instance type is DW Tracker and the status is Connected. To delete the connectors that use other instance types, contact Acquia Support.

To delete a Real-time input connector:

1. Click Integrations > Input Connectors.
2. Click the Real-Time tab.
3. Do one of the following:
   • Click the connector name.
   • In the Actions column, click the Kebab menu icon and click View Connector Details.
4. Click Delete Connector.