This page provides information about the Custom Attributes SFTP integration process for the following scenario:

Data engineers want to:

• Supplement the given data easily and safely by submitting CSV based data feeds with additional pass through parameters.
• Streamline data management processes through SFTP integrations for daily flat file transfer so that they can manage data with more flexibility and less manual tasks.

This page provides information about the Custom Attributes SFTP integration process for the following scenario:

Data engineers want to:

• Supplement the given data easily and safely by submitting CSV based data feeds with additional pass through parameters.
• Streamline data management processes through SFTP integrations for daily flat file transfer so that they can manage data with more flexibility and less manual tasks.

Setting up the CSV feed file

Create a comma-separated CSV feed file that matches the Custom Attributes parameter specification. For more information, see Specifications for surfacing custom attributes.

Formatting guidelines

• File format and naming: Feed files must be in the CSV format with a comma delimiter. Also, the files must adhere to a specific naming pattern.
  For example, the name of the file can be "AcquiaCustomAttribute_xxx_dateandtimestamp.csv", where xxx is the feed table name, and the datestamp and timestamp are in yyyymmdd and hhmmss formats respectively.

  Examples
  AcquiaCustomAttribute_Customer_20240420003314.csv
  AcquiaCustomAttribute_Product_20240420003338.csv
  AcquiaCustomAttribute_ProductCategory_20240420003250.csv
  AcquiaCustomAttribute_Event_20240420003321.csv
  AcquiaCustomAttribute_Transaction_20240420003318.csv
  AcquiaCustomAttribute_TransactionItem_20240420003333.csv

• File column naming:
  • Column names must adhere to the correct naming format to prevent mismatches. For example, if you write "c_customerstringattribute01" in lowercase in your file, the connector does not fail. However, it does not accept the input column. The correct format is "c_customerStringAttribute01".

  • The connector mapping is case-sensitive. Therefore, ensure that the first letter after each prefix of a table name attribute is capitalized. For example, you must capitalize the first letter of the "StringAttribute", "IntegerAttribute", "DecimalAttribute", or "DateAttribute" attribute type after a "c_customer", "c_transaction", "c_transactionItem", "c_product", "c_productCategory", or "c_event" prefix.

    It also applies to non-custom attributes fields such as "SourceCustomerNumber", "Email", and "DeleteFlag".

• The current setup does not support inserting encrypted files. To insert encrypted files, contact the Account team and request for additional configuration.

Example

The following is an example of the content structure of the final customer feed file:

Data parsing considerations

Parse data only for the attributes that you intend to use. Keep unused custom attributes blank or exclude them from the feed file to avoid unintended insertion.

Setting the feed file drop

Place the prepared feed files in the CustomAttributes folder within your designated SFTP file drop space provided by Acquia. For example, the correct folder path must end with Acquia_SFTP/CustomAttributes.

The files are processed according to your tenant's daily workflow schedule.

Verifying the completeness of your file feed

After submitting the files as described in the earlier phases, you can confirm the successful ingestion by validating the file on the CDP UI through Integration Management. For detailed instructions, see Real-time input connectors. To locate the connector, search for SFTP Custom Attributes Standard Input Connector.

Navigating to CDP Studio - Custom Attributes

For detailed instructions, refer to the Custom Attributes page.

Removing test records

After testing, if records have custom attribute values that need removal, note that the system does not permit direct deletion of these values. To remove them, you must delete the entire record from the system.

When preparing the CSV test file, modify the DeleteFlag column as follows:

Example:

• Change the DeleteFlag column to TRUE to remove the record.

• The DeleteFlag column can be used in CSV files sent to the following tables: 

  • Customer

  • Transaction

  • Transaction Item

  • Product

  • Product Category

• The DeleteFlag is not applicable to the Event table as you cannot delete the rows in this table.

• You can delete records in CDP only if they are linked to customers by SourceCustomerNumber or UUID.

Deletion Flow

The delete function activates immediately when the base tables in the Data Warehouse receive the record. To confirm that the record is fully deleted, you must wait for the full refresh. After the full refresh process, the record is removed from the summary tables and completely erased from the data model.

• The record is marked for soft deletion in the base tables. After the refresh process completes, it is then cleared from the summary tables. The compaction workflow handles the actual removal of these soft-deleted records.
• When removing a customer from CDP, you must retain their transactional data to maintain historical sales records. You can label these transactions as Anonymous or Unidentified instead of deleting them. Deleting such transactions can impact historical sales data. The best practise is to retain transactions and label them as Unidentified when there is no linked customer.