This page provides information about the Custom Attributes SFTP integration process for the following scenario:

Data engineers want to:

• Supplement the given data easily and safely by submitting CSV based data feeds with additional pass through parameters.
• Streamline data management processes through SFTP integrations for daily flat file transfer so that they can manage data with more flexibility and less manual tasks.

Setting up the CSV feed file

Create a comma-separated CSV feed file that matches the Custom Attributes parameter specification. For more information, see Specifications for surfacing custom attributes.

Formatting guidelines

• File format and naming: Feed files must be in the CSV format with a comma delimiter. Also, the files must adhere to a specific naming pattern.
  For example, the name of the file can be "AcquiaCustomAttribute_xxx_dateandtimestamp.csv", where xxx is the feed table name, and the datestamp and timestamp are in yyyymmdd and hhmmss formats respectively.

  Examples
  AcquiaCustomAttribute_Customer_20240420003314.csv
  AcquiaCustomAttribute_Product_20240420003338.csv
  AcquiaCustomAttribute_ProductCategory_20240420003250.csv
  AcquiaCustomAttribute_Event_20240420003321.csv
  AcquiaCustomAttribute_Transaction_20240420003318.csv
  AcquiaCustomAttribute_TransactionItem_20240420003333.csv

• File column naming:
  • Column names must adhere to the correct naming format to prevent mismatches. For example, if you write "c_customerstringattribute01" in lowercase in your file, the connector does not fail. However, it does not accept the input column. The correct format is "c_customerStringAttribute01".

  • The connector mapping is case-sensitive. Therefore, ensure that the first letter after each prefix of a table name attribute is capitalized. For example, you must capitalize the first letter of the "StringAttribute", "IntegerAttribute", "DecimalAttribute", or "DateAttribute" attribute type after a "c_customer", "c_transaction", "c_transactionItem", "c_product", "c_productCategory", or "c_event" prefix.

    It also applies to non-custom attributes fields such as "SourceCustomerNumber", "Email", and "DeleteFlag".

• The current setup does not support inserting encrypted files. To insert encrypted files, contact the Account team and request for additional configuration.

Example

The following is an example of the content structure of the final customer feed file:

Data parsing considerations

Parse data only for the attributes that you intend to use. Keep unused custom attributes blank or exclude them from the feed file to avoid unintended insertion.

Setting the feed file drop

Place the prepared feed files in the CustomAttributes folder within your designated SFTP file drop space provided by Acquia. For example, the correct folder path must end with Acquia_SFTP/CustomAttributes.

The files are processed according to your tenant's daily workflow schedule.

Verifying the completeness of your file feed

After submitting the files as described in the earlier phases, you can confirm the successful ingestion by validating the file on the CDP UI through Integration Management. For detailed instructions, see Real-time input connectors. To locate the connector, search for SFTP Custom Attributes Standard Input Connector.

Navigating to CDP Studio - Custom Attributes

For detailed instructions, refer to the Custom Attributes page.

Removing test records

After testing, if records have custom attribute values that need removal, note that the system does not permit direct deletion of these values. To remove them, you must delete the entire record from the system.

When preparing the CSV test file, modify the DeleteFlag column as follows:

Example:

• Change the DeleteFlag column to TRUE to remove the record.

• The DeleteFlag column can be used in CSV files sent to the following tables: 

  • Customer

  • Transaction

  • Transaction Item

  • Product

  • Product Category

• The DeleteFlag is not applicable to the Event table as you cannot delete the rows in this table.

• You can delete records in CDP only if they are linked to customers by SourceCustomerNumber or UUID.

Deletion Flow

The delete function activates immediately when the base tables in the Data Warehouse receive the record. To confirm that the record is fully deleted, you must wait for the full refresh. After the full refresh process, the record is removed from the summary tables and completely erased from the data model.

• The record is marked for soft deletion in the base tables. After the refresh process completes, it is then cleared from the summary tables. The compaction workflow handles the actual removal of these soft-deleted records.
• When removing a customer from CDP, you must retain their transactional data to maintain historical sales records. You can label these transactions as Anonymous or Unidentified instead of deleting them. Deleting such transactions can impact historical sales data. The best practise is to retain transactions and label them as Unidentified when there is no linked customer.

Using Custom Attributes SFTP integration

This page provides information about the Custom Attributes SFTP integration process for the following scenario:

Data engineers want to:

• Supplement the given data easily and safely by submitting CSV based data feeds with additional pass through parameters.
• Streamline data management processes through SFTP integrations for daily flat file transfer so that they can manage data with more flexibility and less manual tasks.

Phase	Task	Description
1.	Prepare the upstream file	Create a comma-separated CSV feed file that aligns with the Custom Attributes parameter specification.
2.	Set a target and schedule your feed file	Set the feed file drop.
3.	Verify the completeness of your feed file	Validate successful ingestion of Custom Attribute feed files into CDP.
4.	Navigate to CDP Studio - Custom Attributes	Configure the Custom Attribute UI naming to change the display name and make it visible in Campaigns+ and Metric. You must use clear and concise names and descriptions as best practices.
5	Remove test records	Delete test records with Custom Attribute values from the system.

Setting up the CSV feed file

Create a comma-separated CSV feed file that matches the Custom Attributes parameter specification. For more information, see Specifications for surfacing custom attributes.

Formatting guidelines

• File format and naming: Feed files must be in the CSV format with a comma delimiter. Also, the files must adhere to a specific naming pattern.
  For example, the name of the file can be "AcquiaCustomAttribute_xxx_dateandtimestamp.csv", where xxx is the feed table name, and the datestamp and timestamp are in yyyymmdd and hhmmss formats respectively.

  Examples
  AcquiaCustomAttribute_Customer_20240420003314.csv
  AcquiaCustomAttribute_Product_20240420003338.csv
  AcquiaCustomAttribute_ProductCategory_20240420003250.csv
  AcquiaCustomAttribute_Event_20240420003321.csv
  AcquiaCustomAttribute_Transaction_20240420003318.csv
  AcquiaCustomAttribute_TransactionItem_20240420003333.csv

• File column naming:
  • Column names must adhere to the correct naming format to prevent mismatches. For example, if you write "c_customerstringattribute01" in lowercase in your file, the connector does not fail. However, it does not accept the input column. The correct format is "c_customerStringAttribute01".

  • The connector mapping is case-sensitive. Therefore, ensure that the first letter after each prefix of a table name attribute is capitalized. For example, you must capitalize the first letter of the "StringAttribute", "IntegerAttribute", "DecimalAttribute", or "DateAttribute" attribute type after a "c_customer", "c_transaction", "c_transactionItem", "c_product", "c_productCategory", or "c_event" prefix.

    It also applies to non-custom attributes fields such as "SourceCustomerNumber", "Email", and "DeleteFlag".

• The current setup does not support inserting encrypted files. To insert encrypted files, contact the Account team and request for additional configuration.

Important

• Any number of custom attributes can be added to your file as required, such as c_customerStringAttribute02, c_customerStringAttribute03. You must use new attributes and avoid reusing the ones that are previously mapped to other fields.
• The order of columns in a file is irrelevant, provided the column names are in the correct format.
• The column values in the feed files must be formatted according to the Data Mapping standards of CDP.  The standard date formats supported by CDP must be used for date columns. For more information, see Date/datetime formats section in Data Mapping.

Example

The following is an example of the content structure of the final customer feed file:

"SourceCustomerNumber"	"c_customerStringAttribute02"	"c_customerDateAttribute03"	"c_customerIntegerAttribute01"
"C04012346178"	"Custom Attribute Value 1"	"02/05/2024"	"123456"
"2023112806010100143796CK"	"Custom Attribute Value 1"	"05/07/2023"	"456789"
"C04018971955"	"Custom Attribute Value 2"		"987643"

Data parsing considerations

Parse data only for the attributes that you intend to use. Keep unused custom attributes blank or exclude them from the feed file to avoid unintended insertion.

Setting the feed file drop

Place the prepared feed files in the CustomAttributes folder within your designated SFTP file drop space provided by Acquia. For example, the correct folder path must end with Acquia_SFTP/CustomAttributes.

The files are processed according to your tenant's daily workflow schedule.

Note

Each file must be less than 2 GB in size.

Verifying the completeness of your file feed

After submitting the files as described in the earlier phases, you can confirm the successful ingestion by validating the file on the CDP UI through Integration Management. For detailed instructions, see Real-time input connectors. To locate the connector, search for SFTP Custom Attributes Standard Input Connector.

Navigating to CDP Studio - Custom Attributes

For detailed instructions, refer to the Custom Attributes page.

Note

Display names can be customized in CDP Studio - Custom Attributes and you can rename the filters for Campaigns+ and Metrics. However, the hardcoded attribute names remain unchanged in the data model. Therefore, the system displays the hardcoded attribute names as c_customerStringAttribute01 across all our data export mechanisms: Snowflake share, cloud exports, and Interactive Queries Unload. You can also notice this on the Business Intelligence tools Interactive Queries as the attribute names directly reference the data model tables and columns.

Removing test records

After testing, if records have custom attribute values that need removal, note that the system does not permit direct deletion of these values. To remove them, you must delete the entire record from the system.

When preparing the CSV test file, modify the DeleteFlag column as follows:

Example:

SourceCustomerNumber	DeleteFlag	UUID	Email	c_customerStringAttribute01	c_customerStringAttribute02
1234567	TRUE	abcd

• Change the DeleteFlag column to TRUE to remove the record.

• The DeleteFlag column can be used in CSV files sent to the following tables: 

  • Customer

  • Transaction

  • Transaction Item

  • Product

  • Product Category

• The DeleteFlag is not applicable to the Event table as you cannot delete the rows in this table.

• You can delete records in CDP only if they are linked to customers by SourceCustomerNumber or UUID.

Deletion Flow

The delete function activates immediately when the base tables in the Data Warehouse receive the record. To confirm that the record is fully deleted, you must wait for the full refresh. After the full refresh process, the record is removed from the summary tables and completely erased from the data model.

• The record is marked for soft deletion in the base tables. After the refresh process completes, it is then cleared from the summary tables. The compaction workflow handles the actual removal of these soft-deleted records.
• When removing a customer from CDP, you must retain their transactional data to maintain historical sales records. You can label these transactions as Anonymous or Unidentified instead of deleting them. Deleting such transactions can impact historical sales data. The best practise is to retain transactions and label them as Unidentified when there is no linked customer.