File Formats

Customer Data Platform (CDP) exports flat-files in compressed CSV format to an Amazon S3 location for each entity selected in the Data Export job.

Format of Compressed Files

Number of files exported

  1. As covered earlier, each Data Export can include attributes across multiple entities.
  2. CDP will export at least one file per entity selected when setting up Data Export. The list of entities available for Data Export can be reviewed here: List of Entities for Data Export
  3. More often that not, there will be multiple files per entity exported depending on the amount of data. When the data being exported is reasonably sized, the platform will break each entity into multiple piece files.

File-naming convention

  1. The convention followed by files exported by CDP is as follows: [FileNamePrefix]_[EntityName]_[TimeStamp]_piece_[X].tar.gz where [FileNamePrefix] is a configurable keyword you can choose, [EntityName] is the name of the entity being exported, [TimeStamp] is the time-stamp for when the export job was executed and [X] is the piece # for the exported file per entity.
  2. An example File Name could be as follows: AgilOneDemoDataExport_customersummary_20171025005013_piece_0.tar.gz

Compression format

  1. CDP uses GZIP compression after archiving CSV files as .tar files.
  2. The resulting file-extension for all exported files is .tar.gz.
  3. Compressing large data files saves Amazon S3 storage and transfer costs for our clients.

Example files dropped on Amazon S3


Format of Uncompressed Files

File-naming convention

  1. Extracting each .tar.gz file unarchives one CSV file.
  2. The convention followed by CSV files after they are extracted is as follows: [FileNamePrefix]_[EntityName].csv where [FileNamePrefix] is a configurable keyword you can choose and [EntityName] is the name of the entity being exported.
  3. An example File Name could be as follows: AgilOneDemoDataExport_customersummary.csv

File format

  • File extension: .csv
  • Column Delimiter: [comma] character (,)
  • Row Delimiter: {LF}
  • Encoding: UTF-8
  • Text qualifiers: [double quotes] character (“)
  • Headers in the first row

Example exported CSV file