Skip to content

Transmogrify my Data

Tilo edited this page Oct 23, 2023 · 19 revisions

Sometimes you want to do a little bit more than just reading the file in as-is.

SmarterCSV 2.x has new built-in mechanisms to do transformations and validations.

Transmofrifier

Transformations / Validations

Both transformations and validations are defined as an array of Procs. e.g.

header_transformations: [ :none, :keys_as_strings ]

This way you can define the sequence in which the Procs are applied.

The special symbol :none allows you to start from [] without any pre-existing default settings, otherwise the Procs you defined will be appended after the defaults.

How a CSV File is processed

When SmarterCSV processes a CSV file, it will run a sequence of transformations and validations.

Either of these 5 transformations/validations are implemented as Ruby Procs, and besides the pre-defined Procs you can create your own customer Procs.

For the header in the CSV File:

  1. Header Transformations, option: :header_transformations
  2. Header Validations, option: :header_validations, typically raise an exception

Please note that :header_validations typically throw a validation error. They are meant to help you bail-out if you determine an incorrect CSV file was uploaded to your application.

If you specify :user_provided_headers, then the two steps above are not processed.

For each row of the CSV File:

  1. Data Transformations, option: :data_transformations, for general transformations
  2. Data Validations

For each hash that was generated for a row:

  1. Hash Transformations, option: :hash_transformations, when you want to transform specific fields
  2. Hash Validations, option: :hash_validations, log errors and warnings

The option :remove_empty_hashes is evaluated after step 3, and if set to true, it will remove any data hashes which would be empty. Default setting is true.