5. Import a File¶
To load a file (CSV, FBV, JSON, or Parquet), follow these steps:
Add a BDomain
Add the CFI BDomain settings
Add a loader
Add the BDomain-specific metadata of the loader
Add a loader column per source column
Create the TSA table
Deliver the file
The examples use the BDomain ‘TESTING’, the loader ‘PERSON’, and the sample CSV file.
5.1. Add a BDomain¶
Make sure the BDomain exists or add it first: Add a BDomain.
Check that the BDomain ‘TESTING’ exists in REDADM.RED_BDOMAINS.
5.2. Add the CFI BDomain Settings¶
Add the CFI-specific BDomain settings: Add CFI BDomain Settings.
Check the settings in CFIADM.CFI_BDOMAIN_SETTINGS.
5.3. Add a Loader¶
Add the loader: Add a CFI Loader.
Check the configuration of the loader ‘PERSON’ in CFIADM.CFI_LOADER.
5.4. Add the BDomain-Specific Metadata of the Loader¶
Each BDomain may have different loader settings, e.g. different file encodings: Add BDomain-Specific Metadata to a Loader.
The result is an entry in CFIADM.CFI_LOADER_BD for the loader ‘PERSON’ and the BDomain ‘TESTING’.
5.5. Add a Loader Column per Source Column¶
Add the column definitions: Add a CFI Loader Column.
Check the column definitions in CFIADM.CFI_LOADER_BD_COLUMNS.
5.6. Create the TSA Table¶
Once all metadata is defined, create the TSA table with API_CFIADM.CREATE_TSA_TABLES.
Example: Create the TSA Table of Loader ‘PERSON’
1EXECUTE SCRIPT API_CFIADM.CREATE_TSA_TABLES(
2 'PERSON' -- p_ldr_name
3 ,'TESTING' -- p_bdomain
4);
This creates the table from the metadata and adds all technical columns (prefixed with RED_).
The table is named <BDOMAIN>_<LDR_NAME> in the schema CFITSA_<SUFFIX>, i.e. CFITSA_TEST.TESTING_PERSON. The names of all TSA tables are listed in CFIADM.CFI_LOADER_DETAILS.
Tip
You can first define the metadata of all loaders and then create all missing TSA tables at once by calling API_CFIADM.CREATE_TSA_TABLES with NULL for both parameters.
5.7. Deliver the File¶
Copy the file into the input directory of the environment instance on the RED server, $REDENV_HOME/data/input (or the upload directory of an sFTP user). The file is picked up within the next minute.
Important
To prevent incomplete files from being loaded, upload the file with the additional extension .tra (e.g. TEST_person_20240101_20240102061500.csv.tra) and rename it to its final name only when the upload is complete. Files ending with .tra are never processed. The file must be readable by the user redsys.
If the file is loaded without errors, it is moved to the directory data/archive; otherwise it is moved to data/error (see CFI process). With a reject limit above 0, rejected rows are written to the error table CFITSA_TEST.TESTING_PERSON_ERR.
The loaded data can be checked in the TSA table and in the import log CFIADM.CFI_IMPLOG_DETAILS. To load the data into the PSA layer, continue with Load CFI Data into the PSA.