topic Input Select Columns in Talend Studio

Input Select Columns

Anonymous — Sat, 16 Nov 2024 08:16:27 GMT

Hello, I'm a bit new to using Talend, but I'm getting the hang of it a little.

I'm creating a job to automatically import files to my sql system, but I'm having some problems with the CSV files.

Each file has over 500 columns, but I only need roughly the first 50 columns. However, when I use the tFileDelimitedImport with a schema of the first 50, the file throws an error and spits out a table with the proper headings, but no data.

I've had success with going into the file and deleting everything to the right of where I've measured the schema, which suggests to me I need to create an import schema for the rest of the file.

However, I don't want to measure out the next 400 column names, even if it's just copy-pasting them, so I'd like to know if there's a way to only import the first X number of columns, or to just import a selected number of columns.

Thank you for your help.

Re: Input Select Columns

Jesperrekuh — Mon, 14 May 2018 18:29:40 GMT

The most easy way is extract complete linestring and split it to array and back to linestring and write to newoutfile

Recode below .. flexible mapping...

// Header ... on first
String[] headerArr = inputrow.completeLineFromFile.split(";"); 
// Main ... rows
String[] rowArr = inputrow.completeLineFromFile.split(";"); 
String[] outputString = {49}
for (int i=0; i<50; i++){
outputString[i]= rowArr[i].trim();
}
rowOutToNewFile.arrayLineCol = Arrays.toString(outputString);

Re: Input Select Columns

TRF — Mon, 14 May 2018 19:32:50 GMT

What if your schema matches the first 50 columns and you untick the option "Check each row structure against schema"?