topic Re: CSV file with random Column position in Talend Studio

CSV file with random Column position

Sanjay5 — Sat, 16 Nov 2024 09:40:23 GMT

Hi,

I want to read a CSV file from shared drive and process the columns in the file.

Name of the columns in the file are same but position on the column is getting changed.

E.g.

File1 has columns as "Username","Model","Drive",

File2 can have "Username","Drive", "Model"

I want to read the corresponding column and process the data.

Could you please let me know how can i handle this situation.

Re: CSV file with random Column position

Anonymous — Tue, 06 Jun 2017 20:48:04 GMT

I have written a tutorial on precisely this here....

https://www.rilhia.com/tutorials/dynamic-column-order

Re: CSV file with random Column position

vapukov — Wed, 07 Jun 2017 03:38:48 GMT

Yes, it great tutorial!

and I was happy to read it when it was written

the only one problem - it really work with only few columns, than start will be more and more complicated (just because a lot of information on screen and a lot of coding)

The more universal solutions:

- pack this logic in java routine which loop over any number of columns. Like parse header and sort columns in all other rows by etalon schema (see p.3)

- convert csv to json, again by routine or by any of command line tools, then parse JSON and not worry about order (I use this)

- and final - the best news :-). All what explained in p.1 - already do the great component from Jan Lolling - tFileInputTextFlat on one of project I extract with this component - more than hundreds columns for KPI from source file with 2000+ columns , work perfect!