topic Re: Ignore Header information on CSV in Talend Studio

Ignore Header information on CSV

Anonymous — Sat, 16 Nov 2024 09:51:25 GMT

Hello dear

I have a csv file that contains dynamique informations on header that we dont know number of line, and then i have columns like that :

line1 Country : French;

line2 Date : 27/04/2017;

lineN (Can be other informations)

lineN+1 ID;Product;Size;

lineN+2 1;TV;32;

lineN+3 1;Phone;N/A;

I want to keep just fields concerned ID,Product and size because i use them on a Tmap

Regards

Re: Ignore Header information on CSV

vboppudi — Thu, 27 Apr 2017 14:04:38 GMT

Hi,

What is your question here?

Regards,

Re: Ignore Header information on CSV

TRF — Thu, 27 Apr 2017 14:11:37 GMT

Here is my proposal,

Initialize a boolean global variable:

Read the file with tFileInputDelimited.

Pass the rows to tJavaRow which to set "headerPassed" to "true" when needed:

Filter rows in tMap:

Here is the result:

Regards,

Re: Ignore Header information on CSV

vboppudi — Thu, 27 Apr 2017 14:38:31 GMT

Hi,

If you don't know header records count and still need to filter header records and load only valid data into target then try below. Here i assumed ID valid value is numeric.

In tMap, used filter condition based on ID is numeric. If it is numeric then pass else reject.

Mathematical.NUM(row1.ID)==1

Regards,

Re: Ignore Header information on CSV

TRF — Thu, 27 Apr 2017 14:42:41 GMT

@vboppudi, Hazardous (IMHO) as you don't know what may arrive before the real header.

The solution I proposed works at any time (any rows number, any content).

Re: Ignore Header information on CSV

Anonymous — Thu, 27 Apr 2017 16:23:11 GMT

@TRF I will test your solution, and i return,

Re: Ignore Header information on CSV

Anonymous — Thu, 27 Apr 2017 16:31:48 GMT

@vboppudi ID is alphanumeric