topic Re: CSV skip selected rows in Talend Studio

CSV skip selected rows

Anonymous — Sat, 16 Nov 2024 04:04:09 GMT

Hi,

i need to skip some rows or use header and remove first row after header.

CSV file format:

first row ignore/remove

second Row is header

third row ignore/remove

4,5.....* rows are data

If I select delimited file and Header = 2, then header is ok, but first data row is "3", that I dont want.

Re: CSV skip selected rows

TRF — Wed, 20 Nov 2019 12:54:42 GMT

tSampleRow is what you are looking for.

Re: CSV skip selected rows

Anonymous — Thu, 21 Nov 2019 06:02:02 GMT

Hi,

I'm guessing, the row you want to skip doesn't have data in the format of the schema provided to the tFileInputDelimited component. If so, you can use the advanced settings of the component. Click on the box "Check each row structure against schema". This is eliminate rows which do not fall under the schema.

If there is another requirement than this, could you please elaborate on that?

Hope this helps.

Regards.

Re: CSV skip selected rows

Anonymous — Thu, 21 Nov 2019 13:13:14 GMT

Hi,

I using tFileInputDelimited, but schema: I must use string for all columns, because in data are this lines and talend can't convert them:

I'm using tfileinputdelimited -> tmap -> DBOutput.

What I want?

-skip first line after header (its third)

-replace "NA" to null (but dont skip this rows)

-remove quotes from date & time and concat to datetime

-use normal data types (double, Date, short)

-unique key is datetime, so keep synchronized csv with db in some periods

Is some easy way, how to do this?

Thank you

Re: CSV skip selected rows

Anonymous — Fri, 22 Nov 2019 02:11:38 GMT

Hi,

Below are my thoughts.

-skip first line after header (its third)

@TRF has already answered your query. You can also use header skip in tfileinputdelimited if you do not want to pick any in between records (like picking first, third , fifth etc.).

-replace "NA" to null (but dont skip this rows)

Read the data and use if expression in tMap to replace with NULL.

-remove quotes from date & time and concat to datetime

if you are reading as String, you can use replaceAll() function in tMap. Lot of examples are already available in community.

-use normal data types (double, Date, short)

Sure 🙂 Once you do the cleansing, either use java data type conversion functions or use tConvertType component.

-unique key is datetime, so keep synchronized csv with db in some periods

Use tUniqRow component to find unique records or tAggregateRow component to group them

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved