topic Re: Validate the CSV file schema dynamically? in Talend Studio

Validate the CSV file schema dynamically?

amit_n — Sat, 16 Nov 2024 08:08:26 GMT

Please let me know how to validate the csv file schema dynamically. Please let me know any one have implemented this in Talend open studio.

Re: Validate the CSV file schema dynamically?

fdenis — Thu, 07 Jun 2018 10:23:38 GMT

open csv file adding one column more.
this one have to be empty.

Re: Validate the CSV file schema dynamically?

amit_n — Thu, 07 Jun 2018 12:51:39 GMT

Please can you explain briefly.

Re: Validate the CSV file schema dynamically?

fdenis — Thu, 07 Jun 2018 13:09:23 GMT

in a csv file

col1;col2;col3

a;b;c

if you open and get the first line (column name)

you an count column (=3)

on a second time if you read a line like this one: e;f;g;h

there is something wrong.

a csv file (first level) is never invalid col1=e col2=f col3=g;h

you can add coma enclosure (CSV Options)

the same line can be wrong

"e";"f";"g";"h" --> ???

"e";"f";"g;h" --> ok

think about what is an invalid csv file?

that's all

Re: Validate the CSV file schema dynamically?

amit_n — Thu, 07 Jun 2018 13:10:58 GMT

Sure . Let me try and get back to u. Thank You.

Re: Validate the CSV file schema dynamically?

amit_n — Wed, 20 Jun 2018 08:38:32 GMT

@fdenis

Please let me know if u have created any sample job for this. So i can create the job based on that.

Re: Validate the CSV file schema dynamically?

fdenis — Wed, 20 Jun 2018 09:45:14 GMT

I can not share pics.
but you are going to success.

Re: Validate the CSV file schema dynamically?

amit_n — Wed, 20 Jun 2018 10:18:16 GMT

Actually my file headers are

emp_id,name,lastname,age,designation

evrytime the column positions will be changing. we dont know how the columns will be in csv file. So how can we achive in this position. We need to validated column header dynamically.

Re: Validate the CSV file schema dynamically?

fdenis — Wed, 20 Jun 2018 11:51:29 GMT

read csv using dynamic type then using java, check all columns are present , then re-order then.

Re: Validate the CSV file schema dynamically?

amit_n — Mon, 25 Jun 2018 11:05:16 GMT

@fdenis

I am not able to make the column as dynamic type. can u share the java code to check all the columns are present.

Re: Validate the CSV file schema dynamically?

fdenis — Mon, 25 Jun 2018 13:03:32 GMT

here is a sample:
https://help.talend.com/reader/g8zdjVE7fWNUh3u4ztO6Dw/GhoMo1chGG3zr0yi8VM__g

Re: Validate the CSV file schema dynamically?

amit_n — Mon, 25 Jun 2018 15:08:08 GMT

I m using Talend open source m not able to find tWriteDynamicFields component. i dint find the tWriteDynamicFields component in SourceForge.net . Please let me know is there any other way.

Re: Validate the CSV file schema dynamically?

fdenis — Mon, 25 Jun 2018 15:16:24 GMT

Samples are only samples. They are not your solution.

this sample use dynamic columns.
you can use them for your task.

an other way is to read only the first (headers) row of your file read the full row and split the result string.

good luck