topic Re: Compare two schemas and data formats in Talend Studio

Compare two schemas and data formats

Anonymous — Sat, 16 Nov 2024 13:54:48 GMT

Hey!
I need to compare schemas of various .csv files (to make sure they have the same no. of columns and rows) and also the data formats (e.g. integer, string or date) in each cell. I want to do this by comparing each schema with a refererence file. Can anyone suggest how to do this?
Thanks!

Re: Compare two schemas and data formats

Anonymous — Thu, 18 Jun 2009 07:16:45 GMT

Hello
Does tFileCompare comonent fit your need?
Best regards
shong

Re: Compare two schemas and data formats

Anonymous — Thu, 18 Jun 2009 07:52:03 GMT

No, tFileCompare compares two files with identical data. In my case, the data is different in the files though the schema has to be the same. I mean to say that each file has to have lets say 9 columns, the one with 10 columns has to be skipped and the data type in any cell under the same column should be same.
If I have 10 files in a folder, I want to skip those with errors and continue with the rest so that my job does not crash.

Re: Compare two schemas and data formats

Anonymous — Mon, 22 Jun 2009 15:28:48 GMT

Hi,
If you define the metadata of CSV file in a CSV file or some where else(outside) then it is possible .
As CSV file does not provide all the metadata, it is impossible to compire the datatype, size of one CSV file to another.
In case you try to get the metadata from data then you may not get correct metadata always from the data available in the CSV file.
Thanks and Regards,
Pravu Mishra.

Re: Compare two schemas and data formats

Anonymous — Tue, 30 Jun 2009 08:19:16 GMT

Hi!
I want to skip the files having 9 columns or 11 columns and pass the files with 10 columns for further processing.. I am trying to do it by tSchemaComplianceCheck.
I created the test schema in repository and kept the check file schema as built-in. Also , I have kept the base schema fields nullable but test schema fields not nullable. But still it doesnt not filter out the corrupt files.
So can you tell me how to do it?

Re: Compare two schemas and data formats

_AnonymousUser — Sun, 13 Sep 2009 19:07:50 GMT

You Have Database tool just for that, its called Columbo. ( http://www.nobhillsoft.com/Columbo.aspx)

Re: Compare two schemas and data formats

_AnonymousUser — Tue, 16 Mar 2010 16:50:04 GMT

Did you ever sort this? I would have assumed Talend could do this, but if you want to compare different data formats in text files or otherwise there are tools out there, e.g.: http://www.citrustechnology.com/solutions/data-comparison/data-transform

Re: Compare two schemas and data formats

Anonymous — Thu, 11 Aug 2016 11:28:17 GMT

Hi guys, is there any tool which is free to do this?