topic Re: Checking the schema of the file using Talend -tSchmeaComplianceCheck or in another way in Talend Studio

Checking the schema of the file using Talend -tSchmeaComplianceCheck or in another way

DM_J — Sat, 16 Nov 2024 00:31:05 GMT

I receive an excel file periodically and I want to upload it to the database and run some automated Talend jobs later.

The problem is sometimes the format or the schema of the file may not correct and it causes the Talend jobs to fail.

I tried to create a new flow to check the file first. The most important checking I need are:

* Checking the column names

* Checking the order of columns

* Checking the type of the columns - especially for the date

So I created a flow to check the schema first. I used tSchemaComplianceCheck to compare with a predefined schema.

Problem with SchemaComplianceCheck

I created a test file.

There is a cell with the wrong format.

When I run the job, I see the message in the output but the rejected row count is 0 while it should be one. I want the flow to die if the file is incorrect ie to add a tDie and die the job

I also don't access the message and it is produced by tSchmeaComplianceCheck . I couldn't find any documentation or examples using this component.

Question 1: How can I access the message?

Question 2: Why the rejected row is 0?

Question 3: Does this component check the order and the name of the schema?

Solving the problem

Question 4: Is there any other way to check the schema?

I am open to any suggestion to solve the problem.

I use Talend 7.

Re: Checking the schema of the file using Talend -tSchmeaComplianceCheck or in another way

Anonymous — Tue, 02 Mar 2021 09:09:57 GMT

Hello,

We are supposing that you have already checked this online component reference about: TalendHelpCenter: tSchemaComplianceCheck

Here is a validation scenario about: TalendHelpCenter: Validating data against schema

Let us know if it helps.

Best regards

Sabrina

Re: Checking the schema of the file using Talend -tSchmeaComplianceCheck or in another way

DM_J — Tue, 16 Mar 2021 18:42:52 GMT

Thank you for the example, I found some other questions with the same problem as mine and it seems some of the issues haven't solved yet from Talend 4, like:

When the row only has a date format issue the count is 0 while it shall be 1

Re: Checking the schema of the file using Talend -tSchmeaComplianceCheck or in another way

Anonymous — Wed, 17 Mar 2021 10:02:08 GMT

Hello,

On which talend 7 build version you got this issue? Talend V 7.4.1 milestone version?

Best regards

Sabrina