Skip to main content
Announcements
Join us at Qlik Connect for 3 magical days of learning, networking,and inspiration! REGISTER TODAY and save!
cancel
Showing results for 
Search instead for 
Did you mean: 
DM_J
Contributor II
Contributor II

Checking the schema of the file using Talend -tSchmeaComplianceCheck or in another way

I receive an excel file periodically and I want to upload it to the database and run some automated Talend jobs later.

The problem is sometimes the format or the schema of the file may not correct and it causes the Talend jobs to fail.

I tried to create a new flow to check the file first. The most important checking I need are:

* Checking the column names

* Checking the order of columns

* Checking the type of the columns - especially for the date

So I created a flow to check the schema first. I used tSchemaComplianceCheck to compare with a predefined schema.

  • Problem with SchemaComplianceCheck

I created a test file.

There is a cell with the wrong format.

 

When I run the job, I see the message in the output but the rejected row count is 0 while it should be one. I want the flow to die if the file is incorrect ie to add a tDie and die the job

 

I also don't access the message and it is produced by tSchmeaComplianceCheck . I couldn't find any documentation or examples using this component.

 

Question 1: How can I access the message?

Question 2: Why the rejected row is 0?

Question 3: Does this component check the order and the name of the schema?

  • Solving the problem

Question 4: Is there any other way to check the schema?

I am open to any suggestion to solve the problem.

I use Talend 7.

Labels (4)
3 Replies
Anonymous
Not applicable

Hello,

We are supposing that you have already checked this online component reference about: TalendHelpCenter: tSchemaComplianceCheck

Here is a validation scenario about: TalendHelpCenter: Validating data against schema

Let us know if it helps.

Best regards

Sabrina

 

 

DM_J
Contributor II
Contributor II
Author

Thank you for the example, I found some other questions with the same problem as mine and it seems some of the issues haven't solved yet from Talend 4, like:

  • When the row only has a date format issue the count is 0 while it shall be 1
Anonymous
Not applicable

Hello,

On which talend 7 build version you got this issue? Talend V 7.4.1 milestone version?

Best regards

Sabrina