[resolved] Cannot check Date pattern using tSchemaComplianceCheck component

[Anonymous]

Hi,
I used tSchemaComplianceCheck component to generate a list of rejected data that violates the schema.
In the tFileInputDelimited, the schema is defined as follows:
ID NUMBER
CREATE_DT STRING
In the tSchemaComplianceCheck, I selected "Custom Defined" option and defined the checked columns
ID; Number; checked Max Length
CREATE_DT; Date; Date pattern: yyyy-MM-dd HH:mm:ss
The data coming in as
ID: 1
CREATE_DT: 01-10-2010 16:12:12
However, this record ID:1 is not rejected from the tSchemaComplianceCheck which cause a Unparseable date error.
How can I make sure the data that is DATE datatype does meet the date pattern requirement?
Thanks in advance!

[Anonymous · Accepted solution]

Hi klmc,
I'm not sure this will help, but I tried the same as you and it worked ok for me. I have the date coming in as a string and used the same custom parameters in the schema check. I am running TIS 4.0.2. This seems to have been a known error in previous versions, so it might be your version: https://community.talend.com/t5/Design-and-Development/date-parse-error-and-tSchemaComplianceCheck/t...
You can reject the invalid dates using tFileInput by changing the type to "Date" and unchecking Die on error. You'll get an unparseable date error on the console, which is not great, but the row will be sent to the reject flow and you can capture the information in a file if you need it.
Regards,
Rick

[Anonymous]

Hi tchd,
Thanks for your help!! Based on your post, I upgraded to TOS 4.1.1; however, tSchemaCompliance component still cannot check the date pattern correctly. Do you know if the issue only happen to TOS but not TIS?
Thanks again!!!

[Anonymous]

Hi klmc,
I've tried the same thing using TOS 4.1.0M4 and it worked the same as TIS, so it doesn't seem to be the version. Here is the data I am using:
ID Date
1 01-10-2010 16:12:12
2 2010-09-17 15:13:21
And the schema has been defined as int and string for ID and for Date respectively. It is tab delimited. I've included a screen shot of my component if that helps.
Also, below is the output from my job:
Starting job ValidateSchema at 22:48 27/10/2010.
connecting to socket on port 4015
connected
1|01-10-2010 16:12:12|2|Date:wrong DATE pattern or wrong DATE data
2|2010-09-17 15:13:21
disconnected
Job ValidateSchema ended at 22:48 27/10/2010.

[Anonymous]

Hi tchd,
I followed your step by step guidelines based on your job and it works great without any issues using TOS 4.1.1!! Really thanks for your help!
Thanks and thanks!