topic Re: How to do Meta data validation with some business condition. in Talend Data Catalog

How to do Meta data validation with some business condition.

SMahadevan1608028474 — Sat, 16 Nov 2024 00:57:04 GMT

Hi All,

I have installed Talend Data Integration Version 7.3.1 version. My source file is (.csv) file and I have around 300 columns. Each and every column will accept various length of data.

In Talend tool I can read the file for meta data using File Delimited and able to see all the 300 columns.

But here my clarifications is how can I validate those columns for Meta data length check. I have given the my requirements below, as a Example.

Requirement 1

In source file Column 1<Employee ID> - The Max length of the characters should accept is 15.

Let's assume my source file contain the length of Employee id 18 characters for records 2 , So I need to parameterization/ set some condition for this field saying that is should not accept more than 15 characters length.

Expecting Result: From the source file I am expecting the 2nd record should validate and highlight.

Requirement 2

While reading the Meta data from source file some of the column the Default condition and some business rule condition I have to check

(E.g.: If Business Region is 'Europe', it should list only the country belongs to Europe region).

Clarifications:

At the moment we are not using any Databases for this Meta data length check and business condition (Requirement 1 and Requirement 2).

I have the (.csv) file with data and I know what is the maximum character length should accept for all the columns. So here am not using any Target system/ Target layer to compare against my source file.

I want to parameterization Or Set my length condition for those columns without using any Database connections.

Is there any option shall I use this without DB connections?

The reason for asking this clarifications is after reading the file using File Delimited in Talend and created the job design(tLogRow_1) and mapped with tFileInputDelimitied_1, Then I have started to Run the job. Getting the below error.

Starting job movies at 09:07 16/12/2020.

[statistics] connecting to socket on port 3617

[statistics] connected

[statistics] disconnected

Job movies ended at 09:07 16/12/2020. [Exit code = 0]

I hope the clarifications is clear , Please let me know if you need any more information.

Re: How to do Meta data validation with some business condition.

Anonymous — Wed, 16 Dec 2020 05:06:16 GMT

Hello,

Could you please take a look at this online documentation about: TalendHelpCenter: Centralizing a Validation Rule to see if it is what you are looking for?

Best regards

Sabrina

Re: How to do Meta data validation with some business condition.

SMahadevan1608028474 — Wed, 16 Dec 2020 08:56:21 GMT

Thanks for your reply on this but I have few clarifications,

Add Validation rule submenu is not yet all displayed for me. I have installed "Talend Open Studio for Data Integration Version 7.3.1".
At the moment we are not using any Databases for this business case condition check and Meta data length checking as (Requirement 1 and Requirement 2).

I want to parameterization Or Set my length condition for those columns without using any Database connections.

Is there any option shall I can use this without DB connections?

I hope the clarifications is clear , please let me know If you need any information.