topic Re: Dynamic Schema: Loop through all fields - tJavaRow in Talend Studio

Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Mon, 08 Jul 2013 21:07:47 GMT

Does anyone know a way to loop through all fields in a dynamic schema input, using the tJavaRow component?
I'm trying to implement a schema validation process, however the source input data utilizes a dynamic schema, so I'm unable to utilize the tSchemaComplianceCheck component. I want to create a custom routine to determine the data type of each record in each field, however I'm not sure how to implement this routine so that it can be applied to all fields in the schema?
Thanks!

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Tue, 09 Jul 2013 02:57:16 GMT

Hi mattirvin
Take a look at my last post in this https://community.talend.com/t5/Design-and-Development/resolved-Generic-Talend-Job/td-p/92354 to know how to get the schema properties and the column value.
Shong

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Tue, 09 Jul 2013 18:41:57 GMT

Thanks Shong. Most helpful.
Is there a method I can use to get the column data type and column length as defined in dynamic schema definition?
ie:
ColumnType = column.GetType();
ColumnLength = column.GetLength()
I have tried to implement this into the example you provided but when I have an integer field and a string field defined in the dynamic schema, both are returned as "id_String" when I use the GetType method.
Is there another way to get the schema metadata so that I can use it to do data validation checks?

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Wed, 10 Jul 2013 08:55:02 GMT

I have tried to implement this into the example you provided but when I have an integer field and a string field defined in the dynamic schema, both are returned as "id_String" when I use the GetType method.

I tested and it works, I have two columns in my table, one type is Integer (11) and another type is varchar(12), the job executes and returns:

Starting job testDynacal at 15:53 10/07/2013.

 connecting to socket on port 3546
 connected
The length of column is: 11column Type is: id_Integer
The length of column is: 11column Type is: id_String
The length of column is: 11column Type is: id_Integer
The length of column is: 11column Type is: id_String
 disconnected
Job testDynacal ended at 15:53 10/07/2013.

Shong

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Fri, 12 Jul 2013 17:57:01 GMT

Hi Shong, Thank you this is helpful. However I noticed that the length printed in your example is always 11 and it doesn't return 12 for the varchar field.
I have also been having this issue where I am unable to retrieve the correct length using the getLength() method.
Can you confirm if this is a bug?

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Mon, 15 Jul 2013 02:40:23 GMT

Hi mattirvin
Maybe I posted a wrong result in my previous post, however, I execute the job again today and see it is getting the right result.
tJavaRow:

int ncount=input_row.c.getColumnCount();
DynamicMetadata column0=row1.c.getColumnMetadata(ncount-2);
DynamicMetadata column1=row1.c.getColumnMetadata(ncount-1);
System.out.println("The column name is:"+column0.getName()+" The length of column is: "+column0.getLength()+" The column Type is: "+column0.getType());
System.out.println("The column name is:"+column1.getName()+" The length of column is: "+column1.getLength()+" The column Type is: "+column1.getType());

If you still have doubts, please show me your details.
Shong

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Thu, 18 Jul 2013 19:33:32 GMT

Hi Shong, I think the difference is that I am using a tSetDynamicSchema component to set the Dynamic field first.
I am also using a tJavaFlex component to extract the schema metadata. The issue I'm having is that the getLength() is not returning the correct length as is defined in the input row to the tSetDynamicSchema.
Does it work for you when using the tSetDynamicSchema component?
Please see my job details below:

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Thu, 25 Jul 2013 19:27:52 GMT

Hi Shong, were you able to replicate this issue on your side using the tSetDynamicSchema component?

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Fri, 26 Jul 2013 12:24:10 GMT

Sorry, I have missed your topic, I will investigate your new problem and come back to you asap!
Shong

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Tue, 30 Jul 2013 20:10:51 GMT

Thanks Shong. Any update so far? I have tried numerous times but still can not get the getType or getLength methods to work with tSetDynamicSchema. The getName() does seem to work though.

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Tue, 30 Jul 2013 22:11:45 GMT

I found the issue and resolved it.
The code for the variable column in the tJavaFlex component was pointing to the dynamic metadata in the tFileInputDelimited component, instead I pointed to the tSetDynamicSchema component and it works.
The tFileInputDelimited was setting all columns to id_String, 100 no matter the dynamic schema definition.

Re: Dynamic Schema: Loop through all fields - tJavaRow

lemufty — Tue, 19 Jul 2016 20:46:40 GMT

hi,
i come on discussion because i have maybe the same problem.
this my csv : id ;name
1;kpk
2; toto
i use dynamic column because my csv structure can change.
this is my workflow: tfileInputDelimited =>tjavaflex=>tmap=tmap;
i arrive to retrieve to tjavaflex all columns from dynamic column of tfileinputdelimited.
so i want to load all columns to tmap and to do transformation.
but how can i do it.
thank you for your help

Re: Dynamic Schema: Loop through all fields - tJavaRow

Anonymous — Fri, 12 Oct 2018 16:38:37 GMT

Hi,

How have you pointed that to dynamicSchema componet. Can you please help with the code.

regards,

Romi

Re: Dynamic Schema: Loop through all fields - tJavaRow

mdeeter — Thu, 03 Jan 2019 17:30:30 GMT

I have a need similar to this job. Is it possible to repost the tJaveFlex screenshot? I am not able to click on it.

Re: Dynamic Schema: Loop through all fields - tJavaRow

mdeeter — Thu, 03 Jan 2019 18:16:04 GMT

My question above was for the site admins.