topic Re: Dynamic Column Names using Context Variables in Talend Studio

Dynamic Column Names using Context Variables

Anonymous — Mon, 24 Sep 2012 11:06:52 GMT

Hi,
I have a very simple and straight forward requirement, I have a file of say 100 columns(maximum) out of which I have a list of columns which I actually need for my downstreams. Per say 20 columns out of 100 and similarly I have 50 odd files to process where the column names and files are different. I know its easy for one job so as to reduce the effort of development I have thought of a Generic Job which can run for all the files which will have a standard 100 column input and output can be decided dynamically on the basis of columns provided for each of the files.
I tried using Context variables for this. For e.g.
context.DynamicHeaderFileOutput=row1.Column1+","+row1.Column2+","+row1.Column25
I have called this context name context.DynamicHeaderFileOutput in the output column expression, But it doesnot print the values as i need? Any pointers or suggestions if I am missing something?
Regards,
Sameer

Re: Dynamic Column Names using Context Variables

Anonymous — Mon, 24 Sep 2012 11:52:38 GMT

Hi,

I tried using Context variables for this. For e.g.
context.DynamicHeaderFileOutput=row1.Column1 "," row1.Column2 "," row1.Column25
Regards,
Sameer

You will get the column value, not the column name with this code. Do you really need to get the column name. This topic will show you how to get the column name.
https://community.talend.com/t5/Design-and-Development/resolved-how-to-get-the-column-names-in-a-data-flow/td-p/99172
If you understand your request well, you need to set a dynamic schema for all files which have different schema. However, 'Dynamic schema' is only available in Talend Enterprise Data Integration.

Re: Dynamic Column Names using Context Variables

Anonymous — Mon, 24 Sep 2012 12:49:16 GMT

Thanks Shong for your prompt response.
I actually need the column values rather the column names. But somehow I am unable to print the column values. It prints the value I have set in the context instead.
For e.g.
First Column is a Numeric Sequence and the other columns are dynamic.
Dynamic Columns are set on the context variables
context.DynamicHeaderFileOutput=row1.Column1+","+row1.Column2+","+row1.Column25

This is my output:
SL NO.|HEADER STATEMENT DATE|STRIKE|EXPIRATION DATE
1,row1.Column1+","+row1.Column2+","+row1.Column25
2,row1.Column1+","+row1.Column2+","+row1.Column25
3,row1.Column1+","+row1.Column2+","+row1.Column25
4,row1.Column1+","+row1.Column2+","+row1.Column25
5,row1.Column1+","+row1.Column2+","+row1.Column25
Regards,
Sameer

Re: Dynamic Column Names using Context Variables

Anonymous — Tue, 25 Sep 2012 03:21:41 GMT

Dynamic Columns are set on the context variables
context.DynamicHeaderFileOutput=row1.Column1+","+row1.Column2+","+row1.Column25
Regards,
Sameer

Hi
This is considered as value to context variable, it is not supported to set dynamic value to context variable in Context viewer. You can do it on a tJavaRow component, for example:
tFileInputDelimited--main--tJavaRow--mian--tLogRow
on tJavaRow:

Context.DynamicHeaderFileOutput=input_row.Column1+","+input_row.Column2+","+input_row.Column3+","+input_row.Column4+....;
System.out.println(Context.DynamicHeaderFileOutput);

Re: Dynamic Column Names using Context Variables

alevy — Fri, 28 Sep 2012 05:16:33 GMT

You can do this with tJavaRow with a series of statements like this:
output_row.line="";
output_row.line =context.FileOutputColumns.contains("Column1,")?(output_row.line.isEmpty()?"":",") input_row.Column1:"";
output_row.line =context.FileOutputColumns.contains("Column2,")?(output_row.line.isEmpty()?"":",") input_row.Column2:"";
...
If you want a more generic approach you would have to use reflection (see https://community.talend.com/t5/Design-and-Development/resolved-Automatically-trim-non-String-fields/td-p/86607 for an example).