topic Re: How to Map Dynamic Columns from an Input CSV file in Talend Studio

How to Map Dynamic Columns from an Input CSV file

jerownimow — Fri, 15 Nov 2024 22:31:58 GMT

I have exactly the same scenario as this article https://www.talend.com/resources/talend-dynamic-schema/.

The field positions on my input are varying so I have to come up of a way to have it fix every time I will receive the source file.

I followed the same procedure but I can't get the correct output. Looks like the issue is on the java code that was provided on the article.

Badly needed a help on this one.

Java Code:

String strArray[] = ((String)globalMap.get("row1.col_list")). split(";");

System.out.println("Target col list: "+((String)globalMap.get("row1.col_list")). split(";"));

String tarArray[] = ((String)globalMap.get("row1.col_list")). split(";");

Dynamic columns = row6.data;

String out="";

for (int i = 0; i < columns.getColumnCount(); i++)

{

DynamicMetadata columnMetadata = columns.getColumnMetadata(i);

String inp_col=columnMetadata.getName().toString();

String col_val=(String)row6.data.getColumnValue(inp_col);

for (int j = 0; j < columns.getColumnCount(); j++)

{

if (inp_col.equals(strArray[j].toString()))

{

tarArray[j]=col_val;

}

for (int t=0; t < columns.getColumnCount(); t++)

{

if (t ==0)

out=tarArray[0];

else

out=out+";"+tarArray[t];

}

row5.data=out;

Set-up on Database:

Output:

*Strange part is that when the col_list was split, it has a weird output:

Re: How to Map Dynamic Columns from an Input CSV file

Anonymous — Fri, 23 Sep 2022 08:29:33 GMT

I just tested the Java code with studio 8 and it works, one thing you need to pay attention to is that the input file should contain the column name in the first line if you are using Dynamic schema to read it, eg:

name;id

AL;1

Mike;2

Regards

Shong

Re: How to Map Dynamic Columns from an Input CSV file

jerownimow — Fri, 23 Sep 2022 12:20:04 GMT

Thank you, @Shicong Hong. Looks good now.

It looks like I changed the Field separator to hyphen so it would appear on the log that the separator is semi-colon which should not be.

I will try this on the actual data.

Re: How to Map Dynamic Columns from an Input CSV file

Anonymous — Sun, 25 Sep 2022 23:40:24 GMT

@Jerome Sabareza , glad to hear that you get it working now. Feel free to post questions if you have any issues.

Regards

Shong