topic Re: How to change column names every refresh in Talend Studio

How to change column names every refresh

Anonymous — Wed, 20 Feb 2019 11:33:30 GMT

Hi. i need to refresh my workflow every month and there will be minor changes in the column headers every time. How do i make this change affect my workflow with minimum effort? i am using the Talend Open studio 6.5. Any suggestions are welcomed.

Re: How to change column names every refresh

Anonymous — Wed, 20 Feb 2019 11:49:59 GMT

Can you give an example of what you mean please? For example, changes in column headings could mean just the header names or could mean that they change order. How will the job know this has happened? What information is there to identify what has changed?

Re: How to change column names every refresh

Anonymous — Mon, 25 Feb 2019 07:40:50 GMT

The order remains same. There will be these small changes, But I want these changes reflected.

Like when I unpivot the table I want to see the changes as my values.

Re: How to change column names every refresh

Anonymous — Mon, 25 Feb 2019 10:58:42 GMT

So you will always have 12 columns? The column headers are simply dates? Do you actually need headers? What exactly are you trying to do with this data? It might be that you do not need to worry about column headers.

Re: How to change column names every refresh

Anonymous — Mon, 25 Feb 2019 11:08:02 GMT

Yes, the structure of the data remains same(i.e., number of columns). Only the column names change. I will need the column names because I will unpivot the data and I want it to look like this. This happens every month. Like there will be 12 refreshes. So is there any way to change only the column names

Re: How to change column names every refresh

Anonymous — Mon, 25 Feb 2019 11:10:44 GMT

I see. So how does your data arrive to Talend? The reason I ask is that there is no need to hard code your column names in Talend. You can omit the column names and simply ensure the first row of data for your output file holds your column names. Does that sound feasible?

Re: How to change column names every refresh

Anonymous — Mon, 25 Feb 2019 11:36:17 GMT

What about the column names stored in the schema? they did not change when i made changes to the file

Re: How to change column names every refresh

Anonymous — Mon, 25 Feb 2019 12:59:57 GMT

The schema column names do not matter, or maybe I do not understand. Why do you think your schema column names need to change? You could have 12 columns called "hello", "world", "empty", "ferrari", "sauber", "rolo", "jumper", "skateboard", "tea", "coffee", "milk" and "sugar", and your output data can be precisely as you want it without referencing any of the column names above.

Re: How to change column names every refresh

akumar2301 — Mon, 25 Feb 2019 15:10:23 GMT

@Aishwarya

One of the solution :

You need to read header as another data record. As the number of Column are fixed (12)

1) create a generic schema (all string )

Col1

Col2

Col12

2) using tMap , split there in 12 different Output for each Col and add a sequence number for each Record and each Column.

So if Input

A,B,C
1,2,3

then there will be 3 Output for each column

RouwNum, Column Number,Col Value

1,1,A

2,1,1

1,2,B

2,2,2

1,3,C

2,3,2

3) Join all output of previous subjob using tJoin and sort on (Colnum and then ID)

1|1|A
2|1|1
1|2|B
2|2|2
1|3|C
2|3|3

4) tdenormisesortrow on Colnum

1|A,1
2|B,2
3|C,3

5) filter id field

A,1
B,2
C,3

So now even if colName change you will have correct result.

so for

X,Y,Z
1,2,3

result will be

X,1

Y,2

Z,3

Hope this will be helpful.

Re: How to change column names every refresh

akumar2301 — Mon, 25 Feb 2019 15:32:46 GMT

Just to add , above solution will work even if input has n number of row.