Feeding variable number of flows with a primary ID into tMap

[Jj5]

I have an excel file that has a primary key and a message field. The message field isn't consistent in the number of records it contains and looks something like this:

Loan number:    Message

1234                  Loan date x

                          customer name x

1235                  Loan Date y

                          customer name y

                          second customer name y

                          loan type y

1236                  Loan date Z

                          Customer name Z

                          second customer name Z

I know how to replicate the primary id (Loan Number) to all the fields in the message column, but I don't know how to define a variable schema like this. Any help would be greatly appreciated.

[Anonymous]

Hi Can you describe your requirement in details? What are you expected result? I don't understand 'a variable schema' you mentioned. Regards Shong

[Anonymous]

Hi Can you describe your requirement in details? What are you expected result? I don't understand 'a variable schema' you mentioned. Regards Shong

[Prakhar1]

I really don't get what exactly you want to do, but If I am right then you can use "Dynamic" type schema. You can select it in the Column type and it will automatically adapt the schema change every-time.

[Prakhar1]

I really don't get what exactly you want to do, but If I am right then you can use "Dynamic" type schema. You can select it in the Column type and it will automatically adapt the schema change every-time.

[Jj5]

Thanks for the reply. It sounds like that's what I'm looking for, but I have no idea how to add a dynamic schema, it's greyed out. I'm using Talend Open Studio for Data Integration 6.3.1 if it matters

tmp1.png
tmp2.PNG

[Jj5]

Thanks for the reply. It sounds like that's what I'm looking for, but I have no idea how to add a dynamic schema, it's greyed out. I'm using Talend Open Studio for Data Integration 6.3.1 if it matters

[Jj5]

Sorry for not being clear, I want to remap the fields to a tFileOutputDelimited output. I need to rearrange the flow so it looks like this:

Loan No |  Loan Date | Customer Name | Second Customer Name | Loan Type

1234 | 01/01/2020 | Bob Smith

1235 | 02/24/2020 | Jeremy Cambell | Jane Cambell | Line of Credit

1236 | 03/12/2019 | Caren Smith | Coleen Martin

By Dynamic I mean that I won't know whether the message field only contains the Loan Date, the Loan Date and Customer Name, etc.

[Jj5]

Sorry for not being clear, I want to remap the fields to a tFileOutputDelimited output. I need to rearrange the flow so it looks like this:

Loan No |  Loan Date | Customer Name | Second Customer Name | Loan Type

1234 | 01/01/2020 | Bob Smith

1235 | 02/24/2020 | Jeremy Cambell | Jane Cambell | Line of Credit

1236 | 03/12/2019 | Caren Smith | Coleen Martin

By Dynamic I mean that I won't know whether the message field only contains the Loan Date, the Loan Date and Customer Name, etc.

[Prakhar1]

Hi @jeanjpotgieter ,

First the Dynamic will not be working with open studio as it is paid version functionality.

Second you can try with tDenormalize to concatenate the message in single line and then read it using tBufferInput/tBufferoutput and using comma separation you can differentiate each value, afterwards for schema change I need to check if it is possible or not. If you know the max number of columns then you can directly create them and then using above procedure you can works for the details where loan number don't have much message those columns will be filled with NULL.