How to Input data that have dynamic column/field/dimension input.

[PMAR191199]

I have a cdr data that the column regularly changes. Sometimes the next row has some column added/deleted. I want to make a talend job that will add more columns when there is a new column added and when the next row has some missing columns it fills the value to NULL for the respected columns.

Here is a

sneak peek of the cdr file.

[Anonymous]

This is a little tricky to respond to as I am not sure of the requirement. You say the columns regularly change and sometimes columns are added or deleted. This can be handled with Talend. You'd need to explain how the data will arrive, but you can certainly dynamically add columns and delete columns.

However, you then ask for missing columns to be given a NULL value. If the columns are always changing, how do we know that a column is missing? Is this a case of if ever a column is added, it needs to be included NULL or not for the rest of time? This is possible, but will grow the columns.

I think you need to give more detail on how the data arrives, how we know what to do and precisely the rules on what may or may not happen.

[PMAR191199]

Actually, the problem is more simple after I saw the data. the problem is I am still new to Talend. There is already a fixed schema so I know which column that I need to extract. The problem is I don't know how to extract a specified pattern with regex from a long string. Ex: I want to extract "TAU=\d*" from every row from that file. do you have any idea of any component in Talend that I can use to extract a specified pattern? I tried tMap and still no light