topic Sort the data in the file which is having different schema. in Talend Studio

Sort the data in the file which is having different schema.

Banupriya — Fri, 15 Nov 2024 23:56:02 GMT

Hi,

I need to sort the data in the file which is having different schema. Schemas we know only on run time. First 4 columns are same. We can not define dynamic schema because it's taking the structure of first row data and appending delimiter to other rows which is having less columns.

Could any one help me here.

Need to sort the data having different schema.

Thanks,

Banupriya

Re: Sort the data in the file which is having different schema.

gjeremy1617088143 — Tue, 20 Jul 2021 07:01:58 GMT

Hi, maybe you can read data row by row in one field then

you parse with a delimiter to see the number of field and you drive data in function to that.

Send me love and kudos

Re: Sort the data in the file which is having different schema.

Banupriya — Tue, 20 Jul 2021 08:35:41 GMT

Thank you for the reply. We are not sure how many columns we are going to get in the file. so we can not define a field counts anywhere.

Re: Sort the data in the file which is having different schema.

gjeremy1617088143 — Tue, 20 Jul 2021 12:54:12 GMT

dynamic shema allow you to do that, you have just to select only one otuput field with dynamic type on the input component, then you can do what you want with :

count the field numbers, add column,supress suppress ,shuffle the column etc.

here a code example to get all column name and values of dynamic object :

if the output field of your input component is for example data (type dynamic):

Dynamic columns = row1.data;

for (int i = 0; i < columns.getColumnCount(); i++)

{

DynamicMetadata columnMetadata = columns.getColumnMetadata(i);

String inp_col=columnMetadata.getName().toString();

String col_val=(String)row1.data.getColumnValue(inp_col);

}

do you have a header in the file or each row have is own schema ?

maybe you could send an exemple of file you have