Define metadata for array of values

[Anonymous]

Hello,

I need to define metadata for array of values and need someway to use it in spark sql. Or atleast tmap

Data

a|b|c|d1|d2|d3|d4|.........d40|x|y|z

i need metadata type for d1...d40

Thanks!

[Anonymous]

Hello,

Do you want to define your Array of values under Metadata in repository tree or Contexts value? Could you please give us more information about your requirement?

Best regards

Sabrina

[Anonymous]

requirement is simple , you can think of it as i have delimited data coming in but  40 of which are  discount %ges for same product on different days.

prodattrib1| prodattrib2|disc1|disc2|disc3| ..........disc40|prodattribx|prodattriby

i want to represent the addtributes   disc1 ...disc40 in a vector or array because i will have to do some decimal calculations based on it. 

ex: 

prodattrib1| prodattrib2|descARRAY[40]|prodattribx|prodattriby

Currently i have 40 attributes for each discount which is really not that friendly especially when i want to loop those values,  i am converting them to stringbuffer  and looping thru -it very awkward.

 in our previous etl tool we could do that so just wondering if i can do with talend.

Thanks!

[Anonymous]

hello,
meta data is, how it can be retrieved, and the various data types (such as objects or arrays) in which it can be returned. Now it's time to learn about the different types of arrays.

When you write an array manually you know what its structure is an what the name of each index is. But when you are building arrays by querying a database, you are going to need to do some detective work to determine the structure of the data returned and the names of the indexes.
patrisia