topic Define metadata for array of values in Talend Studio

Define metadata for array of values

Anonymous — Sat, 16 Nov 2024 08:12:51 GMT

Hello,

I need to define metadata for array of values and need someway to use it in spark sql. Or atleast tmap

Data

a|b|c|d1|d2|d3|d4|.........d40|x|y|z

i need metadata type for d1...d40

Thanks!

Re: Define metadata for array of values

Anonymous — Thu, 31 May 2018 10:31:43 GMT

Hello,

Do you want to define your Array of values under Metadata in repository tree or Contexts value? Could you please give us more information about your requirement?

Best regards

Sabrina

Re: Define metadata for array of values

Anonymous — Thu, 31 May 2018 16:24:50 GMT

requirement is simple , you can think of it as i have delimited data coming in but 40 of which are discount %ges for same product on different days.

i want to represent the addtributes disc1 ...disc40 in a vector or array because i will have to do some decimal calculations based on it.

ex:

prodattrib1| prodattrib2|descARRAY[40]|prodattribx|prodattriby

Currently i have 40 attributes for each discount which is really not that friendly especially when i want to loop those values, i am converting them to stringbuffer and looping thru -it very awkward.

in our previous etl tool we could do that so just wondering if i can do with talend.

Thanks!

Re: Define metadata for array of values

Anonymous — Fri, 01 Jun 2018 02:20:09 GMT

hello,
meta data is, how it can be retrieved, and the various data types (such as objects or arrays) in which it can be returned. Now it's time to learn about the different types of arrays.

When you write an array manually you know what its structure is an what the name of each index is. But when you are building arrays by querying a database, you are going to need to do some detective work to determine the structure of the data returned and the names of the indexes.
patrisia