topic Re: Help filtering and transforming data in Talend Studio

Help filtering and transforming data

Anonymous — Tue, 16 Apr 2019 13:34:02 GMT

I have data in 2 txt files in which I am trying to have Talend sort through a file of data according to the position of data and then filter the Variable names into being columns in an output file.

For example

File 1

Position Variable

1- 2 FIPS State code (00 for US record)
4- 6 FIPS county code ( 0 for US or state level records)

8- 15 Estimate of people of all ages in poverty
17- 24 90% confidence interval lower bound of estimate of people of all ages in poverty

File 2

42	9	5864	4819
42	11	47405	42354

In which the position from file 1 corresponds to the number each digit represents in file 2.

Please help

Re: Help filtering and transforming data

akumar2301 — Tue, 16 Apr 2019 15:32:51 GMT

It is not very clear. Please explain the relation between file1 and File2 with example and your expected output.

Re: Help filtering and transforming data

Anonymous — Tue, 16 Apr 2019 15:46:46 GMT

File 1 has text that explains what the data that is stored in file 2 means.

I want to create a solution that integrates the two files into 1 output file that will make it easier to understand the data.

I am trying to output a file couples both of these functionalities.

Attached is my file 1 and file 2 in separate excel sheets.

SourceData.xlsx

Re: Help filtering and transforming data

akumar2301 — Tue, 16 Apr 2019 16:16:41 GMT

so you want e.g. for

1- 2 FIPS State code (00 for US record)

4- 6 FIPS county code ( 0 for US or state level records)

you want output as

FIPS State code (00 for US record);FIPS county code ( 0 for US or state level records);................................

42;0;..................

42;1;..................

43;132;..................

Re: Help filtering and transforming data

Anonymous — Tue, 16 Apr 2019 16:46:13 GMT

Yes

Re: Help filtering and transforming data

akumar2301 — Tue, 16 Apr 2019 16:58:32 GMT

Can you send me two txt file . Excel is not working

Step 1 )

You need to parse 1st file

output will be

StartPosition;EndPosition;FiledValue

Step2)

Get the postionlength ( EndPosition- Prev record end position)

Length;FieldValue

Step3)

Denomalise both value

Step4) List of FiledValue should be written to fileOutput

Step5) List of Length to globalVar "Pattern"

Step6) use above pattern in tFileInputPositional for File2 and output to fileOutput

Re: Help filtering and transforming data

akumar2301 — Wed, 17 Apr 2019 09:18:12 GMT

attaching a sample job which could help to start.

change tSetGlobalVar

testpositional_0.1.zip

Re: Help filtering and transforming data

Anonymous — Wed, 17 Apr 2019 09:35:03 GMT

How do I open the project that I downloaded from you with my Talend studio?

Re: Help filtering and transforming data

akumar2301 — Wed, 17 Apr 2019 09:42:32 GMT

https://stackoverflow.com/questions/27724071/talend-export-and-import-a-job

Re: Help filtering and transforming data

Anonymous — Wed, 17 Apr 2019 10:01:33 GMT

tried running job and got this in console

Screen Shot 2019-04-17 at 5.00.47 AM.png

Re: Help filtering and transforming data

akumar2301 — Wed, 17 Apr 2019 12:09:39 GMT

weird I used your files only to run the job.

Did you used same file which you attached ?

Re: Help filtering and transforming data

Anonymous — Wed, 17 Apr 2019 12:31:30 GMT

Yes I am. Am i suppose to import the files somewhere ?

Re: Help filtering and transforming data

akumar2301 — Wed, 17 Apr 2019 12:34:13 GMT

Did you change all paths in tsetglobalvar

Re: Help filtering and transforming data

Anonymous — Wed, 17 Apr 2019 12:34:24 GMT

Also I am using a MAC if it matters.

Re: Help filtering and transforming data

Anonymous — Wed, 17 Apr 2019 12:38:56 GMT

I didn't change any paths I just ran the job as is.

Re: Help filtering and transforming data

akumar2301 — Wed, 17 Apr 2019 14:25:13 GMT

Please correct the paths in tsetglobalvar