Hi, I Need to write a Job that will read a file that carries a date in each row. Example: 2015-07-20 bla 2015-07-20 blu 2015-07-21 lala I will then Need to sort each row into a file that is named after the date: File 2015-07-20_file will receive 2 rows, file 2015-07-21 will receive 1 row How can that be done?
This can get complicated.
One you can put all your data rows into an object to read through them. Use the tHashInput and tHashOutput
Then feed that tHashInput to a remove duplicates based on the date column. if you have it, use the tUniqueRow.
Then for each unique date, use a tJavaRow to create a file with the date on it. This is one SubJob
Then another subjob starts and feed it the tHashInput with all the dates in it.
Then use the tJavaRow, to read the date and feed that row to its corresponding Date_File.
this is one quick way to do it. hope this helps.
Hi, thanks for the quick Response. The Problem with that approach might be that the files I am talking about are really large, so I can't really put them into an object. I was hoping to simply extract the date from each row e.g. in a tMap and store that Information in some variable - which I could use in a tFileOutput component to create a dynamic Name. But I don't know how that could work..
I was going to suggest pretty much the same; but was beaten to it.
If your data is too large for HashMap, then try dropping it to a file.
It doesn't look like you've got a delimiter, so I'm guessing you have a fixed length record.
One general approach might be to...
Read the file using tFileInputPositional. Strip out the dates using tMap (othr components can do this too). Make the rows unique tUniqRow and then write then to tHashOutput.
Add a tHashInput and then connect this to tFlowToIterate. Now that you're iterating, you can refer to an entry put on tGlobalMap, so you've got the date.
You can then use this value to fire a single row from tFixedFlowInput, that you can then join back to a second tHashInput.
You should now be extracting the correct data, within each iteration.
You can now write this to a new file, again referencing the date that was put on globalMap, for your file name generation.
