topic Re: Data flow in Talend Studio

Data flow

HookemHorns — Sat, 16 Nov 2024 01:09:50 GMT

So I created a data flow recently that I believe to be a little clunky. I have several dbinputs, create tables, and dboutputs. I basically create a table that I want to insert into, select what I want in dbinput and then insert it into said created table using dboutput. This will create a lot of tables and I was wondering if there was a way where I could create these tables in memory and only create one final table at the end that I could have inserted into the database? What tool should I use in the palette?

Re: Data flow

vikramk — Fri, 06 Nov 2020 05:42:38 GMT

Hi @Not defined Not defined ,

If you want to cache to memory, you could possibly use tBufferInput and tBufferOutput, thashinput and thashoutput components as pairs will help you cache the data to memory as output and you can retrieve whenever you need from cache again. Please go through below url.

https://help.talend.com/reader/wDRBNUuxk629sNcI0dNYaA/Rqfj~ssCzCvBc~_TNU7Psg

Please mark it as solution if it helps.

Re: Data flow

HookemHorns — Fri, 06 Nov 2020 18:43:24 GMT

Hi @Vikram Kumar

My example is a little bit different since I'll be outputting the final result to a db table. Also I don't get how you can read these tables from memory. Sorry little new at Talend and trying to piece everything together. Trying to duplicate a data flow I currently have in magic ETL (DOMO). Thanks!