Difference between tcreatetemporaryfile and Buffer componets

[Anonymous]

Hi Team

in Talend we have Tcreatetemporaryfile for storing data in temp files and get discarded once they are processed

So is it  best practice practice use Tcreatetemporaryfile or buffer components  to storing temporary data

please suggest .

Thanks

Saranya

[fdenis]

buffer are to share data between jobs and sub-jobs.(I think there is some trouble managing schemas when you try tu use multiples buffer into the same job. [there is only one buffer by job ] )
Temporary files are real file used to hold data during a process.
You can also use a db Table if you want.

Regards

[Anonymous]

Hi 

Thank for your replay

ok Leave about Buffers

Lets take Hash Components(Hashinput/HashOutPut). we can temp data in multiple stages of different schemas. we can store and access them.

Here is it better to use hash component or tCreateTemporary File.

Thanks

Saranya 

[Anonymous]

Quick answer: it really depends on what you're doing

The hash output/input components will store data in memory. If you're not storing a lot, these can be very convenient and fast. However, you do need to be mindful of memory usage and not load too much data into the hashes.

Using a temp file is almost always preferred for larger amounts of data as the data is written to the filesystem instead of memory. This will be slower, but will allow your jobs to process more data without running out of memory.

[fdenis]

better is temp file to free up some memory.