Do not input private or sensitive data. View Qlik Privacy & Cookie Policy.
Skip to main content

Announcements
Qlik Open Lakehouse is Now Generally Available! Discover the key highlights and partner resources here.
cancel
Showing results for 
Search instead for 
Did you mean: 
Soumya_M
Contributor II
Contributor II

To insert records from file to table

Q. A job is already in place, input data - transformations - output data. I have a new file with some records, the moment tFileExists component detects the file in the given location, the file should be transformed into a table, all the records must go through the same transformation and give the output. How can i achieve this ? 

Labels (3)
5 Replies
Anonymous
Not applicable

The easiest way would be to convert the file's data into the format of your existing job's input data (and to the same source), then use the existing job.

Soumya_M
Contributor II
Contributor II
Author

Can you elaborate it please?

I need to automate the job.

Anonymous
Not applicable

It is a little hard to elaborate as each of the decisions on how to do this depend on several factors. For example, what products are you using? Are you using Talend Open Studio or a subscription version? Do you want this to run in a batch way (a job that is started and stopped) or do you want this to be triggered by the presence of a file? Those are just a few of the questions.

 

If you want this to be triggered by the presence of a file, then you will probably be best served by using a route with the ESB product. If you are happy to have this process scheduled to run (maybe every hour or even every minute or 5 minutes), this can be done with a job (and would be a little easier). To reuse your existing job, you will need to know how to get your new data picked up by that job. Now, that will depend upon where your existing job picks its data up from.

Soumya_M
Contributor II
Contributor II
Author

Okay, I'm using Talend Open Studio for Data Integration 8.0.1

The job has to be triggered by the presence of a file, it should automatically give the output without minimal or no user interaction.

I have given certain location in local system [in tFileExists component], from where it picks up the file.

 

Anonymous
Not applicable

OK, you have two choices. You either schedule the job to run every few minutes or hours (depending on the frequency of the file delivery) or you use the ESB product and build a route to continually monitor the folder to pick up the files as they arrive.