topic Re: Interaction with front end application instantly in Talend Studio

Interaction with front end application instantly

Anonymous — Thu, 24 May 2018 13:27:31 GMT

Here's the case, Whenever the data update happens in front end application, it will come to Database. Then talend has to process the below scenario /Job and then send instant update to front end. Reason for this query is, the below job process is taking too much of time and my client is not ready to wait that much but client wants immediate impact. The below process is taking more than 10min to complete. I just want to know, Is it TALEND suits for this kind of requirement?

Currently i got the below flow in the job

DBTable-->tMap-->OutputExcel (Here some formula will be calculated and bring 2 extra columns to the subjob)

subjob

InputExcel-->tMap--DBTable

Re: Interaction with front end application instantly

Jesperrekuh — Thu, 24 May 2018 14:52:56 GMT

Two things:
1 -How much does it take to go from frontend -> database, looks like updates within database takes some time?
2 - Is your process scheduled for like to run every 10 minute?

I dont get the excel part? is that the frontend?
Why not make a view and an ODBC connection, instant update no Talend needed

Re: Interaction with front end application instantly

Anonymous — Thu, 24 May 2018 15:14:38 GMT

1 -How much does it take to go from frontend -> database, looks like updates within database takes some time? [Ans] Front end to database update is happening instantly. Talend work is coming after it. From DB i am picking the data, sends to excel and then again back to DB. When i tried with normal excel it worked fast, but when i used the real excel which has macros, the job is taking time.
2 - Is your process scheduled for like to run every 10 minute? [Ans] No, its not scheduled. Its still in development stage. While testing i found this issue.

I dont get the excel part? is that the frontend? [Ans] Front end is a web application, whenever something updates it comes to DB. From here the talend job is coming in the picture. Below is the flow.

DBTable-->tMap-->Excel (Data in 8 column will be entered, 9th and 10th column is having formula which will be calculated and taken to the DB table)

SubJob

Excel-->tMap-DBTable

Note: Both DB Table are same, Both excel are also same. I am calling the excel in subjob because, the main excel calculated some value in 9th and 10th column and that needs to be updated in DB table.

Why not make a view and an ODBC connection, instant update no Talend needed [Ans] can you please elaborate?

Re: Interaction with front end application instantly

Jesperrekuh — Thu, 24 May 2018 22:42:37 GMT

With an ODBC connection you could access the database directly from Excel and load the data, in example pivot and create graphs... Google it please, to see the possibilities. And a DB-view is something you create within your database, its purpose is to have a specific view on the data. Within the view you make the calculations and joins to other data (if necessary).

As what Im reading here... the workflow you are working on is not very efficient ... and is planning for failure. There're two workflows, Frontend->db->excel and the second one calculations and back into the db. Why on earth are you calculating within excel? What is so specific that its needed to support this kind of work?

Re: Interaction with front end application instantly

Anonymous — Fri, 25 May 2018 07:28:50 GMT

Even i had a similar questions, but this requirement says we cant skip this excel. Also, We cant implement the excel's formula in talend. The calculation should be done in excel only and store the updated data in DB, and this should be done instantly as soon as we receive data from front end. The entire process should be done in seconds. Hope this is not possible in talend, right? thanks for your inputs, i will google it for ODBC connections.