topic Re: Transfer data from data server to warehouse (postgresql) in Talend Studio

Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 06:36:11 GMT

Hello,

I am new to Talend and needed a help with transferring data from data server (postgresql) to warehouse server (postgresql). A step-wise elaboration will be appreciated.

Till now what I've been able to achieve:

1. Make DBConnections for data server and warehouse server and load their respective schemas. (tPostgresqlInput)

2. Insert a tMap between the two and map the columns of data server table and warehouse table.

But, when I run the job, it shows the following screen (in the picture), and when I try to check the warehouse table, I am not able to see any data there, i.e. table is empty.

Thanks!

Re: Transfer data from data server to warehouse (postgresql)

manodwhb — Thu, 07 Feb 2019 06:41:19 GMT

@Harguna ,you need to commit the data once it is loaded.find the below screenshot. until unless you commit you will not see any data ,if your job has completed successfully.

Re: Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 06:52:28 GMT

Thank you for the response. I tried your method but the following error occurred (attached)

Screenshot 2019-02-07 at 12.20.21 PM.png

Re: Transfer data from data server to warehouse (postgresql)

manodwhb — Thu, 07 Feb 2019 06:54:56 GMT

@Harguna ,in which component you were getting this error and check in the code tab for the error?

Re: Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 06:59:10 GMT

Here is the code version of the error:

Also, does Postjob needs to be configured or something? I just dragged and added the two components you asked me to and linked them via trigger.

Screenshot 2019-02-07 at 12.27.36 PM.png

Re: Transfer data from data server to warehouse (postgresql)

manodwhb — Thu, 07 Feb 2019 07:03:04 GMT

@Harguna Can you show in design which component is failed?

Re: Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 07:03:11 GMT

Here is m workspace for the job:

Screenshot 2019-02-07 at 12.32.17 PM.png

Re: Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 07:05:23 GMT

The code says postjob component, but in the workspace, a red alert symbol is on commit component (attached images):

Screenshot 2019-02-07 at 12.27.36 PM.png
Screenshot 2019-02-07 at 12.32.17 PM.png

Re: Transfer data from data server to warehouse (postgresql)

manodwhb — Thu, 07 Feb 2019 07:07:26 GMT

@Harguna ,Understand your problem,since you have not specified connection in commit component.

Re: Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 07:11:57 GMT

Do I have to configure the prefab or the postgresqlconnection component?

Re: Transfer data from data server to warehouse (postgresql)

manodwhb — Thu, 07 Feb 2019 07:14:20 GMT

@Harguna ,yes ,you need configure connection component or if you have created metadata for the target DB you can drag that into designer there you can specify postgresqlconnection component.

Re: Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 07:30:02 GMT

New error in prejob and old error in post job are still there. I am attaching the screenshot.

Btw, let me clear it out what I've done overall:

1. Added prejob component and connected with target DB postgresqlconnection.

2. added source and target postgresql DBs as tpostgresqlinput and tpostgresqloutput, connected via tmap where columns are mapped.

3. added postjob component and commit component.

Screenshot 2019-02-07 at 12.55.42 PM.png

Re: Transfer data from data server to warehouse (postgresql)

manodwhb — Thu, 07 Feb 2019 07:32:36 GMT

@Harguna ,show me the design with error component? have you select prostgresqlsonnection in the component list of tPostgresqlCommit?

Re: Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 07:37:15 GMT

Yes, I did add postgresqlconeection in postgresqlcommit.

Here are screenshots of the error and the designer:

Screenshot 2019-02-07 at 1.03.28 PM.png
Screenshot 2019-02-07 at 1.04.36 PM.png

Re: Transfer data from data server to warehouse (postgresql)

manodwhb — Thu, 07 Feb 2019 07:40:13 GMT

@Harguna ,I am not able to download the attachment,can you re-share the job design?

Re: Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 07:42:45 GMT

Here you go:

Btw, thank you so much for the help

Screenshot 2019-02-07 at 1.11.23 PM.png
Screenshot 2019-02-07 at 1.11.31 PM.png

Re: Transfer data from data server to warehouse (postgresql)

manodwhb — Thu, 07 Feb 2019 07:50:21 GMT

@Harguna ,you have correctly the Commit component. you need to see the error from

Window>>Show View>>General>>Error Log

Re: Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 07:58:58 GMT

If you check the error window in previous screenshots, you'll see that it says some java compatibility issues, or some plug-in issues (screenshot attached here). Any ideas about this?

Screenshot 2019-02-07 at 1.28.22 PM.png

Re: Transfer data from data server to warehouse (postgresql)

manodwhb — Thu, 07 Feb 2019 09:15:35 GMT

@Harguna ,it is difficult from that screenshot,could you please share the ecah component settings and schema?

Re: Transfer data from data server to warehouse (postgresql)

Anonymous — Thu, 07 Feb 2019 10:14:59 GMT

I updated JVM version and that error stopped showing but a new error has come which says that permission to the database denied. I don't know if the problem is at my end or is it something with Talend. Thanks for the help. I really appreciate it!