topic Re: Issue with data loaded in table using tpostgresoutput component in Talend Studio

Issue with data loaded in table using tpostgresoutput component

Anonymous — Sat, 16 Nov 2024 03:40:13 GMT

Hi Team,

I would like to share one of the critical issue which I am noticed in TOS.

We used tpostgresoutput component to process the data into target Postgres DB however, when we fetch the data from the table. We have notice below issue.

1. If we select the table based on column then no rows return.

2. sometime select count(*) from table doesn't return any record however, table have data.

Again, when we use tpostgresrow component and manually put Insert query then all looks good.

INSERT INTO vmrctta1.test
(test1, test2, test3, test4)
VALUES('"+test.test1+"', '"+test.test2+"', '"+test.test3+"', '"+test.test4+"')

So, anyone in community does came across with such issue with tpostgresoutput as tpostgresrow performace is very slow but data is perfectly fine and we are looking for solution with tpostgresoutput.

Re: Issue with data loaded in table using tpostgresoutput component

Anonymous — Mon, 06 Jan 2020 13:07:55 GMT

Hello,

Do you use tDBCommit components between your operations?

Regards

Pierre

Re: Issue with data loaded in table using tpostgresoutput component

Anonymous — Mon, 06 Jan 2020 13:20:07 GMT

Hi,

Could you please detail the design on high level.

Is your answer is to improve the performance of tpostgresrow component?

Or suggestion on issue with tpostgresrowoutput.

Re: Issue with data loaded in table using tpostgresoutput component

Anonymous — Mon, 06 Jan 2020 13:36:29 GMT

It is for the tpostgreoutput issue

Re: Issue with data loaded in table using tpostgresoutput component

Anonymous — Mon, 06 Jan 2020 15:37:39 GMT

Hi,

Could you please check whether you are doing commit after loading the data?

If you are using Connection component, please select Auto commit option.

But if you need to do rollback and commit manually, please add the components like below.

Please also select the Die on Error option in tDBOutput component. Could you please let us know the output after these changes?

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Re: Issue with data loaded in table using tpostgresoutput component

Anonymous — Mon, 20 Jan 2020 03:36:40 GMT

Hi Nikhil,

We are committing in batches to improve the performance.

Please check and let me know in case you see any issue.

Is it anything to do compatability with Talend 7.2.1 version? Is there possibility that tpostgresoutput version works well with other TOS older version?

Sample mapping with property details

Re: Issue with data loaded in table using tpostgresoutput component

Anonymous — Mon, 20 Jan 2020 06:55:05 GMT

The JDBC driver collects all data until the end and then starts sending them out to the application your Talend job Prepaid Gift Balance