topic Re: DB Migration: Last table data is blank (Output DB) in Talend Studio

DB Migration: Last table data is blank (Output DB)

Anonymous — Sat, 16 Nov 2024 02:54:10 GMT

Hi,

Using the following job and settings, I was able to dynamically migrate a whole database from MSSQL to MySQL.

However, I noticed that the last table's data is blank (only the table structure and column is migrated).

Please refer to the screenshots and attached job file below.

**The table and its schema is migrated, but no data inside the table. // I'm using a dynamic schema.

Job Design and DBInput

dbinput

DBOutput

When I tried to logged each component, I found that the last table's data is loaded and pass on from tDBInput to tMap and also from tMap to tDBOutput, as the last table's data showed up in the log.

However, when I check the actual output database, only the last table contains no data at all.

Output DB's 1st Table - Have data

2nd (last) table - No Data

==============================================================

Re: DB Migration: Last table data is blank (Output DB)

manodwhb — Tue, 24 Mar 2020 04:30:00 GMT

For the last table, is there any error from log?

Re: DB Migration: Last table data is blank (Output DB)

Anonymous — Tue, 24 Mar 2020 04:54:21 GMT

@manodwhb
do you mean the whole run log??
If yes, only the tables' data are printed (both).

Re: DB Migration: Last table data is blank (Output DB)

manodwhb — Tue, 24 Mar 2020 04:55:18 GMT

What has printed for the last table?

Re: DB Migration: Last table data is blank (Output DB)

Anonymous — Tue, 24 Mar 2020 05:18:05 GMT

@manodwhb

This is the log for the job run...

and here is single table job run.

the table data is logged here, however, no data is imported to the database. (only schema and column headings)

Re: DB Migration: Last table data is blank (Output DB)

manodwhb — Tue, 24 Mar 2020 05:22:05 GMT

As per the log there is no error right and even it has printed to console that last table data right? If yes it bit confusing. Can you try to run the had last table hard-coded one for testing to see that why it is not inserting data?

Re: DB Migration: Last table data is blank (Output DB)

Anonymous — Tue, 24 Mar 2020 05:35:19 GMT

@manodwhb

yes, there is no error in the console.
can you elaborate more about 'hard-coded one'?

Here I tried to do a debug run

and somehow nothing shows up in row5?

can you tried importing this job? please check the attached file.

Migration_DB_Parent_Job.zip

Re: DB Migration: Last table data is blank (Output DB)

Anonymous — Tue, 24 Mar 2020 06:07:42 GMT

@manodwhb

I figured out the reason why the last table data is not inserted.

Now that I change tDBOutput_1's setting,, from use existing connection (tDBConnection_2) to manual setup in tDBOutput and it work now.

Do you happened to know why? Does it have something to do with the trigger 'OnComponentOK'?

dbinput

Re: DB Migration: Last table data is blank (Output DB)

manodwhb — Tue, 24 Mar 2020 06:09:19 GMT

one thing could be it might rollback? But not sure how it could have triggered.

Re: DB Migration: Last table data is blank (Output DB)

manodwhb — Tue, 24 Mar 2020 06:10:31 GMT

use the on component trigger from tdboutput.

Re: DB Migration: Last table data is blank (Output DB)

Anonymous — Tue, 24 Mar 2020 06:20:15 GMT

@manodwhb

I tried to use on component trigger from tdboutput as suggested however there's an error

Re: DB Migration: Last table data is blank (Output DB)

manodwhb — Tue, 24 Mar 2020 06:26:07 GMT

You are doing wrong your dbconnection component should trigger before tdbtablelist.

Re: DB Migration: Last table data is blank (Output DB)

Anonymous — Tue, 24 Mar 2020 07:02:52 GMT

@manodwhb

Here I tried connecting it like this and there's no error.

However, no data is inserted to the output database (not only the last one but all)

Re: DB Migration: Last table data is blank (Output DB)

manodwhb — Tue, 24 Mar 2020 07:05:42 GMT

If you used tdbconnection component you no need to connect to tdbouput, since you are using Torrin, which will execute first.

Re: DB Migration: Last table data is blank (Output DB)

Anonymous — Tue, 24 Mar 2020 07:22:38 GMT

@manodwhb

but when I use existing connection from tdbconnection component, the last table data is not inserted (that's the initial setting)

Re: DB Migration: Last table data is blank (Output DB)

manodwhb — Tue, 24 Mar 2020 07:27:56 GMT

if that is the case built in of tdbouput and do not use the existing connection.