topic [resolved] logging failed rows? in Talend Studio

[resolved] logging failed rows?

Anonymous — Sat, 16 Nov 2024 13:38:10 GMT

I am currently running a script which takes many rows from csv and inserts each into a mysql database.
When some of these rows fail to properly insert due to whatever reason (for example a string where an integer should be) I am going to need to log these failed rows in some manner. Is there some existing functionality/solution for logging these failures within the system, possibly passing them via email to the administrator of the system?
Thank you in advance.

Re: [resolved] logging failed rows?

Anonymous — Tue, 22 Dec 2009 04:08:29 GMT

Hello guy
You can get the reject rows and log them to a file, use a tSendemail to send a email attachs the reject file at the end of job.
Best regards

shong

Re: [resolved] logging failed rows?

Anonymous — Wed, 23 Dec 2009 14:57:36 GMT

is it possible for me to see the tMap settings from this example Shong? I am still unable to get such errors to not cause the whole job to fail.

Re: [resolved] logging failed rows?

Anonymous — Wed, 23 Dec 2009 15:11:14 GMT

Hello
Define one column as key and it is unique in table, when you insert a dupicate key, it will throw such error.
Best regards
shong

Re: [resolved] logging failed rows?

Anonymous — Wed, 23 Dec 2009 15:55:33 GMT

no unforunately I get this, along with an exit of the job.
Exception in component tMysqlOutput_1
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry 'xxxxxxxxx@msn.com' for key 2
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:353)
at com.mysql.jdbc.Util.getInstance(Util.java:336)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1016)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:957)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2938)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1601)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1710)
at com.mysql.jdbc.Connection.execSQL(Connection.java:2436)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1402)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1694)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1608)
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:1593)
at test_project.fe_user_creation_0_1.fe_user_creation.tFileInputDelimited_1Process(fe_user_creation.java:4702)
at test_project.fe_user_creation_0_1.fe_user_creation.runJobInTOS(fe_user_creation.java:7393)
at test_project.fe_user_creation_0_1.fe_user_creation.runJob(fe_user_creation.java:7298)
at test_project.kreko_0_1.Kreko.tRunJob_11Process(Kreko.java:1882)
at test_project.kreko_0_1.Kreko.tFileList_1Process(Kreko.java:767)
at test_project.kreko_0_1.Kreko.tRunJob_1Process(Kreko.java:499)
at test_project.kreko_0_1.Kreko.runJobInTOS(Kreko.java:2200)
at test_project.kreko_0_1.Kreko.main(Kreko.java:2092)
Exception in component tRunJob_11
java.lang.RuntimeException: Child job running failed
at test_project.kreko_0_1.Kreko.tRunJob_11Process(Kreko.java:1902)
at test_project.kreko_0_1.Kreko.tFileList_1Process(Kreko.java:767)
at test_project.kreko_0_1.Kreko.tRunJob_1Process(Kreko.java:499)
at test_project.kreko_0_1.Kreko.runJobInTOS(Kreko.java:2200)
at test_project.kreko_0_1.Kreko.main(Kreko.java:2092)

Re: [resolved] logging failed rows?

Anonymous — Wed, 23 Dec 2009 15:58:19 GMT

And I had somehow forgotten to uncheck Extend Insert, apologies again Shong
Thank you