topic job works in TOS but breaks while using ETL in Talend Studio

job works in TOS but breaks while using ETL

Anonymous — Sat, 16 Nov 2024 09:30:22 GMT

Hi!

I have a Talend Job that is working well when I execute it on TOS. But when i use the ETL in the company software, it breaks.

I have an error message saying that there is an error in a function I developped (used in tMap).

Why the job is working in TOS but not when using the ETL? And how can I know how to fix it?

I replaced my function by a number (the function returns a double) and it works. So I can confirm the issue is the function.

Does anyone know what to do?

Thanks

Re: job works in TOS but breaks while using ETL

vapukov — Wed, 19 Jul 2017 13:30:22 GMT

Issue not in environment, issue - in the data

If Your function return double - for 99,99% some of values in production environment have NULL values and not NULL when You run it locally

in this case - You need add NULL handle to Your function (same for divide by 0)

what logic must be? it depend from what is Your goal:
- You can return null as well
- You can auto-replace for default value

- etc

Re: job works in TOS but breaks while using ETL

Anonymous — Wed, 19 Jul 2017 13:57:16 GMT

Hi,

I already considered this case. My function calculate a rate. If one of the entry variables is null, it returns null.

For the division, I use a function that returns 0 if one of the variable is null or equals 0.

And in my tMap (and my schemas), the variables affected are nullable.

The error message is not a NullPointerException so I think it must be something else.

I try to get my error message again and i'll post it within an hour. It might help.

Edit : got an issue independant from Talend. I'll post the error as soon as I can.

Re: job works in TOS but breaks while using ETL

Anonymous — Wed, 19 Jul 2017 15:46:21 GMT

I cut some lines (confidential infos) but this is a large part of the error generated

GRAVE : routines/ValueUtil :
at routines.clientUtil.getRate(myRoutine.java:263)
at alimentation.client_myAlim.tFileInputDelimited_2Process(myAlim.java:5435)
at alimentation.client_myAlim.tFileList_2Process(myAlim.java:1847)
at alimentation.client_myAlim.runJobInTOS(myAlim.java:35459)
at alimentation.client_myAlim.runJob(myAlim.java:35284)
at com.client.framework.feed.ExternalFeedPluginManager.runJobTalend(ExternalFeedPluginManager.java:196)
at java.lang.Thread.run(Thread.java:745)

routines.ValueUtil from [Module "deployment.0-feed_20170719_162239.ear:main" from Service Module Loader] :
at org.jboss.modules.ModuleClassLoader.findClass(ModuleClassLoader.java:205)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassUnchecked(ConcurrentClassLoader.java:455)
at org.jboss.modules.ConcurrentClassLoader.performLoadClassChecked(ConcurrentClassLoader.java:404)
at org.jboss.modules.ConcurrentClassLoader.performLoadClass(ConcurrentClassLoader.java:385)
at org.jboss.modules.ConcurrentClassLoader.loadClass(ConcurrentClassLoader.java:130)
at routines.clientUtil.getRate(clientUtil.java:263)
at alimentation.client_myAlim.tFileInputDelimited_2Process(myAlim.java:5435)
at alimentation.client_myAlim.tFileList_2Process(myAlim.java:1847)
at alimentation.client_myAlim.runJobInTOS(myAlim.java:35459)
at alimentation.client_myAlim.runJob(myAlim.java:35284)
at com.client.framework.feed.ExternalFeedPluginManager.runJobTalend(ExternalFeedPluginManager.java:196)
at java.lang.Thread.run(Thread.java:745)

The datas source in TOS and with the ETL is exactly the same. So, I don't get why it's working in TOS and not with the ETL...

Re: job works in TOS but breaks while using ETL

Anonymous — Wed, 19 Jul 2017 20:08:41 GMT

Please raise a support ticket on this if it is not working in your subscription version.

Re: job works in TOS but breaks while using ETL

Anonymous — Mon, 31 Jul 2017 09:39:13 GMT

Hi!

I completely forgot to reply!

Well, the error wasn't understandable but, in the end, the mistake was mine.

I had the case where one of my value was just a blank space... So it wasn't null but it wasn't an expected content so it broke. I used the TRIM function to solve it.

So, like almost every time, it wasn't a bug, it was just an uncomplete function.

But thanks for your replies and your reactivity, it's cool to know that we have people ready to help newbies like me when we need

Re: job works in TOS but breaks while using ETL

vapukov — Mon, 31 Jul 2017 09:54:11 GMT

@EmilieR wrote:

Hi!

I completely forgot to reply!

Well, the error wasn't understandable but, in the end, the mistake was mine.

I had the case where one of my value was just a blank space... So it wasn't null but it wasn't an expected content so it broke. I used the TRIM function to solve it.

So, like almost every time, it wasn't a bug, it was just an uncomplete function.

But thanks for your replies and your reactivity, it's cool to know that we have people ready to help newbies like me when we need

welcome back! 🙂

it is return us to original error reason:

"different data on dev machine and in live environment",

NULL or empty - it just details, but most important - always need have proper test data set