topic Re: tFileFetch Thumbnail cannot be generated on the fly in Talend Studio

tFileFetch Thumbnail cannot be generated on the fly

solquest — Sat, 16 Nov 2024 00:19:31 GMT

Hey there.

I have a job for getting products' images url from xml files, for fetching them and then for uploading to S3.

In one case I get this error in the fetching part:

Exception in component tFileFetch_3 (ProductsImages)

java.lang.Exception: Method failed: HTTP/1.1 409 Thumbnail cannot be generated on the fly for asset(id: 9iBn2rnqq8G924MJvaky36, path: /INTERLACED/PUBLIC/PRODUCTS/8056300956327-1.png, v=1, c=2, p=1).

at mio.productsimages_0_1.ProductsImages.tFileFetch_3Process(ProductsImages.java:7927)

at mio.productsimages_0_1.ProductsImages.tFileInputDelimited_3Process(ProductsImages.java:6089)

at mio.productsimages_0_1.ProductsImages.tFileInputXML_1Process(ProductsImages.java:5135)

at mio.productsimages_0_1.ProductsImages.tS3Connection_1Process(ProductsImages.java:1432)

at mio.productsimages_0_1.ProductsImages.tJava_2Process(ProductsImages.java:1287)

at mio.productsimages_0_1.ProductsImages.runJobInTOS(ProductsImages.java:11651)

at mio.productsimages_0_1.ProductsImages.main(ProductsImages.java:11283)

while the rest of the products are fine.

Can anyone tell me what I should check?

In tFileFetch_3 component I have

protocol = https,

URI = ((String)globalMap.get("planogramThumbnailUrl"))

Destination directory = context.workingDir + "/tmp/output/"+context.imagesFileName+"/img"

Destination filename = globalMap.get("imgName") + "_thumb_" + globalMap.get("planogramThumbnailProgr") + ".jpg"

and all other flags unchecked apart from Die on error,

while in Advance tab I have timeout = 30000 and all other flags unchecked

Thanks.

Re: tFileFetch Thumbnail cannot be generated on the fly

gjeremy1617088143 — Sat, 03 Apr 2021 16:24:06 GMT

Hi, the file is .png and you want To fetch it in .jpg . Are the other images you try to fetch are jpg or png ?

Send me love and kudos

Re: tFileFetch Thumbnail cannot be generated on the fly

solquest — Sat, 03 Apr 2021 19:29:50 GMT

Well, in the thumbnail URI I'm receiving I have '..._8056300956327-1_thumb.jpg?...' but actually when trying to open it I get the error "The requested resource is not available in its current state.", so it definitely looks like a problem of the image itself.

I wonder if there's a way for detecting this issue before entering tFileFetch_3, for avoiding to receive an error and stop the job...

Thanks!

Re: tFileFetch Thumbnail cannot be generated on the fly

solquest — Sat, 03 Apr 2021 23:23:14 GMT

I've tried to uncheck Die on error on tFileFetch_3 and to do:

tFileFetch_3 -> OnComponentError -> tJava for writing a message in a log file, but seems it's never going there, not even for those images that were killing the job...

And the same thing when using OnSubjobError

What am I missing?

Re: tFileFetch Thumbnail cannot be generated on the fly

gjeremy1617088143 — Sun, 04 Apr 2021 08:34:12 GMT

Hi solquest, when you uncheck Die on error in your tFileFetch component , is your exception is visible on the console ? If yes you can use a tlogcatcher to catch java exception.

Re: tFileFetch Thumbnail cannot be generated on the fly

solquest — Sun, 04 Apr 2021 09:27:09 GMT

Thanks!

With

tFileFetch_3 -> OnComponentOK -> tlogCatcher -> Main -> tJava

I managed to write those error in a separate log file.

Thanks for helping.

Re: tFileFetch Thumbnail cannot be generated on the fly

gjeremy1617088143 — Sun, 04 Apr 2021 09:48:10 GMT

You dont have to Link tfilefetch to tlogcatcher , tlogcatcher is an entry component link only his output. Tlogcatcher catch all log of your job : info, error,fatal error, java exception etc

Re: tFileFetch Thumbnail cannot be generated on the fly

solquest — Sun, 04 Apr 2021 21:19:52 GMT

Even easier then.

Thanks again!

Re: tFileFetch Thumbnail cannot be generated on the fly

solquest — Mon, 05 Apr 2021 00:06:50 GMT

Hmmm, I've just tried, but seems that tLogCatcher is catching things only if I have set Die on error:

In job Stats & log I have Use logs (tLogCatcher) = true, On Console = true, Catch runtime errors = true, Catch user errors = true, Catch user warnings = true.

I've set tLogCatcher -> Main -> tLogRow (table) without any input on tLogCatcher and I don't have that table in logs (but there I have the error

java.lang.Exception: Method failed: HTTP/1.1 409 Thumbnail cannot be generated on the fly for asset(id: 8_gKCQsp4P9A7Ss9y_miDF, path:)

If I set Die on error = true in tFileFetch component, I get the table in log:tLogRow_1 |

|=------------------+------+--------+----------+-------+---------------------+-------+--------+--------------+------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---=|

But in this way I can't catch all the errors, cause job stops at first one.

I've tried printing ((String)globalMap.get("tFileFetch_ERROR_MESSAGE")), but again is null unless Die on error is true.

What can I do for saving all the errors messages (so with Die on error = false)?

Thanks.