topic tWebService and temporary folders in Talend Studio

tWebService and temporary folders

Anonymous — Sat, 16 Nov 2024 09:46:23 GMT

Hello Community,

I have several jobs running in Talend Runtime (invoked by route/cTimer or service). In most cases everything work perfect, but sometimes I have exception:

Exception in component tWebService_2
java.lang.SecurityException: Unable to create temporary directory,C:\Users\*******\AppData\Local\Temp\wsdl149493219078118022
	at org.talend.webservice.helper.ServiceDiscoveryHelper.createTempWsdlDir(ServiceDiscoveryHelper.java:101)
	at org.talend.webservice.helper.ServiceDiscoveryHelper.<init>(ServiceDiscoveryHelper.java:89)
	at org.talend.webservice.helper.ServiceDiscoveryHelper.<init>(ServiceDiscoveryHelper.java:77)
	at local_project.sendcustomer_0_1.SendCustomer.tHashInput_2Process(SendCustomer.java:8915)
	at local_project.sendcustomer_0_1.SendCustomer.tHashInput_1Process(SendCustomer.java:6978)
	at local_project.sendcustomer_0_1.SendCustomer.tMSSqlInput_1Process(SendCustomer.java:3554)

I guess it is related to parallel jobs execution. There is similar topic https://community.talend.com/t5/Design-and-Development/Unable-to-create-temporary-directory/td-p/102668, but no solution.

I also noticed that tWebService is not deleting temporary files, there are a lot of wsdlxxxxxxxxxxxxxxx folders in Temp.

It happens even if I specify temp folder in advanced properties.

Talend Open Studio 6.3.1

Windows Server 2012

JDK 1.8

Runtime is running as windows service

Any advice how to deal with this issues (both exception and not removed temp files)?

Thank you in advance!

Re: tWebService and temporary folders

Anonymous — Thu, 18 May 2017 08:55:42 GMT

Hi,

The exception is random?

Are you excecuting in parallel multi instances of twebservice? This issue is probably caused by calling a webservice in parallel.

Have you tried to disable the parallel call and use just one thread to see if it works?

Best regards

Sabrina

Re: tWebService and temporary folders

Anonymous — Mon, 22 May 2017 12:37:45 GMT

Hi Sabrina,
Thank you for your reply.

The exception is random, right. I do not know what you mean with parallel call.

Scenario, transfer data from database to webservice call:

1. Load data from database (example: students), for each row invoke web service call.
2. OnSubJobOK triggers load other data (example: exams for those students) and for each row invoke web service call.

So I have tWebService1 on 1st part and tWebService2 on second.

Re: tWebService and temporary folders

Anonymous — Mon, 22 May 2017 13:21:28 GMT

Additional info here, it might happen that other Jobs with tWebService component are running in parallel. Jobs are started from Service and service is responding on external requests.

Re: tWebService and temporary folders

Anonymous — Sun, 04 Jun 2017 13:22:07 GMT

I managed to solve both issues, so just to share solution if someone find it useful.

Yes, exception was caused by parallel execution. I wasn't able to avoid parallel execution, but I was able to specify UNIQUE temp folder for each tWebService component as:

"/tmp/" + pid+ "/" + currentComponent

Second issue with too many temp folders was caused by standard behavior of java temp files. Folders was deleted once I stop runtime. I don't plan to restart my runtime at all, so in PreJob I create temp folder "/tmp/" + pid and delete it in PostJob.

Hope someone will find this useful.