tSOAP - access the response header

[Anonymous]

Hello,

i have been searching around now for quite a time, I want to communicate with a SOAP Webservice where the process is the following:

1. connect with the Webservice via Login Credentials sent in the body part
2. receive a Cookie from the header of the response to the login message
3. use the received cookie for further requests on the webservice

This is a quite common scenario.

Component options which i have tested:

- tSOAP component - no chance to get the HTTP header information out of that

- tWebservice component - same.

- tHttpRequest - no chance to get the HTTP header out of the response, only the body part - WTF is this component else for instead of raw HTTP requests?!?

- tESBConsumer component with hacks to get to the HTTP header - does somehow not work for me (see https://community.talend.com/t5/Design-and-Development/resolved-SOAP-header-in-tJavaRow/m-p/78503#M3...)

Can someone suggest a solution? Reading out a cookie from a HTTP response Header is not that uncommon and i think Talend should support this kind of stuff out of the box, tell me when i'm not right. My alternative currently is to just use Java Code for building a component which should already exist?!?

[Jesperrekuh · Accepted solution]

Ooooh dear..... well ... uhhhm....  tSystem and cUrl

Make sure the cUrl is called like this, assuming your on some sort unix/linux distro, so in tSystem:

new String[]{"bash","-c","curl -k -L -v -c "+((String)globalMap.get("cookiesFolder"))+"/cookie.txt https://somefreakingurl/login -d 'login=login&username="+context.Usr+"&password="+context.Pwd+"'" }

search internet for tSystem and bash array because of loading the same bash environment when you start your talend job from command line. 

I quit using all of the Talend components... cUrl exactly because of what  you experienced.

[Anonymous]

This isn't a hack, it is just using the HEADERS globalMap value returned by the tESBConsumer component (if headers are returned). The code below should simply list the key values to all headers returned. You should be able to progress from here. This assumes that your tESBConsumer is tESBConsumer_1.

java.util.Map<String,java.util.List<String>> headers = ((java.util.Map<String,java.util.List<String>>)globalMap.get("tESBConsumer_1_HEADERS"));

java.util.Iterator<String> it = headers.keySet().iterator();

while(it.hasNext()){
	System.out.print(it.next());
} 

[Anonymous]

Hello Dijke,

thanks for your reply. I got the tSystem component running in general, but with my curl string (which works fine on the command line) the tSystem component can't handle whitespaces somehow. That is the String i have in my tSystem component:

"curl -k -L -s -c cookie.txt --header \"Content-Type: text/xml;charset=UTF-8\" --header \"SOAPAction: login\" --data @login.xml https://theSoapEndpointIamCalling.com"

so this is my error message:

curl: no URL specified!
curl: try 'curl --help' or 'curl --manual' for more information
/bin/bash: login --data @login.xml https://theSoapEndpointIamCalling.com: No such file or directory
|curl: no URL specified!

it looks like the call is cutted after SOAPAction: and then "login" is called with parameters, which is not the expected behaviour.

Do you have a hint for getting this working? My stupid simple solution would be just calling a bash script which is calling the curl command, but i would like to avoid having additional scripts.

[Anonymous]

Hello rhall_2_0,

i tried the tESBConsumer component solution, but until now i did not managed to get the correct SOAP message for the request.

My SOAP request looks like the following:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:urn="urn:Myprovider_myreport">
   <soapenv:Header/>
   <soapenv:Body>
      <urn:login>
         <user>myuser</user>
         <pass>mypassword</pass>
         <network>mynetwork</network>
      </urn:login>
   </soapenv:Body>
</soapenv:Envelope>

I am just trying to load the XML file with the content from above with a tFileInputXML component, loop over the soap envelope and then push the content from the body in the payload, but i only receive errors from the API stating login errors (when cUrl'ing with that file, everything works fine and i receive a login OK response and a cookie).

Is there something wrong with my input for the tFileInputXML?

btw, I'm using Talend Real-time Big Data Platform 7. I have quite a bunch of similar jobs and was hoping that the tSOAP component is just "complete" in terms of usage, instead I now must go with an tESBConsumer workaround or a tSystem curl workaround - I am already thinking of coding an addition to the tSOAP component which fullfils my needs to access the responding HTTP header for cookie extraction...

Thanks already for all the people replied here, the Talend community is quite active now in comparsion to a few years ago I hope to be able to give something back, I'll scan the incoming questions and will document / explain my solutions here as complete as I can.

[Jesperrekuh]

No idea... 

Did you call cUrl using the array call like my example, or just directly the command?

Seems like they (tSystem vs bash prompt ) differ.

In other examples I've seen the soapaction within quotes, like:

\"SOAPAction: "urn:loginRequested"\"

Maybe play around with single quotes... any documentation on your wsdl part, because your soapaction could also be a full url?

[Anonymous]

OK, first of all you need to see that you are getting the correct data from the tFileInputXML. To do this, remove the tESBConsumer and connect a tLogRow to the tFileInputXML. This will print out anything retrieved from the file to the output. I suspect that you are either not retrieving anything or you may not be retrieving the correct section of the XML. Since your XPath seems to show you want the Body, I actually think you are probably struggling with missing namespace information in your XPaths. There is actually an option on the tFileInputXML's Advanced Settings to ignore namespaces. 

Play around with those settings with the tLogRow connected. Don't connect the tESBConsumer until you are getting the XML snippet you expect

[Anonymous]

Hello rhall_2_0,

i tried to get the payload (and additionally the header, which i do not need - i just need the header of the server response with the Set-Cookie attribute for further API calls) as debug output as seen below. I expected an <urn:login> part, but i just got an <login> part. I used the "Namespace Ignore" option in the additional settings and used the Encoding Type: UTF-8 as the XML file is UTF-8-encoded. Result below.

Without having the "Namespace Ignore" option on, i get a result which is nearly what i wanted - just that i have the additional attribute xmlns:run="urn:blablabla" at the <urn:login> - that is okay (see below).

But I got a leading <?xml version="1.0" encoding="UTF-8"?> which i do not need and which results in errors of the API. I tried that log output directly as the body part with SOAP UI and it gives me Encoding errors; when i remove the <?xml ...?> tag it works fine and gives me a successful login response as expected.

Final (?) question now: How do i remove the <?xml version="1.0" encoding="UTF-8"?> Tag from the payload stream (type document) before it enters the tESBConsumer component?

tESBConsumer_4_debug.png

[Anonymous]

The XML header is added automatically to the XML document to make it XML compliant. Can you tell me whether your payload column (being returned by the tFileInputXML) is a Document or a String? It should be a Document. The ESBConsumer (assuming it has been configured appropriately via the "Service Configuration" button) should be able to handle the column if it is a Document.

[Anonymous]

The Output is a Document. But as a document with a leading <?xml ...?> definition the API will not accept the request and give me errors. My idea was now to use a simple tReplace component to find and delete the <?xml version="1.0" encoding="UTF-8"?> String from the Document, but as the tReplace does not accept Documents as Input, it gives me errors. Is there an easy way or do I need to build a document to string -> replace -> string to document construction for that, and when i need to do that, which components can i use to transform document to string and vice versa?