topic Re: Getting json response from url. in Talend Studio

Getting json response from url.

Anonymous — Sat, 16 Nov 2024 12:26:22 GMT

Hi all,
I am trying to get a json response by connecting to a url. Using tJson with URL selected yields a 401 error. Using tRest yields a 401 error as well. However the exact same url string returns me the correct json response in both curl and through my web browser. Not sure what else to try.
Here is the format of the URL I have used:
"http://user ass@myurl.com/api/v1/products"
"https://user ass@myurl.com/api/v1/products"

Any help would be greatly appreciated.

Re: Getting json response from url.

Anonymous — Thu, 19 Jan 2012 19:38:03 GMT

You may have a problem with escape chars ( / ).

Re: Getting json response from url.

Anonymous — Thu, 19 Jan 2012 20:02:09 GMT

Thanks saburo for the idea.
I did use tFilefetch with this string " http://myurl.com/api/v1/products" and selecting Need Authentication in the advanced field worked. Which solves the problem of getting json but does not solve the problem of uploading a json file.
I tried tRest does not have the Need Authentication and doing "http://user ass@myurl.com/api/v1/products" gives me a 401. In tRest I can specify what I send back in the http body but it won't authenticate..

I did try escaping the slashes and same thing. If it was an escape issue would forward slashes still cause it???

Re: Getting json response from url.

Anonymous — Thu, 19 Jan 2012 20:53:20 GMT

I found weird results with forward and backslashes, never really figured out the exact rule.
Something is happening sometimes because forward slashes are converted into backslashes for cross system compatibility (I guess).
Anyhow, it's not your case since with the same url you managed to get the json, plus 401 is specific to authentication.
REST authentication seems to be a bit more complex than sending username assword@
see here
http://www.berenddeboer.net/rest/authentication.html
and here :
https://developer.atlassian.com/display/FECRUDEV/Writing+a+REST+Client+in+Perl
I never used rests, so I cannot help with this one, but based on what I saw in the links, it seems you need to setup specific authentication headers which will contain username and password in base64 encoding...
If the component is not designed to do that... well, it may require a bit of work to extend it for that purpose.

Re: Getting json response from url.

Anonymous — Thu, 19 Jan 2012 21:07:51 GMT

Aha...
I have been trying to set the http headers but I don't really know what to put there and the RFC 2617 which explained it left me a little bit more confused. The base64 encoding is not a problem because I already have the string encoded.
In the tRest component I have tried:
URL : " https://myurl.com/api/v1/products/"
HTTP METHOD: POST
HTTP HEADERS:
name value
"Authorization" "Basic encodedUserName:encodedPassword" -> I know these are the right credentials because copy paste onto browser returns json string as well as doing it through curl.
Http Body:
{
price:19.99
}
But I get a 401. Does any one know/understand how to pass the credentials in an HTTP header?
By the way thank you for taking the time to help me out I really appreciate it.

Re: Getting json response from url.

Anonymous — Tue, 24 Jan 2012 00:04:07 GMT

I think I have exhausted other options.
So to write back a message via REST architecture I used the tJava component with the following code in it:

try {
 
		java.net.URL url = new java.net.URL("https://yourURL.com/api/v1/products/1/");//the 1 is the product id
		
		java.net.HttpURLConnection conn = (java.net.HttpURLConnection) url.openConnection();
		conn.setDoOutput(true);
		conn.setRequestMethod("PUT");
		conn.setRequestProperty("Content-Type", "application/json");
	
		
		String username = "someUser";
		String password = "somePassword";
		
	sun.misc.BASE64Encoder enc = new sun.misc.BASE64Encoder();
        String userpassword = username + ":" + password;
        String encodedAuthorization = enc.encode( userpassword.getBytes() );
        conn.setRequestProperty("Authorization", "Basic "+encodedAuthorization);
 		conn.setAllowUserInteraction(true);
 		
 		String input = "{\"available\":\"14\",\"price\": \"14.99\"}";
		
 
		java.io.OutputStream os = conn.getOutputStream();
		os.write(input.getBytes());
		os.flush();
		os.close();
		/*if (conn.getResponseCode() != java.net.HttpURLConnection.HTTP_CREATED) {
			throw new RuntimeException("Failed : HTTP error code : "
				+ conn.getResponseCode());
		}
 		
 	
 		*/
		java.io.BufferedReader br = new java.io.BufferedReader(new java.io.InputStreamReader(
				(conn.getInputStream())));
 		
		String output;
		System.out.println("Output from Server .... \n");
		while ((output = br.readLine()) != null) {
			System.out.println(output);
		}
 
		conn.disconnect();
 	}
	 catch (IOException e) {
 
		e.printStackTrace();
 
	 }

I hope someone finds this useful. Currently I don't have the time nor the know-how to extend the tRest component to make it work with credentials. But hopefully higher powers can.