topic Re: TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem in Talend Studio

TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

Anonymous — Tue, 08 May 2012 13:37:04 GMT

Mission:
Download with Talend Components tFileFetch CSV Reports from a Redmine System
Redmine needs Login, is configured to use HTTPS and has redirects
tried Implementation#1:
tFileFetch_1
using POST, redirect=true, PostVars with LoginCredentials, save Cookie, single Cookie true, Url: Login Page
|
|
onSubjobOK Link
|
|
tFileFetch_2
using Get, readCookie, URL: Redmine Path to the CSV data
ISSUE #1: There isContinue_tFileFetch_1 an exception on: https://XXX/redmine/login
java.lang.Exception: Method failed: HTTP/1.1 500 Internal Server Error
tried Implementation#2:
same as #1 without onSubjobOK Link
#ISSUE #2: tFileFetch_2 causes an exception "Method failed: HTTP/1.1 406 Not Acceptable"

QUESTIONS:
How to implement this job scenario?

WIN 7 - Version: 5.0.2 Build id: r78327-20120216-0244 - JAVA

Re: TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

Anonymous — Wed, 09 May 2012 07:17:50 GMT

Hi
Just use only one tFileFetch.
Need Authentication: true.
Support Redirection: true.
Besides, the URI shouldn't be Login Page. It should be like this "http://talendforge.org/file_fetch.txt".
Regards,
Pedro

Re: TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

Anonymous — Wed, 09 May 2012 09:00:38 GMT

Hi Pedro,
thanks for your reply.
I already did this in my first tries.
tFileFetch:
Authentication (unnecessary as it is HTTP Authentication but anyway ticked on): Login Credentials
Use Post: provided Logincredentials + plus redirection url info to Url for downloading the csv
Write / read cookie (write for login, read for the redirect)
ticked on redirect
I did also test a lot of scenarios e.g. with the url:
- direct CSV download URL
- login URL and redirect url
- etc
But this implementation did not work. I encountered the issue, that the session cookie was not set properly and the redirect failed. Therefore I started the test for an implementation similiar as mentioned in the Component Guide PDF, provided by Talend.
Status: Issue is still unresolved

Re: TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

Anonymous — Wed, 09 May 2012 09:11:16 GMT

Hi
Could you tell me some info to reproduce this issue?
Or I can figure out how to fix it.
Regards,
Pedro

Re: TOS 5.X tFileFetch dowloading CSV from Redmine System HTTPS problem

Anonymous — Wed, 09 May 2012 13:27:43 GMT

Hi Pedro
I did following tests:
Using one tFileFetch and configured as following:
https
PostMethod
PostVariables: Logincredentials as HTML Formfieldnames from the LoginPage
Save Cookie: true (creation by Login)
Read Cookie: true (reading while redirecting to requested page)
Single Cookie header: true
Response to Console: true
Support Redirect: true <--- check Exception Test 2
Additional: Authentication on, Login Credentials filled out
Test1: URL to get the CSV of Tickets for a particular Tracker:
URL: https://XXX/redmine/projects/ABC/issues.csv?c%5B%5D=project&c%5B%5D=tracker&c%5B%5D=status&c%5B%5D=priority&c%5B%5D=subject&c%5B%5D=author&c%5B%5D=assigned_to&c%5B%5D=updated_on&c%5B%5D=category&c%5B%5D=cf_16&f%5B%5D=status_id&f%5B%5D=tracker_id&f%5B%5D=&group_by=&op%5Bstatus_id%5D=%2A&op%5Btracker_id%5D=%3D&set_filter=1&v%5Bstatus_id%5D%5B%5D=1&v%5Btracker_id%5D%5B%5D=18

Result: Exception:
There isContinue_tFileFetch_1 an exception on: https://#####.....
java.lang.Exception: Method failed: HTTP/1.1 406 Not Acceptable
at jobexport.loadredmine_0_1.LoadRedmine.tFileFetch_1Process(LoadRedmine.java:517)
at jobexport.loadredmine_0_1.LoadRedmine.runJobInTOS(LoadRedmine.java:834)
at jobexport.loadredmine_0_1.LoadRedmine.main(LoadRedmine.java:691)

Test 2: Url to Webpage with the Tickets and try to save the Webpage
URL: https://XXX/redmine/projects/YYY/issues?set_filter=1
Result: Exception:
org.apache.commons.httpclient.HttpMethodDirector isRedirectNeeded
INFO: Redirect requested but followRedirects is disabled
Redirect to: https://XXX/redmine/login?back_url=https%3A%2F%2FXXX%2Fredmine%2Fprojects%2FYYY%2Fissues
Status Line: HTTP/1.1 200 OK
*** Response Header ***
Date: Wed, 09 May 2012 12:15:02 GMT
Server: Mongrel 1.1.5
Status: 200
ETag: "53d25a9c8844ce24f82e333c2b1c45af"
X-Runtime: 12
Content-Type: text/html; charset=utf-8
Content-Length: 3794
Cache-Control: private, max-age=0, must-revalidate
Set-Cookie: _redmine_session=BAh7CDoQX2NzcmZfdG9rZW4iMTFCNDRsM1lsUS9lMVlZblFRcmpickFOWWpXN3NVVjI5dkYwRWs4aXdZUkU9Og9zZXNzaW9uX2lkIiU4Y2U2MWNhNmNlYjA2ZWM3NzJjNzcwMjczOTk0NWRhZCIKZmxhc2hJQzonQWN0aW9uQ29udHJvbGxlcjo6Rmxhc2g6OkZsYXNoSGFzaHsGOgplcnJvciIqQmVudXR6ZXIgb2RlciBLZW5ud29ydCBpc3QgdW5nw7xsdGlnLgY6CkB1c2VkewY7CFQ%3D--30b10957e5df0449265ba58a6b25203bbf8f0173; path=/; HttpOnly
I got a 200 OK Response for a LoginPage on which I was redirected as I have to log in first. (Kindly note the url addition back_url)
Thanks For Support