topic sharepoint integration with Talend Studio in Talend Studio

sharepoint integration with Talend Studio

sushantk19 — Fri, 15 Nov 2024 22:49:42 GMT

hello friends,

How can we access sharepoint location (https path) from Talend studio to download one of the files to our local system? what talend studio components could be used?

Re: sharepoint integration with Talend Studio

Anonymous — Thu, 16 Jun 2022 09:27:04 GMT

Hello,

You are able to download a file from SharePoint using a tFileFetch component.

Please take a look at this KB article for details.

https://community.talend.com/s/article/How-to-download-a-file-from-SharePoint-using-tFileFetch-eq7eb

Best regards

Sabrina

Re: sharepoint integration with Talend Studio

sushantk19 — Mon, 20 Jun 2022 07:02:51 GMT

@xdshi: yes, i have seen this article before. I did the steps mentioned as above. i could see my job suceeds and creates a file in destination folder also, but its a blank .xls file. please see the log from talend studio after running this job:

Checking ports...

Sending job 'job_000_Sharepointaccess_Test' to server (52.58.77.6:8004)...

File transfer completed.

Deploying job 'job_000_Sharepointaccess_Test' on server (52.58.77.6:8003)...

Running job 'job_000_Sharepointaccess_Test'...

Starting job job_000_Sharepointaccess_Test at 08:58 20/06/2022.

Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.formatMsgNoLookups=true

[statistics] connecting to socket on port 3823

[statistics] connected

Status Line: HTTP/1.1 200 OK

*** Response Header ***

Cache-Control: no-store, no-cache

Pragma: no-cache

Content-Type: text/html; charset=utf-8

Expires: -1

Strict-Transport-Security: max-age=31536000; includeSubDomains

X-Content-Type-Options: nosniff

X-Frame-Options: DENY

Link: <https://aadcdn.msftauth.net>; rel=preconnect; crossorigin

Link: <https://aadcdn.msftauth.net>; rel=dns-prefetch

Link: <https://aadcdn.msauth.net>; rel=dns-prefetch

X-DNS-Prefetch-Control: on

P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"

x-ms-request-id: d543e8c2-6178-4524-8e37-478b89233300

x-ms-ests-server: 2.1.12890.7 - WEULR2 ProdSlices

X-XSS-Protection: 0

Set-Cookie: buid=0.AV4A-DtSG8LfBU2Tiz8XRdyx3gMAAAAAAPEPzgAAAAAAAABeAAA.AQABAAEAAAD--DLA3VO7QrddgJg7Wevr6nvWIVqm7ilhWBtXss2ULVipc16pv_JfhXq5QcCsqmlB16jWEtyD7G0-wo1LvnY2MFiu50NZ3HMZyJaaE5oXinSRxtf0wW4DKG00or6TshUgAA; expires=Wed, 20-Jul-2022 06:58:35 GMT; path=/; secure; HttpOnly; SameSite=None

Set-Cookie: fpc=AlSS84PARpBKmH_3IALO9vzdRiBqAQAAAJoRQtoOAAAA; expires=Wed, 20-Jul-2022 06:58:35 GMT; path=/; secure; HttpOnly; SameSite=None

Set-Cookie: esctx=AQABAAAAAAD--DLA3VO7QrddgJg7Wevrk-DgCICtEstBxpD3Tlh0INpuLmu2_Gq1HVWlcquqLATy_dhYqZGUW89-XiHSp7iut3ZraEJgYijEBHO0jrn4XXUJSsE195oDNXRt4V2sndhFRaPaJcpxFNQI35E7F5mLOmoV8aPTIh5IQpRDq0i96cf6TcFv5vkURFTV_Mn1ZKQgAA; domain=.login.microsoftonline.com; path=/; secure; HttpOnly; SameSite=None

Set-Cookie: x-ms-gateway-slice=estsfd; path=/; secure; samesite=none; httponly

Set-Cookie: stsservicecookie=estsfd; path=/; secure; samesite=none; httponly

Date: Mon, 20 Jun 2022 06:58:34 GMT

Content-Length: 194664

Jun 20, 2022 8:58:35 AM org.apache.commons.httpclient.HttpMethodBase processResponseHeaders

WARNING: Cookie rejected: "$Version=0; esctx=AQABAAAAAAD--DLA3VO7QrddgJg7Wevrk-DgCICtEstBxpD3Tlh0INpuLmu2_Gq1HVWlcquqLATy_dhYqZGUW89-XiHSp7iut3ZraEJgYijEBHO0jrn4XXUJSsE195oDNXRt4V2sndhFRaPaJcpxFNQI35E7F5mLOmoV8aPTIh5IQpRDq0i96cf6TcFv5vkURFTV_Mn1ZKQgAA; $Path=/; $Domain=.login.microsoftonline.com". Illegal domain attribute ".login.microsoftonline.com". Domain of origin: "login.microsoftonline.com"

[statistics] disconnected

Job job_000_Sharepointaccess_Test ended at 08:58 20/06/2022. [Exit code = 0]

as i am not strong in web technologies, i could undrestand the above log correctly. do you know why would it genererate a blank file? what is missing from my side?

Re: sharepoint integration with Talend Studio

Anonymous — Mon, 20 Jun 2022 09:05:07 GMT

Hello,

Is your url OK with you? I see there is no forbidden error from your job.

Could you please post some screenshots of component settings in this thread?

Best regards

Sabrina

Re: sharepoint integration with Talend Studio

sushantk19 — Mon, 20 Jun 2022 09:25:09 GMT

@Xiaodi Shi : please find the screenshots attached:

Re: sharepoint integration with Talend Studio

sushantk19 — Mon, 20 Jun 2022 13:00:53 GMT

also, is it possible to load the data from sharepoint directly to a relational database table??

Re: sharepoint integration with Talend Studio

Anonymous — Tue, 21 Jun 2022 03:34:49 GMT

Hello,

Are you using https or http Protocol from the list of tfilefetch component?

So far, talend don't support for transferring data by air. You have to download files from sharepoint to local and then load them into a relational database table.

Best regards

Sabrina

Re: sharepoint integration with Talend Studio

sushantk19 — Tue, 21 Jun 2022 06:12:36 GMT

@xdshi: ok thanks. That answers one of my questions. I tried using both http and https. which one should i use?? the result is same whether i use http or https.

Re: sharepoint integration with Talend Studio

Anonymous — Tue, 21 Jun 2022 06:46:12 GMT

Hello,

It should be https Protocol for your URI field. As the URL is quite long, is this URL working fine on browser?

Have you tried to check "Die on error" option in basic settings of tfilefetch component to see if there is any error message printed on console?

Best regards

Sabrina

Re: sharepoint integration with Talend Studio

sushantk19 — Tue, 21 Jun 2022 07:47:33 GMT

1.ok. i used https protocol now.

2.The URL passed in my talend tFetch component is slightly different than the actual URL. I have referred to your below link to get my URL:

https://community.talend.com/s/article/How-to-download-a-file-from-SharePoint-using-tFileFetch-eq7eb

please find the details of URL in attached file.

Let me know if something is wrong here.

3.i unchecked the "die on error", but there is no error in console.

below is the message on console:

Checking ports...

Sending job 'job_000_Sharepointaccess_Test' to server (52.58.77.6:8004)...

File transfer completed.

Deploying job 'job_000_Sharepointaccess_Test' on server (52.58.77.6:8003)...

Running job 'job_000_Sharepointaccess_Test'...

Starting job job_000_Sharepointaccess_Test at 09:42 21/06/2022.

Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.formatMsgNoLookups=true

[statistics] connecting to socket on port 3432

[statistics] connected

Jun 21, 2022 9:42:54 AM org.apache.commons.httpclient.HttpMethodBase processResponseHeaders

WARNING: Cookie rejected: "$Version=0; esctx=AQABAAAAAAD--DLA3VO7QrddgJg7Wevr0n67-qbnLKU5khlnzSJkU6Ov-E3LJFkFarkZ80WU5DcPsOKxpWYtlopLYSQCmV0y2MYBnN5kuSvfe3a7dNvPbtTt9LAuwMIzePwnUqlLU1W0_j2wQMedTjieKy9wdNykRpC12KWZUrybXK4HsMcV5H9_X4PoPOs90C4ggcTfiaogAA; $Path=/; $Domain=.login.microsoftonline.com". Illegal domain attribute ".login.microsoftonline.com". Domain of origin: "login.microsoftonline.com"

[statistics] disconnected

Job job_000_Sharepointaccess_Test ended at 09:42 21/06/2022. [Exit code = 0]

Re: sharepoint integration with Talend Studio

sushantk19 — Tue, 21 Jun 2022 07:52:13 GMT

also, i am using google chrome as my browser to access this link. does that make a difference if i pass "Mozilla/5.0" in my header??

Re: sharepoint integration with Talend Studio

Anonymous — Tue, 21 Jun 2022 08:30:04 GMT

Hello,

"User-Agent" header is OK with your use case.

Is this website available in google chrome?

Does it work well when using tHttpRequest component?

Best regards

Sabrina

Re: sharepoint integration with Talend Studio

sushantk19 — Tue, 21 Jun 2022 09:46:57 GMT

yes. website is available in google chrome.

when i run this URL using thttprequest component(without checking the die on error option), it still generates a corrupted file. the log is shown below:

Sending job 'job_000_Sharepointaccess_Testing1' to server (52.58.77.6:8004)...

File transfer completed.

Deploying job 'job_000_Sharepointaccess_Testing1' on server (52.58.77.6:8003)...

Running job 'job_000_Sharepointaccess_Testing1'...

Starting job job_000_Sharepointaccess_Testing1 at 11:44 21/06/2022.

Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.formatMsgNoLookups=true

[statistics] connecting to socket on port 3917

[statistics] connected

[statistics] disconnected

Job job_000_Sharepointaccess_Testing1 ended at 11:44 21/06/2022. [Exit code = 0]

Re: sharepoint integration with Talend Studio

sushantk19 — Tue, 21 Jun 2022 09:55:38 GMT

i have tried using both the components thttprequest and tFetch but still unsuccessful.

Re: sharepoint integration with Talend Studio

Anonymous — Wed, 22 Jun 2022 03:58:58 GMT

Hello,

What does your downloaded file look like? We are suspecting the format could not supported by tfilefetch component.

Best regards

Sabrina

Re: sharepoint integration with Talend Studio

sushantk19 — Wed, 22 Jun 2022 06:19:19 GMT

as seen from my talend component screenshot, i am trying to generate .xlsx file(please see attached). i also tried to generate .xls. it opens but without any data. isnt .xls or .xlsx supported?

Re: sharepoint integration with Talend Studio

sushantk19 — Thu, 23 Jun 2022 06:35:41 GMT

@Xiaodi Shi : do you have any further updates on this?

Re: sharepoint integration with Talend Studio

Anonymous — Thu, 23 Jun 2022 06:59:39 GMT

Hello,

I'm discussing your issue with our team and then come back to you as soon as possible.

Best regards

Sabrina

Re: sharepoint integration with Talend Studio

sushantk19 — Tue, 28 Jun 2022 14:44:29 GMT

@xdshi: do you have any update on this ?