sharepoint integration with Talend Studio

sushantk19 · ‎2022-06-15

hello friends,

How can we access sharepoint location (https path) from Talend studio to download one of the files to our local system? what talend studio components could be used?

Anonymous · ‎2022-06-16

Hello,

You are able to download a file from SharePoint using a tFileFetch component.

Please take a look at this KB article for details.

https://community.talend.com/s/article/How-to-download-a-file-from-SharePoint-using-tFileFetch-eq7eb

Best regards

Sabrina

sushantk19 · ‎2022-06-20

@xdshi: yes, i have seen this article before. I did the steps mentioned as above. i could see my job suceeds and creates a file in destination folder also, but its a blank .xls file. please see the log from talend studio after running this job:

Checking ports...

Sending job 'job_000_Sharepointaccess_Test' to server (52.58.77.6:8004)...

File transfer completed.

Deploying job 'job_000_Sharepointaccess_Test' on server (52.58.77.6:8003)...

Running job 'job_000_Sharepointaccess_Test'...

Starting job job_000_Sharepointaccess_Test at 08:58 20/06/2022.

Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.formatMsgNoLookups=true

[statistics] connecting to socket on port 3823

[statistics] connected

Status Line: HTTP/1.1 200 OK

*** Response Header ***

Cache-Control: no-store, no-cache

Pragma: no-cache

Content-Type: text/html; charset=utf-8

Expires: -1

Strict-Transport-Security: max-age=31536000; includeSubDomains

X-Content-Type-Options: nosniff

X-Frame-Options: DENY

Link: <https://aadcdn.msftauth.net>; rel=preconnect; crossorigin

Link: <https://aadcdn.msftauth.net>; rel=dns-prefetch

Link: <https://aadcdn.msauth.net>; rel=dns-prefetch

X-DNS-Prefetch-Control: on

P3P: CP="DSP CUR OTPi IND OTRi ONL FIN"

x-ms-request-id: d543e8c2-6178-4524-8e37-478b89233300

x-ms-ests-server: 2.1.12890.7 - WEULR2 ProdSlices

X-XSS-Protection: 0

Set-Cookie: buid=0.AV4A-DtSG8LfBU2Tiz8XRdyx3gMAAAAAAPEPzgAAAAAAAABeAAA.AQABAAEAAAD--DLA3VO7QrddgJg7Wevr6nvWIVqm7ilhWBtXss2ULVipc16pv_JfhXq5QcCsqmlB16jWEtyD7G0-wo1LvnY2MFiu50NZ3HMZyJaaE5oXinSRxtf0wW4DKG00or6TshUgAA; expires=Wed, 20-Jul-2022 06:58:35 GMT; path=/; secure; HttpOnly; SameSite=None

Set-Cookie: fpc=AlSS84PARpBKmH_3IALO9vzdRiBqAQAAAJoRQtoOAAAA; expires=Wed, 20-Jul-2022 06:58:35 GMT; path=/; secure; HttpOnly; SameSite=None

Set-Cookie: esctx=AQABAAAAAAD--DLA3VO7QrddgJg7Wevrk-DgCICtEstBxpD3Tlh0INpuLmu2_Gq1HVWlcquqLATy_dhYqZGUW89-XiHSp7iut3ZraEJgYijEBHO0jrn4XXUJSsE195oDNXRt4V2sndhFRaPaJcpxFNQI35E7F5mLOmoV8aPTIh5IQpRDq0i96cf6TcFv5vkURFTV_Mn1ZKQgAA; domain=.login.microsoftonline.com; path=/; secure; HttpOnly; SameSite=None

Set-Cookie: x-ms-gateway-slice=estsfd; path=/; secure; samesite=none; httponly

Set-Cookie: stsservicecookie=estsfd; path=/; secure; samesite=none; httponly

Date: Mon, 20 Jun 2022 06:58:34 GMT

Content-Length: 194664

Jun 20, 2022 8:58:35 AM org.apache.commons.httpclient.HttpMethodBase processResponseHeaders

WARNING: Cookie rejected: "$Version=0; esctx=AQABAAAAAAD--DLA3VO7QrddgJg7Wevrk-DgCICtEstBxpD3Tlh0INpuLmu2_Gq1HVWlcquqLATy_dhYqZGUW89-XiHSp7iut3ZraEJgYijEBHO0jrn4XXUJSsE195oDNXRt4V2sndhFRaPaJcpxFNQI35E7F5mLOmoV8aPTIh5IQpRDq0i96cf6TcFv5vkURFTV_Mn1ZKQgAA; $Path=/; $Domain=.login.microsoftonline.com". Illegal domain attribute ".login.microsoftonline.com". Domain of origin: "login.microsoftonline.com"

[statistics] disconnected

Job job_000_Sharepointaccess_Test ended at 08:58 20/06/2022. [Exit code = 0]

as i am not strong in web technologies, i could undrestand the above log correctly. do you know why would it genererate a blank file? what is missing from my side?

Anonymous · ‎2022-06-20

Hello,

Is your url OK with you? I see there is no forbidden error from your job.

Could you please post some screenshots of component settings in this thread?

Best regards

Sabrina

sushantk19 · ‎2022-06-20

@Xiaodi Shi : please find the screenshots attached:

sushantk19 · ‎2022-06-20

also, is it possible to load the data from sharepoint directly to a relational database table??

Anonymous · ‎2022-06-20

Hello,

Are you using https or http Protocol from the list of tfilefetch component?

So far, talend don't support for transferring data by air. You have to download files from sharepoint to local and then load them into a relational database table.

Best regards

Sabrina

sushantk19 · ‎2022-06-21

@xdshi: ok thanks. That answers one of my questions. I tried using both http and https. which one should i use?? the result is same whether i use http or https.

Anonymous · ‎2022-06-21

Hello,

It should be https Protocol for your URI field. As the URL is quite long, is this URL working fine on browser?

Have you tried to check "Die on error" option in basic settings of tfilefetch component to see if there is any error message printed on console?

Best regards

Sabrina

sushantk19 · ‎2022-06-21

1.ok. i used https protocol now.

2.The URL passed in my talend tFetch component is slightly different than the actual URL. I have referred to your below link to get my URL:

https://community.talend.com/s/article/How-to-download-a-file-from-SharePoint-using-tFileFetch-eq7eb

please find the details of URL in attached file.

Let me know if something is wrong here.

3.i unchecked the "die on error", but there is no error in console.

below is the message on console:

Checking ports...

Sending job 'job_000_Sharepointaccess_Test' to server (52.58.77.6:8004)...

File transfer completed.

Deploying job 'job_000_Sharepointaccess_Test' on server (52.58.77.6:8003)...

Running job 'job_000_Sharepointaccess_Test'...

Starting job job_000_Sharepointaccess_Test at 09:42 21/06/2022.

Picked up JAVA_TOOL_OPTIONS: -Dlog4j2.formatMsgNoLookups=true

[statistics] connecting to socket on port 3432

[statistics] connected

Jun 21, 2022 9:42:54 AM org.apache.commons.httpclient.HttpMethodBase processResponseHeaders

WARNING: Cookie rejected: "$Version=0; esctx=AQABAAAAAAD--DLA3VO7QrddgJg7Wevr0n67-qbnLKU5khlnzSJkU6Ov-E3LJFkFarkZ80WU5DcPsOKxpWYtlopLYSQCmV0y2MYBnN5kuSvfe3a7dNvPbtTt9LAuwMIzePwnUqlLU1W0_j2wQMedTjieKy9wdNykRpC12KWZUrybXK4HsMcV5H9_X4PoPOs90C4ggcTfiaogAA; $Path=/; $Domain=.login.microsoftonline.com". Illegal domain attribute ".login.microsoftonline.com". Domain of origin: "login.microsoftonline.com"

[statistics] disconnected

Job job_000_Sharepointaccess_Test ended at 09:42 21/06/2022. [Exit code = 0]

Azure

Studio

v7.x