topic Re: how extract a table that exists in url in Talend Studio

how extract a table that exists in url

Anonymous — Sat, 16 Nov 2024 05:37:40 GMT

extract a URL table
hello to all
I want to extract a table that exists in the following url,

https://security-tracker.debian.org/tracker/status/itp

which component I can use with which configuration

How I would like the output to look like:

package

issue

debainbug

I thank you in advance for your time and help

Re: how extract a table that exists in url

fdenis — Tue, 11 Jun 2019 11:21:39 GMT

trest is a good one

Re: how extract a table that exists in url

Anonymous — Tue, 11 Jun 2019 12:42:29 GMT

just as @fdenis said, I suggest tREST to get the web page,
tFileInputRegex to parse the file and extract columns as you want

Or, you check on exchange.talend.com to find suitable component for web scraping.

Re: how extract a table that exists in url

Anonymous — Tue, 11 Jun 2019 12:48:50 GMT

I thank you for your answer but by using trest I couldn't extract a table from the site https://security-tracker.debian.org/tracker/status/itp and save it in my database

Re: how extract a table that exists in url

Anonymous — Tue, 11 Jun 2019 13:30:33 GMT

I thank you for your answer

I checked on exchange.talend.com ; I would like to use tHttpTableInput to get the table in the below link https://security-tracker.debian.org/tracker/status/itp

but I didn't succeed I don't know how I configure this component

regards

Re: how extract a table that exists in url

fdenis — Tue, 11 Jun 2019 13:43:53 GMT

what is the error message?

Re: how extract a table that exists in url

Anonymous — Tue, 11 Jun 2019 14:03:50 GMT

it's my site and the table I want to extract https://security-tracker.debian.org/tracker/status/itp

it's the configuration : thanlks

regards

Re: how extract a table that exists in url

fdenis — Tue, 11 Jun 2019 15:24:37 GMT

I cannot get this component from open studio 7

Re: how extract a table that exists in url

Anonymous — Tue, 11 Jun 2019 15:27:48 GMT

it's the configurationDownload tHTTPTableInput component from https://exchange.talend.com to extract URL data because Talend doesn't have a standard component for it

Step 1: Once you have downloaded the component from http://talendforge.org/exchange/, simply add it to the folder \talend\plugins\org.talend.designer.components.localprovider_7.1.1.1.20181026_1147\components
of Talend, then relaunch Talend.
Step 2: put this tHTTPTableInput in a specific folder, i. e. my components.
Then: Under Windows / Preferences, under Talend / Components, select the myComponents folder as the User component folder.
Click on Apply, then Ok.

Re: how extract a table that exists in url

fdenis — Tue, 11 Jun 2019 15:31:15 GMT

so to have support on this get file by trest then open it as xml it will be more processionnal.

Re: how extract a table that exists in url

Anonymous — Tue, 11 Jun 2019 15:43:21 GMT

I am not sure what you mean by this. Can you explain this with the photo please

thanks

regards

Re: how extract a table that exists in url

Anonymous — Tue, 11 Jun 2019 15:43:50 GMT

I think it will be difficult for you to do data scraping with talend.

Data scraping depend on the struture of the html page (the attached file is the structure of your web page)

I don't think you can find any generic component to do what you are trying to do.

I suggest you go on freelancer.com, get someone who can provide to you a simple java code to scrape that page (it will not work on other pages), then you can integrate it in talend.

Re: how extract a table that exists in url

Anonymous — Wed, 12 Jun 2019 14:36:33 GMT

Hi,

I could see that you are currently planning for JSON parsing of the message which is the right approach rather than trying to parse from the output HTML page.

https://community.talend.com/t5/Design-and-Development/parsing-and-extract-json/m-p/164151#M99914

So I hope you do not have any outstanding query in this post.

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved 🙂

Re: how extract a table that exists in url

Anonymous — Wed, 12 Jun 2019 15:35:37 GMT

thank you for your attention, in fact these are two different things I need data extraction from a json file and also I need data extraction from url and also I need data extraction from pdf in different sites and in the end I will integrate all the tables I have extracted so for me it is necessary that I find a way to extract the data or by using the tHTTPInputtable component or in other way

regards