topic Re: Loading file names from URL in Connectivity & Data Prep

Loading file names from URL

dafnis14 — Mon, 10 Sep 2018 05:58:21 GMT

Hi,

Mu app loads images from URL based on a string that I build in script for each item:

https://ccc.com/images/upload/products/new-products/3/XXX.jpg

While a URL is defined for each item, not all items have images yet.

I would like to retrieve the existing file names so I can add a filter for the user to select only items that have images.

How can I load the files or their names from the URL?

Thanks!

Re: Loading file names from URL

jonathandienst — Mon, 10 Sep 2018 06:15:54 GMT

There is no built in way to iterate over files in a http location. Does that url supply a directory if you query at a higher level? eg https://ccc.com/images/upload/products/new-products/3/

If so, use this URL to get the directory, parse the content and load the relevant images.

Re: Loading file names from URL

dafnis14 — Mon, 10 Sep 2018 06:46:47 GMT

Hi Jonathan,

When I try to access the link of directory, I get the following error:

You don't have permission to access /images/upload/products/new-products/3/ on this server.

Assuming there are permissions, how can I query and parse the content?

Thank you!

Re: Loading file names from URL

jonathandienst — Tue, 11 Sep 2018 10:14:55 GMT

That normally means that there is no directory. There is no standard governs how the directory appears. If its text, you can load as a text source. Otherwise, use and XML load to load from the web page.

Re: Loading file names from URL

dafnis14 — Tue, 11 Sep 2018 15:06:48 GMT

Hi Jonathan,

Thanks for clarifying.

I would recommend the customer to flag an item with an image in their ERP and I'll use this field for filtering.

Looks to me much easier to handle...:-)