Skip to main content
Announcements
See what Drew Clarke has to say about the Qlik Talend Cloud launch! READ THE BLOG
cancel
Showing results for 
Search instead for 
Did you mean: 
nathan_roadshow
Contributor
Contributor

Loop through a HTML document

Hi all, I'm trying to loop through various html's, e.g:

Through a local xlsx file which looks like below which is loaded in:

ChannelChannel_LinksMarketHD
ABC1https://www.ontvtonight.com/au/guide/listings/channel/1990/abc1.htmlFree TVFALSE
ABC-HDhttps://www.ontvtonight.com/au/guide/listings/channel/2984/abc-hd.htmlFree TVTRUE
ABC-COMEDY-ABC-KIDShttps://www.ontvtonight.com/au/guide/listings/channel/2043/abc-comedy-abc-kids.htmlFree TVFALSE
ABC-MEhttps://www.ontvtonight.com/au/guide/listings/channel/2096/abc-me.htmlFree TVFALSE
ABC-NEWShttps://www.ontvtonight.com/au/guide/listings/channel/2149/abc-news.htmlFree TVFALSE
SEVENhttps://www.ontvtonight.com/au/guide/listings/channel/1639/seven.htmlFree TVFALSE
7-HDhttps://www.ontvtonight.com/au/guide/listings/channel/2890/7-hd.htmlFree TVTRUE

 

Due to the webpage, i'm getting 'BAD GATEWAY', and then it will end the execution. I am also going through various HTML's and therefore random ones fail and interrupt the chain each time.

For example I am currently running:

For i = 1 to NoOfRows('AUChannelTable')

Let vURL = Peek('Channel_Links',($(i)-1),'AUChannelTable');
Let vChannel = Peek('Channel',($(i)-1),'AUChannelTable');
Let vChannelID = SubField(SubField(vURL,'channel/',2),'/',1);
Let vMarket = Peek('Market',($(i)-1),'AUChannelTable');
Let vHD = Peek('HD',($(i)-1),'AUChannelTable');

Table:
LOAD
now(1) as QVD_Load_Time
, '$(vChannel)' as Channel
, '$(vChannelID)' as CID
, '$(vMarket)' as Market
, '$(vHD)' as HD_Flag
, "Time"
, "TV Show"
, Date(Today(1))&'%'&'$(vChannelID)'&'%'&[TV Show]&'%'&"Time" as Key
FROM [$(vURL)]
(html, codepage is 28591, embedded labels, table is @1);

Next i

 

What would be the best way to write a loop to, if fails, repeat the attempt to access the html?

I will therefore be able to iterate through each row and if fail, retry?

 

Apologies for the difficult question but it has been bugging me!

0 Replies