topic Re: robots.txt in Management & Governance

robots.txt

implanet — Sat, 16 Nov 2024 20:43:33 GMT

Hello,

Our qlik sense installation can be reached from outside of the company trought firewall NAT rules.

How to add a robots.txt file to prevent any crawling from search bot?

Thanks.

Re: robots.txt

Giuseppe_Novello — Wed, 05 Jun 2019 15:02:45 GMT

With Qlik Sense out of the box, I do not think it would be possible. You might need to implement a reverse proxy with that, that utilize IIS and with IIS you can implement Robot.txt in "inetpub/wwwroot/" folder. As Far I know, we do not carry documentation to do so, it is best to reach your Qlik Account manager so you can have a conversation with a pre-sale architect or consulting services.

Gio

Re: robots.txt

implanet — Wed, 05 Jun 2019 15:13:07 GMT

Thank you Giuseppe for your reply.

It makes sense even if I didn't want to take the reverse proxy way to implement this.

Maybe this feature could be added in the next version, could you take it care of it?

Thanks again.

Re: robots.txt

Giuseppe_Novello — Wed, 05 Jun 2019 15:23:55 GMT

I can make an ideation; however, I know that it won't fly as much, since their option, also speaking with another colleague you can also do the same with Nginx. Once you make the URL public facing it is matter of the customer's security team to implement measure to prevent security issues. Anyways, I'll submit it, but I can't promise it will fly very high.

Gio

Re: robots.txt

Giuseppe_Novello — Wed, 05 Jun 2019 15:28:08 GMT

I have submitted the idea: https://qlik.ideaglow.com/forums/idea/0/All-Ideas/2674/Qlik-Sense-and-Robots-txt

Note this is internal site, so you won't be able to see it, but it for reference.

gio

Re: robots.txt

implanet — Wed, 05 Jun 2019 15:42:00 GMT

You're right about it's the matter of the customer to implement the security.

But as Qlik provide a web server engine bundled in the product, it would make sense they provide also the tools to manage it correctly.

Maybe a solution could be to separate the web server from the whole qlik sense product, maybe as an option during the installation (some people prefering IIS or apache or nginx) and this would easy up the web maintenance.

BTW thanks again for your feedback.

AB.