What is the url and hostname for the custom login page? Are you inputting the full url address with https? You will want to do this.
Is your custom login page hosted the same server as the QlikView web server components or does it have a certificate trust from the web server hosting the custom login page to the QlikView web server components?
What is the domain the certificate is trusted to? If you are going to localhost and the certificate isn't named for localhost then you will get the privacy warning.
You may want try navigating to the fully qualified domain of the server to see if that resolves the certificate errors.