This sounds like something on the mac side of things. I did a quick google on the error message and came up with this:
sorry to say my mac skills are lacking lately I was an old school mac user back in college (Mac Centris 650 :-) http://www.everymac.com/systems/apple/mac_centris/stats/mac_centris_650.html )
I just tested on my girlfriends iMac running OSX Leopard v10.5.6 (Java applet; v18.104.22.168_16) and I do not get the same results as you do. It works fine for me when i go to http://demo.qlik.com and choose any of the java versions of the demos.
Can you verify the versions of java and mac os you are currently running?
I have solved the issue. There are a few steps to follow :
1-Il the local policies -> security -> Network Lan manager -> choose LM and NTLM protocol
2-In the QVJava use the basic authentication+anonymous (to avoid double auth NT and Java)
2-Force the domain to be the Server machine name otherwise Safari on Leopard will use the IP which is not OK