AFAIK that's not possible with ad groups. Are the user member of a ad group which aren't allowed to access this sheet you couldn't give them access in any way - one access denial meant it's always denied regardless if there are further authorizations.
This meant you need to use single user instead of user groups (this mustn't be done manually - there are ways to read an ad, for example Search Recipes | Qlikview Cookbook) or more practically by 5 users: you used a visibility-condition for this sheet like:
if(match(osuser(), 'user1', 'user2', ...), true(), false())
So if I can resume, here are the possibilities:
1) We use publisher -> we create manual groups (you can't use AD groups)
2) We have an enterprise edition (not SBE) and we create manual groups (you can't use AD groups)
3) We have SBE edition (no DMS possible) and we add users one by one
I wonder why we can't use AD groups, it would be so much easier as it's already set up and no risk of error. It could directly be managed by help desk.