Throw those users-to-be-denied-access out of the AD group(s)?
Or better: create a special distribution group per document in AD and add that group to the Distribution list instead of groups that are too permissive? Disadvantage: if you don't take care of AD, you'll have to ask a sysadmin to add/remove people from those groups...
And probably best but not the easiest technique: use Section Access and load permissions from your own DB. That way, you still have the management of document access rights in your own hands instead of leaving it in the hands of sysadmins or any other IT person that takes care of AD?
I dont think there is a "Deny" list when doing distributions from the QMC.
I agree with Peter that you should create a group with all users who should have access. If thats not pratical you could use Section access and read users from the ADgroup and then use an CSV or Excelfile to eclude specific users