If I am understanding you correctly, your Qlik System is running slowly and this is not a Qlik App issue.
I think you want to add a filter to your User Directory Connector.
You should look at the advanced section of the UDC configuration and add a filter string in the LDAP Filter Field.
Building the filter strings are tricky but your AD administrator should be able to help
We have a group named QV_Users that includes anybody who has access to the system