Currently QR only allows user permission management either at server or task level and there is no way to prevent usage of configured endpoints that connects to controlled source or targets.
Having the ability to manage user permissions at endpoint level (source or target) would be key for shared QR servers that are used by multiple teams.