The Qlik Sense map visualization does not seem to be able to ignore selections. It would be nice if this object type would be able to ignore selections in the expression, in the same way as other objects type in Qlik Sense. It seems like this functionality was supported in earlier versions of Qlik (support case from 2020 here).
NOTE: Upon clicking this link 2 tabs may open - please feel free to close the one with a login page. If you only see 1 tab with the login page, please try clicking this link first: Authenticate me! then try the link above again. Ensure pop-up blocker is off.