The Qlik Sense map visualization does not seem to be able to ignore selections. It would be nice if this object type would be able to ignore selections in the expression, in the same way as other objects type in Qlik Sense. It seems like this functionality was supported in earlier versions of Qlik (support case from 2020 here).