You could use the option Rajesh mentioned, but then your other charts will respond to nothing anymore and become static charts.
Since the purpose of Qlik Sense is to create interactive and responsive dashboards, a better option is to just exclude the filters used in the drill-down.
Lets say you have a drill-down Productgroup > Product. You must add "Productgroup=, Product=," to the set analysis in your measure expression to achieve this functionality. Add this to every chart which you don't want to react to the active filters.