the variable is not the problem... Date use in Set Analysis is very tricky. In order to compare dates in set analysis, you have to format the value EXACTLY the same way your Date Field it formatted - I mean, even the difference between 'M/D/YYYY' and 'MM/DD/YYYY' can break it.
If your Date Field "ObserverDate" is formatted as 'MM/DD/YYYY' (look it up in Document Properties, Number) - then format your variable value the same way:
date(varLookBackDate +(only (FacilityTimezoneOffset) / 24) - (varHours / 24), 'MM/DD/YYYY')
Then it should work, provided that everything else is in order.