I have sorted it now, bit of a bodge to be honest.
If there is a neater way I would like to hear.
As we are talking days of the week (small number and not likely to change any time soon) I made an inline table with the days and used that as the dimension. In the expression I did the following:
=sum(if(DayName = countDayInline, aggr(sum(ActivityCount), DayName, OpeningHours), 0))
Seems to do the job.