You can do the following to improve the performance.
1. While final load just load the columns you required.
2. Create 2 separate dashboards 1 for aggregation and one for detailed data.
3. Create QVD with raw data and process the data loaded from QVD's and then created another version of QVD's with aggregated or any other additional columns required.
4. Now use the final qvd's created above in the 2 separate dashboards 1 for aggregation and another for Detailed data.