There are four approaches that you can take
1) Create a master calendar and create a master time table (this can save you a lot of space as it won't have to repeat the same time information for each of the date)
2) Create a master calendar based on date and time
3) Since you only want to create the values that are available in your database, take a distinct resident load of your date and time fields in another table and do all the date time manipulation in that table letting them link on date or time.
4) Do all the manipulation of date and time in your fact table itself.