Simply put, the file is there because it exists in the server folder (i.e. whatever folder is defined in the QVS --> Folders tab). You can either manually delete it or remove all users' security permissions to the file, which would also result in it not being displayed in the AP. If the files are named logically, you can also write a batch script that would delete old files. I hope I understood your question correctly...
Create a new mount in the AP. This mount will contain all the files that will be outputted daily, as well as all the files from the day before.
Go to QEMC --> System --> Supporting Tasks tab. Expand the tree and create a new External Program task with the following Command Line Statement: del "d:\test\*.*" /Q (where d:\test is replaced by the path to the mount created in Step #1). Give the task a name. Under the Triggers tab, create a Schedule trigger that will run the task on the same schedule as you are currently using to reduce & distribute documents. Save the task.
Modify your Publisher task to output the reduced files to the new mount.
Inside the Publisher task, click the Triggers tab. Delete the existing Schedule trigger, and add a new trigger that will run On Event From Another Task. Make the task run on Successful completion of the new task you created in Step #2.
If you would like Publisher to reduce and distribute the documents even if deletion of the files failed for some reason, create a second trigger that will make the task run on Failed completion of the above task.
Save the Publisher task.
That should do the trick--all the files from the previous day will be deleted before new ones are outputted.