When you distribute a document, the Publisher creates the QVW file with the name of the distribution task (or whatever name you set in the Distribution Task). This qvw is saved on the server, usually in a ServerMountedQVW folder or wherever you point your access point.
So, to delete the older named version of the document, you have to find that folder that contains the distributed documents and delete all the associated files. For example, if you distribution task is called Sales_Application (and that is the name as it appears on the Access Point), then you will need to delete all the files starting with Sales_Application... in the folder where Publisher is distributing. (Don't delete the QVW that you are developing!)
Hope this helps.