Since access is granted rather than denied you will need to disable the security rules: StreamEveryone and StreamEveryoneAnonymous which will kill access to Everyone. The complicated issue here is that you will then need to create a security rule which grants access to Everyone for all the other users. So short answer: It's possible, but will be difficult to maintain. It seems easier to just not publish to the Everyone stream, in my opinion.
