As long as any logging action means access to write (and read) on disk, the more you log the slower your system will go. But, what the actual impact in a real life scenario? It will depend on your hardware, specially hard disk specifications or network capabilities if you are using a shared resource. If concurrency is low and number of charts is small, that should not be a problem at all, or vice versa, if there is a lot of users at the same time using a huge number of docs with lots of charts, logging may cause the system to slow down.
I'd strongly recommend you to take a look at the QlikView System Monitor. It's a really powerful application that reads your server logs and draws charts based on that info. Plus, you can modify it to fit your needs.
Hope that helps.