I can only guess here: Since you have tried it and without the WHERE clause, it is fast, I assume that it's not the SQL part causing problems.
What does the timestamp consist of? Is it the usual >> DD.MM.YYYY hh:mm:ss <<?
In that case, that is a so-called high_cardinality field as it has very many different values. Is new data loaded at least once a day? Then you could re-format it and drop the day and just keep the time_part, that might make it faster.
Generally, disintegrating it into a date and a time might make it faster as then there will be significantly fewer different values of that field. That is generally recommended in loading data.