If you are not naturally fast have no fear, there are ways that you can build fast relexes and many of these techniques have nothing to do with drumming. You can take up tap dancing or a martial art.
There are also many games that can help you build your reflexes. One of the best I’ve found is a simple little toy called a BOP-IT. Try one!