but still Firefox and Safari might do their own thing, especially Safari which is known to be couple years late to implement features.
If you want to have full control over the player, then you need to remove the control attribute, and add your own controls with html, css and javascript. This method requires a good level of javascript especially, as video can be a pain to deal with and playback errors must be handled properly.