A lot of times, I ride Gelato in the freewheel mode (i.e., single speed), especially if I am going after the big hills.
That means that I'm just riding with one brake, and that would be the front brake.
But that's not really a problem you see.
Fixed gear riding is all about control - speed up or slow down by applying pressure to the pedals.
Single speed is more about momentum - keep your speed up so that you can make it over the next big hill.
So when Gelato and I are freewheeling it, I really don't use the brake except in heavy traffic because braking just slows you down anyways.
Make momentum your friend and ease up on the brake(s).