Removing the Lock Button on Lee Breech Lock Challenger - But Should You?

I'm removing the Spring Loaded Retainer/Release Button on my Lee Breech Lock Challenger Press, but should you have it removed as well? In my opinion, if you are using the Lee Spline Drive Bushing, removing the button would make life easier. And from my understanding, it should not affect consistency either. Comment Below: What's Your Process with Reloading?