Knuckleheads. You'll be perfectly fine with them off... it will change the handling dynamics of the car without a doubt, but only the driver can label it as handling like ass. It'll make the car more likely to oversteer... some people like that... some people don't. It is entirely up to the Driver.
You can leave them off for as long as you want... it doesn't matter. I know a lot of Auto-Xers that remove their front bar completely to emphasize the effects of their rear bars. Its all about a driver's preference.