The reason the CSS cache needs refreshing is because Drupal optimizes all individual CSS files from various modules and themes into one CSS file which is optimized into a single file.
So that this file is not recompiled every page load, which would loose the benefit of optimization, Drupal needs to know when the CSS file has changed in order to recompile this. At which cache refresh seems like the ideal time. To turn this off - rather than turn off caching completely you can simply:
Go to /admin/settings/performance where there is a field labeled "Optimize CSS files":
Disable this whilst you are doing your development and making changes to your CSS file. Then when in production and most of your CSS is set then you can enable it. I highly recommend the performance gains this brings in the loading of your pages.
http://stackoverflow.com/questions/4436496/drupal-disable-css-cache