You cannot just cold turkey off the caffeine, that is when the headaches start.
The safest way to do it is to taper off slowly, reducing your intake on a daily basis. If you are used to drinking 24 oz, cut that down by 6 oz, etc. Replace the caffeine boost with a boost from herbal tea (mint works well) or find something else to take its place. I was able to switch to sparking flavored waters (Dasani, Schweppes, Poland springs, Bubly, LaCroix) all make decent flavors, so I was able to trick my taste buds a bit. It helped me kick the soda and coffee habit entirely in a month. If headache still comes on for you, then take an ibuprophen to combat it, but by the end of a month you should not be having headaches or any withdrawal symptoms.