Many times, a headache is from a normal cause that you can treat yourself. Some especially common causes during pregnancy are dehydration (you need more water now!) and caffeine withdrawal (for those who abruptly cut out coffee). For those who are already susceptible to migraines, unfortunately, pregnancy may make them worse. Finally, for those who wear glasses, pregnancy can actually cause relaxation of the muscles around your eyes that lead to your vision changing, and therefore eye strain headaches.
To treat, Tylenol or magnesium supplements are your best bets. Speak with your doctor if you have persistent or sudden severe onset headaches, especially during the third trimester, as this could be a sign of a more serious issue.