No one is positive of the origin but entymologists have two strong guesses. I copied the following from the site listed in my sources:
"This self-referential American term is of uncertain origin. There are two leading hypotheses with several other less-likely contenders.
The OED2's earliest usage cites are from the 1680s. They refer to a pirate (at least it is probably to one individual) named Yankee Duch (1683),Captain Yankey (1684), and Captain John Williams (Yankee) (1687). The next earliest reference is an estate inventory from 1725 listing a slave named Yankee.
The earliest recorded usage of the term for Americans in general is in a 1758 letter by General James Wolfe, the hero of the battle of Quebec, in which he uses it as a pejorative term.
The song Yankee Doodle dates from 1775 and was intended to be insulting. Following the battle of Concord, during which the retreating British played it on the route back to Boston, the Americans adopted the tune as their own and the term began to acquire a complimentary sense.
This, however, may not be earliest usage of Yankee in a positive sense. In 1789, William Gordon published a history of the American Revolution in which he credits a Cambridge, Massachusetts farmer named John Hastings with using Yankee as an adjective meaning excellent as far back as 1713. John Hastings actually existed, but we have no other sources that credit his usage of the term, which is in contradiction to the general usage during the Colonial period.
The leading hypothesis as to its origin is that it is from the Dutch janke, meaning a diminutive of the name Jan. The OED2 favors this explanation, as does American Heritage and Ayto.
The second leading hypothesis is that it is from Jan Kaas, literally John Cheese, a nickname for the Dutch that parallels the British John Bull. Usage of Jan Kaas has been dated in Europe to the 1650s. The term could have been applied to Dutch pirates in the Caribbean (hence the 1680s references) and later shifted to New Englanders. Mencken favors this explanation, saying that the term was probably applied by Dutch New Yorkers to New Englanders "whose commercial enterprise outran their moral scruples." Thrifty New Englanders like Hastings may have taken this as a compliment.
Other, less likely origins have also been suggested:
The earliest suggestion comes from Thomas Anburey, a British officer serving under Burgoyne in 1789. He claims it comes from the Cherokee word eankke meaning coward. Supposedly, it was first applied by Virginians to New Englanders who refused to help them in their war with the Cherokees. No other reference to the Cherokee word has been found, however.
Others starting with the Rev. John Heckewelder (1819) and James Fenimore Cooper (1841), claim it derives from an American Indian corruption of the word English. Various supposed Indian words, such as Yengees, are claimed to support this hypothesis.
Washington Irving, in his Knickerbocker's History of New York, facetiously claims it comes from a MaisTchuseg (Massachusett) word Yanokies meaning silent men. Some have taken this to be a serious suggestion.
Another hoax appeared in an 1810 Boston newspaper. It claimed that it derived from a Persian word, jenghe, meaning warlike man or swift horse. The article was a parody of Noah Webster's writings and, again, some have taken it seriously.
The Pennsylvania Evening Post in 1775 suggested that it came from the name of an Indian tribe, the Yankoos, which meant invincible ones. Despite the patriotic sympathy exhibited by the paper, there is no other evidence of the existence of this tribe.
Various British dialectical words have also been suggested. Yankee was supposedly a Lincolnshire word for gaiters or leggings. In Scots, yankie means a forward, clever woman and yanking is an adjective meaning pushy, forward. Another dialect word, jank means excrement, although this one is pronounced with the /j/ sound, not the /y/. "
I am in favor of the second explanation that it was a nickname for dutch pirates in the Carabean that began to be percieved respectably and was taken as a nickname for Americans, and later Northerners.