John was a Tory. According to Loyalists of the American Revolution he was born in America and lived on the Mohawk River where he had 510 acres. He owned an additional 300 acres of uncultivated land in Holland's Patent. He was imprisoned by "the Whigs" from May 12, 1776 to March 4, 1777; after being released on bail he joined the British at Niagra and served under Colonel Butler.

He went with Colonel Barry St. Leger on the expedition to Fort Stanwix and on August 6, 1777, fought at the Battle of Oriskany (Aurisco) against American forces commanded by his brother, General Nicholas Herkimer (1728-1777). Nicholas was mortally wounded in the battle.

He served afterwards in the Commissariat and as a Captain in the bateaux. His property was confiscated. His losses included one of his slaves, Richard Buck, who died while serving with Butler's Rangers. He estimated his losses at £2,939 sterling and received £1,815 sterling in compensation.