It all happened during the Revolutionary War, a famous New Jersey jurist, Judge Imlay, hadn’t yet decided which side to chose revolutionaries or loyalists, so when Big George(yes Washington) talked to one of the Judge’s slaves he asked him what was the judge’s side. Washington actually fell in love with what the slave had to say. Washington said it so many times that the sentence became standard in our language. What the slave did say was “Until my master knows which is the strongest group, he’s staying on the fence.”
Imlay was born in Imlaystown, notice the similarity between the family name of James and his hometown. The town is now historical preserved for it’s historical importance in our society.