Yes, just like when you bought the land you were under many other non-HOA restrictions, like you couldn't install a waste processing plant where your house is.

Many Americans idea of freedom is completely out of touch with the reality on how land contract law works.