Property rights are not like other rights, contrary to what Madison and a lot of modern political theory says. If I have the right to free speech, it doesn't interfere with your right to free speech. But if I have property, that interferes with your right to have that property, you don't have it, I have it. So the right to property is very different from the right to freedom of speech. This is often put very misleadingly about rights of property; property has no right. But if we just make sense out of this, maybe there is a right to property, one could debate that, but it's very different from other rights.