Racists are nuts, they literally make up scenarios in their heads of things that happen and you can always tell when they're full of shit. Being a black bloke myself who has a house in a affluent area has becoming a running joke my partner and I have about the people who think we don't own our own house. I very much enjoy calling people out in my HOA meetings and FB groups. The funniest thing to me is how shocked people are when ethnic minorities own a nice piece of property.
No matter how much people try and alleviate equity you can't stop us from thriving.