If it helps any, this is a very common question in the states. Its rarely ever a malace thing, but a curiosity thing. Everyone has interest in people’s lineage (regardless of color). Even if it goes back 300 years. Maybe it has a different implication in France though.
You dont gotta explain yourself for nobody