I think the problem is because of people from other countries displaying their flags, not necessarily that it's an American flag. If flags are allowed, they won't be just American.

Would you be offended by a Mexican flag hanging at your neighbors house?