Activists are flooding the immigration minister’s western Sydney electorate with flyers and banners labelling him a “racist”.