Removing gloves before eating is a strict rule. But if you have a glass or a canape or a fork in your hands, that is an obvious and polite excuse for not shaking hands. Just practice the regretful ...
Cloth gloves can keep your hands warm in winter, and they're breathable in summer, but they're really only useful as gifts. They look good, but they aren't great for heavy-duty gardening.