Sunshine Polishing Cloths Safe for Gemstones?

by Jackie Davidson.
(New Jersey, USA)

I use gemstones, sterling silver and crystal/glass in my pieces. If I use a Sunshine Cloth to polish the sterling silver, it will also rub against the gemstones, even if I try to avoid them — is that OK?

I read somewhere that you should avoid that, but it seems nearly impossible.

Jackie Davidson