I would ask a smoker to change shirts and wash hands before holding my baby. I have no issues with this and to me it makes perfect sense. We had this issue and I feel your pain but the conversation with our smoking family members actually went well and even those I thought would be upset were not!
This is what I do. My mother in law gives me grief, but I'm just very firm with her and won't let her touch my daughter until she has washed and put on one of DH's t-shirts. On the other hand a good friend of mine recently took up smoking again and she is more than willing to honor my request. I just put out a shirt and have it waiting for when they arrive. My friend automatically goes and washes and changes her top. I have everyone who holds the baby wash their hands first; even DH so the t-shirt for smokers is only a small extra step. My MIL always needs reminding, but she's just difficult. It's your baby's health so don't be afraid to stick up for it.