I wouldn't switch thermometers mid cycle. Just change the batteries in your old one.
It doesn't matter so much whether it's got the exact correct temperature, but whether it catches shifts in temp, right? Whether it's 97.3 degrees or 105 jelly beans, we just need it to say "things changed by 4 units since yesterday."
Keep checking temp with both, and see if the new one and old one shift at the same time, about the same amount. It'd be normal for them to not display the same temp.