To truly eliminate all dairy, you would also need to cut out any foods that have dairy derivatives, like casein, whey, any ingredient that starts with "lacto". Unfortunately, you'll be amazed to find that tons of foods have dairy in them! If you feel like that's too difficult to handle right now, you could try just eliminating the main dairy foods to see if there's a change. But if there's no change, you couldn't eliminate the possibility of a dairy intolerance all together because he may be one of those babies who's sensitive to even small amounts. So then you'd need to try eliminating the dairy ingredients too. It's up to you if you want to just bite the bullet and eliminate everything right away, then you'd know for sure.
BTW, it can take up to 2-3 weeks to get all the dairy proteins out of your body, so it's wise not to judge the results until then. Those dairy proteins stick around in your system longer than most other foods!
The mucousy stools and congestion can certainly be indicative of an allergy. Though, do you think it's possible he just has a cold too? Then the extra mucus would show up in his stool.