I think I would have to ask him to leave for the night if he I just going to harass you. All will work out. It will be ok
The only time you should look in your neighbor's bowl is to make sure that they have enough. You don't look in your neighbor's bowl to see if you have as much as them.