To be fair, the buyer must agreed to a fixed price and paid a certain amount of deposit to show his seriousness. The buyer may has his own concern, as he had not seen the watch personally, then he got to paid a deposit upfront.
My suggestion is to make a video call, present the watch to the buyer, have a good chat and see whether he's serious buying or not.
I had once travelled to Melaka to meet a potential buyer. In the end the buyer did not take my watch and said sorry to me. I had just wasted my time, petrol and toll. However, I did enjoy a good nyonya cuisine there
If you want to meet the buyer in Johor, just lower your expectation. Be ready whether he's buying or not. If he continue with the purchase, then it's good for you. If not, just take this as your day trip to Johor. Whatever the outcome was, you'll definitely less frustrated than me.