Intereresting pakornlam. If you can travel to Muse without a permit on public transport, then there's presumably little stopping you doing that and then just turning up at the border and asking to leave? I can't see that they'd make you go all the way back to Lashio for a permit and a taxi. It might still be rather complicated trying to cross the other way, from China to Burma though.
There are NO checkpoints anywhere after Lashio until you reach Mile 105, which is the start of the Mu-se economic zone. It's at that point that all foreigners, including Chinese MUST present themselves to the immigration authorities stationed there. If you are heading to China, they will also look at your permit and allow you to continue. If you don't have a permit, they will still call their colleagues down at the border, ask you to check-in with them there and fill out the guest registration at your hotel. They will also mention that you must head back the way you came. At immigration itself, you won't be allowed to leave without a permit. So even if you managed to avoid mile 105, which probably isn't too difficult if you hide in the car you are travelling in provided they don't force it to stop, you still wouldn't be permitted to leave the country. I know it's silly, but for decades Myanmar has been like this. 4 Thai-Myanmar international border crossings have only been open since Aug 2013 and this one is said to be one of the next to be upgraded.
It is possible that after the border is upgraded to international status, the mile 105 checkpoint will cease for foreign travelers and just remain for goods transport vehicles to make sure they have permission to carry their cargo either further into Myanmar or to the Muse economic zone and China. We will see what happens AFTER the border opens, perhaps in April or so.