You can just stop paying fees, but the company will stay in the Registry for years to come. Given that WY Registry is publicly available, someone, for whatever reason, could in 4 years go to the Registry and see you had LLC in that State. If you decide to dissolve it for good (and pay dissolution fees), LLC will be deleted from the Registry. Out of curiosity, how much are they asking for dissolution?
Otherwise, you should not have problems if you decide just to let it go, no penalties, no charges, no travel ban etc.