Well, for pretty much all western countries you declare a new residency, not a new tax residency.
Exceptions, and notes:
Ive learned Spain frequently asks residents/citizens that have left to prove that they live in the new country, and the best way to prove it is with a tax residency certificate from the new country.
Hungary has a quasi taxation by citizenship system, but it doesn't apply for countries Hungary has a double tax treaty with. So it means one has to be a tax resident in the new foreign country for the double tax treaty to be︀ relevant, not just a resident. (Unless one has dual citizenship, in which case Hungary's taxation︁ by citizenship doesnt apply)
Norway has a similar system to Hungary, but only for the︂ first three calendar years after moving out of Norway. (Being a dual citizen doesnt help︃ in Norway's case though.)
US has taxation by citizenship, so to avail benefits of double︄ tax treaties, one has to be a tax resident in the new foreign country. Just︅ for the FEIE exemption maybe one doesnt need to be a tax resident in a︆ foreign country though, Im not sure.
Australia has this domicile test, meaning the one's permanent︇ place of abode has to be in one foreign country outside Australia, and one can't︈ just travel around. But it doesn't specifically say one has to be a tax resident︉ abroad to have a permanent place of abode abroad, but I guess a tax certificate︊ from abroad would help if Australian authorities look into your affairs.