I'm guessing you are on T-Mobile (I think there are not that many carriers left that support wifi calling (the new UMA).
I think, from when Rogers had UMA/Wifi-Calling, that somewhere you set two choices:
Wifi Preferred or Mobile Network Preferred. Unfortunately, the wifi preferred assumes strong enough signal. So assuming that more access points / repeaters are out of the question, I think another option might be an app that disconnects wifi at a certain signal threshold.
I have an app that sets priority on wifi but not quite what we need here. Mine will switch to another AP if the signal dips (at a threshold I configure) but another AP with better signal is present. I have 3 AP's in the house so it works for me. What we have to find for you is an app that is similar that will drop wifi if the signal is below a certain dbm.
This app "ByeFi" (lol) sounds hopeful: