That's tricky, and might depend on the device/software you are using.
I almost suggested in that last post that using one of those app lock apps to lock the system settings app might do it: if the user can't access Settings > Applications it will be hard for them to delete an app. But then I tested on my phone and found that a long press on an app icon gave me the option to uninstall it. However, further testing shows that's true for Nova launcher (my default), but the device's built-in launcher does not offer that, so if I were using the default software the idea might work. You can still remove icons from the homescreen, but that doesn't delete the app (in general: if for example you are using a Xiaomi device with it's MIUI interface it might. It's been a long time since I played with MIUI). So locking the settings app might work, but you'll have to test on your device.
However, it also depends on what you are trying to protect against. Accidentally removing an app from the screen is something a kid can easily do inadvertently, but in most cases that doesn't delete the app. A launcher that allows you to lock the desktop is also a good protection against that (Nova does this, but it's easy to unlock if you want to. I think the unlock option is harder to find on some others). Locking the settings app is really protecting against deliberate deletion, but in most cases it's hard to delete apps accidentally. However, it's also harder to protect against deliberate deletion (the device administrator thing would only slow someone down, since that can be deselected easily).
I don't know whether using a guest account on the device would be helpful? I've never really used those so can't speak from experience, but the idea is that it would let you keep your data and setup separate from what they are able to use. It was originally a tablet-only feature, but has been available on phones in more recent Android versions.