You can disable the "Google" app: that really will disable assistant. I do that on my Pixel and don't even receive prompts to turn it on.
If you use a Google search bar widget that will also stop working. Personally I can't stand those things, and it's one reason I don't use the Pixel launcher (which won't let you remove it), but if you do use it this is the drawback. Apart from that I've found no other consequences of disabling the Google app, but it may be that there's some other Google feature I don't use that would be affected.
The annoying thing for me is that if you ever turn Assistant on in order to test it it enables every possibly type of data collection and logging, changing whatever Google settings you might previously have had. But turning it off does not turn any of that stuff back off again - you have to dig through a maze of account settings to undo that stuff. Not surprising really: it's easier for Google to code it this way and serves their interests, but it's one reason why I will never test Assistant again.
