Need audio player app with reliable playback

Poweramp has a good history on being a solid media player, have you looked into the app's Settings menu that might relate to options like 'Always start over' or 'Resume'?

I use the MX Player Pro app as my primary media player and the VLC app on occasion, both are consistently good at resuming playback with videos and audio files.

 

This should do what you want: https://play.google.com/store/search?q=casse o player 2022&c=apps&hl=en it doesn't need the latest version of Android - I have it working well on an old phone on Android 4.1.1.
Here's another similar app: https://play.google.com/store/apps/details?id=nava.apps.cassetterecorder
The Cassette metaphor works very well, the pause/play really does work.

 

I've been using Musicolet for some time and has been great. It's feature rich with no ads and does resume playback. (4.7 stars)
https://play.google.com/store/apps/details?id=in.krosbits.musicolet