(which gave me a list of the mounts, so I could choose the right mount points for /system, which was not mtdblock4 for me, but rather mtdblock1)
# mount -rw -o remount /dev/block/mtdblock1 /system
and after that the adb push worked.
Yay, thanks for the hints in this thread.
(still don't know why adb shell sysrw didn't work)