I was also thinking something like an old iPhone with e.g. the Tascam dictaphone app or even just one of the cheaper stereo field recorders from Tascam or Zoom. They can be had really cheap second hand, since it’s not like it needs many fancy functions.
On a slightly different note, you could consider looking into LittleBits synth set and other build sets. There’s lots of options for making sound-related stuff and it’s basically made for kids + your kids seem to have the right age for it (mine is still just a bit too young, but as soon as I can excuse myself wanting these sets with it being relevant for her, it’ll happen ASAP).