I discovered that my problem was that whatever version of ssh-keygen that I used created the files with the wrong filenames... The files initially created where %USER_HOME%.ssh\ida_rsa and ida_rsa.pub, but git expected them to be id_rsa and id_rsa.pub.
I solved the problem by running git bash, THEN running ssh-keygen