EUREKA!
Apparently, you can use plink as the main ssh client and just load your keys in pageant (if you're like me, you already do):
You can do that by setting the GIT_SSH env variable to plink.exe path like so:
set GIT_SSH=C:\Program Files\PuTTY\plink.exe
or, you can use plink from TortoiseGit:
set GIT_SSH=c:\Program Files\TortoiseGit\bin\TortoisePLink.exe
Credit: Original solution taken from this blog post