There are probably a billion guides for this already, but whatever. If you DON'T have a ~/.bash_profile (a file that gets executed every time you start cyg):

touch ~/.bash_profile
chmod a+x ~/.bash_profile

Now that you have the file, add this to it:

SSHAGENT=/usr/bin/ssh-agent
SSHAGENTARGS="-s"
if [ -z "$SSH_AUTH_SOCK" -a -x "$SSHAGENT" ]; then
	eval `$SSHAGENT $SSHAGENTARGS`
	trap "kill $SSH_AGENT_PID" 0
fi

This will start up ssh-agent for each Cygwin shell you have open. Close your Cygwin shell (if one is open) and open a new one. Now type:

ssh-add ~/.ssh/id_rsa
[enter your password]

Voila! No more typing your stupid password every time you need to ssh somewhere. Note that if you close the Cygwin window, you'll have to ssh-add your key again! This is good security...you can close the window when you're done and someone who happens on your computer sitting there won't have password-less access to any of your secure logins.