You said your baby is 4 weeks old, so I wouldn't feel frustrated about pumping that 2 oz. over 20 minutes. My pediatrician said that to get a rough idea of how much a baby needs at each feeding, take their month of age (yours would be one) and add one to it for a total number of ounces to be taken per feeding (one plus one, so that'd be 2 oz). It's my belief you are making enough milk after all! Babies that young just don't take a lot of milk until their stomachs grow larger.
Hang in there, and for what it's worth, try to visualize some kind of calming image while you try to nurse. I try closing my eyes along with a few deep breaths to just relax despite the baby crying or what not. Babies can sense any kind of anxiety or frustration, so the more you try to maintain calm (even if you are stressed! I can relate!) the more likely it is your baby will stop giving you so much of a hard time. It's worth a try anyhow!
You're a great mom to be continuing to try to nurse despite these challenges. It really DOES get better and will be very worth all the efforts!