Gosh, my story is SO similar to yours!
We needed a nipple shield for 3-4 weeks before we eventually weaned from it, and we were very worried about SIDS and heard about the whole pacifier thing and ultimately waited until our son was 8 weeks to try.
He never took to the pacifier. I'm not sure if things would have been different if we had tried to introduce it sooner, but that is what happened in our situation. All of the previous posters bring up very valid points about SIDS and the risks. I would make a decision based upon how well your baby is nursing now. My son was still having trouble nursing at 6 weeks, but we were both a lot more comfortable around 8-10 weeks.
In some situations it would be helpful if he would take a pacifier, but mostly, I'm happy he is not attached to one. You sound like you are very committed to breastfeeding, so if you decide to introduce one now, and it causes trouble, you would be able to take the correct steps to get your baby back on the breast. Obviously avoiding any "nipple confusion" would be best, but I think whatever decision you make is a good one!