Don't be mad at yourself! You have just been doing what you needed to to get through this very difficult time!
Can you afford to rent a hospital grade pump?
People have gotten their babies back to the breast after a much longer time on bottles than that - and it's great that you got him to sort of latch on. You will get there!