Try to think of it this way: in traditional societies, babies are cared for by aunts, grandmothers, older siblings, etc. It's normal to hand your baby to someone else for several hours. Basically, that's all you're doing - you're giving your LO to another caretaker and then picking up where they left off in the afternoons. In the meantime, you're making some money to help bring up your child. Best of luck - I know it's terribly hard.
It is so very, very hard, but you will make it! And...the fact that you miss him so much shows how much you love your lo.