When you've had oversupply for a long time, "balancing out" can feel like your supply is tanking. It's so common that my LC actually warned me, while I was still suffering from oversupply, that a day would come when my breasts wouldn't feel uncomfortably full and I wouldn't pump twice as much as my baby needs, and she warned me: "That's not you losing your supply! That's NORMAL. Don't panic."
If you feed on demand on weekends, you may lose that "extra" four ounces. If you want to continue to make more than your baby eats, you will eventually have to pump on weekends to keep that up. That's not your supply tanking - that's just your supply adjusting to what your baby actually eats.