I'd say ignore the person who says 'he's relying on you' - in the case of an owned horse yes he is but not a part loan one, methinks someone didn't read all you're post lol
I too agree that moving on might be the answer. I know where you're coming as we all want to do the best we can for the animals we care for. I loan a mare two afternoons a week - I already have two equines, a horse and a pony, since the pony is too small (she's a driving mare) and the horse is singular I borrow another to ride out with my partner twice a week. It doesn't cost me anything other than yard work. and sometimes they leave a lot to do, I've no worries as I look after them as if they were my own but it gets noted you can bet!
If these people you ride for have other horses they can produce a heck of a lot of muck in a day so two barrow loads would be one day for say four horses out all the time...does that sound about right? A horse produces a pile about once every 1- 2 hours on average so by counting piles and dividing by horses you can get a rough guess of how long it's been left.
I think you've answered your own question yourself when you say you've been 'paying to poo pick' and will all over winter. Riding is the reward for the hard work, and if you're not going to be able to for most of the winter evenings, and weekends are never guaranteed to be fine enough for riding let alone if the field is in the same state as most get into (please tell me you're not on clay...?) Is there nowhere else to ride? if the field is the grazing or the future hay crop I can pretty well guarantee there's going to be a comment about it being 'too wet' and that'll put the kibosh on all your riding. If there was road riding and or some off road usable winter rides that would be one thing. We do have to curtail the riding during winter but not lose it altogether. It's possible to deck the horse out in waterproofs but if you're going to cut up the land too much that's not viable. It's probably worth you having a chat with the owners and seeing if there's a compromise, they could genuinely not know you're not happy - or you're finding the poo picking a bit much, suggest the waterproofs (and volunteer to tack clean right after you ride if that's viable - you must take care of their tack at all costs and if it gets wet it'll need care immediately)
You say 'after school' which means you're still school age, there will come a time when you have to go to uni or get a job and you may not be able to ride as much as you'd like so IMO move on now - don't get tied into this horse who you may well love and click with but he's not the only one out there. I know it makes you feel mercenary, but your riding career deserves more at this point. There are other loan horses out there just crying out for a dedicated carer like yourself (the people I borrow the mare from have had a series of no hopers so if you turn up and poo pick even when you don't ride then you're one up on them - apparently one of them moved a horse in then cleared off for months on end)
Getting out of the agreement by rights you should give a months notice (is there a written loan agreement? best to check details) and stay in contact with them, you can always blame increasing school homework/commitments - might be worth asking for a reference too so that you can wave it at the next future loan horse's owners (this will also do two things, make them note you more and what they say will reflect how they feel about you as a carer - I think you probably know what I mean). There is no shame in taking a decision that protects yourself, not everyone is in a position or willing to slog through winter - and it can be a slog.. The horse sounds like he'll be ok - he's fed, watered and exercised and I assume his other rider will take care of him well too, and maybe if you leave the owners will think twice about allowing a minor to do two barrowloads of muck picking for in effect nothing.(assuming it's not just a normal day's load)
I wish you all the best - hope it works out ok :)