At a minimum, you'd need access to both sides of the fence for any treatment or repair. You'll need to get your neighbor's permission to do so, of course. While you're over in his yard, bring your rake and rake out all the leaves - you'll need to do this anyway to get access to the bottom of the fence and allow it to dry out before any repairs can be made or coatings added. Raking up his leaves will build goodwill and will help his shed last longer, too, since it appears to have wood siding. You may want to mention this to your neighbor, as those leaves that are rotting out your fence will also be rotting out his shed. While he may not care, he probably will, and this would be encouragement for him to rake this area out in the future since it's self-serving.
If your neighbor is unreasonable and won't let you into his yard to access the other side of the boards, you'll have to disassemble from your side. While you've got the fencing off, you can either hold out an olive branch and rake up and dispose of all the leaves collected there, or you can escalate the battle by blowing them out of the little gap and all over his yard. Your call... ;)
Once you've got access to the fence and you've got it dried out, you'll need to assess the condition of the boards. If they're still reasonably sound, but just discolored, you can clean them with a pressure washer, bleach or a wood brightener. If they're no longer sound, you'll have to replace them.
I'd recommend a good, regular maintenance regime of applying a wood sealer or paint, at your choice to the wood. One thing to keep in mind is that the end-grain of the wood will suck up much more water than the face-grain will. The advantage you've got is that it will also suck up much more sealant/paint than the face will. Use this to your advantage and make sure you put several coats on the end-grain of the boards. This will help fill up all those pores that will otherwise suck up water.
You could, of course, replace all the fence boards with a composite or PVC fencing material if you desired to do so. At this particular time, lumber prices in the US have skyrocketed, to the point where my mother just received a quote to redo her deck & stairs that was cheaper to do with Trex™ decking than lumber. Last I looked, composite decking was nearly twice the price of wood. YMMV...