It depends a lot on the kind of traffic you expect and the condition of the ground. I'm going to write from my experience, outside Seattle, WA, USA, where the subsoil is glacial till.
For light foot traffic, pea gravel can be nice. Rocks are under 1", uniform in size, and have smooth surfaces. Every time you step on it, the rocks will shift, softening the footfall. This is also noisy and causes the gravel to spread out. You can contain it with edging, or just let it spread and refresh it periodically. It's not good for wheeled traffic, as the wheel will just sink in.
If you want to push a heavy wheelbarrow, ride a bicycle, or drive on gravel, you want to pick something with sharp edges. It also helps if the rocks are a mix of sizes. They will lock together over time, providing a sturdy surface.
If the ground is soft and collects water during your rainy season, smaller gravel will just get pushed in to the mud with traffic. An initial layer of larger rock (say, 3" minus) will stabilize things, and a top layer (say 3/4" minus) will give a smooth surface. Mud will still work its way to the surface, but the ground will remain stable. In very wet soft ground, I've heard that a layer of 3" - 5" crushed rock works well under the 3" minus, but I haven't tried it.
To keep mud from working its way up, put a layer of landscape fabric down first.
Look at the shape of the land and the history of draining. Avoid low, wet areas, or reshape them to drain elsewhere. Consider how your traffic will wear and shift the ground, and how that will affect draining in the future, too (e.g. tires leaving ruts).
It a good idea to remove topsoil and other organic matter first. Keep it, though, and use it elsewhere.
Wood exposed to earth, or even gravel, as you discovered, is a compromise at best. You can hedge your bet by using treated wood. Use wood with the maximum amount of preservative. "Outdoor wood" is not enough, the preservative levels must be at least adequate for ground contact. Preferably wood treated for foundation service should be used. Doing something to ensure any water that does contact the wood can easily drain away is also important, as well as trying to prevent water contact in the first place.
9" of soil does not impart much of a lateral load, so almost any retaining structure will suffice. Still, it would be better to not involve the fence if possible IMO. How far back from the existing structure would depend on what you end up doing to retain the extra earth. You could extend the existing structure, which would obviously have 0 offset. At the other extreme, relying on gravity to retain the earth, such as placing a row of large stones, you would want to see a minimum 45 degree slope between the existing wall and the tops of the new structure.
By using an impervious retaining material like stone or concrete blocks, you can backfill with plain soil, no gravel is required for minimal height walls. Taller structures often backfill with stones to prevent hydrostatic pressure from pushing on the wall. When back filling against wood, do everything reasonably possible to prevent water from contacting the wood, even if it is treated.
Best Answer
If you were pouring a slab over this -- and required decades of stability -- you might need to worry about your fill. Organic matter and other debris that will break down over time is appropriate for topsoil but not for fill.
Your project is going to be less sensitive to settling. You should compact your fill well as you place it. It looks like you have way less than a cubic yard in your photo, so you can easily do it with a hand tool, like a tamping bar or a lawn roller (see equipment rental yards for the roller.) Make more than one pass of fill-compact-more-fill in the deepest area. A few inches at once should compact well with a small lawn roller and won't take all day!