Steaming and seasoning veggies in a pressure pot

One thing we have found that helps in adding flavor during the steaming process is to slice garlic thinly, and line the bottom of the steamer basket with the garlic. Then afterwards, toss the garlic in with the potatoes and add salt/pepper/etc. (I'd probably add paprika, onion powder, and a pinch of cayenne.)

I would imagine orange or lemon peel might work the same as garlic when steamed, although you might get better results tossing the steamed veggies with some zest.

You can also toss the steamed potatoes with vinegar or lemon juice (whether or not it's combined with oil is up to you) along with the spices.

Another option would be to bake your potatoes rather than steam them. Just wash, poke, and bake at 350F for 1 hour. There's no more fat, but the flavor and texture are VERY different. Just make sure not to wrap them in foil, since you won't get the same results. (Wrapping potatoes in foil is essentially steaming them, and the crust won't get crisp.)

I haven't seen your recipe, but from looking at others, steamed pudding takes on the order of a couple hours of steaming. Doing it in the oven won't be any faster or less boring than steaming it - it may even be slower, since heat transfer from steam is pretty efficient. 45 minutes at 120°C would probably leave it undercooked, and if you cook at a higher temperature, you'll be missing the point - steaming for a long time lets it cook slowly and more evenly. You could work out how to cook it hotter and faster, but it wouldn't be a steamed pudding anymore, it'd be a "normal" pudding.

If you want to do it in the oven, you should use a water bath to keep it from getting too hot - put the vessel with the pudding into a larger vessel, and add water to come most of the way up the sides of the pudding. Then you can increase the temperature of the oven, perhaps to 150-175°C, without overheating the pudding. But I doubt you'll be able to get the cooking time very much less than the original cooking time with steam without significantly changing the results.

Finally (thanks Elendil), steaming helps keep the pudding moist, and an oven even with a water bath is not a terribly humid environment, so it might still be drier than desired. Covering tightly, and possibly adding a tiny bit more liquid, might help with that.