Cookies – How to convert a British biscuit recipe to a moister American style cookie

Try using lemon zest (lemon peel shavings) and lemon or lime juice. You might try sprinkling the zest over the top of the cookies right before you put them in if you don't get enough flavor from adding it to the batter

Yes, the original recipes involved an enormous amount of trial and error. People baked, swapped recipes, and the good ones were desirable and became widespread, while the bad ones died out. The advanced knowledge of food chemistry wasn't even available at the time - Hannah Glasse published her cookbook more then 100 years before Mendeleev published the periodic table.

Luckily, you don't have to go through the same process if you want to invent a good recipe now. There are only a few basic ingredients in baking - eggs, sugar, flour, water - and you can be sure that all possible ways to combine them, as well as most secondary additions and substitutions, have been thoroughly explored. People have distilled the knowledge of this exploration in books, and you can use this basic knowledge to build new recipes which will function with a high probability.

Before you develop a baking recipe, there are four things you have to know.

1. Technique. If you want to create a cookie recipe, you will have to use creaming. If you want to make a new kind of eclair, you have to know how to prepare a basic pate a choux. There are cookbooks teaching these techniques, some of them are explained in questions here on SA.

2. Base ratio. For most baked goods, there are ratios which give you the best results. For example, for a crepe you want 1 part flour, two parts egg and two parts liquid. As long as you keep that, you can let your imagination run free by using different liquids, or adding spices, or even plopping pieces of fruit in the pan and pouring the batter over them. You can learn about this from Ruhlman's book Ratio, or use a known-good recipe for a plain variant of the good you are trying to make as your starting point.

3. Flavor combinations. You can add any ingredients you want, but there is no guarantee they will work well together. Choosing the right ones is a combination of talent (being able to imagine what a combination will taste like before you have had it) and experience. You should try to be more analytical about the flavors of things you eat - which tastes can you distinguish? Which aromas? What makes them go well together? Is it their similarity, or the contrast? - and when you have done that long enough, you will be able to predict the goodness of a combination. A book to help you along is The Flavor Bible, which describes good combinations. You can also take popular combinations and transfer them across types of food. For example, if you like apple-cinnamon pie, you can try to make cinnamon cookies glued in pairs with apple butter.

4. Ingredients' structure and their role in the baked good. This knowledge is maybe the least usual among home cooks. When you create new recipes, you will always dilute the base formula, or use substitutions. If you don't have this knowledge, it will be hit-or-miss whether your new recipe will work or not. It is a bit like rebuilding a house's interior without knowing which walls are load-bearing and which aren't. If you know what each ingredient does in a recipe, you will know when a substitution is possible and when it isn't. For example, many people will tell you that applesauce is a substitute for eggs. And you can indeed bake a cake with applesauce instead of eggs, but don't try to whip a mayonnaise with it. This is because eggs have a different role in cake and mayonnaise. Learning about these roles requires a lot of effort, and a curiosity about these things. Most people will find it easier to just try whatever they feel like, and live with the occasional failed new-recipe attempt. If you would rather learn about them, I can recommend two sources. One is Cooking for geeks by Jeff Potter, which is short and doesn't go into the depth of things, but offers lots of useful information and is easy to read, or the really exhaustive book called On Food and Cooking by Harold McGee, which runs to about 800 pages and goes into detail most people don't want to know about, but is a must-read if you are fascinated by food science.

And, of course you can always just run into the kitchen and experiment. Don't forget to document along the way so you can reproduce your results! Basically, you can always have luck and hit a good new recipe, the knowledge I listed above (combined with some baking experience) helps you increase your first-try success rate from about 2-3% as a full amateur to somewhere about 80% or more. It is your decision how to divide your time between reading and baking, any combination can function, depending on your learning style.