You should be able to get a reasonable steak stovetop using a cast iron grill pan, if you have a strong enough exhaust. Oil the cast iron pan (with canola or such), then heat it very hot, until it starts to smoke. Make sure the meat is completely dry on the outside (wipe with a paper towel, water will prevent browning) and gently place in the pan. Leave it there for a minute or two (it'll smoke quite a bit!), rotate 90° to get the nice grill marks. Leave for another minute or two. Flip, and repeat for the other side.
It may splatter, have a splatter screen handy. It will smoke, quite a bit, make sure the exhaust is on high.
You will probably have a medium-rare steak now. Using a thinner cut will make it more well done (you can cut a thick steak in half with your chef's knife, making two thin steaks); so will plopping it in the oven (not sure if you want the oven before or after searing, I like 'em medium rare...).
Remember to let the meat rest for 5 minutes or so before serving.
Also, if your steaks are coming out ridiculously tough, you're probably using the wrong cut of meat, or some terrible grade. What cut are you using?
[edit: I should note that, in case it wasn't obvious, the burner should be up pretty high]
Edit by rumtscho There is a reason this answer specifies a cast iron pan. If you are limited in your cookware choice, pay attention to the maximum temperature your pan can tolerate. If you are using a non-stick pan, you have to go on medium heat and wait longer, else you'll damage the pan. (And sorry, but you can't get it as tasty as on high heat that way).
Goat is pretty mild, as gamey meats go, so it is possible that you just aren't fond of goat? I know people who love lamb and so I turn them on to goat, which is milder, but if you eliminate what many would call gaminess, you have effectively blanded out your goat completely.
One technique that can work with your ground goat is to brown the meat, then rinse it with water, in a colander, to remove excess grease. Much of the gaminess is in the fat.
The main trick is just to work with recipes that are designed for that taste. Goat is great in curry, for instance. The shepherd's pie that you describe is also good for goat because the blandness of the potatoes offsets the stronger taste of the goat. You wouldn't want to make a traditional shepherd's pie, for instance, with chicken breast, or pork loin. Not enough flavor.
To get your meat filling to hold together better, when you brown the meat, add a couple of tablespoons of flour and cook it in with the fat. You are basically making a roux in place which will then thicken as your pie bakes, holding the moisture rather than letting it run away. If you do the rinsing idea I mentioned above, then you would do this step after rinsing, although you might need to add a little oil to the mix.
Best Answer
Older goat tends to have a very strong flavor, hence why many suggest braising in strong flavors. You can braise your kid following these recipes - just remember to use 3/4th the amount of strong seasonings as you would for a regular goat.
As for grilling the meat, you could try to marinate the meat overnight with herbs and spices. Unlike chicken, which is a fairly blank canvas taste-wise, goat has a strong taste (even kid meat has a slight gamey/goat-y flavor). As I said, if grilling, marinate.
If you're braising or stewing, toss in a beer, some mint or basil and a few other herbs.
For the ribs - you could french them to get rid of some of the connective tissue and then marinate and roast them at low heat for a few hours to break down some of the connective tissue.
Good luck and great eating!