You will need:
1 - Rivets or screws. I would recommend rivets for aluminum gutter
2 - An outlet for the new downspout, if you can't reuse the old one
3 - A piece of metal slightly bigger than the width of your downspout
4 - Aviation snips, if you currently don't own any
5 - Gutter sealer or solder bars
6 - A new downspout, if needed
7 - As tester101 mentioned, you might need to change the slope of the gutter.
Now, All gutters should be sloped, but, this is not always done and is not always that big of a deal if the gutters is not that long. However, if it is sloped already, then you will most definitely need to remove the hangers and change the slope from one side to the other.
Directions:
1 - Start by removing the old downspout, however attached it is. If soldered you will need a torch or a soldering iron. If not, rivets or screws are easy to remove.
2 - Remove the outlet.
3 - Patch the hole with a piece of sheet metal, copper aluminum or steel
4 - If copper, solder the patched metal for that you will also need an soldering iron. To solder the copper together you also need soldering fluid, or acid. A torch will heat the metal too much and you won't be able to solder it together. If aluminum or steel, rivet it together and seal it with gutter sealer.
5 - Adjust gutter slope if necessary
6 - Using the newly purchased or older outlet, cut the new hole for it on the other side of the gutter. Use the outlet as a jig. Place over the bottom side of the gutter, as it will be easier to cut the hole. Using a scratch all or sharpie, mark the outlet on the gutter. Make sure you mark the INSIDE of the outlet.
7 - Place new outlet. Example
8 - Solder it if copper, or rivet it and seal it if aluminum or steel.
9 - Attach the downspout.
This should be done at your own risk. There's a reason professionals do it daily.
Have fun!
Best Answer
If you want to repair the leaking pipe where it enters the foundation, I'd use hydraulic cement to seal it. As for the downspout into the cast iron pipe, there are transition pieces that should tie them together more cleanly and tightly - or you could fashion something from ABS or PVC sewer pipe/fittings and downspout fittings. Otherwise, my thought on this one is: "If it ain't broke (or not causing any major problems), don't fix it"!
As DaveM stated, it appears that water from your downspouts is flowing into the sewer system. Although generally not allowed today, it was apparently legal when the house was built, so I'd leave it alone - unless you have flooding problems in your basement or crawlspace. Making any changes to this system would typically require you to follow current codes which could end up being very costly - or at the very least, time-consuming.