Electrical – this electrical box with a button

That looks like a doorbell transformer. Those smaller wires should lead to your doorbell chime unit (assuming you have a doorbell). The junction box is there simply to provide power to the doorbell transformer.

From the picture it looks like you have 12/3 and 14/2 in that box, but it is difficult to tell. How confident are you the larger wires are larger than #12?

Figure out what breaker is supplying power to that circuit. As long as it is sized appropriately for the smallest conductors on the circuit, it will be fine. For example, if that is indeed a mix of #12 and #10 copper, your breaker must be 20A or smaller.

Also as a side note, NM cables entering that metal junction box need to be secured with a cable clamp. Any unused knockout holes need to be plugged. It looks like you've got a couple of code violations there at the top of your picture.

To identify the wire gauge, the best way might be to take the cover off of the breaker box (advise to do this with power off for safety, but power can be left on - proceed at your own risk) and see if there are any indications on the wire sheathing. If there isn't, use a guide like to identify the gauge of wire.

As for copper vs. aluminum wire, aluminum might be cheaper in larger sizes. However, aluminum requires some special anti-oxident joint compound when you make connections with aluminum wire (see Ideal no-alox for an example of this stuff). If you don't use the compound, its possible that the connections will fail over time.

EDIT: Using the info you provided in the comments, and the wire voltage drop calculator at http://www.southwire.com/support/voltage-drop-calculator.htm, assuming:
- Single Phase 240V
- Direct Burial
- no more than 3% voltage drop
- 2 AWG aluminum wire
Your wire gauge will allow you to maximum run somewhere in the 75-80 amp range. I wouldn't go all the way to 100 amps, but 60 amp service should work. You'll need 1 AWG wire to run 100 amps safely that distance.