I've forgotten the formal proof for this, but hopefully this is correct:
Consider a D6 (for the sake of concrete language).
When you roll a 1, you reroll the die and keep the result. This produces an average value of 3.5, and happens 1/6 of the time.
When you roll a 2, you reroll the die and keep the result (even if it's lower). This produces an average value of 3.5, and happens 1/6 of the time.
When you roll a 3, you keep the result. This produces an average value of 3, and happens 1/6 of the time.
And so on.
This gives the following formula for the average of the D6: \$ (3.5 + 3.5 + 3 + 4 + 5 + 6) / 6 = 4.1\bar{6}\$.
Working similar formulas for the other dice, we get this table:
\begin{array}{lccc}
\hline
\text{Die} & \text{(standard) Avg.} & \text{GWF Avg.} & \Delta \\
\hline
\text{d4} & 2.5 & 3.00 & 0.50 \\
\text{d6} & 3.5 & 4.1\bar{6} & 0.6\bar{6} \\
\text{d8} & 4.5 & 5.25 & 0.75 \\
\text{d10} & 5.5 & 6.30 & 0.80 \\
\text{d12} & 6.5 & 7.3\bar{3} & 0.8\bar{3} \\
\hline
\end{array}
Dice are independent. 2D6 will have an average value of \$2 \cdot 4.1\bar{6} = 8.3\bar{3}\$.
Common weapon average damage (Great Weapon Fighting):
\begin{array}{lcc}
\hline
\text{Weapon} & \text{Avg. GWF dmg} & \text{improvement w/ GWF}\\ \hline
\text{Greatsword (2d6)} & 8.3\bar{3} & 1.3\bar{3} \\
\text{Greataxe (1d12)} & 7.3\bar{3} & 0.8\bar{3} \\
\text{Longsword (1d10)} & 6.30 & 0.80 \\
\text{Double-bladed Scimitar (2d4)} & 6 & 1 \\
\text{Smite (level 1, 2d8)} & 10.50 & 1.50 \\
\qquad \text{(+ weapon damage)} \\ \hline
\end{array}
Observations:
The ability works out to about a +1 to damage.
It scales to almost a +3 when smiting. The more dice you add (high level smite, for example), the better the ability.See errata, below
The bonus is "swingy." It can range from a -2 to a +10 on 2D6, for example.
Errata
In April of 2016, Jeremy Crawford ruled that additional dice from abilities like smite can not be re-rolled by Great Weapon Fighting.
Best Answer
1 + Str modifier, as the SRD says
Some features (monk, etc) may change the amount of damage caused.
The current PHB errata from WoTC updates what's in the PHB on page 195, and states the unarmed strike proficiency as shown in the SRD. It also states: