Why, after the London Fork, the average gas price of mined blocks is so lower than the needed gas price to have transactions included

Let's quickly understand how the fee is changed in EIP-1159.

1. Base Fee: Base fee is determined by the network itself.
2. Max Priority Fee: Optional. Determined by the user and paid directly to the miner.
3. Max Fee Per Gas: This is the absolute maximum fee you want to pay per unit of gas.

In the EIP-1159 transaction; we supply maxFeePerGas and maxPriorityFeePerGas. The base fee is automatically calculated by the network.

Now the base fee can increase/decrease but we have already supplied the max fee we want to pay in maxFeePerGas. So if baseFee increases in the next block, the priority fee can be adjusted to make sure the total fee never crosses the max fee.

Base Fee + Priority Fee <= Max Fee

Now about calculating the fee;

we need to get the base fee from the blockchain node and we can then decide how much max fee we want to make. Ideally, if you want your tx to be valid even after 6 blocks (assuming the base Fee increases to a maximum per block); you can use given formulae to calculate the max fee.

Max Fee = (2 * base fee) + Priority fee

So in terms of how do we get each field:

baseFee = await (web3.eth.getBlock("pending")).baseFeePerGas
priorityFee = choose yourself
MaxFee = (2 * baseFee) + priorityFee