[Ethereum] How to configure max txpool memory size

txpool

What is the default txpool memory size in geth? How can I change the txpool memory size?

According to What is the max size of transactions can clients like geth keep in txpool?, "The number of transactions in the pending transaction pool is effectively limited by memory." I wonder what the default setting is and how to change it.

Best Answer

Geth

The default config values associated with the transaction pool can now be changed from Geth's CLI:

TRANSACTION POOL OPTIONS:
  --txpool.pricelimit value    Minimum gas price limit to enforce for acceptance into the pool (
default: 1)
  --txpool.pricebump value     Price bump percentage to replace an already existing transaction (default: 10)
  --txpool.accountslots value  Minimum number of executable transaction slots guaranteed per account (default: 16)
  --txpool.globalslots value   Maximum number of executable transaction slots for all accounts (default: 4096)
  --txpool.accountqueue value  Maximum number of non-executable transaction slots permitted per account (default: 64)
  --txpool.globalqueue value   Maximum number of non-executable transaction slots for all accounts (default: 1024)
  --txpool.lifetime value      Maximum amount of time non-executable transaction are queued (default: 3h0m0s)

While this doesn't allow you to limit exactly how much memory is used, increasing, for example, accountslots or globalslots will lead to an increase in used memory, and vice versa.

Parity

Similarly, Parity has the following CLI option:

 --tx-queue-size LIMIT          Maximum amount of transactions in the queue (waiting
                                 to be included in next block) (default: 1024).

Summary

It is limited by your computer's memory.

From my basic testing in What happens when a transaction nonce is too high?, I crashed geth with 400 x 64 transactions with a data payload of 4,500 bytes when running in 4 Gb RAM.

I am assuming that your question is asking what is the maximum number of transactions that can be stored in the transaction pool. If you are asking what is the maximum size of a single transaction, the answer is at Is there a limit for transaction size?.

Details

There is a pool for pending transactions and a queue for transactions that are not ready to be processed.

The data structure representing the transaction pool in geth can be seen at go-ethereum - tx_pool.go, lines 54 to 77:

// TxPool contains all currently known transactions. Transactions
// enter the pool when they are received from the network or submitted
// locally. They exit the pool when they are included in the blockchain.
//
// The pool separates processable transactions (which can be applied to the
// current state) and future transactions. Transactions move between those
// two states over time as they are received and processed.
type TxPool struct {
    config       *ChainConfig
    currentState stateFn // The state function which will allow us to do some pre checks
    pendingState *state.ManagedState
    gasLimit     func() *big.Int // The current gas limit function callback
    minGasPrice  *big.Int
    eventMux     *event.TypeMux
    events       event.Subscription
    localTx      *txSet
    mu           sync.RWMutex
    pending      map[common.Hash]*types.Transaction // processable transactions
    queue        map[common.Address]map[common.Hash]*types.Transaction

    wg sync.WaitGroup // for shutdown sync

    homestead bool
}

From stackoverflow.com - Maximum number of elements in map, the maximum number of elements that can be stored in a Go map data structure is 2147483647 for 32-bit architecture and 9223372036854775807 for 64-bit architecture.

Pending Transaction Pool

Pending transactions are stored in TxPool.pending that is a map(transaction hash -> pointer to transaction). In go-ethereum - tx_pool.go, there is no limit check to the number of transaction when adding transactions to this data structure.

The number of transactions in the pending transaction pool is effectively limited by memory.

Transaction Queue

Queued transactions are stored in TxPool.queue that is a map(from address -> map(transaction hash -> pointer to transaction)). In go-ethereum - tx_pool.go, there is a check that the number of transactions from the same From address does not exceed maxQueued = 64.

The number of transactions in the transaction queue is effectively limited by memory, with the limitation that there can only be a maximum of 64 transactions for transactions with the same From address.

What is the difference between a pending transaction and a queued transaction?

Pending transactions are transactions that are ready to be processed and included in the block.

Queued transactions are transactions where the transaction nonce is not in sequence. The transaction nonce is an incrementing number for each transaction with the same From address.

For example:

Transaction from account 0xaaaa...aaaa with nonce 0 has been included in the blockchain.
Transaction from account 0xaaaa...aaaa with nonce 1 has been included in the blockchain.
10 transactions from account 0xaaaa...aaaa with nonces 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 as sent to an Ethereum node. These are placed in the transaction queue as the transaction from account 0xaaaa...aaaa with nonce 2 has not been seen by the Ethereum node.
Once the transaction from account 0xaaaa...aaaa with nonce 2 is added to the transaction pool, the 10 transactions with nonces 3, 4, 5, 6, 7, 8, 9, 10, 11 and 12 will be moved from the queue to the pending transaction pool and all 11 transactions are ready to be processed and inserted into into the blockchain (provided there is enough gas).

Pending Transactions – Difference Between ‘Pending’ Block and Txpool

It looks to me as though this is a bit of an optimisation/edge case that is causing some confusion - that the node assumes that the network that you are connected to is functioning normally i.e. that blocks will be mined by rational actors.

Initially I was under the impression that the txpool is where all the pending transactions are stored on a node

This is correct.

...that the pending block is only present when a node is mining (i.e. the block that's being solved for)

This, specifically, is the part where I think the node assumes that it's part of a network that is or will be functioning correctly in the future.

To help clarify, we can run through some scenarios:

In the case above: where there are fewer transactions (TXs) than could be expected to fit into a block (based on the gas limit) then it is safe to assume that which ever node goes on to mine the next block will put all of the TXs from the mempool into the block (after all, there's an economic incentive for the miner to maximise their profit by including as many TXs as possible). So the node can make an intelligent guess at what the next block will look like.
In the case where there are more TXs in the mempool than would be able to fit into the block, then it's more difficult for predict which TXs will be included in the block (as reordering/modifying the list of TXs impacts the mining of the block), but it's probably safe to assume that any miner will be aiming to include the TXs with the largest fees to maximise their profit.

Could someone explain the difference between these two concepts in the context of pending transactions, the account nonce and pending transaction broadcasting?

To summarise and hopefully answer these questions:

Valid TXs (either created on this node or received from a Peer) will be added to the node's, local mempool until they are included in a block (whether that block is mined by this node or another on the network). At which point the TX are removed from the mempool and included in the block.

Since we can expect that most of the nodes are rational and behaving correctly; it's possible to make an intelligent guess as to which TXs (from the mempool) will be included in the next block.

As an account makes TXs, the account's nonce will increase to enforce the ordering of TXs for that account. Even if the past TXs have not been included in a block, the account can continue to make further valid TXs as long as the nonce is incremented. They can appear in the mempool and could be expected to appear in the next block - depending on gas fees, num TXs in the mempool etc.

New TXs that are submitted on a node will be broadcast to all or a random subset of its peers (depending on how the node is coded). The new, pending block will only be broadcast to a node's peers when it is mined by a mining node.