Solidity – What is the Maximum Size of Memory in Solidity

evmgo-ethereumsolidity

I am digging into memory and the free memory pointer in solidity.

I have been wondering what the maximum size of memory in a single execution of a function in solidity?

As I know, when memory is used, the free memory pointer address is automatically updated at 0x40. I am very curious of what the end of the free memory pointer address will be ?

I tried to figure it out on the remix ide, just like below, but I often got out of gas.

  assembly {
        mstore(0x40, 0xFFFFFF)
    }

It seems I can set the memory pointer onto the maximum number of 32bytes

like this
mstore(0x40, 0xFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF)

So does it mean by that the maximum size of memory is 32 bytes??

If anybody knows, please let me know it and the reason why?

Best Answer

I just want to extend the answer about theoretical memory size (32-byte address, 2^256 bytes because it's byte-addressable). In reality, if you look at the entire blockchain, it has several limits:

it's bounded by the implementation of the client.
it's bounded by the block gas limit.

For 1., the official Go Ethereum implementation uses Uint64 for memory size, so the memory size is bounded by 2^64 bytes: Go Ethereum Mstore implementation For 2., it's the real limit that you will hit much sooner before the 1. comes, and it's blockchain-dependent and function-type dependent.

For example, the Ethereum blockchain now has ~30M gas (Jan 2023). In practice, the transaction can only spend less than that. So with the approximate equation (X/32)^2/512 + 3X/32 = 30M you found X is approximately a bit higher 0x3C0000. It's the real limit for the current Ethereum blockchain. For BSC, it's higher (~140M Block gas limit).

Function to test gas limit :

contract Test{
    function test()public {
        assembly {
            mstore(0x40, 0x3C0000)
        }
        // The above mstore doesn't cost much gas but it affects the subsequent mstore and mem alloc (the free mem pointer)
        string memory dummy = "test";
        // MSTORE for dummy will extend mem and spend nearly 30M gas.
    }
}

Another concern is the view function when using eth_call directly, as far as I know there is no official upper limit gas spent by eth_call and hence your function may hit the client gas limit in 1.. Unofficially the client or RPC provider will have their own limit on that (e.g. 550M on alchemy). Different clients may have different implementations on this.

Related Solutions

Solidity Log Events – How to Split Calldata Bytes to Multiple Log Events?

Arbitrary Split approach

Edit: Added this section to address arbitrary split destinations

By the time you are splitting, you should know the length of the target data. Below is an example implementation of copying the bytes to their destination inside solidity, which should be trivially extendable to N buckets.

pragma solidity ^0.4.15;

contract HelpLogs {

  event LogFirstHalf(bytes _data);
  event LogSecondHalf(bytes _data);

  function logit(bytes data) external {
      uint midpoint = data.length / 2;
      bytes memory data1 = new bytes(midpoint);
      for (uint i = 0; i < midpoint; i++) {
          data1[i] = data[i];
      }
      bytes memory data2 = new bytes(data.length - midpoint);
      for (i = 0; i < data.length - midpoint; i++) {
          data2[i] = data[i + midpoint];
      }
      LogFirstHalf(data1);
      LogSecondHalf(data2);
  }
}

Note that the gas usage is higher than it needs to be, because it works byte-by-byte. It would be faster to use 32-byte words, with bitmasking. A good reference is memcpy from Arachnid's solidity-string utils library.

[Edit: old] Fixed Bucket Approach

You can split the data externally without the 21 kgas overhead. Send the pre-split data as two parameters to a single function call:

pragma solidity ^0.4.15;

contract HelpLogs {

  event LogFirstHalf(bytes _data);
  event LogSecondHalf(bytes _data);

  function logit(bytes dataPart1, bytes dataPart2) external {
    LogFirstHalf(dataPart1);
    LogSecondHalf(dataPart2);
  }
}

This will cost less gas than splitting inside the EVM.

Solidity – Why Solidity Never Frees Memory: Understanding the Mechanism

Solidity always places new objects at the free memory pointer and memory is never freed (this might change in the future).

This means simply that memory management in Solidity is currently very rudimentary. The compiler maintains a pointer to the end of the last allocated block and that's it. Allocating a new memory variable means moving that pointer forward.

The compiler never tries to reclaim already allocated memory. Even when all references to your memory variable go out of scope, there's no mechanism that would detect it and mark the space as available for reuse.

This is true for the duration of a single external call to the contract. Each new call starts with zeroed memory and freshly initialized free memory pointer.

There are some operations in Solidity that need a temporary memory area larger than 64 bytes and therefore will not fit into the scratch space. They will be placed where the free memory points to, but given their short lifetime, the pointer is not updated. The memory may or may not be zeroed out. Because of this, one should not expect the free memory to point to zeroed out memory.

This part refers to a small optimization the compiler performs when it needs a very short-lived memory block for the current operation. Instead of properly allocating it by updating the free memory pointer, it just writes past the pointer, knowing that the memory there is not in use. The text warns you that if you try to read from this area of unallocated memory using inline assembly, you might find some leftovers from such an operation and you should not assume that the memory there is always zeroed-out.

Best Answer

Related Solutions

Solidity Log Events – How to Split Calldata Bytes to Multiple Log Events?

Solidity – Why Solidity Never Frees Memory: Understanding the Mechanism

Related Topic