[Ethereum] the array size limit of a returned array from a contract function call in Solidity

arrayscontract-invocationgasparityscalability

I have a contract storing rdf data in various mappings. It is all working lovely except for large array returns. I have functions to return arrays of hashes matching certain criteria. I have found, through testing, that a function call to the contract that returns an array of 36,000 bytes32 hashes works fine. However, a function call to the contract that should return an array of 100,000 records fails. An example function I am calling is:

// return a bytes32 array of hashes matching a subject search hash
function getSubjectArray(bytes32 which) public constant returns (bytes32[]) {
    return subjectindex[which];
}

I was originally testing using web3 function calls to the contract, then I decided to test using direct curl commands to see if it was a web3 issue. But it behaves the same using curl. A result array of 36,000 hashes returns from the curl call successfully, but a result array of 100,000 returns just '0x'.

curl --data '{"id":1,"jsonrpc":"2.0","method":"eth_call","params":[{"to":"0x0a0bdad3a719adacf6ef29299b25f4799c3961ba","data":"0xd6cf19ffa6ccec97d46345da54fa1ac0a269b860a855d19895705920f2fdfb4eb786817a"},"latest"]}' -H "Content-Type: application/json" -X POST 127.0.0.1:55200

{"jsonrpc":"2.0","result":"0x","id":1}

I have stepped though this large results array using a separate function passing an index, so I know all the 100,000 records are there to be returned. So it seems to be a limit on the size that an array returned from a function call to a contract can be.

Can anyone confirm if there is a limit that an array being returned from a function can be?

Best Answer

The limit is how much gas is provided: gas is needed even for eth_call / constant functions: Can Solidity constant functions be arbitrarily complex?

The gas needed to be increased, example in bold:

curl -H "Content-Type: application/json" -X POST 127.0.0.1:55200 \
--data '{"id":1,"jsonrpc":"2.0","method":"eth_call","params": [{"to":"0x0a0bdad3a719adacf6ef29299b25f4799c3961ba","data":"0xd6cf19ffa6ccec97d46345da54fa1ac 0a269b860a855d19895705920f2fdfb4eb786817a", "gas":"0x4000000000000"},"latest"]}'

@tomusdrw from Parity also had a good explanation of eth_call:

Yes, reads are indeed free, but eth_call is just creating a "virtual" transaction that gets executed in the context of the blockchain. It's not subject to block gas limit that you tried (4.7M), but still gas calculation is being done for anything that eth_call does (for instance to prevent infinite loop). Our default gas limit was 50M and taking out ~64k of bytes32 was hitting the limit, you should try specifying something much larger like 2**50

Related Solutions

Solidity Contract Invocation – Handling String Parameter in Raw Call

You have to properly encode the parameter using the ABI encoding. Something like

instance.call.gas(gas).value(price)(
    bytes4(keccak256("method(string)")),
    u256(0x20),   // pointer to data
    param.length, // length of data
    param         // actual data
);

could work, but it would probably be better to either use inline assembly for that (because it does not include any magic) or wait a little more and use the function abi.encode which should hopefully be available soon.

In general, using low-level calls is discouraged and should only be used of there is no other way, for example if you do not want to propagate a revert condition.

You can do the same (apart from not propagating the revert) using

interface MethodStringInterface {
    function method(string);
}
contract MyContract {
    function myFunction(address _a) {
        MethodStringInterface(_a).method("abc");
    }
}

or even better:

interface MethodStringInterface {
    function method(string);
}
contract MyContract {
    function myFunction(MethodStringInterface _a) {
        _a.method("abc");
    }
}

Solidity – How to Add Data to a Dynamic Array

-1 doesn't do anything in this case. The push method returns the array length. This value minus one gives the index of the element that was pushed (e.g. 0 because length will now be 1 and the index starts counting at 0). You're not doing anything with the return value, so it's just wasting gas to compute the offset.

The MyBytesArray() function that you get for "free" owing to the keyword public accepts one argument for the index and returns one element at a time. I suspect index is defaulting to 0 because it's overlooked when you call it. If so, that returns the first element. If you ask for the second element (index 1) then you'll get the second element.

On the caller side, using a Truffle contract abstraction, it would look something like:

var myContract; var rowToGet = 0; // 1,2,3 var instance = MyContract.Deployed() .then(function(instance) { myContract = instance; return myContract.myBytesArray(rowToGet); }) .then(function(response) { console.log(response); // one element of the array }); }

It might help to think of public as giving you a "free" function that looks like this:

function MyBytesArray(uint index) public view returns(bytes32) { return(MyBytesArray[index]); }

By convention, your variable name should be camelCase, e.g. myBytesArray.

You can return the entire array but this is expensive, possibly wastefully so, and there are concerns about scaling. That said, you could add a function:

function getMyBytesArray() public view returns(bytes32[]) { return myBytesArray; }

Hope it helps.

Best Answer

Related Solutions

Solidity Contract Invocation – Handling String Parameter in Raw Call

Solidity – How to Add Data to a Dynamic Array

Related Topic