[Ethereum] What design patterns are appropriate for data structure modification within Ethereum smart contracts

contract-designdesign-patternssolidity

I am looking to hear about design patterns that people have implemented within Ethereum smart contracts to allow for post deploy modification of data structures.

For example, say I have a contract which contains a struct defining an Address. Should I down the line realise that I want to add an email address property to the Address.. Is there any pre-deployment design pattern consideration that would allow for me to do this.

A little more complex of an example.. If I had two properties of a contract named 'question' and 'answer', and all of a sudden I want to have multiple possible answers.. how would one go about making such a change?

My issue/concern is that if you interface with said contract through a browser for example, you could simply update the contract that the front end points to (post update).. BUT how do you resolve the problem of maintaining any contract data across update deployments..?

Thanks
T

Best Answer

You need to put in consideration the following:

This has to be planed from the start. You will need to design your smart contract taking in consideration the following 5 points:

You have to have a good testing strategies and tactics. Because the cost of updating the your smart contract can really ruin your life.
Keep your smart contracts modular and fairly separate rules and logic from data structure. So if you will need to change something, you will change just the related contract and you will not need to change many or all contracts.
You should be prepared by having an emergency stop or circuit-breaker to be able to stop all operations during any migration. Because you do not want to be in a situation where people can still update/insert data to the old version of the smart contract while you are migrating and thereafter.
You should previously provided the ability to read all the data from your smart contract. Of course you can do a permissioned read by restricting reading all data to the owner or any other trusted user or even another smart contract. You will need to read from the old version of your smart contract and insert in the new version.
You will use of of the following strategies to communicate with your smart contract. I copied them from Smart Contact Best Practices:

Upgrading Broken Contracts

Code will need to be changed if errors are discovered or if improvements need to be made. It is no good to discover a bug, but have no way to deal with it

...

However, there are two basic approaches that are most commonly used. The simpler of the two is to have a registry contract that holds the address of the latest version of the contract. A more seamless approach for contract users is to have a contract that forwards calls and data onto the latest version of the contract.

Example 1: Use a registry contract to store latest version of a contract

In this example, the calls aren't forwarded, so users should fetch the current address each time before interacting with it.

contract SomeRegister {
    address backendContract;
    address[] previousBackends;
    address owner;

    function SomeRegister() {
        owner = msg.sender;
    }

    modifier onlyOwner() {
        require(msg.sender == owner)
        _;
    }

    function changeBackend(address newBackend) public
    onlyOwner()
    returns (bool)
    {
        if(newBackend != backendContract) {
            previousBackends.push(backendContract);
            backendContract = newBackend;
            return true;
        }

        return false;
    }
}

There are two main disadvantages to this approach:

Users must always look up the current address, and anyone who fails to do so risks using an old version of the contract

You will need to think carefully about how to deal with the contract data when you replace the contract

The alternate approach is to have a contract forward calls and data to the latest version of the contract:

Example 2: Use a DELEGATECALL to forward data and calls

contract Relay {
    address public currentVersion;
    address public owner;

    modifier onlyOwner() {
        require(msg.sender == owner);
        _;
    }

    function Relay(address initAddr) {
        currentVersion = initAddr;
        owner = msg.sender; // this owner may be another contract with multisig, not a single contract owner
    }

    function changeContract(address newVersion) public
    onlyOwner()
    {
        currentVersion = newVersion;
    }

    function() {
        require(currentVersion.delegatecall(msg.data));
    }
}

This approach avoids the previous problems but has problems of its own. You must be extremely careful with how you store data in this contract. If your new contract has a different storage layout than the first, your data may end up corrupted. Additionally, this simple version of the pattern cannot return values from functions, only forward them, which limits its applicability. (More complex implementations attempt to solve this with in-line assembly code and a registry of return sizes.)

Regardless of your approach, it is important to have some way to upgrade your contracts, or they will become unusable when the inevitable bugs are discovered in them.

I created a story on Medium for this with title: Essential Design Consideration for Ethereum dApps (1): Upgradeable Smart Contracts

Related Solutions

[Ethereum] way to return data from a fallback function

As outlined here by chriseth (who created Solidity):

Fallback functions cannot have return values.

It does look like your result2 value is the msg.data value. That said, you have not outlined how you are calling the client function - it is possible that returning msg.data is a fallback of the library/tooling that you are using.

[Ethereum] Address checksum Solidity implementation

I went ahead and put together a working example of EIP-55 in Solidity in case it's useful to anyone that comes across this. It should also be pretty straightforward to create a library that extends address with a few of these methods.

pragma solidity ^0.5.1;


/**
 * @dev This contract provides a set of pure functions for computing the EIP-55
 * checksum of an account in formats friendly to both off-chain and on-chain
 * callers, as well as for checking if a given string hex representation of an
 * address has a valid checksum. These helper functions could also be repurposed
 * as a library that extends the `address` type.
 */
contract AddressChecksumUtils {
  /**
   * @dev Get a checksummed string hex representation of an account address.
   * @param account address The account to get the checksum for.
   * @return The checksummed account string in ASCII format. Note that leading
   * "0x" is not included.
   */
  function getChecksum(
    address account
  ) external pure returns (string memory accountChecksum) {
    // call internal function for converting an account to a checksummed string.
    return _toChecksumString(account);
  }

  /**
   * @dev Get a fixed-size array of whether or not each character in an account
   * will be capitalized in the checksum.
   * @param account address The account to get the checksum capitalization
   * information for.
   * @return A fixed-size array of booleans that signify if each character or
   * "nibble" of the hex encoding of the address will be capitalized by the
   * checksum.
   */
  function getChecksumCapitalizedCharacters(
    address account
  ) external pure returns (bool[40] memory characterCapitalized) {
    // call internal function for computing characters capitalized in checksum.
    return _toChecksumCapsFlags(account);
  }

  /**
   * @dev Determine whether a string hex representation of an account address
   * matches the ERC-55 checksum of that address.
   * @param accountChecksum string The checksummed account string in ASCII
   * format. Note that a leading "0x" MUST NOT be included.
   * @return A boolean signifying whether or not the checksum is valid.
   */
  function isChecksumValid(
    string calldata accountChecksum
  ) external pure returns (bool ok) {
    // call internal function for validating checksum strings.
    return _isChecksumValid(accountChecksum);
  }

  function _toChecksumString(
    address account
  ) internal pure returns (string memory asciiString) {
    // convert the account argument from address to bytes.
    bytes20 data = bytes20(account);

    // create an in-memory fixed-size bytes array.
    bytes memory asciiBytes = new bytes(40);

    // declare variable types.
    uint8 b;
    uint8 leftNibble;
    uint8 rightNibble;
    bool leftCaps;
    bool rightCaps;
    uint8 asciiOffset;

    // get the capitalized characters in the actual checksum.
    bool[40] memory caps = _toChecksumCapsFlags(account);

    // iterate over bytes, processing left and right nibble in each iteration.
    for (uint256 i = 0; i < data.length; i++) {
      // locate the byte and extract each nibble.
      b = uint8(uint160(data) / (2**(8*(19 - i))));
      leftNibble = b / 16;
      rightNibble = b - 16 * leftNibble;

      // locate and extract each capitalization status.
      leftCaps = caps[2*i];
      rightCaps = caps[2*i + 1];

      // get the offset from nibble value to ascii character for left nibble.
      asciiOffset = _getAsciiOffset(leftNibble, leftCaps);

      // add the converted character to the byte array.
      asciiBytes[2 * i] = byte(leftNibble + asciiOffset);

      // get the offset from nibble value to ascii character for right nibble.
      asciiOffset = _getAsciiOffset(rightNibble, rightCaps);

      // add the converted character to the byte array.
      asciiBytes[2 * i + 1] = byte(rightNibble + asciiOffset);
    }

    return string(asciiBytes);
  }

  function _toChecksumCapsFlags(address account) internal pure returns (
    bool[40] memory characterCapitalized
  ) {
    // convert the address to bytes.
    bytes20 a = bytes20(account);

    // hash the address (used to calculate checksum).
    bytes32 b = keccak256(abi.encodePacked(_toAsciiString(a)));

    // declare variable types.
    uint8 leftNibbleAddress;
    uint8 rightNibbleAddress;
    uint8 leftNibbleHash;
    uint8 rightNibbleHash;

    // iterate over bytes, processing left and right nibble in each iteration.
    for (uint256 i; i < a.length; i++) {
      // locate the byte and extract each nibble for the address and the hash.
      rightNibbleAddress = uint8(a[i]) % 16;
      leftNibbleAddress = (uint8(a[i]) - rightNibbleAddress) / 16;
      rightNibbleHash = uint8(b[i]) % 16;
      leftNibbleHash = (uint8(b[i]) - rightNibbleHash) / 16;

      characterCapitalized[2 * i] = (
        leftNibbleAddress > 9 &&
        leftNibbleHash > 7
      );
      characterCapitalized[2 * i + 1] = (
        rightNibbleAddress > 9 &&
        rightNibbleHash > 7
      );
    }
  }

  function _isChecksumValid(
    string memory provided
  ) internal pure returns (bool ok) {
    // convert the provided string into account type.
    address account = _toAddress(provided);

    // return false in the event the account conversion returned null address.
    if (
      account == address(0)
    ) {
      // ensure that provided address is not also the null address first.
      bytes memory b = bytes(provided);
      for (uint256 i; i < b.length; i++) {
        if (b[i] != hex"30") {
          return false;
        }
      }
    }

    // get the capitalized characters in the actual checksum.
    string memory actual = _toChecksumString(account);

    // compare provided string to actual checksum string to test for validity.
    return (
      keccak256(
        abi.encodePacked(
          actual
        )
      ) == keccak256(
        abi.encodePacked(
          provided
        )
      )
    );
  }

  function _getAsciiOffset(
    uint8 nibble, bool caps
  ) internal pure returns (uint8 offset) {
    // to convert to ascii characters, add 48 to 0-9, 55 to A-F, & 87 to a-f.
    if (nibble < 10) {
      offset = 48;
    } else if (caps) {
      offset = 55;
    } else {
      offset = 87;
    }
  }

  function _toAddress(
    string memory account
  ) internal pure returns (address accountAddress) {
    // convert the account argument from address to bytes.
    bytes memory accountBytes = bytes(account);

    // create a new fixed-size byte array for the ascii bytes of the address.
    bytes memory accountAddressBytes = new bytes(20);

    // declare variable types.
    uint8 b;
    uint8 nibble;
    uint8 asciiOffset;

    // only proceed if the provided string has a length of 40.
    if (accountBytes.length == 40) {
      for (uint256 i; i < 40; i++) {
        // get the byte in question.
        b = uint8(accountBytes[i]);

        // ensure that the byte is a valid ascii character (0-9, A-F, a-f)
        if (b < 48) return address(0);
        if (57 < b && b < 65) return address(0);
        if (70 < b && b < 97) return address(0);
        if (102 < b) return address(0); //bytes(hex"");

        // find the offset from ascii encoding to the nibble representation.
        if (b < 65) { // 0-9
          asciiOffset = 48;
        } else if (70 < b) { // a-f
          asciiOffset = 87;
        } else { // A-F
          asciiOffset = 55;
        }

        // store left nibble on even iterations, then store byte on odd ones.
        if (i % 2 == 0) {
          nibble = b - asciiOffset;
        } else {
          accountAddressBytes[(i - 1) / 2] = (
            byte(16 * nibble + (b - asciiOffset)));
        }
      }

      // pack up the fixed-size byte array and cast it to accountAddress.
      bytes memory packed = abi.encodePacked(accountAddressBytes);
      assembly {
        accountAddress := mload(add(packed, 20))
      }
    }
  }

  // based on https://ethereum.stackexchange.com/a/56499/48410
  function _toAsciiString(
    bytes20 data
  ) internal pure returns (string memory asciiString) {
    // create an in-memory fixed-size bytes array.
    bytes memory asciiBytes = new bytes(40);

    // declare variable types.
    uint8 b;
    uint8 leftNibble;
    uint8 rightNibble;

    // iterate over bytes, processing left and right nibble in each iteration.
    for (uint256 i = 0; i < data.length; i++) {
      // locate the byte and extract each nibble.
      b = uint8(uint160(data) / (2 ** (8 * (19 - i))));
      leftNibble = b / 16;
      rightNibble = b - 16 * leftNibble;

      // to convert to ascii characters, add 48 to 0-9 and 87 to a-f.
      asciiBytes[2 * i] = byte(leftNibble + (leftNibble < 10 ? 48 : 87));
      asciiBytes[2 * i + 1] = byte(rightNibble + (rightNibble < 10 ? 48 : 87));
    }

    return string(asciiBytes);
  }
}

Best Answer

You need to put in consideration the following:

Related Solutions

[Ethereum] way to return data from a fallback function

[Ethereum] Address checksum Solidity implementation

Related Topic