Ethers.js – Inconsistency in Recovered Account when Verifying EIP-712 with JavaScript and Python

apeethers.jspython

I have a client that signs EIP-712 messages:

eip_712.js (dep: ethers: 5.7.1)

const { Wallet } = require("@ethersproject/wallet");
const { _TypedDataEncoder } = require("@ethersproject/hash");

console.log("EIP-712 JS");

// ! this is a test private key, DO NOT USE IT
const privKey = '0x0123456789012345678901234567890123456789012345678901234567890123';
const signer = new Wallet(privKey);
console.log("Signer is: ", signer.address);

// ! You need to fix your ethers version because _signTypedData is experimental
// ! feature.
// * https://docs.ethers.io/v5/api/signer/#Signer-signTypedData

const domain = {
    chainId: 0,
    name: '',
    verifyingContract: '0xCcCCccccCCCCcCCCCCCcCcCccCcCCCcCcccccccC',
    version: ''
};

// The named list of all type definitions
const types = {
    Player: [
        { name: 'wallet', type: 'address' },
        { name: 'email', type: 'string' }
    ],
    Message: [
        { name: 'player', type: 'Player' },
    ]
};

// The data to sign
const value = {
    player: { 
        wallet: "0x14791697260E4c9A71f18484C9f997B308e59325",
        email: "[email protected]",
    }
};

console.log(
    "Signing this digest: ", _TypedDataEncoder.hash(domain, types, value)
);

signer._signTypedData(
    domain,
    types,
    value
).then(signature => {
    console.log("Signature is: ", signature);
}).catch(e => {
    console.log("Something went wrong", e);
});

And now I am trying to verify the from account in python:

eip_712.py (deps: eip712 = "0.1.4", eth-account = "0.7.0")

#!/usr/bin/env python
from eip712.messages import EIP712Message, EIP712Type
from eth_account import Account

print("EIP-712 PY")

class Player(EIP712Type):
    wallet: "address"
    email: "string"


class Message(EIP712Message):
    _chainId_: "uint256" = 0
    _name_: "string" = ''
    _verifyingContract_: "address" = '0xCcCCccccCCCCcCCCCCCcCcCccCcCCCcCcccccccC'
    _version_: "string" = '' 

    player: Player 

signature = bytes.fromhex('f65e247fc142c6fa98febae4dba127be1694e83a5fce938c4389945594a5acd454771cdf810b90f2f83d60c69f8d576d5ff8dab957f57b2a247c86baffae6d011c')

message = Message(player=Player(
    wallet='0x14791697260E4c9A71f18484C9f997B308e59325',
    email='[email protected]'
))


# this is the digest that the user signs on the client side
print('Digest that was signed: ', message.body.hex())

from_account = Account.recover_message(message.signable_message, signature=signature)
print('Account that signed the message: ', from_account)

The outputs for the above two are:

EIP-712 JS
Signer is:  0x14791697260E4c9A71f18484C9f997B308e59325
Signing this digest:  0x044b3a930b5d539e1ec6f5ab7de69a1c6c0f103a11c6377adbd81e92763df9be
Signature is:  0xf65e247fc142c6fa98febae4dba127be1694e83a5fce938c4389945594a5acd454771cdf810b90f2f83d60c69f8d576d5ff8dab957f57b2a247c86baffae6d011c

EIP-712 PY
Digest that was signed:  e5477989520456a98eb64ab80304b4e9b1858fe542121f4722a0ea1e12aed8c9
Account that signed the message:  0xE01Fc9697648F725409D75B08f5e7D98CddD13bD

Looks like the digest these two methods are signing is different?

EDIT:
The digest on the Python side isn't actually the digest that is used to recover the account. I went into eth_account lib and printed the actual digest. This is message_hash on line 450 in the account.py file. The actual digest that is used is: 0xa7875cf3863821279f6b8d2c254e34caec0e7376cb37fda6387150b0ec3aaf96. This implies that eth_account is using a different digest to the one that ethers.js produces. I shall hardcode the ethers.js digest into eth_account to verify that if indeed the digest were correct, whether it would accurately recover the address.

EDIT:
Indeed, if I use the digest from ethers.js, then eth_account is recovering the correct signer address. This implies that if we figure out why eth_account is producing a different digest, then we are in business.

EDIT:
I have verified that not using eip712 library and just encoding the structured data directly, produces the exact same incorrect digest on the Python side. i.e. the below is also invalid:

signable_message = eth_account.messages.encode_structured_data(
        {
            "message": {
                "player": {
                    "wallet": "0x14791697260E4c9A71f18484C9f997B308e59325",
                    "email": "[email protected]"
                }
            },
            "types": {
                "EIP712Domain": [
                   {"name": "chainId", "type": "uint256"},
                   {"name": "name", "type": "string"},
                   {"name": "verifyingContract", "type": "address"},
                   {"name": "version", "type": "string"}
                ],
                "Player": [
                    {"name": "wallet", "type": "address"},
                    {"name": "email", "type": "string"}
                ],
                "Message": [
                    {"name": "player", "type": "Player"}
                ]
            },
            "primaryType": "Message",
            "domain": {
                "chainId": 0,
                "name": "",
                "verifyingContract": "0xCcCCccccCCCCcCCCCCCcCcCccCcCCCcCcccccccC",
                "version": ""
            }
        }
    )

EDIT: I am noticing discrepancy in encoding the header of the message. Ethers.js encodes domain into: 0x8b73c3c69bb8fe3d512ecc4cf759cc79239f7b179b0ffacaa9a75d522b39400fc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a4700000000000000000000000000000000000000000000000000000000000000000000000000000000000000000cccccccccccccccccccccccccccccccccccccccc, whilst eip712 (and eth_account) encode it into: 0x03fdd899fb21de70ced90f32a1783b1f2014b8b662f8233df477aeefb0ff0e4a0000000000000000000000000000000000000000000000000000000000000000c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470000000000000000000000000ccccccccccccccccccccccccccccccccccccccccc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470

And the above appears to be a discrepancy in the order of encoding in conjunction with encoding the primary type differently.

EDIT: having noticed the discrepancy in encoding the header of the message, and noticing that only one field from the EIP712 domain is required, I managed to achieve the same encoding using a single value in the header. Namely, instead of using all the: chainId, verifyingContract, name and version, I am only using name, which achieves parity in encoding. This, therefore, suggests that to fix the discrepancy between the two, some sorting must be applied. Either on the ether.js side or python side. Given ethers.js already does field name sorting, and because it is more widely used on the client side, it makes sense to fix the sorting on the Python side.

EDIT: there appears to be no issue about encoding and having the body, they match. Therefore, I can retrieve the correct signer on the Python side.

EDIT: I have made a repo that reproduces all of the above, you can find it in this commit: https://github.com/nazariyv/eip712-discrepancy/commit/c7baa4630cbb5f1d01b5f123ea0727d527fb25c4

Best Answer

The reason you cannot retrieve the correct signer is due to the discrepancy in encoding the EIP712Domain. It appears that because ethers.js does sorting on the types, but Python does not, you are getting different encoded header which causes the problem. There is no issue on the actual value of the message, at least, not in this example (there may still be an issue there, too).

This is not really a solution, but for my purposes it was sufficient: drop all the fields on EIP712Domain and only use name.

For eip_712.js

const domain = {
    //chainId: 0,
    name: 'RKL Player Data',
    //verifyingContract: '0xCcCCccccCCCCcCCCCCCcCcCccCcCCCcCcccccccC',
    //version: ''
};

For eip_712.py

class Message(EIP712Message):
    _name_: "string" = "RKL Player Data"
    # _chainId_: "uint256" = 0  # noqa
    # _name_: "string" = ""  # noqa
    # _verifyingContract_: "address" = (  # noqa
    #     "0xCcCCccccCCCCcCCCCCCcCcCccCcCCCcCcccccccC"
    # )
    # _version_: "string" = ""  # noqa

    player: Player

And you will achieve parity in messages and therefore will be able to correctly recover the signer.

EDIT: indeed, the problem was the ordering of the header fields in Python. If you change the ordering to what you see here, you can recover the sender on Pyhton side no problem: https://github.com/nazariyv/eip712-discrepancy/tree/fix

Related Solutions

Web3.py – Recovering JavaScript Signed Messages in Python

This is what ended up working:

async function create_signature(message, accounts) {
    var hex = ''
    for(var i=0;i<message.length;i++) {
        hex += ''+message.charCodeAt(i).toString(16)
    }
    var hexMessage = "0x" + hex
    var signature = web3.eth.personal.sign(hexMessage, accounts[0])
    return signature
}

And on the Python side, we recovered the address like this:

def recover_address(message, signature):
    message_hash = defunct_hash_message(text=message)
    address = w3.eth.account.recoverHash(message_hash, signature=signature)
    return address

If anyone knows a better way to calculate the hexMessage in the JavaScript code, please respond with a comment!

Web3.py – Python Web3 Account Sign_Message with Nonce

Under the hood, it's just one message that you need to sign.

Some background on how it works. Let's use Nansen as an example as it's the closest to the one on your screenshot.

When you click "Sign in with Wallet" (e.g. MetaMask), the app server generates a message with a time to live & a nonce. Here's what the message looks like in the MetaMask UI:

And here's what's actually sent to you from the server:

app.nansen.ai wants you to sign in with your Ethereum account:\n0x9729187D9E8Bbefa8295F39f5634cA454dd9d294\n\nBy signing this transaction you are allowing Nansen to see the following: your wallet address.\n\nURI: https://app.nansen.ai\nVersion: 1\nChain ID: 1\nNonce: BELEGLDkqN5\nIssued At: 2024-01-15T09:19:11.000Z

So, you need to sign the entirety of this message as one with your private key:

from eth_account.messages import encode_defunct
from web3 import Web3

private_key = '1e5a89614d79cab1e39a45f5b116498a38d373ba117a05b35df397f42d7a0c94'

# The message you received from Nansen
message = "app.nansen.ai wants you to sign in with your Ethereum account:\n0x9729187D9E8Bbefa8295F39f5634cA454dd9d294\n\nBy signing this transaction you are allowing Nansen to see the following: your wallet address.\n\nURI: https://app.nansen.ai\nVersion: 1\nChain ID: 1\nNonce: BELEGLDkqN5\nIssued At: 2024-01-15T09:19:11.000Z"

# Encode the message
encoded_message = encode_defunct(text=str(message))

# Sign the message with your private key
signed_message = Web3().eth.account.sign_message(encoded_message, private_key=private_key)

print("Signature:", signed_message.signature.hex())

Once you get the signature, you need to post it back to the app server within the time to live period so that you get signed in.

Best Answer

Related Solutions

Web3.py – Recovering JavaScript Signed Messages in Python

Web3.py – Python Web3 Account Sign_Message with Nonce

Related Topic