web3.py – Resolving ‘filter not found’ Error in web3.eth.getFilterChanges

filtersweb3.py

I am listening ethereum network through local geth node. I connected to geth through web3.py. I started a job in python code that periodically gets new blocks from filter. Here is the code:

def main():
    web3 = Web3(HTTPProvider(cfg.eth_node_url))        
    web3_confirmed_filter = web3.eth.filter('latest')

    scheduler = BlockingScheduler()

    scheduler.add_job(
        func=eth_listener_job.process_blocks(),
        args=[web3, web3_confirmed_filter],
        trigger=IntervalTrigger(seconds=cfg.pending_trx_scan_interval),
        id='block_processor',
        name='Load new blocks',
        replace_existing=True)   

    scheduler.start()

Here how job method looks like

def process_blocks(web3, web3_filter):
    print('Scanning for new blocks...')

    ....
    trx_blocks = web3.eth.getFilterChanges(web3_filter.filter_id)
    ....

Sometimes I get error on line

trx_blocks = web3.eth.getFilterChanges(web3_filter.filter_id)

ValueError: {'code': -32000, 'message': 'filter not found'}

What might be a reason for this error?

Best Answer

Summary

The node (geth) is deleting the filter, maybe due to inactivity, so returns filter not found for that ID.

Why?

Informally, filters work over json-rpc work like so:

  1. Send to node: "Create a filter with parameters X, Y, Z"
  2. Node replies: "Filter installed, with ID 456"
  3. Send to node: "Any new information on filter ID 456?"
  4. Node replies: "Yes, these changes: [...]"
  5. Time passes...
  6. Node decides to uninstall filter 456
  7. Send to node: "Any new information on filter ID 456?"
  8. Node replies: "I have no record of filter 456"

Unfortunately, the spec doesn't have much to say about when the node is allowed to uninstall the filter. The node can do it at will, and without notice. So in the end, the only thing you can do is check for the error, and recreate the filter if it is gone... (ugh, I know)

Web3.py plans

There is an open issue to stop using that json-rpc filter mechanism altogether, and keep all the filter state on the web3.py side. It was inspired by exactly the issue that you brought up.

Related Topic