[Ethereum] How fetch transaction history for account quickly using Web3j

go-ethereumjavarinkebytransactionsweb3j

Intro

We use Web3j java library to work with Ethereum network. I use Rinkeby test network wit Geth client at the moment. I got familiar with blockchains only a week ago, so terms I use can be incorrect.

Brief

Almost all required features are clear for me except transactions history. I've stucked at this point. What I have now is two test methods.

Examples

  • First piece looks up through all history by do post filtering, so it requires a lot of time to get result.

    @Test
    public void testReplayTransactions() throws Exception {
        EthBlock earliestBlockResponse = web3j.ethGetBlockByNumber(DefaultBlockParameterName.EARLIEST, false).send();
        EthBlock.Block earliestBlock = earliestBlockResponse.getBlock();
        LOGGER.info("EARLIEST block number: {}", earliestBlock.getNumber());
    
        EthBlock latestBlockResponse  = web3j.ethGetBlockByNumber(DefaultBlockParameterName.LATEST, false).send();
        EthBlock.Block latestBlock = latestBlockResponse.getBlock();
        LOGGER.info("LATEST block number: {}", latestBlock.getNumber());
    
        DefaultBlockParameter startBlock = DefaultBlockParameter.valueOf(latestBlock.getNumber().subtract(BigInteger.valueOf(1000)));
        DefaultBlockParameter endBlock = DefaultBlockParameter.valueOf(latestBlock.getNumber());
        //DefaultBlockParameter startBlock = DefaultBlockParameter.valueOf(earliestBlock.getNumber());
        //DefaultBlockParameter endBlock = DefaultBlockParameter.valueOf(earliestBlock.getNumber().add(BigInteger.valueOf(100000)));
    
        Subscription subscription = web3j
            .catchUpToLatestTransactionObservable(DefaultBlockParameterName.EARLIEST)
        //.replayTransactionsObservable(startBlock, endBlock)
            .filter(transaction -> {
                return KnownPeers.initialPeer.getAddress().equals(transaction.getFrom())
                    || KnownPeers.initialPeer.getAddress().equals(transaction.getTo());
            })
            .doOnError(throwable -> LOGGER.error("Error occurred", throwable))
            .doOnCompleted(() -> LOGGER.info("Completed"))
            .doOnEach(notification -> LOGGER.info("OnEach"))
            .subscribe(transaction -> {
                LOGGER.info("Transaction {} of block {}. From {} to {}",
                    transaction.getHash(),
                    transaction.getBlockNumber(),
                    transaction.getFrom(),
                    transaction.getTo());
            });
    
        Thread.sleep(10000);
        subscription.unsubscribe();
    }
    
  • The next one queries for logs, but nothing returned in the first few seconds. So I guess either it also looks at all transactions or logs are something other.

    @Test
    public void getLogs() throws Exception {
        PeerInfo peerInfo = KnownPeers.initialPeer;
        EthFilter filter = new EthFilter(
            DefaultBlockParameterName.EARLIEST,
            DefaultBlockParameterName.LATEST,
            Arrays.asList(peerInfo.getAddress())
        );
    
        LOGGER.info("Start fetching logs");
        Subscription subscription = web3j
            .ethLogObservable(filter)
            .doOnError(throwable -> LOGGER.error("Error occurred", throwable))
            .subscribe(log -> {
                LOGGER.info("Transaction {} of block {}. From {} to {}",
                    log.getTransactionHash(),
                    log.getBlockNumber(),
                    log.getAddress(),
                    log.getType());
            });
    
        Thread.sleep(100000);
        subscription.unsubscribe();
    }
    

Found links

Question

How fetch all transactions for an account quickly. Maybe not all, but 10-20 at once. Max timeout – less than 1-2 secs. This resource can do it.

Update 1

As I see list account transaction is an opened issue for web3j with a pretty long history. Also found related topic at stackexchange.
There is some Etherchain API where they have GetAccountTransactions REST method, but haven't grasped yet how it is related to my task. I see Powered by Parity v1.7.8 in the footer

Best Answer

There is no workaround besides of the links you´ve posted. It will depend of your CPU power, network and database. I believe they (etherscan team) gets data from the blockchain and stores in an indexed database and uses some analytics tools to display data to users.

Related Topic