How to use AsyncHTTPProvider in web3py?

Question

I am trying to use the AsyncHTTPProvider in a project, and here is my test code:

import asyncio
from web3 import Web3
import asyncio

async def main():
    w3 = Web3(Web3.AsyncHTTPProvider("http://127.0.0.1:8545"))
    coinbase = await w3.eth.coinbase
    print(coinbase)

if __name__ == "__main__":
    asyncio.run(main())

I used ganache-cli for the local provider.

I couldn't find any examples online of people using the AsyncHTTPProvider, even in the web3py github I couldn't really find an example that I could understand (reading pull requests of large libraries is very tedious)

When I run this code, I get the following error:

Traceback (most recent call last):
  File "test.py", line 11, in <module>
    asyncio.run(main())
  File "C:\Program Files\Python37\lib\asyncio\runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "C:\Program Files\Python37\lib\asyncio\base_events.py", line 587, in run_until_complete
    return future.result()
  File "test.py", line 7, in main
    coinbase = await w3.eth.coinbase
  File "D:\Name\Ethereum\FirstProject\env\lib\site-packages\web3\eth.py", line 282, in coinbase
    return self.get_coinbase()
  File "D:\Name\Ethereum\ArbitrageMax\env\lib\site-packages\web3\module.py", line 57, in caller
    result = w3.manager.request_blocking(method_str, params, error_formatters)
  File "D:\Name\Ethereum\FirstProject\env\lib\site-packages\web3\manager.py", line 154, in request_blocking
    response = self._make_request(method, params)
  File "D:\Name\Ethereum\FirstProject\env\lib\site-packages\web3\manager.py", line 133, in _make_request
    return request_func(method, params)
TypeError: 'coroutine' object is not callable
sys:1: RuntimeWarning: coroutine 'AsyncBaseProvider.request_func' was never awaited

looking at the error log it seems despite using the AsyncHTTPProvider, 'w3.manager.request_blocking' is still being used, which suggests it is still using the synchronous api

I ran the following code to ensure the web3 object's provider is actually async:

import asyncio
from web3 import Web3
import asyncio

async def main():
    w3 = Web3(Web3.AsyncHTTPProvider("http://127.0.0.1:8545"))
    print(type(w3.provider))
    # coinbase = await w3.eth.coinbase
    # print(coinbase)

if __name__ == "__main__":
    asyncio.run(main())

output:

<class 'web3.providers.async_rpc.AsyncHTTPProvider'>

looking in the web3py repository's tests it seems that I am doing everything right, since the tests are interacting with the w3.eth async api the same way I tried:

class AsyncEthModuleTest:    
    ... # other tests

    # test of get balance which involves awaiting coinbase
    @pytest.mark.asyncio
    async def test_eth_get_balance(self, async_w3: "Web3") -> None:
        coinbase = await async_w3.eth.coinbase  # type: ignore

        with pytest.raises(InvalidAddress):
            await async_w3.eth.get_balance(  # type: ignore
                ChecksumAddress(HexAddress(HexStr(coinbase.lower())))
            )

        balance = await async_w3.eth.get_balance(coinbase)  # type: ignore

        assert is_integer(balance)
        assert balance >= 0

link to pull request where tests were added (issue #2056)

The only issue is that I couldn't figure out where is the declaration of the 'async_w3' object

the version of web3 I started my project is: web3==5.20.1 I updated to the latest release to see if it would fix the issue (web3==5.23.0) and nope, still same old error

Resurrection

Answer 1

I got an answer in the web3 github:

This is the correct syntax to create the provider (I haven't tested it yet though)

from web3 import Web3
from web3.eth import AsyncEth

w3 = Web3(Web3.AsyncHTTPProvider("http://127.0.0.1:8545"), modules={'eth': (AsyncEth,)}, middlewares=[])

If anyone in the future finds this question: keep in mind that as of now async is a work in progress. that may change in the future and you'll need to check the docs

supported middlewares

Resurrection

Answer 2

The approach posted by @Ramgos is no longer the way to do this; as of the time of this post (2023-10-06), the way to do this is as follows:

from web3 import AsyncWeb3
w3 = AsyncWeb3(AsyncWeb3.AsyncHTTPProvider('http://127.0.0.1:8545'))

AyncWeb3 will take care of overriding the modules etc.

(taken from the documentation)

Smirch