MismatchingStateError: mismatching_state: CSRF Warning! State not equal in request and response

Question

This is driving me absolutely crazy and preventing me from being able to do local dev/test.

I have a flask app that uses authlib (client capabilities only). When a user hits my home page, my flask backend redirects them to /login which in turn redirects to Google Auth. Google Auth then posts them back to my app's /auth endpoint.

For months, I have been experiencing ad-hoc issues with authlib.integrations.base_client.errors.MismatchingStateError: mismatching_state: CSRF Warning! State not equal in request and response. It feels like a cookie problem and most of the time, I just open a new browser window or incognito or try to clear cache and eventually, it sort of works.

However, I am now running the exact same application inside of a docker container and at one stage this was working. I have no idea what I have changed but whenever I browse to localhost/ or 127.0.0.1/ and go through the auth process (clearing cookies each time to ensure i'm not auto-logged in), I am constantly redirected back to localhost/auth?state=blah blah blah and I experience this issue: authlib.integrations.base_client.errors.MismatchingStateError: mismatching_state: CSRF Warning! State not equal in request and response.

I think the relevant part of my code is:

@app.route("/", defaults={"path": ""})
@app.route("/<path:path>")
def catch_all(path: str) -> Union[flask.Response, werkzeug.Response]:
    if flask.session.get("user"):
        return app.send_static_file("index.html")
    return flask.redirect("/login")


@app.route("/auth")
def auth() -> Union[Tuple[str, int], werkzeug.Response]:
    token = oauth.google.authorize_access_token()
    user = oauth.google.parse_id_token(token)
    flask.session["user"] = user
    return flask.redirect("/")


@app.route("/login")
def login() -> werkzeug.Response:
    return oauth.google.authorize_redirect(flask.url_for("auth", _external=True))

I would hugely appreciate any help.

When I run locally, I start with:

export FLASK_APP=foo && flask run

When I run inside docker container, i start with:

.venv/bin/gunicorn -b :8080 --workers 16 foo

Dichroscope

Ah, important extra information. If i start with --workers 1, I don't have this problem. What am I missing? – Dichroscope

how you fix the error can you guide me – Lastditch

Answer 1

Issue was that SECRET_KEY was being populated using os.random which yielded different values for different workers and thus, couldn't access the session cookie.

Dichroscope

I had the 'CSRF Token Invalid' problem after moving my code from 'traditional' deployment (flask+uwsgi+nginx) to docker(gunicorn). It's the only thing that worked. – Tannatannage

i am facing the same problem in fast api how i can fix this – Lastditch

The way to fix this is to generate the random secret key once and then copy/paste it as a string value into your application's configuration. If you regenerate the value each time the application loads, then each instance of your application will have different secret keys, which will cause these types of errors. – Benzyl

I am also having this same issue using fast API. Running from localhost if that makes a difference. – Hord

Answer 2

How I Fix My Issue

install old version of authlib it work fine with fastapi and flask

Authlib==0.14.3

For Fastapi

uvicorn==0.11.8
starlette==0.13.6
Authlib==0.14.3
fastapi==0.61.1

Imporantt if using local host for Google auth make sure get https certifcate

install chocolatey and setup https check this tutorial

https://dev.to/rajshirolkar/fastapi-over-https-for-development-on-windows-2p7d

ssl_keyfile="./localhost+2-key.pem" ,
 ssl_certfile= "./localhost+2.pem"

--- My Code ---

from typing import Optional
from fastapi import FastAPI, Depends, HTTPException
from fastapi.openapi.docs import get_swagger_ui_html
from fastapi.openapi.utils import get_openapi

from starlette.config import Config
from starlette.requests import Request
from starlette.middleware.sessions import SessionMiddleware
from starlette.responses import HTMLResponse, JSONResponse, RedirectResponse

from authlib.integrations.starlette_client import OAuth

# Initialize FastAPI
app = FastAPI(docs_url=None, redoc_url=None)
app.add_middleware(SessionMiddleware, secret_key='!secret')




@app.get('/')
async def home(request: Request):
    # Try to get the user
    user = request.session.get('user')
    if user is not None:
        email = user['email']
        html = (
            f'<pre>Email: {email}</pre><br>'
            '<a href="/docs">documentation</a><br>'
            '<a href="/logout">logout</a>'
        )
        return HTMLResponse(html)

    # Show the login link
    return HTMLResponse('<a href="/login">login</a>')


# --- Google OAuth ---


# Initialize our OAuth instance from the client ID and client secret specified in our .env file
config = Config('.env')
oauth = OAuth(config)

CONF_URL = 'https://accounts.google.com/.well-known/openid-configuration'
oauth.register(
    name='google',
    server_metadata_url=CONF_URL,
    client_kwargs={
        'scope': 'openid email profile'
    }
)


@app.get('/login', tags=['authentication'])  # Tag it as "authentication" for our docs
async def login(request: Request):
    # Redirect Google OAuth back to our application
    redirect_uri = request.url_for('auth')
    print(redirect_uri)

    return await oauth.google.authorize_redirect(request, redirect_uri)


@app.route('/auth/google')
async def auth(request: Request):
    # Perform Google OAuth
    token = await oauth.google.authorize_access_token(request)
    user = await oauth.google.parse_id_token(request, token)

    # Save the user
    request.session['user'] = dict(user)

    return RedirectResponse(url='/')


@app.get('/logout', tags=['authentication'])  # Tag it as "authentication" for our docs
async def logout(request: Request):
    # Remove the user
    request.session.pop('user', None)

    return RedirectResponse(url='/')


# --- Dependencies ---


# Try to get the logged in user
async def get_user(request: Request) -> Optional[dict]:
    user = request.session.get('user')
    if user is not None:
        return user
    else:
        raise HTTPException(status_code=403, detail='Could not validate credentials.')

    return None


# --- Documentation ---


@app.route('/openapi.json')
async def get_open_api_endpoint(request: Request, user: Optional[dict] = Depends(get_user)):  # This dependency protects our endpoint!
    response = JSONResponse(get_openapi(title='FastAPI', version=1, routes=app.routes))
    return response


@app.get('/docs', tags=['documentation'])  # Tag it as "documentation" for our docs
async def get_documentation(request: Request, user: Optional[dict] = Depends(get_user)):  # This dependency protects our endpoint!
    response = get_swagger_ui_html(openapi_url='/openapi.json', title='Documentation')
    return response


if __name__ == '__main__':
    import uvicorn
    uvicorn.run(app,    port=8000,
                log_level='debug',
                ssl_keyfile="./localhost+2-key.pem" ,
                ssl_certfile= "./localhost+2.pem"
                )

.env file

GOOGLE_CLIENT_ID=""
GOOGLE_CLIENT_SECRET=""

Google Console Setup

Lastditch

Security question: Can anyone set the user field of the session? I worry Depends(get_user) is not offering the type of "protection" people think it is – Seibold

@Seibold if you encode token with private key so it will not be decoded the get_user is in background can me modify so it will decode token with private key and show user detail that we have encoded at the time of creation of token – Lastditch

Thanks for this answer. It worked locally but on deployment, it still gets the error. – Forbore

@RobinMuhia254 can you share the error – Lastditch

Answer 3

@adamcunnington here is how you can debug it:

@app.route("/auth")
def auth() -> Union[Tuple[str, int], werkzeug.Response]:
    # Check these two values
    print(flask.request.args.get('state'), flask.session.get('_google_authlib_state_'))

    token = oauth.google.authorize_access_token()
    user = oauth.google.parse_id_token(token)
    flask.session["user"] = user
    return flask.redirect("/")

Check the values in request.args and session to see what's going on.

Maybe it is because Flask session not persistent across requests in Flask app with Gunicorn on Heroku

Adverbial

thanks. Here are the results: - Docker + gunicorn (16 workers) TrhHO856SVe7EA9INtHKZ85rnieDKS None authlib.integrations.base_client.errors.MismatchingStateError: mismatching_state: CSRF Warning! State not equal in request and response. - Docker + gunicorn (1 worker) enq7AxTUI5lfhTvDwvMKRe6pn5Hah8 enq7AxTUI5lfhTvDwvMKRe6pn5Hah8 - Flask webserver eqahv9lpZOHpPmTabsqBNSDtEN4TEI eqahv9lpZOHpPmTabsqBNSDtEN4TEI Why does the 16 worker scenario cause the issue?! I still only make 1 request. – Dichroscope

@Dichroscope I'm trying to reproduce the issue with github.com/authlib/demo-oauth-client/tree/master/… gunicorn app:app -w 16 --bind=0.0.0.0:5000 But I can't reproduce it. When does it happen? I tried many times, there is still no error. – Adverbial

It happens immediately - as soon as I boot the webserver with 16 workers, if I try to visit the homepage - which redirects me to login, i always get the mismatched CSRF error. – Dichroscope

@Dichroscope follow authlib twitter account, we can arrange a zoom meeting to debug this problem. I'll DM you. – Adverbial

@Dichroscope can you create a sample github repo to reproduce the issue? Maybe it is because this #30985122 – Adverbial

thanks so much for your help and for offering a zoom to fix. As it happens, your link was incredibly useful - I feel stupid, it was because i was using os.random to generate the SECRET_KEY and so this was inconsistent across workers and thus, they couldn't access the session state and hence the CSRF. Changing this to a fixed string fixed the problem - thank you again! – Dichroscope

Answer 4

I have encountered the same problem in FastAPI. What works for me is setting the same secret key in both - sessionMiddleware and oauth.register - places:

In respective python module:

# Set up OAuth
config_data = {'GOOGLE_CLIENT_ID': GOOGLE_CLIENT_ID, 'GOOGLE_CLIENT_SECRET': GOOGLE_CLIENT_SECRET}
starlette_config = Config(environ=config_data)
oauth = OAuth(starlette_config)
oauth.register(
    name='google',
    server_metadata_url='https://accounts.google.com/.well-known/openid-configuration',
    client_kwargs={'scope': 'openid email profile'},
    authorize_state='Oe_Ef1Y38o1KSWM2R-s-Kg',### this string should be similar to the one we put while add sessions middleware
)

In main.py (or wherever you declare app = FastAPI()):

app.add_middleware(SessionMiddleware, secret_key="Oe_Ef1Y38o1KSWM2R-s-Kg")

Urbanist

Answer 5

When running gunicorn with multiple workers this problem showed up (not when running it with only a single worker). Upon closer inspection I realized that some of the links created in flask were set like this: redirect_uri=url_for("callback", _external=True) Flask uses the currently served scheme, which in my case was "http" (running behind the gunicorn server). However, the gunicorn communication with the auth0 server (this was my issue where the csrf token error popped up) was done using https (served by gunicorn). Setting the following solved the problem: redirect_uri=url_for("callback", _external=True, _scheme="https").

I hope this helps someone.

Standard

Answer 6

I fixed this problem by hard-refreshing my app in my browser. It seems I had changed some of the client IDs in my code and restarted the app, but I was still clicking the login button from an out of date version of my own app in my browser.

Bascomb

i get this a lot during development if i've failed to fully authenticate and it seems like my brower cookies are all messed up. killing the browser and restarting make this error go away. – Ligurian

Answer 7

well, this is the least recommended method, but nothing else worked for me.

In the oathlib\oath2\rfc6749\parameters.py, under "parse_authorization_code_response", I commented out the following lines:

if state and params.get('state', None) != state:
    raise MismatchingStateError()

I'm not trying to integrate with a web app, simply writing a python script to build a youtube playlist.

Agincourt

Answer 8

I got the same error message with my implementation with django oauth client (Authlib). In my case, it was due to an invalid local domain as a redirect URL ('http://exampledomain:8000' instead of 'http://exampledomain.com:8000').

When I corrected it, it worked:

ultimapi = settings.oauth.create_client('ultimapi')

def user_authorization(request):
    if (request.user.is_authenticated):
        return (redirect('home'))
    return ultimapi.authorize_redirect(request, 'http://exampledomain.com:8000')

def authorization_callback(request):
    info = ultimapi.authorize_access_token(request)
    print(info)
    return (redirect('http://localhost'))

Heavensent

Answer 9

Another thing that can go wrong, specifically for the ClassLink OAuth provider:

If ClassLink is not configured correctly then the button in a user's launchpad home page for your app will link to your app without including "state" in the query string.

This can cause confusion if you have a "log in with ClassLink" button on your own site that works, but then some users report that they can't log in without mentioning that they're logging in using the button in ClassLink's site.

A possibility in this case, though I haven't tried it, may be to redirect the user to the beginning of your OAuth process when MismatchingStateError is raised.

Likeminded