state is echoed back in the query string sent to the redirect_uri. It has two purposes:
The original use, as per its name is to transmit state information from the initiating webpage to the redirect_uri. For example, I have a process that sends users a link that allows them to link their account to some other resource. The link contains information (a token) that describes that resource. So the user clicks the link and they are taken to a web page, where they are redirected to the authentication server. I need that token to make its way through the authentication process back to the redirect_uri, so that the business logic behind the redirect_uri can finish linking the resources to the users account. I do this by using "state" to carry that information. Given "state" has no size limit, if can be used to carry all sorts of very useful information.
As a means to protect against XSRF... It is very easy for a third party to forge a request to the authentication server and trick your redirect_uri into accepting the response. If the state information is a randomly generated value and the redirect_uri is in some way able to validate it as being its own, then you can protect yourself against this.
It is a misconception that the random value must be bound to or stored in some session. Though this is a valid way of verifying the state, it is flawed. If there is a delay in authentication (for example a user hanging in the login screen for several hours), then such a validation method would fail if the session had expired. It would also fail if the user had initiated multiple logins by mistake or by double clicking.
It is far better imho to use a private key to sign the random value. For example take the SHA1 or SHA256 hash of a random number + the private key as the signature. Make the state value a combination of the the random number and the signature. Noting that you would convert the binary value to Base64. It is common to use a full stop as a separator in the same manner as a JWT:
signature = BASE64_SHA256(random + privateKey)
state = random + "." + signature
You verify this by splitting the state value at the full stop as follows:
verifyValue = BASE64_SHA256(split(state,".")[0] + privateKey)
If verifyValue = split(state,".")[1]
then the state is valid. No pesky session storage, so validation is much faster.
It is perfectly feasible to combine the two purposes into one. That is you can take the known value that you want your redirect_url to know about, add the random number and then add the signature as a SHA256 hash of both values. E.g:
signature = BASE64_SHA256(data + "." + random + privateKey)
state = data + "." + random + "." + signature
This is super secure because all the state information is fully signed and protected, which doesn't happen if you use a simple security token stored against the session (as per the method in OpenID Connect: Create an anti-forgery state token)
10/11/2023: IMPORTANT Addendum
The privateKey is server side. So this method is suitable for Step A of Authorization Code Grant as per RFC 6749:
In the initiating step A you need to post a request to your own server to generate the signed state field. Your server returns a 303 redirect with Location: set to the authentication server with the signed state field.
For example:
HTTP/1.1 303 See Other
Content-Type: text/html;charset=ISO-8859-1
Location: https://authenticate.somehost.com/Authenticate/?client_id=someid&redirect_uri=https://www.myhost.com/returnpage&state=FFcI1gmokWkJpjjy3pG6WPl7KryZVxPGSOx9F4WvaY8.gcnIBIUpBzOh3e-iwUq1Y_gsK-Y5SCSS3ahGrIaU6zQ&code_challenge=dOTqvgv0B5p4Nur2RrkWjysAMCSNaeu73pv3xWBBWPQ&code_challenge_method=S256