Various issues with refresh token rotation

[mtt87]

Description 🐜

Took me a bit of time but I think I finally understood why we were seeing so many errors related to refreshing the token in our app.

I'm following the suggested setup to refresh tokens https://next-auth.js.org/tutorials/refresh-token-rotation which works fine.

The issue is the broadcasting that if you have more than 1 tab open with your web app, it will trigger a session check for EVERY tab and then you end up triggering the refreshAccessToken() function multiple times simultaneously with the same set of parameters (same refresh_token)

This creates an issue because the first request will be successful and obtain a new access_token paired with a refresh_token while all the other calls that were made will get a 400 since the old refresh_token has just been invalidated by the first successful call.

Simplest way to fix this would be to have the option to disable broadcasting and let each tab try to refresh the session only when focused.

EDIT: 3 scenarios where this is a problem

• broadcasting getSession event when a session refresh is already in flight
• opening a new page when a session refresh is already in flight
• focusing a different tab when a session refresh is already in flight

Check here for a lock mechanism proposed solution:
#2071 (comment)

How to reproduce ☕️

In the app try to set a clientMaxAge of 10 seconds

      <AuthProvider
        session={pageProps.session}
        options={{
          clientMaxAge: 10,
        }}
      >

Then add a log in [...nextauth].js jwt callback

    async jwt(token, user, account, profile) {
      console.log('JWT')

Open 2 tabs and move between them, you will notice that every 10 seconds there are 2 calls being made to refresh the session and 2 console logs

JWT
JWT

I can create a codesandbox later after I finish work if needed but it's pretty simple to verify I believe 😄

I'm using "next-auth": "3.23.3"

[balazsorban44]

If true, that would be a very nice fix! So basically there is a race condition here, where two calls made to the /ap/auth/session endpoint will both try to refresh the token, and the second one finishing fails. Right?

[mtt87]

Yes correct, let me explain it better with a practical example.

Imagine you issue access_token that expire after 10 minute and a refresh_token that lasts for 1 month that you can use to refresh your access_token.

The way it works is that when you exchange the refresh_token you obtain a new access_token but also a new refresh_token, which means that the previous refresh_token is now invalid.

Now imagine a timeline where you have 3 tabs open with the same app my-app.com

00:00 the user opens my-app.com on a tab and login into the app, the user now has a session that contains

{ access_token: "at1", refresh_token: "rt1", expire_in: "15min" }

00:01 the user opens another tab with my-app.com and after a brief check, the user is already logged in because there is a valid session
00:02 the user opens another tab with my-app.com and after a brief check, the user is already logged in because there is a valid session
00:03 the user open another tab with a different website and starts reading an article for 5 minutes
-- 5 min later --
05:00 the user decides to go back to tabA where your app my-app.com is open, this triggers a broadcast of the session event to tell all the other tabs where your app is open to check the session
05:01 all the 3 tabs have made a call and the token is still valid so nothing happens, just to confirm we still have the originally exchanged access_token and refresh_token

{ access_token: "at1", refresh_token: "rt1", expire_in: "15min" }

05:02 the user opens another tab and keeps browsing
-- the user goes out for lunch, 30 min later --
35:00 the user is back from lunch and selects one of the my-app.com open tabs
35:01 the tab broadcasts the session event and all 3 tabs are checking at the same time if the session is valid
35:02 the access_token is now expired so all the 3 check session calls have triggered refreshAccessToken()

1st request is successful: exchange the refresh_token rt1 with a new access_token and refresh_token so the jwt callback is returning the updated token as following

{ access_token: "at2", refresh_token: "rt2", expire_in: "15min" }

2nd and 3rd request fail with 400 because they are trying to use the refresh_token rt1 that has just been invalidated so the jwt callback is returning the updated token as following

{ access_token: "at1", refresh_token: "rt1", expire_in: "15min", error: "RefreshAccessTokenError" }

Now we have 3 big problems:

1. This happens in a window of milliseconds and it can trigger a rate limit, imagine you have 5 tabs open it's 5req/s, generally these auth endpoint have a rate limit to avoid bruteforcing etc
2. We added error to the token and the client now believes that the the refresh token exchange was unsuccessful (we send the user to the login again)
3. We have lost forever the latest correct access_token and refresh_token because the last call that was made returned the previous token with the error

// The function was called with the old token containing "at1" and "rt1"
{ ...token, error: "RefreshAccessTokenError" }

[balazsorban44]

Love the details, make sense! I'll come back to this, thank you!

At the end of the day, I think the main problem here is not the broadcasting, but the race condition. Similar to #1455

[mtt87]

Yes but unfortunately the race condition is not solvable in a stateless serverless environment that’s why I think the only way to solve it is to prevent this from happening due to the broadcasting.

To solve this at the server level we would need a persisted state of the refresh token requests that are in flight in order to stop the subsequent request and instead wait but it starts to become very complex :)

[balazsorban44]

I see, and I was thinking the same. fixing it by making it possible to disable broadcasting is half way, but what about clientside code where after some stale time, let's say two requests are going to an external api and they use the access token from session?

I know react-query does fetch deduplication, so maybe we need to do something similar in our client code?

[mtt87]

I’m using SWR for the requests which works similarly to react-query for the deduplication and that’s fine. The problem imho is not deduplicating on the client side for a single page but trying to deduplicate for all the pages.

I’m trying to think at the problem from a different perspective: how about the broadcast is not saying “hey all tabs please refresh the session” but rather “hey I’m
TabA and I’m going to refresh the session”.
Then all the other tabs can wait for either a success event, and after that they can refresh their session or a fail event which means they should logout.

[balazsorban44]

Don't know if SWR does that, but react-query deduplicates site-wise, using a React Context. (Or by pages I guess you meant tabs?)

What you are saying how to go about broadcasting does sound interesting, and I would rather not introduce new options, if we can work something out transparently for this.

[mtt87]

What you are saying how to go about broadcasting does sound interesting, and I would rather not introduce new options, if we can work something out transparently for this.

Going to bed now, I’ll write down a proposal tomorrow with some scenario

[mtt87]

On my side the main question for you @balazsorban44 is:

What problems are we trying to solve broadcasting the getSession event?

I see, and I was thinking the same. fixing it by making it possible to disable broadcasting is half way, but what about clientside code where after some stale time, let's say two requests are going to an external api and they use the access token from session?

This problem has nothing to do with next-auth because you can have 100 requests to external API at the same time and they will not trigger the jwt() callback even if you are using a lambda to proxy the requests, I just tested the scenario where I have a serverless API that's using getToken() and it doesn't trigger jwt()

async function handler(req: NextApiRequest, res: NextApiResponse) {
  const token = await getToken({
    req,
    secret: process.env.NEXTAUTH_SECRET,
    encryption: true,
    signingKey: process.env.NEXTAUTH_SIGNIN_KEY,
    encryptionKey: process.env.NEXTAUTH_ENCRYPTION_KEY,
  })
  // call the API with the access_token contained in the encrypted token object
  ...
}

---

Writing down the full picture so if someone wants to read they have more context.

Refresh token rotation scenario

A session is based on session cookies that are set on successful login by the server lambda.

__Secure-next-auth.session-token
__Host-next-auth.csrf-token
__Secure-next-auth.callback-url

The most common scenario is the oAuth based authentication where an oAuth provider on successful login will issue an access_token and a refresh_token.

If you want to have a completely stateless authentication (no databases needed) then you will add these tokens to the JWT.
Due to the nature of these artifacts, it's not a good idea to put them plain unencrypted in a JWT but rather use an encrypted JWT (JWE).

Next-auth supports this by adding

jwt: {
  encryption: true,
  secret: process.env.NEXTAUTH_SECRET,
  signingKey: process.env.NEXTAUTH_SIGNIN_KEY,
  encryptionKey: process.env.NEXTAUTH_ENCRYPTION_KEY,
},

Next-auth uses the jwt() callback to control this, so in the case of a scenario where you have a refresh token you want to add a check to make sure if the access_token is expired, try to refresh it and return the new access_token and refresh_token.

Note: the jwt() callback is triggered every time the user is checking the session, either by calling getSession() or with the useSession() hook.
What they do is calling GET /api/auth/session behind the scenes.

async jwt(token, user, account, profile) {
  // Return previous token if the access token has not expired yet
  if (Date.now() < token.expiresAt) {
    return token
  }
  // Access token has expired, try to update it
  return refreshAccessToken(token)
},

refreshAccessToken() is a function to exchange the refresh_token for a new pair of access_token and refresh_token that are returned and included in the updated encrypted JWT (JWE).
Important: the previous refresh_token is instantly invalidated.

async function refreshAccessToken(token: JWT) {
  try {
    const refreshedTokens = await request(...)
    return {
      ...token,
      accessToken: refreshedTokens.access_token,
      expiresAt: Date.now() + refreshedTokens.expires_in * 1000,
      refreshToken: refreshedTokens.refresh_token ?? token.refreshToken, // Fall back to old refresh token
    }
  } catch (err) {
    return {
      ...token,
      error: 'RefreshAccessTokenError',
    }
  }
}

Next-auth broadcasting

Next-auth creates automatically a broadcast channel on every tab you open with the same web app.
https://github.com/nextauthjs/next-auth/blob/main/src/client/index.js#L39
Every tab is listening for 2 session events:

• getSession: this is broadcasted whenever you unfocus and focus back one of the tabs where your web app is running, telling the other tabs that they should check their session and in the scenario where you have 3 tabs open it will trigger simultaneously 3 API calls to check the session
• signout: this is telling the other tabs that the user is not longer logged in so they should sync and logout the user as well

Broadcasting issue with refresh tokens

If you have 3 tabs checking the session and triggering the jwt() callback 3 times in the same moment, it means that there are 3 refreshAccesstoken() being called, which means that there are 3 API calls to exchange the token, all of them with the same refresh_token currently available.

1st call: 200 successful, receives the new access_token and refresh_token that are returned and added to the encrypted JWT to substitute the existing access_token and refresh_token.
2nd call: 400 error, the refresh_token used to call the function is now invalid as it's just been used. We add the key error: "RefreshAccessTokenError" that's added to the session so the frontend now believes that the user needs to signin from scratch.
3rd call: 400 error, same as above

The problem is that the 2nd and 3rd call are basically "overwriting" the 1st call and now the latest JWT contains the error and the wrong tokens. The refreshed tokens are now lost and unrecoverable.

Serverless stateless environment

It's important to understand that we are in a stateless serverless environment.
We cannot assume that the 3 check session calls mentioned above are going to the same lambda function, they could be different lambda function and all of them are isolated.
We want to keep a stateless auth so there are no databases involved and nothing can be persisted due to the ephemeral nature of a lambda function.

Proposed solution: option to disable broadcasting some events

If we add an option to disable the broadcasting of the getSession event, we can prevent this from happening at the root.
What does it mean in a scenario where we have 3 tabs?
1st tab that gets focus will check the session and trigger a refresh token and update the session accordingly.
2nd tab and 3rd tab are not aware this happened BUT because updating the session with the new tokens basically means to update the cookies, the 2nd and 3rd tab got the session updated automatically in case they have some background stuff (for example they are polling an API from time to time)
When the user opens the 2nd tab it will check its session and nothing happens as the cookie has already been updated with the new tokens not yet expired, same would happen to the 3rd tab.
Since we are still broadcasting the signout event, in case the user decides to logout or their tokens are expired and can't be refreshed, all the other tabs will still be notified.

[mtt87]

@balazsorban44 sorry to bother you, did you have a chance to read my last comment?
My question for you: what problems are we trying to solve broadcasting the getSession event?

[balazsorban44]

Will have a proper read of this in the next few days, I hope. I actually just had to deal with this problem at work, came to a conclusion and used some different approaches, but ultimately I think supporting token rotation will be something that will need some more thinking. This has certainly become rather a feature request than a bug.