How is the token cache stored?

Discussions
1
  1. Is the cache stored for about an hour deletion timeout?
  2. Is the cache stored separately for the account or separately for the chat?
  3. Let’s say that I have finished my work in chat-1. If I give a task in chat-2 with the same model, will my token cache in chat-1 be untouched until the deletion timeout?
  4. Let’s say that I have finished my work in chat-1. If I give a task in chat-2 with the other model, will my token cache in chat-1 be untouched until the deletion timeout?
2 Likes
3

hi @Artemonim I will check with the team for the actual duration.

  1. The duration depends on the provider with default being 5 minutes and the 1h cache being charged at higher rate.
  2. The cache is per chat thread as offered by AI provider.
  3. Yes it would be untouched.
  4. Yes it would be untouched.
2 Likes
4

Feature request: Put a timer to let the user know the exact time of cache deletion.

5
  1. Create a feature request.
  2. I assume that it will be difficult (and not very necessary) to make it work stably.
6

Most users do not run over the cache time and it would need extra explaining why there is a timer at all when users should not need to think about it.

1 Like
7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.