[Feature Request] (Local) token counter as implemented with Xenova/the-tokenizer-playground

Implementing a toggle to add a token counter based on the model and corresponding context window would be amazing (for the Pro subscription).
The space can be found here, have not looked into how it exactly works:

3 Likes

This is a great idea

1 Like

I just came across this topic, after I had posted something similar, just linking for reference:

2 Likes