[Feature Request] (Local) token counter as implemented with Xenova/the-tokenizer-playground

[litecode]

I just came across this topic, after I had posted something similar, just linking for reference:

Display number of tokens in current chat

I am not sure if this information is easily available, but I just noticed that when mentioning files with the @ feature in Ctrl + L in Long Context Chat mode, there is a handy feature that displays the token length of the file next to the file name. That is very cool. It made me wonder if it would be possible to display the cumulative token length of the chat somewhere - i.e. where the value was updated as the chat progressed, just to see how large the chat was actually getting. In case it ma…