You misunderstood. They output one single token and then re-ingest the entire input + that extra token to generate the next one. For token 3 that’s the entire input + token 1 and token 2
If the way you write text is to go and re-read the entire conversation from scratch, just to type another letter you are doing it wrong


I know for a fact that for every single letter I type I don’t have to re-read though this entire comment chain. That’s as simple as I’m able to make this comparison for you. If you still can’t understand I’m sorry for your family