The archived chats are only found in the agents side bar.
But you first have to be in the new layout mode, which requires clicking on this double arrow (if you see it)
If you ever want to go back to classic layout with the double arrow, have to press Ctrl+E (windows) or Cmd + E (mac)