When messages are received in the queue, they will go to all agents. An agent will need to take the chat (the button next to the 3 dots on the top) and resolve it.
Add the new agents into the queue, make sure they are logged in to the queue, and when a new message comes everyone should now see it.