Websites area >3-dots menu on the site > manage users

You can remove the client that is added to this site and then invite that person as a collaborator without removing the Client role

Client role is tied to a username while the Collaborator one to an email so you just need to create new Collaborator Role and the assign it to a website