I figured this out.
The All users page does not list all users. It lists only the users who can add or change pages on the site. That means viewers are not listed. If you want to change a viewer to another role so they can add or update pages, you have to send them another invitation for the new role at which point they show up in the All Users list. There, you can change their role to Editor, Author, Contributor, or Administrator (but not to viewer). My guess is that if you want to change somebody in the All Users list to a viewer, you have to delete them and send another invitation for the viewer status.
I think this is a confusing design.