Hello there, Hyhonfire
You can indeed do that by creating a Custom Menu and adding your Pages to it but not adding the static page that serves as your Home page. You can do that by Appearance> Custom Menus in your dashboard.
If your home page is the page that shows some posts, it can get trickier with the theme you're using. Check this guide: http://en.support.wordpress.com/menus/
If, by removing the Home button, you mean removing the Navigation Pane itself. Create a Custom Menu with no entities inside it and use it as your primary navigation.
Hope that helps. Thanks!