Replacing the links is not okay. They are now dynamic and change when a site is customized. If the links are replaced manually, then they won't get updated if we need to make changes to them later or changes based on the customization status of each site.
Just changing the word "Blog" to "Site" or "Website" would be okay if you can find a way to do that.
[EDIT: To clarify the statement above, the WordPress.com Terms of Service have been modified to say "Footer credits and the WordPress.com toolbar may not be altered or removed regardless of upgrades purchased." That means altering the wording in footer credits is not supported. Part of the reasoning for this is that WordPress.com reserves the right to change the text in the credits as needed (i.e. sometimes we may need to add links to font attribution or something else or we may decide to change "Blog at WordPress.com" to say something slightly different) and if you modified the wording, it would be subject to change and it would most likely break any wording modifications you added via CSS.]