First scenario
A file is deleted from the media library and then a new one with the same name is uploaded shortly after that.
All posts, pages, etc. that we make on our blogs are kept in a MySQL database as are the names of all files that we upload (images, videos, audio files, Word documents, etc.). With databases, what happens is that if you delete something, it is not immediately deleted, it is "marked" for deletion and is then no longer available. On some preset schedule, the database software "packs" the database and all the stuff "marked" for deletion is then removed. This is done to conserve server resources and is especially important when you have very large databases such as those at wordpress.
If you delete a file from your media library, it does not immediately go away as I said, but is marked for deletion. If you try to upload a file with the same name, the database will not let you because even though it is a new file, and the old one of the same name is marked for deletion, the database will not allow that file to be over written and so it ads a number to the end of the file name.
Second scenario
A file with the same name exists in the database from a previous upload and is not deleted. Then a new file with the same name is uploaded.
WordPress will not overwrite an existing file in your media library. What it will do is upload the new file with the same name and put a number after it thus preserving both.
If you think about this, it is a good thing. Perhaps you mistakenly name a new file for upload with the same name as an older upload, and if they allowed overwriting, you could overwrite an older file that was important and possibly not something that could easily be recreated (perhaps it was accidentally deleted from your computer).
Now this situation could be handled a little differently and when wordpress detects the fact that you are trying to upload a new file with the same name as one that already exists in your media library, they could put up an alert box and give you the choice of overwriting the file, or changing the name. While this may sound like the perfect solution, I have been called on many times when someone has overwritten a file on their computer "by mistake" even though this type of warning came up and because they were not paying attention, they just said OK. Result: Old file gone. In some instances I've been able to recover the file if they call me right away and do not do anything else on their computer, but if they don't notice for a few days or a week, there is little chance of recovering it.