I agree with WastingTime completely.... and to follow up on what he mentioned, there are some links we may prefer to, or even forget to set as "open link in a new page." If those links are clicked, you or your readers lose your/their place in the infinite scroll and are forced to start all over again.
Other issues are the sidebar. We obviously use the sidebar for pertinent information we want the reader to see and eventually click on. The side bar gets lost after one or two posts.
On top of that, if you have multiple posts with multiple videos, each time a video is played, Adobe plug ins eat more and more of your memory up without resetting because you've gone to a new page. As a result, the further along you scroll the more your system resources are drained, and if there's too much of a strain on system resources subsequent videos won't even play properly.
I personally don't like it one bit, and at the very least, I think people should be given an easy option to turn it on or off. The whole point to blogging is to be unique, and forcing everyone into the same format takes away people's ability to customize their blog to achieve any sort of uniqueness.