Oh, this is going to get interesting real fast.
Legally, you don't need anyone's permission to link to them.
Mark has stated that linking to warez will get your blog shut down asap. The videos on Youtube are usually there without permission. Same idea. :)
One thing that comes to mind. As I understand the Youtube setup, the uploader has to mark if their video is to be publically accessable. That means the uploader is giving permission for other sites like wp.com or daria.be to embed and use the video in their own sites. I would think that would mean legal permission to use and display.