Thanks. So, on to the questions...
"Do I have to go manually change the Max Width settings for each and every post that requires this?"
No. The media settings are supposed to be the defaults that apply for every new post you create. Changing those settings doesn't affect images in already published posts (except if you delete and re-insert the images). And changing the settings all the time to adjust the images in each new post isn't a reasonable way to solve your problem. What you need to do is set the default widths that would suit the majority of cases.
The width of your main column is 570px. When you insert an image that's wider than that, and select the Full Size option, the theme will insert a version that's scaled down to 570px, so that the image will cover the whole width of the post. That's ok for plain images but wrong for captioned images: the extra frame pushes the image a little to the right, and the rightmost part of the image as well as the right side of the frame get cut off. The 558 I suggested in my first reply is the width that will produce correct full-width captioned images. If you never want full-width captioned images, then set a smaller width for the Large option, and an even smaller one for the Medium option. To give you an idea of what you should be aiming for, the first two captioned images on your post "Please Sir, I Want Some More" are 400px wide.
"How do I insert a captioned photo in the right size if I can't scale it down after inserting it?"
When I wrote you can't do that, I meant you can't do it the way you've been doing it (clicking on the image in the Visual editor and clicking the percentage tool). As I explained, this will change half of what it should, and you've seen the result: captioned images that spill outside their frame. To change the width of already inserted captioned images, you need to switch the editor to HTML and modify the image shortcode. The shortcode for a (linking) captioned image is like this:
[caption id="attachment_NNN" align="WHATEVER" width="WWW"]<a href="LINK URL"><img src="IMAGE URL?w=WWW" alt="" title="FILENAME" width="WWW" height="HHH" class="size-WHATEVER wp-image-NNN" /></a> CAPTION TEXT[/caption]
To effectively change the width, you need to change all three instances of WWW.
"How do I solve my problem of whole paragraphs of text disappearing, and other images scrambling, when I hit publish?"
Since you say "hit publish", I assume you're referring to drafts. If so, this has nothing to do with images, captions or resizing: it's a glitch with drafts that has been going on for months. Many users have complained about this (even about drafts disappearing altogether), and so far WP has nothing to say. Just avoid using drafts: when you start a new post, write a couple of words or lines, set the post visibility to private, publish the post, continue editing and updating, turn to public when finished. If you've got completed drafts you haven't published yet, my condolences! Before you attempt to publish them, switch the editor to HTML, copy all, paste and save in a text file so that you'll have a backup if the draft vanishes.