While I'm not sure which page you are asking about, I'll tell you what I see on this page:
where you have so far inserted 2 images, side by side. They are both uploaded as full sized 1024 wide but set to display at 210px wide.
My first thought is that each image is too wide to display 4 pictures across the width of your theme:
Remarks: quadruple number = narrow layout with sidebar / narrow layout without sidebar / wide layout with sidebar / wide layout without sidebar
Each image is too wide to fit four across your column width. And that does not even take into account padding which the theme's CSS may incorporate into image display.
wide layout with sidebar max width = 774
divide by 4 and that is the maximum width per image, without padding.
The more accurate way to insert images 4 across the column, once you determine the size (width) you need, is to work in the html editor. Insert one image after the other without linebreaks between them. It is very possible to do it in the visual editor as well, aligning all to the left, and making certain that you don't press return between any of them, but the visual editor does not allow you to see precisely where your cursor is.
I do see the difference in the way the captions display. I'm not sure, but that could be caused by different ways you placed the images or perhaps by switching from the visual to the html editor?
It is best to place your images and text and once you get that looking the way you like, place captions in the image editor at the very end. The reason is that with the visual editor, you cannot be absolutely precise and certain about where your cursor is.
If it were me placing 4 pictures in a row, then I would probably opt for making a table. If you want more information about that, then let us know.