Hmm. So, part of it could be the "No signal" status which causes
Blue Iris to auto-generate an image and I'm not sure how it handles different aspect ratios.
But auto-cycle can also yield results like this when you use an H.264 streaming method. H.264 video can't change resolution midway through, so Blue Iris just initializes the video encoder at the resolution of the regular group frame, and any camera that doesn't have a perfectly matching aspect ratio will be rendered with black bars within the video stream. Then when the video is displayed in UI3, if the video size doesn't match your viewport size, you can get even more black bars outside of the video stream. In just the right combination, one set of black bars will be on the sides and the other set will be top and bottom, and presto, your video is letterboxed.
When you stream in JPEG mode, each frame is independent of every other frame, so it can be rendered at the most appropriate resolution and shown with only one set of black bars.
To improve the behavior you get, I suggest you go here and change the group frame size to 1920x1080 or whatever best fits your monitor's size. Do be aware that the higher the resolution you set, the more CPU time it will require to encode your video. 1920x1080 is usually manageable. 3840x2160 (4K) might not be. (note: you
can type in the frame size box, you don't have to select from the pre-filled values)