Color4K-X - one man's journey towards clarity

I just wanted to share a crude methodology which I stumbled into while trying to figure out why my live feed/videos were always oddly pixelated - almost as though the lens was being hit with a fine rain mist. You could see it everywhere, but certain places in frame it was annoyingly evident.

This became my approach -

• I had already added a time overlay to my camera feed, and I noticed that the seconds would freeze, then accelerate .. back and forth.
• Based off the above observation, I soon realized that some of that seemed tied to the 'rain mist' pixelation in my frame.
• I did the usual chasing of my tail, and I was probably into my 3rd hour on my second day when I realized something. The pixelation was repetitive, and predictable. You had to blow it up sometimes to really observe that, but it was clear.
• Once I saw it was repeating in a cyclic fashion I also caught on that the longer the pixelation duration, the worse the image quality was. More mist/pixelation/focusing-unfocusing and so on, however you want to describe it.
• So I decided that if I could somehow gradually shorten it's period, the pixelation might gradually diminish.
• I soon realized that raising the bit rate was slowly shortening the pixelation duration.
• I kept it in H.264, General, 4K res, CBR, Custom bit rate with 30FPS (which I have now dropped to 20)
• I set about raising the bit rate - first by 1024, then 512, 256, 128, 64, 32 and then 16 over the course of a hour or so. Each time I'd blow up the same section of the frame to see how a section of blowing grass looked.
• As I raised the bit rate, the grass crept towards being 'quieter' with less noise, and clearer motion, without as much pixelation and stuttering. *And the out-of-focus/pixelation phase was becoming shorter and shorter.
• I stopped when I was satisfied and had a bit rate of 6696 to show for it. Each blade of grass was clear, and it's waving motion was smooth, and that cyclic focus/defocus/pixelation was minimized. I did all this at night as well.

I'm certain as a lay person I didn't stumble upon the best methodology for this, but it served it's purpose. I am sure I will get back at it to refine it a bit more but for now everything looks so much better than prior and I have to magnify things beyond x12 to begin seeing ugly pixelation degradation.

This is where I'm at now - I dropped the FPS recently after once again reading on here that maxing out FPS was just a huge waste. I'm down from 30 to 20. That appears to mitigate stuttering as well. And once my FPS dropped to 20, it issued a new bit rate of 6400, which I have just left. I will begin using my approach to hone in on the perfect image Q in the days to come. For now I'm happy. *It also appears my Synology NAS, and camera are happier when the iFrame rate is 2x the FPS for some reason. I didn't input this; I noticed the camera software jumped it up while I was adjusted the bit rate, and when I dropped FPS down to 20, I looked down and there was a new iFrame rate of "40", again 2x the FPS.

Here, below, are the various settings that I've landed on. If anyone sees anything which catches their eye, like a setting which seems way off just holler. I am all ears, and happy to entertain changes in my .. journey towards clarity.







And a screen shot from just now -



Peace

I just tried raising the bit rate up to 8064, and then did a short movie. The motion is pretty good I think. Definitely not perfect quite yet, but not bad. I think the aberrance I see at one point may be from the WDR level. I know I'm pushing that.

View attachment Bit rate raised to 8064 Not bad but not perfect May be WDR though .mp4

Hi nightlife

Usually the advice on here out to match the fps and iframe rate. In your case it should be 20.

I'll leave it to those more knowledgeable on the subject to offer an explanation why

And this is a 16x magnification screen capture of the rocks to the upper right in the frame above. It reveals that pulsing I was referring to. It's like every 2 seconds the image snaps back to the right; repeat. This, I think might be improved by tweaking the frame rate up or down.

View attachment Camera bit rate = 8064 = pulsing pixelation period ~ 2 secs.mov

The one to one ratio of frame and iframe rates is based on the fact that Blue Iris starts detection with a full frame, iframe. That produces the best motion detection in Blue Iris. Since Blue Iris isn't being used here it's not as important in this case.

The "snapping back" effect is a result of the iframe rate, IMHO. As the iframe rate goes up that will increase and there's no real way, that I know of, to avoid it. It's a result of the full frame being "painted" every two seconds, frame/iframe ratio of 2:1.

Griswalduk said:

Hi nightlife

Usually the advice on here out to match the fps and iframe rate. In your case it should be 20.

I'll leave it to those more knowledgeable on the subject to offer an explanation why

Click to expand...

Great observation, and I totally agree with that. I've seen that repeated many, many times here as well. Initially it was one of those data points that I was tweaking, and I did try it initially matching the FPS, and even slowly raised it up to the full 150, and back again. But at some point the software (Dahua) bumped it up to double the FPS, and I don't know why. Once that happened I just adhered to it's will, and continued on. I also noted, as you likely did that the Substream was also bumped x2 on it's own. That is a change I did not make either.


So I am curious to know why the camera did that - could it be tied to the bitrate? But I have to recall that once I dropped to 20 FPS, the camera bumped the iFrame to 40 on it's own, and that was even with the bit rate of 6400 which it also auto-filled. So some of these changes 'seem' to be of the cameras choosing.

@sebastiantombs thanks ...every days a learning day

Again, @Wildcat_1 is the real expert on all of this. I'm just a "duffer" at best.

That is precisely what I love about this forum - the advice flows so freely, and is so on point that we all benefit. Every day there is another detail to hammer out, to research and to put to bed before going on to the next. I laugh every time I google something and this forum pops up at the top of the results. I naturally click on the link, and am .. home once again.


And this freely flowing exchange of information trickles down, until there is an army of IPC users stepping in to lend a helping hand for the next new guy. Of all of it, I am deeply appreciative.

+1 to the above

Dahua cams auto set the iframe as double, at least that has been my experience.

I would suggest bumping the bitrate up. Most of us find the 4MP needs 8192, so the 8MP will need more. My 4K/X works well at 12000, but each field of view is different. Looking at yours I think a higher bitrate would help especially with more FPS.

Work on your other parameters to try to eliminate need for backlight. Sometimes we need it but get it as low as possible.

do you have any screenshots with normal picture settings? (no wdr, no vivid, all around 50)
for me the colors look a bit awful (reddish etc), not sure why you set it like this. also wdr tends to wash out everything

user8963 said:

do you have any screenshots with normal picture settings? (no wdr, no vivid, all around 50)
for me the colors look a bit awful (reddish etc), not sure why you set it like this. also wdr tends to wash out everything

Click to expand...

Heading into supper time here, but leave this with me, and tomorrow around the same light I will try as you request.


But odd that you think it's a bit awful (reddish) because for me being, perhaps more familiar with the colour of the cliffs, lillac bushes and so on would have to say the vast majority of the colours are spot on. Some things like the dead balsa in the lower right corner do seem quite red though. It's always nice to burn everything to the ground once one thing has improved, to see if that one tweaked setting was throwing any of the others off.


I'm game. I'll post whatever I get tomorrow. Thanks for your input.

wittaj said:

Dahua cams auto set the iframe as double, at least that has been my experience.

I would suggest bumping the bitrate up. Most of us find the 4MP needs 8192, so the 8MP will need more. My 4K/X works well at 12000, but each field of view is different. Looking at yours I think a higher bitrate would help especially with more FPS.

Work on your other parameters to try to eliminate need for backlight. Sometimes we need it but get it as low as possible.

Click to expand...

Awesome. Ok, I was wondering about it's limits. I was thinking there might be a point of diminishing returns, but I'll kick it up several notches tomorrow in the daytime. Bitrate up, with backlight need dropping. Check.


Thanks!

I'd suggest a few setting changes. Are you testing these with motion in the scene? Or just after a pretty image? Not everyone has the same requirements is why i ask. For most its about crisp clear realistic images of possible bad guys or other events.

1. Bitrate at least 8192 but 10240 is even better
2. image controls- those look severe to me but try putting all to 50 and tweaking after you get the rest cleared up
3. exposure- daytime I run cust range 0-2. At 16.33 you're only allowing 1/60. It can run a much faster shutter than that
4. DNR at 100? Holy blur Batman! Dial both 3d and 2d back to under 50. I run 30/20 daytime
5. WDR- unless necessary shut it off. Never over 20. It introduces artifacts and blur

Like I said, we all have different requirements and very different scenes. For me its about ID of humans and vehicles. YMMV

View attachment 4116PLocal_ch1_20211204153540_20211204153618.mp4

+100 to what @bigredfish said above.

DNR should never be that high, it'll artificially soften/smooth up your picture so much. You should play with the extremes of those settings to see for yourself what it does so you can learn what they do. Look at a pic with DNR at say 20 then move it to 100 and see what it does to your shot.... I run the minimum DNR I can based on my exposure and gain... I usually end up ~40 or lower.

Just from looking at your configuration screenshots and reading about your bitrate setting methodology, it is clear you like to experiment. I suggest you try maxing out the bit rate at least once, just to see if it improves the detail capture very much (when you have digitally zoomed in a lot of course).

Yep, I would agree given what you are trying to capture - critters and such - to see what max bitrate does.

It also sounds like you were struggling with i-frames causing image quality to shift temporarily. This can be really confusing when you don't know where it is coming from! I wrote quite a bit about this issue in a wiki article a few years ago. I wrote the article for Blue Iris users, but some of the information is relevant to any IP cam.

I began adjusting my setting today to hone in on a 'pulseless' image. I can say this, because of the Synology Surveillance Stations own requirements, this all became incredibly tedious. After a few hours without success, I honed on the fact that for whatever reason I had to apply congruent changes to Sub Stream 2, and 3. At first I was simply focused on the Main Stream, and might try a 1:2 FPS:I Frame interval, but because I didn't apply that same 1:2 math to the Sub Streams I was doomed I think. Near hour 4 I discovered that in actuality a 1:4 FPS:I Frame Interval was required to illicit a cessation in the "pulsing". Also to sate both the NAS, and the camera it seems a FPS of 20 may be my sweet spot. After that I said heck, and bumped up the bitrate to 12288 to see how everything made out, and voila I have honed in on what for now appears to be the optimal setup. There is a blizzard stirring outside right now so it's difficult to truly know if the "pulsing" has stopped for good, but focusing on the feed for a minute at a time appears as though it's vastly diminished, or absent.

Next I adjusted video, and returned everything to 50%, turned off WDR, and turned off backlighting. Went from vivid to standard as well. Under exposure I have shutter speed 0-2, gain 0-16, exposure comp @ 50%, and the NR are the following - 3D = 30, 2D = 20.

Video setting are as follows:



Here is a screen capture once done as above - looking at the live feed after shooting this screen capture reveals very high quality video, with a perfectly in sync clock, and zero pulsing noted through the snow fall. I'll tray and do a video capture to check..



And a video capture from a minute ago which will lay out the results thus far - I sense aberrance but it may not be the I Frame Internal but other video settings. I'll wait till the blizzard passes before proceeding further.

View attachment Post tweaks of FPS, I Frame Interval (1 to 4) bitrate much higher etc.mov