Visible (and very annoying) Key Frame Pulses In All Streams After Recent Update...

[Pogo]

Everything was just fine and my higher frame rate live streams were as smooth as water flowing. The sub streams were even respectable in their own right -- until I updated to 5.6.5.9 yesterday.

I'd been running nicely with 5.5.8.2 which I've remained with faithfully since my last attempted upgrade to 5.5.9.6 back in August. I experienced major stability problems with my setup for whatever reason(s) and I wasted no time trying to figure it out. Just crossed my fingers and rolled back to 5.5.8.2 and had been rock solid since -- and I've been very happy with the overall performance of my setup.

So yesterday was the day I thought it would probably be a good idea to try bringing things up to date..., well..., just because? The upgrade went smoothly with no obvious hitches, but after the restart and everything was fired back up I immediately noticed pulsing in my normal 12 cam live stream view on a 24" display (which doesn't exactly provide exceptional detail with that view in the first place). Solo'd a couple cams and the same thing, only larger. Main streams, sub streams, 2MP, 5MP, didn't matter. The pulses seemed to clearly be at my two I-frame intervals of one and two seconds corresponding with the cameras set up as such. I realize you can sorta see the key frame effect in a clear hi-res stream, but It literally makes a clean static view look like a 1 or 2fps stream.

I really had no inclination to spend all night pissing up a rope trying to troubleshoot it beyond toggling some settings and slamming the hood a few times. I figured reverting to the previous version would get me back to normal, but the issue is still present even after doing so. I've killed encoding completely with no change. Restarted all cameras. Stopped and restarted BI. Rebooted. Did my normal monthly system maintenance routine including a Windows update. Did all the usual stuff one would normally try under the circumstances. Still pecking at it for possible common clues, but pretty much blind in the dark at this point.

As usual, I'm at your mercy for any suggestions or enlightenment that may remedy the situation -- short of a registry enema and starting from scratch.

TIA, gang

 

[Flintstone61]

Somebody will chime in,,,I have no idea about this situation......Unless trying a 1 to 1 keyframe might be something to troubleshoot with to see what happens.

 

[Pogo]

So after almost 48hrs. since this began, I can definitely say it is a somewhat dynamic condition and is now barely discernable on some cameras including the one that was initially the most obvious and caught my attention originally. The others still exhibit the pulsing to what I would consider an annoying level with no obvious rhyme or reason or real commonality among those cameras.

And as we so often do, blaming a software change or upgrade can be a convenient excuse for not completely dotting "I"s, crossing "T"s and paying enough attention to details or following the pesky directions. Not that I didn't have good reason based on previous experience to suspect the upgrade as possibly having at least something to do with this new condition -- and since it became obvious immediately after said upgrade --, but I think it's only fair to try it again to see if the condition comes back as pronounced as it was the first time -- especially since it has diminished somewhat and an A/B comparison at this point could probably provide a relatively fair assessment of things..., unless of course the current condition is the result of lingering mouse tracks left behind from the upgrade even after rolling back to the earlier firmware. Beyond my pay grade to make that determination, but could still be a distinct possibility with no way to know without another easily observable performance change one way or the other.

Anyway, nothing to lose but time since nobody wants to take a shot at an educated explanation otherwise.

 

[Pogo]

No major change with the exception of realizing I DID lose the most recent 10 days worth of clips upgrading the first time. Recovered after trying pretty much all the database tools and have no idea which one(s) were successful.

Not a very impressive performance over the past coupla days for the vaunted BIue Iris.

 

[Mike A.]

I've seen the typical pulsing with individual cams but not for all cams as you're describing. Search for video pulsing here and you'll find various things to try.

Not sure what to suggest if you're seeing it across all cams. Maybe something common about how you have the cams/BI set up. Post the cam's video settings for one that shows the effect and the video settings from BI and maybe we'll see something.

 

[Pogo]

Thanks for the input. The issue has been put on the back burner for the time being, though I've been keeping a pretty close eye on it. It's almost definitely key frame related and varies from camera to camera (in intensity) vs. being a blanket effect across all cameras as might be the case with a general video display setting, bad cable, etc.

Playing with the bit rates as mentioned in one of the posts that popped up in your suggested search will be an interesting test. As mentioned, the pulsing does have a dynamic character depending on scene, color, activity, all those things. I use a combination of CBR and VBR largely dependent on what the defaults of the camera happen to be unless there is a specific reason to change -- which there may indeed be now. Not sure I agree with the assertion indicating this is just how H264/H265 looks as presented by one poster though. How it works maybe. Thought it was supposed to be the exact opposite in how it looks.

I'll be able to devote some time to it over the next couple days and see where it goes.

Thanks again.

 

[Mike A.]

Maybe also send a note to Ken and ask if something changed in that version since you were good before. Probably won't get a quick response but might save some tail chasing.

 

[bp2008]

I would be shocked if the Blue Iris update has anything to do with this, since Blue Iris doesn't affect how a camera encodes its video stream. It is simply coincidence that you noticed it after a BI update.

In my experience this issue only arises because of a suboptimal encode, which happens before the video even leaves the camera. You should open some of the RTSP streams in VLC media player or any other player independent of Blue Iris, so you can verify that it has the same pulsing effect as you see in Blue Iris. Also try from another device, like a tablet running a different operating system (not Windows), to further verify that it is in fact the camera doing this.

Since it affects all (or most?) of your cameras at the same time, I think it is probably affected by the lighting in your environment. E.g. under a particular lighting situation, the scene is more difficult to encode cleanly, so you end up with more obvious pulsing at keyframes.

Some people go on a wild goose chase thinking they can blame a bad cable, network switch, or something of that nature, but that is a misconception about how video is transmitted by IP cameras. It is not like analog video where electrical/RF noise has a very real impact on the video quality. With IP camera video, it is essentially impossible for data loss or corruption or an insufficient link speed to cause an effect that is anything like this.

I wrote about the pulsing keyframe issue at length in this wiki page, but unfortunately sometimes you just can't get rid of it entirely. In particularly bad cases, different firmware on the camera can be beneficial, but that is only if the firmware happened to include better encoder tuning than the previous firmware (which is unlikely but I have seen it before with an older dahua 4k bullet).

 

[Pogo]

I don't think my missing clips was just a coincidence any more than my immediately noticing this issue after said upgrade. The lighting or camera encoding didn't change that fast. And the camera that initially caught my eye was an analog camera hanging off an Amcrest (Dahua) DVR and processed through that. No IP involved aside from the link to my BI server. I run everything set to H264 due to the limitation of my particular i7. My IP cams are a combination of basic junk according to most opinions, but have worked well and looked pretty good for my purposes as I experiment and tweak along the way toward upgrading to better devices.

The above said, I did back peddle a bit in my second post and am clearly open to suggestions and any educated experience in further troubleshooting to a hopeful resolution -- or at least a logical and provable reason for what I'm seeing.

Would love to read your wiki but the link repeatedly fails.

 

[bp2008]

Ah, sorry, my comment above was only about the pulsing keyframe problem, not about missing clips.

Blue Iris does not use a robust database system, and clip database corruption is alarmingly common. The usual fix is to right click a clip and choose Database > Repair/Regenerate. It takes some time to complete, during which new recordings can't be made. But it tends to fix the clip database at least until the next corruption occurs.

In particular, there was an update within the past few months that subtly breaks existing clip databases. Somewhere in the ballpark of the 5.6.0 update I think, but I don't remember for sure. Anyway every machine I've updated past a certain point will start having intermittent issues with clips not appearing in the list, and the issue goes away after I regenerate the database.

 

[bp2008]

Would love to read your wiki but the link repeatedly fails.

Weird, if you open the wiki link in the ipcamtalk menu at the top of the site, look for the "Troubleshooting Streaming Problems" page.

 

[Pogo]

Cool. That worked as did the link in your post afterward. Indications would suggest either a Google or Bitdefender flag on the link in your post initially, but that's a whole other subject about unintended results of preventive security mechanisms. (Yawn.)

A great general explanation of the pulsing issue in your wiki. It's basically what I'm experiencing to varying degrees, but was definitely on the exaggerated side after the upgrade and remained so until it eventually settled down. If not BI, could it have possibly been the Intel HA process itself after the reboot? I get the reasoning behind it not likely being a BI issue, but the coincidence certainly involves something that occurred during or after the upgrade -- which does also include external video processing elements that I neglected to consider.

Regardless, I guess my greater confusion is about what seems to be the general acceptance of the artifact at all in any form. It was so minimal across my entire setup that I only really noticed it at all on a couple of Wyze cams that hadn't been converted yet to POE ethernet devices (which turns them into totally different little rascals, btw). As mentioned, all else was liquid smooth to my eye otherwise.

I appreciate the input and am anxious to inspect the individual camera details (where available) and tweak accordingly to observe any major changes -- hopefully positive ones.
Gonna be kinda hard to do in a blizzard, tho. May be a few days before I can see the results of my efforts!

Thanks.

 

[bp2008]

If not BI, could it have possibly been the Intel HA process itself after the reboot

I suppose it is a possibility. But unlikely.

 

[Pogo]

So after some experimenting and following the various suggestions, I can definitely say VLC is delivering very clean streams in all cases except intentionally introduced extremes. There is still a hint of key frame, but it's much more along the lines of barely noticeable vs. annoying.

A/B comparisons of the same optimally adjusted camera streams between BI and VLC (as displayed on the BI server machine) yield observably different results with VLC delivering a smoother stream. Note I used the term "observably", not "dramatically". That said, there still appears to be a certain degree of what I would consider instability in the consistent rendering of the key frames as viewed with BI. It's sometimes quite a bit more noticeable than other times without much rhyme or reason for such a change.

At this point I'm pretty much resolved to accepting it as perhaps environmental variables unique to the various cameras and locations -- that I simply never noticed before my recent BI upgrade (...., ahem).

It was not a noticeable annoyance before it became one. There's not much more I can do or say except to thank everyone for their input and suggestions. It has certainly been an educational exercise that I will continue diddling with here and there to hopefully determine a common thread that actually explains it.

Thanks again for the help.

 

[looney2ns]

So after some experimenting and following the various suggestions, I can definitely say VLC is delivering very clean streams in all cases except intentionally introduced extremes. There is still a hint of key frame, but it's much more along the lines of barely noticeable vs. annoying.

A/B comparisons of the same optimally adjusted camera streams between BI and VLC (as displayed on the BI server machine) yield observably different results with VLC delivering a smoother stream. Note I used the term "observably", not "dramatically". That said, there still appears to be a certain degree of what I would consider instability in the consistent rendering of the key frames as viewed with BI. It's sometimes quite a bit more noticeable than other times without much rhyme or reason for such a change.

At this point I'm pretty much resolved to accepting it as perhaps environmental variables unique to the various cameras and locations -- that I simply never noticed before my recent BI upgrade (...., ahem).

It was not a noticeable annoyance before it became one. There's not much more I can do or say except to thank everyone for their input and suggestions. It has certainly been an educational exercise that I will continue diddling with here and there to hopefully determine a common thread that actually explains it.

Thanks again for the help.

Did you report this to BI support?

 

[Pogo]

It seems to have been implied that I'm just chasing my tail trying to solve a problem that isn't generally perceived as one, so no, I've only tried the few suggestions given here and intended to continue on as I said -- just keep dickin' with the situation to see if there's an obvious common thread I'm simply overlooking.

One comment I will offer is that a couple cheapie ZOSI 2MP analog cams hanging off an Amcrest DVR I'm playing with show the least amount of visible disturbance while providing a surprisingly decent image for their particular application. Pretty ironic.