Network Congestion

[TheBobC]

Hoping someone can point me in the right direction. Here is my system

Cameras(all connected via wifi):
4 Dlink 5222Lb PTZ
2 Ctronics CITPC-590c PTZ

Head End:
Intel NUC7i5
Ubuntu 22.04
Agent DVR

Routing:
Asus ZenWifi XT8 Base Unit
Asus ZenWifi XT8 Mesh Node 1
Asus AC-RT3100 Mesh Node 2

Internet:
Comcast 1 Gig Cable

Other Devices:
Approximately 50 other various devices such as computers, RPI for storage server control, Alexa devices, etc. 12 or so of these are ethernet and the rest Wifi.

I got Agent DVR setup and functioning mostly as hoped. All was happy. Then, after 48 hours, my network started to have huge congestion issues. Connecting to other devices in network, and getting any kind of internet was very, very slow. I did a speedtest and it was down to around 11 Mbps or so (from its usual 950+). This was true of a device on ethernet as well as wifi. After some troubleshooting, I narrowed it down to the newly added Agent DVR setup. I did some reading, and followed some advice to separate out the cameras and Agent DVR to a different subnet. So my normal network is in the 10.0.0.1 range. I set the cameras and the ethernet card in the NUC to the 10.0.1.1 range (cameras still on the same SSID as everything else). I also set the Wifi in the NUC with a setting in the 10.0.0.1 range(ethernet was 10.0.1.1) so I could communicate with it. I put one computer on the network that happened to have two NIC cards with its second card also on the 10.0.1.1 range so could see/troubleshoot the cameras directly. (the Agent DVR NUC is headless). This functioned but sadly in the same way as previous. Within a day my network was unusable. Immediately after unplugging the NUC, full usability was restored.

Is there a better way? What is dragging my network down so much?

Quick note, as I illustrated in my rough drawing, the cameras are connected to all three of the mesh nodes due to placement around the home.

Open to ideas.

 

[wittaj]

You are lucky you haven't had issues before now! You now reached the point where one more device is too much.

Wifi and cameras do not go together.

There are always ways if you don't want to run an ethernet cable.

You need power anyway, so go with a powerline adapter to run the date over your electric lines or use a nano-station.

Maybe you are fine now one day with wifi cams, but one day something will happen. A new device, neighbors microwave, etc.

Cameras connected to Wifi routers (whether wifi or not) are problematic for surveillance cameras because they are always streaming and passing data. And the data demands go up with motion and then you lose signal. A lost packet and it has to resend. It can bring the whole network down if trying to send cameras through a wifi router. At the very least it can slow down your entire system.

Unlike Netflix and other streaming services that buffer a movie, these cameras do not buffer up part of the video, so drop outs are frequent, especially once you start adding distance. You would be amazed how much streaming services buffer - don't believe me, start watching something and unplug your router and watch how much longer you can watch NetFlix before it freezes - mine goes 45 seconds. Now do the same with a camera connected to a router and it is fairly instantaneous (within the latency of the stream itself)...

The same issue applies even with the hard-wired cameras trying to send all this non-buffer video stream through a router. Most consumer grade wifi routers are not designed to pass the constant video stream data of cameras, and since they do not buffer, you get these issues. The consumer routers are just not designed for this kind of traffic, even a GB speed router.

So the more cameras you add, the bigger the potential for issues.

Many people unfortunately think wifi cameras are the answer and they are not. People will say what about Ring and Nest - well that is another whole host of issues that we will not discuss here LOL, but they are not streaming 24/7, only when you pull up the app. And then we see all the people come here after that system failed them because their wifi couldn't keep up when the perp came by. For streaming 24/7 to something like an NVR or Blue Iris, forget about it if you want reliability.


This was a great test that SouthernYankee tried and posted about it here:

I did a WIFI test a while back with multiple 2MP cameras each camera was set to VBR, 15 FPS, 15 Iframe, 3072kbs, h.264. Using a WIFI analyzer I selected the least busy channel (1,6,11) on the 2.4 GHZ band and set up a separate access point. With 3 cameras in direct line of sight of the AP about 25 feet away I was able to maintain a reasonable stable network with only intermittent signal drops from the cameras. Added a 4th camera and the network became totally unstable. Also add a lot of motion to the 3 cameras caused some more network instability. More data more instability.
The cameras are nearly continuously transmitting. So any lost packet causes a retry, which cause more traffic, which causes more lost packets.
WIFI does not have a flow control, or a token to transmit. So your devices transmit any time they want, more devices more collisions.
As a side note, it is very easy to jam a WIFI network. WIFI is fine for watching the bird feed but not for home surveillance and security.
The problem is like standing in a room, with multiple people talking to you at the same time about different subjects. You need to answer each person or they repeat the question.

Test do not guess.

For a 802.11G 2.4 GHZ WIFI network the Theoretical Speed is 54Mbps (6.7MBs) real word speed is nearer to 10-29Mbps (1.25-3.6 MBs) for a single channel

And @TonyR recommends this (which is the preferred way IF you want to do wifi)

The only way I'd have wireless cams is the way I have them now: a dedicated 802.11n, 2.4GHz Access Point for 3 cams, nothing else uses that AP. Its assigned channel is at the max separation from another 2.4GHz channel in the house. There is no other house near me for about 300 yards and we're separated by dense foliage and trees.

Those 3 cams are indoor, non-critical pet cams (Amcrest IP2M-841's) streaming to Blue Iris and are adequately reliable for their jobs. They take their turns losing signal/reconnecting usually about every 12 hours or so for about 20 seconds which I would not tolerate for an outdoor surveillance cam pointed at my house and/or property.

But for me, this works in my situation: dedicated AP, non-critical application and periodic, short-term video loss.... if any one of those 3 conditions can't be achieved or tolerated, then I also do not recommend using wireless cams.

 

[TheBobC]

There you go. Thank you for the very thorough explanation and examples. I guess if I went with a situation where it was only view on demand i would not have the issues? Meaning put the cameras back on the main subnet and they are there to view if I want to, but no DVR?

 

[wittaj]

Yes, the NVR now makes it streaming 24/7 to the DVR and your system simply cannot keep up. It eventually loses enough packets that you get this issue.

 

[SodFather09]

I will never understand why people install Wi-Fi cameras, you need to power them, so why not use data and power in one cable ?

 

[TheBobC]

Well, maybe you would understand if you were in my place. The property is large with multiple legs of power. Thus running network over power has never been fruitful for me here. It is also situated in such a way as to be prohibitively expensive to run cat 6 to each camera. I have had the six cameras functioning for some time on demand on Wi-Fi without any problems. We were having some issues in the neighborhood that made me want to step up to DVR, and I had an old unused NUC available. As the answer above indicated, seems that was one step over the line.

 

[wittaj]

Powerline adapters run the data over your existing electric lines and is much better and stable than wifi. I have been using a pair for years for that problem run.

 

[SodFather09]

Well, maybe you would understand if you were in my place. The property is large with multiple legs of power. Thus running network over power has never been fruitful for me here. It is also situated in such a way as to be prohibitively expensive to run cat 6 to each camera. I have had the six cameras functioning for some time on demand on Wi-Fi without any problems. We were having some issues in the neighborhood that made me want to step up to DVR, and I had an old unused NUC available. As the answer above indicated, seems that was one step over the line.

What your saying makes zero sense, you still need to run a wire if it’s Wi-Fi or Poe . So why wouldn’t you run one cable with data and power? I actually find it much harder to install 12v camera over a Poe camera. I have a somewhat large home 4800 sq ft and all my cameras have cat 6 connections, and each room in the house also has a cat 6 and a cat 7 cable in it and that includes all three bathrooms . You can get high quality pure copper cat 6 on Amazon for cheap. Just run the cable or if you can’t do it buy all the cable and have someone do it for you.

 

[Coldair]

what does task manger say about your network use?

 

[kd5mdk]

So any lost packet causes a retry, which cause more traffic, which causes more lost packets.

Has anyone set cameras to transmit over UDP? That seems much more suitable for a streaming use case than TCP.

For a 802.11G 2.4 GHZ WIFI network the Theoretical Speed is 54Mbps (6.7MBs) real word speed is nearer to 10-29Mbps (1.25-3.6 MBs) for a single channel

You're not going to get it everywhere in your house, but having an 802.11ax network is now much cheaper and higher performance than what you would see with g or n, and is not a significant cost compared to what I'm seeing people quote for cameras.

 

[TheBobC]

what does task manger say about your network use?

When I placed them on a different subnet, the task manager no longer included them.

 

[Coldair]

When I placed them on a different subnet, the task manager no longer included them.

what are they without them and what does the other subnet say?

I have 18 cameras on like with several of them being 8 mp my network is on 100 meg traffic