Stunnel [HTTPS] connected users showing up as server's own IP in Connections

[kennnnnnnnnnnneth]

I have BI running on a Win10 box, and for the most part I've got everything humming along smoothly. I have Stunnel set up for HTTPS webUI access on my LAN. I have now set up a VPN gateway on my network that can talk to my BI server's subnet. That is all working fine - I can VPN in from work, navigate to BI's address with HTTPS, see my cams - all good, working as expected.

Naturally, the next step when exposing a server to the world is to ensure that it is secure, and even more importantly, make sure we can detect if (when) something worms its way past the VPN and starts hammering away at my BI login.

This is where stuff stops working as expected.

When I log into BI directly to the unsecured HTTP address (bypassing Stunnel directly to BI), I can see each of the user logins and their respective source IP addresses. This is as expected. If I see a suspect IP and block it using the "Limit Access by IP Address" function, it blocks it. All good. See screenshot "bi_http.png" which shows two logins and their respective IPs.

When I log into BI using the secured HTTPS address (translating thru Stunnel), I can see each user login, but their source IP address is the same as the server address!! At first this was 127.0.0.1 (localhost), the Stunnel default (and perfectly fine for my use). So, based on the best advice offered on Stunnel.org and roughly 3 hours scouring this very forum, I changed my stunnel config file to specify my host's actual IP. Unsurprisingly this did not solve the problem at all - The users still all show up under the same IP, but that IP is now the host IP I entered into the config file. This is equally useless for identifying intruder's IP. The biggest issue with that obviously is that I (and BI) cannot ban IPs, automatically or manually, because it would be banning its own localhost IP, causing all sorts of problems. See screenshot "bi_https.png" which shows the two logins appearing as the localhost IP.

I have searched for hours and it appears I am the only person on the planet experiencing this issue with Stunnel or BI.

Am I screwed? Does using Stunnel permanently nuke any ability to manage (or even view!) connections in BI, thereby making literally all of BI's Advanced Web Server tools pretty much useless?



Meaningful bits of Stunnel config:

debug = info
output = stunnel.log
[blue-iris]
accept = 443
connect = 81
cert = stunnel.pem

 

[bp2008]

You need to figure out how to have stunnel add an X-Forwarded-For header to all the requests (google should help). Also make sure Blue Iris has the "Use X-Forwarded-For headers" box checked in BI Settings > Web server > Advanced.

Also you should be aware that the X-Forwarded-For header can easily be spoofed and Blue Iris doesn't have any protection against that, so if your BI instance is publicly accessible on its plain HTTP port then it presents an attack vector to bypass authentication if you have anonymous authentication enabled for LAN users. All the attacker needs to do is guess what your LAN IP range is and spoof a header saying that is his address.

This vulnerability has been around forever.... I'm gonna send a note to BI support to see if they'll fix it by providing an IP whitelist for trusting that header.

 

[TL1096r]

Stunnel was nothing but a hassle for me, and like you, I had issues that no one else seen. I setup OpenVPN through the Asus router and now I use the OpenVPN app - so far it was the best solution and no issues.

 

[kennnnnnnnnnnneth]

You need to figure out how to have stunnel add an X-Forwarded-For header to all the requests

Did some checking around - turns out this was supported via a patch developed by the HAProxy team, but that patch only went up to Stunnel 4.32 which is ancient, and Stunnel otherwise doesn't natively support X-Forwarded-For at all. Stunnel does have a different setting which should work, transparent = source which rewrites the address on the encapsulated connection to that of the client IP, but that is only available on Unix based OSes - and since BI only runs on Windows, that puts us in a bit of a pickle.

There is one workaround I am considering, but which may require a bit of messing around - Installing the WSL (Windows Subsystem for Linux), and installing and running Stunnel inside an Ubuntu shell. That would, presumably, allow me to set up and use the transparent = source option. I will update if this is successful.

 

[bp2008]

You could use something like nginx instead. It has a Windows version I think, but I never used it on Windows. Using UI3 with a reverse proxy server · bp2008/ui3 Wiki

 

[kennnnnnnnnnnneth]

Thanks bp2008 -

Short Answer - I will be going the nginx route.


Long Answer - There is no combination of technologies which will accomplish a transparent proxy to BI via Stunnel. This is because BI does not support receiving those modified packets Stunnel is sending. I can’t see inside BI and why exactly it is discarding them, but I watched with Wireshark as the packets were passed to UI3, yet it did not reply. The behavior was the same as when UI3 detected bad or suspicious packets - I think (though I can’t be certain) that UI3 sees those packets with the rewritten source address as broken or malicious.

I had installed Stunnel4 in WSL2, which is basically a light VM, an Ubuntu instance specifically, and had set up a blind transparent proxy from the Windows side (using netsh) to send all :443 (HTTPS) traffic to the WSL2 Ubuntu instance. This was working fine. I then set up Stunnel inside Ubuntu to take the :443 traffic from the Windows side, and pipe it out to :81 (HTTP) back on the Windows side to talk to BI. So far, so good - just a standard Stunnel setup, and worked the same as if it was running natively in Windows. Access was good.

Where it broke down was “transparent = source”. Unlike Windows, this didn’t throw an error in Ubuntu, because Stunnel supports ”transparent = source” in a linux Kernel. Great! So we go to test and…. Nothing.

After hours of fiddling and testing, I discovered the following:

1. Stunnel works just fine when run thru the WSL2 Ubuntu instance on my BI server. Even uses the same .conf and .pem.
2. Stunnel‘s “transparent = source” option, which does not work on the native Stunnel Windows app, does work in Stunnel in the WSL2 Linux kernel.
3. Not only does the transparent option work in WSL2, but using Wireshark I could even see the new (correct!) source address on the packets being sent from Stunnel to UI3.
4. While the “transparent = source“ option is set, UI3 will refuse to reply; when I delete the transparent option from stunnel.conf and restart it, UI3 will reply.
5. This shows, definitively, that the failure is occurring in UI3. My assumption is that because “transparent = source” has to rewrite the source IP of the packet, it is doing so in an invalid manner, or UI3 isn’t equipped to handle a rewritten source IP outside of X-Forwarded-For (this is where nginx comes in).

Hopefully someone in the future will see this thread and save themselves a headache.

 

[bp2008]

@kennnnnnnnnnnneth

I think you are getting "UI3" mixed up with "Blue Iris's web server". UI3 runs in a web browser, not in Blue Iris. Anyway what you've said doesn't make much sense.

I'm not familiar with the transparent mode of stunnel, but if it makes stunnel try to forward the IP packets as if stunnel was a router, then I'm not surprised it doesn't work. Windows is probably receiving the TCP SYN packet and sending the ACK directly to the client and stunnel is not intercepting it because, I don't know.

 

[kennnnnnnnnnnneth]

I think you are getting "UI3" mixed up with "Blue Iris's web server". UI3 runs in a web browser, not in Blue Iris. Anyway what you've said doesn't make much sense.

Yes, you are right - UI3 is a front end, served to the client by the BI web server. Saying UI3 was slightly inaccurate on my part, but as they are part of a functional technology stack and operate as a single unit, in this particular situation the distinction between "UI3" and "BI Web Server" is entirely semantic.

I'm not familiar with the transparent mode of stunnel

That's OK - I have become extremely familiar with Stunnel's inner workings over the last few days. I've been using proxies of various scopes for years; since this is my first adventure running both the proxy server and client on the same machine, and specifically my first time utilizing Stunnel (and its delightful flaunting of accepted standards), I have read every last scratch of documentation, every forum post, and every blog available on the topic. I have also, since last posting, reached out to Michał Trojnara, the man who wrote and maintains Stunnel. Despite not holding a commercial license or a support plan, he (or someone he works with) was gracious enough to reply and confirm some things:

• Stunnel does not officially support X-Forwarded-For headers, except via HAProxy's haxy patch. The latest Stunnel version to support the patch is over 10 years old, and so it is not an option for security and stability reasons.
  • There are other third party patches as well; one is part of a prohibitively expensive package, one includes a routing "feature" that breaks my iptables mangle table, and the other two were unstable and crashed Stunnel immediately.
• Stunnel, when compiled and run on a *nix kernel (this means NOT Windows), DOES (as of 2018) support a similar function to pass the source IP to the destination - enter "transparent = source". This is what I was attempting to do with my WSL2 instance.
• Stunnel's transparent mode does try to forward the packets as if stunnel was a router, but rather than adding a header that can be understood as communicating the source IP while maintaining the correct reply route (such as what happens with X-Forwarded-For headers), Stunnel's transparent mode rewrites the source address in its entirety. This is observable in wireshark captures.

Armed with this official data from Michał, I believe you are correct-

• WITHOUT transparent = source, Stunnel receives a SYN from client 192.168.5.10, passes the packet to BI with source 127.0.0.1, BI sends its stuff back to 127.0.0.1 where stunnel is waiting to help bridge the gap back to the client.
• WITH transparent = source, Stunnel receives a SYN from 192.168.5.10, passes the packet to BI with new source IP 192.168.5.10, and BI replies directly to 192.168.5.10 because it has no real way to know that it was supposed to reply to Stunnel at localhost. This is where X-Forwarded-For headers would solve the problem, but unfortunately as was confirmed above, that is simply not an option with Stunnel.

In conclusion, I was wrong to blame BlueIris for the failure - While BI's network stack is anemic, it complies with standards, and can't be expected to go beyond the basic implementation included in the package. So onward to Nginx, a solution which, while being a much larger pain to set up and keep running, I am far more familiar with, and will support those aforementioned X-Forwarded-For headers.

Hope that clears up what I was trying to communicate, and explains for yourself and others exactly why Stunnel can't act as a true transparent proxy for BI.