Recently I had to transfer a NetScaler VPX configuration to a MPX appliance. This was not much of a hassle as most of the configuration can just be re used but be sure to check as i.e. the ntp configuration did not quite reach the new appliance.

Nevertheless as always I was not only applying the old configuration but also doing some optimizing. The number one desire was to use rules with Exchange 2010 SMTP. The challenge was not indeed to create a rule within Exchange but rather that the rule was based on client addresses!

There was already a load balancing virtual server created which worked perfectly but you have to keep in mind that with a standard load balancing virtual server the Exchange servers will always only notice the NetScalers IP address.

First things first you will have to make sure that the clients IP address will reach the Exchange servers. This can easily be done by choosing your Service or Service Group and check „Use Source IP“ (USIP) within the Advanced settings. After completing this single configuration your source IP will be transmitted to your Service i.e. in my case Exchange Servers. You will notice that communication with your virtual server is broken now (test environment or at least test virtual server!). This problem arises as your server will now answer directly to your requesting client as he now knows its IP address and tries to start a direct connection. As your client did never start a direct connection with your server it will drop all packages and data will be lost in space.

To overcome this problem you will need to use your NetScaler as a gateway for your defined servers. This would mean you will have to make sure that your NetScaler is able to route all packages as your gateway does and you always will have to recall that those servers are not using your standard default gateway anymore. If only some clients are using a such configured virtual server you can use manual defined routes also.

As you can see this might not be always an option as in addition to changing the gateway on your production servers reverse traffic will flow through the NetScaler again and often you just want a load balancer but will not use any optimization so why spend your precious throughput on useless data pass through?

From this point on in my example the Exchange server will get the clients IP address and is able to use rules to check i.e. wether the client resides in the internal or external network and use different policies then. If you will just test this scenario use a route for your test client and sure a test virtual server for load balancing. This will make sure your production environment is not being affected.

The communication flow so far is IP based so your server will always have to answer through your NetScaler as he did start the communication. To break down this barrier we will have to make the server think that communication originated from the client itself and the client that it still communicates with the virtual server so packages will not be dropped.

This can be done by changing the redirection mode of your virtual server from IP based to MAC based or Direct Server Return (DSR). You will accomplish this by choosing „MAC Based“ in your virtual servers advanced settings Redirection Mode option. The change from IP to MAC based forwarding is all there is to do on the NetScaler. From now on all packages send from the virtual server to your server i.e. Exchange will still get inserted the clients IP address as of USIP enabled on the services but in addition with MAC based redirection enabled the NetScaler will replace its own MAC in every package with the one of your services server.

You have to know that this will only work with protocol type ANY, so you might have to change your services server and/or virtual server. If you will not you will get an error message, while trying to switch your virtual server to MAC based redirection.

Your services server will now think that communication came directly from your client but will never answer this request. This behavior is due to the fact that your services server will not own the virtual servers IP thus dropping the package. A network trace will show that from your NetScaler there will be three SYN attempts but you will never see an ACK from your server.

First we will have to make the server own your virtual servers IP address but not make it addressable from outside itself. You sure know that this would give your network a headache and you do not want that. Add a loopback adapter on your services server uncheck all protocols leaving IPv4 checked. Configure your virtual server IP address and subnet mask and you are done here. You should choose a sensible name for the loopback adapter so you will recognize it at once. This will not just make configuration easier but also troubleshooting in case 😉

You can add the loopback adapter within your device manager choosing add legacy hardware. The loopback adapter can be found with choosing Microsoft as manufacturer.

At this second your services server also owns the NetScaler virtual server IP address but the sad news it it will mostly still not answer if your services servers are beyond Microsoft Windows Server 2003. If you are using a newer version your TCP/IP stack will be secured and your server will never answer a request on your production LAN network adapter.

This will lead to some configuration on your production and loopback adapter. To understand what you will be configuring you might wanna read a bit in here.

netsh interface ipv4 set interface "Your production network adaptor name" weakhostreceive=enabled
netsh interface ipv4 set interface "Your loopback network adaptor name" weakhostreceive=enabled
netsh interface ipv4 set interface "Your loopback network adaptor name" weakhostsend=enabled

The magic is done!

Be sure if you tested with manual routes or NetScaler as your gateway before to revert those settings otherwise you might run into serious problems. Your client will now still communicate with your load balancing virtual server thus leverage its capabilities. Your services servers however will answer directly to your client.

To sum up the needed configuration tasks:

  1. Create your Load Balancing virtual server with services servers
  2. Check USIP on every Services server or your Service Group
  3. Change redirection mode on your virtual server to MAC based
  4. Create a loopback adaptor on every services server with the virtual servers IP address and uncheck any other protocol
  5. Configure wakhostreceive and send on your services servers network adaptors as described
  6. Test

In a follow up article I will show you the necessary alteration to use the same technique for Citrix Provisioning Services TFTP.

Always mind your network!

Getagged mit

27 Responses to HowTo load balance while preserving a clients source IP but not using the NetScaler as your Gateway

  1. Evren sagt:

    This worked very well with netscaler and 2 Virtual servers in esx but did not work for 2 physical hp servers.

    • Stefan Holste sagt:

      Hi Evren,

      I am not sure why it should not but maybe you are using some special LB network drivers which might prevent this from working.
      If you are still interested you might disable any HA or similar in your network configuration and just use one single network card for testing and see what happens.

  2. David Garcia sagt:

    Great post, very informative. Worked for me perfectly!!!

    Many Thanks!!!

    • http://www./ sagt:

      i love this drawing, and it reminds me of the feeling of one of the greatest places on earth, the mutter museum. when i lived in philly i used to spend some of my lunch breaks there. i did a couple pieces for ‘the ganzfeld’ based on exhibitions there.nice.renee

    • http://www./ sagt:

      Hej SanneSÃ¥fremt det er 4 Hour Body du følger (hvilket det ser ud til) er rugbrød med lavt sukkerindhold “tilladt”. NÃ¥r han skriver “brød” begrundes det med at det er fyldt med simple kulhydrater. Det er rugbrød ikke pÃ¥ samme mÃ¥de. Grunden til at han ikke nævner det er, at rugbrød kun spises i Danmark. Det er tæt pÃ¥ umuligt at opstøve udenfor landets grænser.

    • I read the news reports Saturday about the Texas man who was jailed for 83 days for skipping jury duty.Yep, 83 days. In 2009 — not 1909.So, for 83 days, the man didn’t go to work, lost his job, couldn’t provide for his family and acquired a record.Don’t even get me started on Texas billing rape victims for their rape kits. I’ve seen two polls by national polling organizations on the topic of Texas seceding from the Union and both polls indicate a majority of the American people would like to see it happen. Washington DC is ready and willing to take Texas‘ place at the Union table.

  3. David Garcia sagt:

    I only did this for the SMTP vserver and services. For the rest of the services (OWA, AES) it does not matter the source IP.

    • Stefan Holste sagt:

      Hi David,

      you are correct as mostly you would like to write some policy about who is allowed to send mails and who is not.

  4. Johnd338 sagt:

    I genuinely enjoy studying on this website, it holds good content. Never fight an inanimate object. by P. J. O’Rourke. kefedfedeadf

    • You’re so interesting! I do not suppose I’ve read through anything like that before. So nice to find somebody with some unique thoughts on this subject matter. Seriously.. thank you for starting this up. This web site is one thing that’s needed on the internet, someone with some originality!

    • such a cool post! I had no interest in the LA Marathon (no particular reason) until I saw all of this…it looks awesome and really get to run through a lot of the cool LA areas! AWESOME…THANKS!

  5. Allen Fox sagt:

    Best netscaler/exchange post ever. Thank you!

  6. Eric Covert sagt:

    You are the bomb! Walked straight through your article and fixed an issue with Right Fax we have been dealing with for the last 6 months. You deserve a medal or at the very least a bacon sandwich!

    • Stefan Holste sagt:

      Hi Eric,

      I would like that bacon sandwich very much but I am guessing we are some kilometers apart 😉

      Great to hear I could help!

  7. Chris sagt:

    Great article – However as I have no idea about exchange, I am missing the concept of which services/protocols in exchange are to be created on the Netscaler. The OWA is the one that is talked about most, but how do you configure SMTP, RPC, AB, POP3 etc or am I overcomplicating it? As mentioned elsewhere the official Citrix Ex2010 guide is broken and does not work.
    Is there an article that you know of, that has an overview of which services can and should be load balanced/configured on the netscaler for EX2010/2013?

    • Stefan Holste sagt:

      Hi Chris,

      apparently you are right that most guides do have some flaws in one part or another. This is why you have to read some and see what fits best in general and for your environment.

      It is also a good idea to dig a bit deeper in your NetScaler configuration as sometimes you are able to optimize some described configuration with some features or only a but more knowledge.

      For example most guides are missing to use pattern sets which are great if you would like to filter for more than one URL part but then initiate always the same action.

      Pattern sets are for example great for content switching policies or to write a negative policy.

      As I do have some web services only available via https I did create a policy which does https rewriting but only for those which are available via https and note those listed in the pattern set.
      I guess I should write a short article about that.

      As soon as I did implement Exchange 2013 I will see wether there is some use to write an article about it.

      Stay tuned even if I am unfortunately not in the situation to update this site regularly.

  8. […] Before we go further I highly encourage you to read this Blog post by Stefan Holste. The rest of the steps in my guide are lifted right from his solution but if you want to know WHY we are doing this go read it. It’s well written and also shows how to set this up. You can find it on his blog article titled HowTo load balance while preserving a clients source IP but not using the NetScaler as your Gateway […]

  9. Rosario sagt:

    I needed roughly 2 years to find a solution for smtps to Exchange 2010 CAS-Server. I started on NetScaler MPX 8200, Version 10.1x – we did not succeed not even with the Citrix partner firm that helped us install/configure the whole thing. Same situation in February, as we started over from scratch with another partner firm with Version 10.5.56-12 – But I found a solution configuring two load-balancing-virtual servers, one for port 465 and one for port 587, both using SSL_TCP with correct Subject-Alternate-Name certificate. Unfortunately both the service-groups have to be configured to use TCP. I tried with SSL_TCP and also with SSL_BRIDGE, but without success. I disabled all legacy SSL-Protocols to only leave the TLS-protocols to avoid poodle and other flaws.
    In the IIS-SMTP-RECEIVE-LOG you will only see the NetScaler’s subnet-IP. But in the syslog Entries on the NetScaler I am able to see the client-source-IP. I think I do not need any more. Please correct me if I am mistaken.

  10. Rosario sagt:

    Ohhh, one caveat though: despite seeing the client-source-IP on the NetScaler side I can not make a direct correlation with IIS-Log-Entries, unless time-stamps correspond more or less. So having the client-source-IP directly in the IIS-Logs would for certain be a good thing to have. So I will have to consider USIP mode. The whole NetScaler is already configured to use MacBasedForwarding. But I would have to make according tests to make smtp work the way explained here.

    • Stefan Holste sagt:

      Hi Rosario
      you will only need the USIP if you want the Client IP being seen by the backend. This is useful if you for example like to filter who is allowed to send mail via smtp and your smtp server uses the client IP to determine wether it allows or denies to send an Email.
      If you are not using the clients IP on your backend servers then you mostly will not need to preserve the clients IP within the TCP stream.
      If it is an http service you might wanna try inserting the x-forwared-for header or a header your web service is able to evaluate.

  11. Jørn Breinholt Frandsen sagt:

    It seems to me, that by using the MAC based redirection mode You presuppose that the Loadbalancer and the Services are on the same VLAN, or am I missing something?

    As I understand the destination IP from the LB to the Service is now the LB-VIP address and forwards the packet using the MAC address of the Service and that can only be done within the same VLAN.

    If I am wrong please correct me and if I am right please tell me what to do if the LB and the Services are on different VLAN’s maybe on different WAN locations.

  12. […] HowTo load balance while preserving a … – 17 Responses to HowTo load balance while preserving a clients source IP but not using the NetScaler as your Gateway […]

  13. […] HowTo load balance while preserving a … – 18 Responses to HowTo load balance while preserving a clients source IP but not using the NetScaler as your Gateway […]

  14. Henrik sagt:

    Stefan, I’m having trouble getting this to work when also using a content switch and two arm mode. Is CSW not supported with DSR?

  15. rosielu69 sagt:

    Порно фото галереи, более 500 тысяч фотографий

  16. ScottUneve sagt:

    Hi. Really good, useful post, and a little out of the box. I learned something new today!

Schreib einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind markiert *