TCP-proxy Client-IP

Community Forum Forums Thunder and AX Series General TCP-proxy Client-IP

Tagged: ,

This topic contains 29 replies, has 3 voices, and was last updated by avatar diederik 3 weeks, 3 days ago.

Viewing 15 posts - 1 through 15 (of 30 total)
  • Author
    Posts
  • #15192
    avatar
    flaneur
    Member

    Hi.
    We have SLB Virtual Service with type TCP-Proxy and we’ve also enabled Insert Client IP option in TCP-Proxy template but all traffic is forwarded from AX1030 internal IP .The question is how we can get source client IP ?

    #15202
    avatar
    ericn
    Member

    If you’re using a vport of type TCP, e.g., port 25 tcp, then you need to look in the TCP header for the information. Similar to an HTTP Template which inserts the client ip in the HTTP Header as a Header Field, a TCP Template mucks with the Options in the TCP Header.

    In my experiment, the client IP address was inserted as a TCP option just after the Window Scale option (plus a 0x01 pad).

    1c 07 01 64 40 01 64 which corresponds with the IP address 100.64.1.100
    1c – option type (not recognized by my wireshark)
    07 – length (7 bytes)
    01 – pad
    64 40 01 64 – Client-IP in hex.

    #15212
    avatar
    flaneur
    Member

    Thank you for your reply but the problem is how our proxy under AX1030 can define the source client IP when we have virtual service with type tcp-proxy ?

    #15222
    avatar
    diederik
    Member

    You need to find out what options your proxy supports.

    If your proxy can not read the inserted Client-IP in the TCP-Options, then you need to use another way.
    If your proxy supports reading the X-Forwarded-For HTTP header, you need to make sure the AX1030 inserts it.

    Normally you would set you Virtual Service to use HTTP/HTTPS and inserting the headers is very easily done in the HTTP Template.

    If for some reason you can not use HTTP, you could look at using aFlex to insert the header in the raw TCP data stream.

    Is there a particular reason why you are using TCP-Proxy and not HTTP?

    #15232
    avatar
    flaneur
    Member

    It seems that our proxy can’t read the inserted Client-IP in the TCP-Options or I can’t find the way how to do this,we use envoy for proxy under AX1030.Unfortunately we can’t use HTTP because we have a lot of connections to our service and in case we’ll use HTTP I think AX1030 can’t handle such amount of connections.Can you help with aFlex, how can I insert the header in the raw TCP data stream ? Maybe you have any example ?

    #15242
    avatar
    diederik
    Member

    Why do you think the A10 can not handle the nr of connections?

    The use of aFlex is going to be much more heavy on your system than using HTTP.
    The aFlex would have look at all packets and buffer data for every connection that needs to have te header inserted.
    Actually it will need to add the header multiple times, every time a new request is coming in, even if it using an existing connection.

    What does the Memory and Data CPU usage look like on peak moment?
    If your device can not handle the extra processing required for HTTP, you should not use aFLex.

    You could change your network setup, so that the return traffic from your proxy goes back to the A10… so you can remove source-NAT.

    What other services are you running on the AX 1030? If you are also using it for a VIP with SSL-Offload, I would strongly suggest upgrading to a newer system as the SSL chipset on the AX is extremely outdated.

    #15252
    avatar
    flaneur
    Member

    It’s difficult to say how many connections we have but I can say that we throughput 100k requests per second and we will have more.We don’t terminate SSL on AX1030 side,that deal our envoy proxy.Actually I thought about how to remove source-nat but for some reason I didn’t find the way how make our service works without source-nat.
    Now when we use TCP type of our virtual service CPU and Memmory usage are very-very low.I can’t find any benchmarks for AX1030 to see how many HTTP connections it can handle and I decided to use TCP to hedge.

    #15262
    avatar
    diederik
    Member

    In an ideal situation the AX1030 should be able to do about 400K new TCP connections per second.
    And depending on the re-use of these connections well over 1 million HTTP Requests per second. I expect it should be able to handle over 6 million concurrent connections.
    But again that all depends on your setup.

    If I were you, I would set the port type to HTTP and use the template, and then see what the performance impact is.

    And if you expect to grow more than the unit can possible handle, get the Thunder 1040S :)

    #15272
    avatar
    flaneur
    Member

    Thank you for your suggestion but our hoster can only provide us AX1030 and these is the only option for us.

    #15282
    avatar
    flaneur
    Member

    Maybe you can help how I can get rid of source-nat ? Maybe this somehow will help me ?

    #15292
    avatar
    diederik
    Member

    I actually already told you :)

    You could change your network setup, so that the return traffic from your proxy goes back to the A10… so you can remove source-NAT.

    Source-Nat is used to make sure the traffic from the A10 to the Proxy also comes back to the A10.

    The only way to remove the need for Source-Nat is by making sure the Proxy sends the return traffic back to the A10.

    So if you know the client IP’s, configure the proxy in such a way that for the client IP’s as destination it uses de A10 as next hop. If the Proxy and A10 are not part of the same broadcast network, you also need to fix the routing on the devices between the A10 and Proxy.

    #15302
    avatar
    ericn
    Member

    The tcp-proxy template and tcp-proxy vport handle insert-client-ip in the same way as the tcp template and vport: by inserting the client ip as a TCP Option in the TCP Header.

    As do how to read the TCP Option at your application server, I’ve heard of customers using firewall logging to record the value. I’m not sure what possibilities exist for the Application to read the value directly from the TCP socket (I took a seminar on UNIX tcp socket programming some 16 years ago but never used the knowledge).

    The vport type HTTP with an HTTP template with insert-client-ip sounds like it may be the way to go. It’s been optimized pretty well over the past decade.

    I agree with diederik that Source-NAT is largely a solution to a routing problem: how does one get the server’s reply packets back to the ADC in order to do whatever L4/L7 magic your need done without making the ADC the default gateway for all traffic from the server. There are other uses as well, but the return traffic challenge is the big one. If you can solve the problem with routing, then you don’t need insert-client-ip

    If the information so far isn’t enough to resolve the design/config challenge, you might be dealing with a situation perhaps too complex to handle in an online forum. Maybe contact A10 Professional Service? I would advise that especially if you wish to use the aFlex based solution.

    • This reply was modified 1 month, 2 weeks ago by avatar ericn.
    • This reply was modified 1 month, 2 weeks ago by avatar ericn.
    #15352
    avatar
    flaneur
    Member

    Switched to HTTP and faced with another issue,so we can’t terminate SSL on ADC side cause we have smth like 1500 different certificates and the amount of certificates are growing every day,we terminate SSL on our proxy side.And as I understand we can’t bypass through ADC SSL traffic without termination, am I right ?

    #15362
    avatar
    diederik
    Member

    You have a loadbalancer in front of a proxy in front of websites you are hosting yourself?
    I suppose the proxy is doing caching?

    You can actually have the ADC bypass SSL traffic… but that means you still can not add the HTTP Header, as for HTTP Header insertion you need to alter the payload of the packets which are SSL encrypted.

    Without knowing the total structure of your setup, I believe you only have 2 options.
    Either remove the need for Source-NAT by possibly moving the location of the A10 ADC or the proxy, as the proxy is doing SSL Termination the proxy is the most logical device to do HTTP Header updates and Source-NAT.

    Or use insert-client-ip and find out how you can make your Proxy read the Client-IP’s from the TCP Options.

    Like ericn suggested, you might want to get in touch with you A10 Sales/Systems Engineer so they kan have a look at your setup and possible help getting A10 Professional Services involved.

    #15372
    avatar
    flaneur
    Member

    The scheme is like below:
    ADC with external IP address and Source-NAT to the internal IP address -> 2 Envoy proxies in Service Group that are balanced with the help of ADC by TCP type.
    The only way that can help us is to remove the Source-NAT from this chain but how can we route traffic from external IP address to our internal IP addresses ?

Viewing 15 posts - 1 through 15 (of 30 total)

You must be logged in to reply to this topic.

Comments are closed.