Bug: wrong network interface used when forwarding CORBA traffic

[stanislaw55]

Hi,
I've found (in my opinion) bug in Tango Gateway.
When Device Server and client are in different networks (say A and B) and Tango Gateway is in both of them (has network interface with different addresses in both A and B), it can choose wrong interface to forward traffic.
From client point of view, it looks like Device Server is completely down despite is running properly.

Client in network B connects to network interface of gateway B. Gateway then chooses to forward traffic to interface for network A.

It's non-deterministic bug and it happens for me when testing with Docker from time to time.
I'll try to come up with a fix

[vxgmichel]

Hi @stanislaw55 :)

That's an interesting point. I had a quick look and it turns out it's possible to bind the client socket to a specific interface before connecting:

    reader, writer = await asyncio.open_connection(
        some_host, some_port, local_addr=(local_interface, 0))

This would have to be added here:

tango-gateway/tangogateway/gateway.py

Lines 78 to 79 in 5170d47

	reader, writer = yield from asyncio.open_connection(
	host, port, loop=loop)

Also, note that the server interface can already be specified using the -b/--bind argument:

tango-gateway/tangogateway/cli.py

Lines 35 to 38 in befdbec

	parser.add_argument(
	'--bind', '-b', metavar='ADDRESS', default='0.0.0.0',
	action=EnvDefault, envvar='TANGO_GATEWAY_BIND',
	help='Specify the bind address (default is all interfaces)')

Accepting both interfaces as command line arguments would be pretty neat! I'm not sure what the terminology should be though:

tango client <-> tango gateway server | tango gateway client <-> tango server

How should the left/right part of this diagram be called:

• external interface / internal interface,
• tango client interface / tango server interface
• gateway server interface / gateway client interface
• something else?

[stanislaw55]

Hi @vxgmichel
thanks for rapid answer!
I've been inspecting the code and I suspect that order of interfaces matter. If first interface in machine/container is the internal one (the one in network with Device Server and real database),
then this one is taken into account when binding address - it is clear from code.

I forgot to mention previously that the problem is only related to events using ZMQ. Synchronous communication using CORBA works just fine.

Bind address works as local_addr, but only for the very first connection.

I agree that some kind of proper terminology is a real need for this project. Personally I have something like network A with real TANGO database and only server and network B whit just TANGO clients. Tango Gateway is sitting in between.

I'll try setting local_addr and report back.

[stanislaw55]

@vxgmichel, the code you pointed to only checks connectivity. The actual connection is made in here (I think)

tango-gateway/tangogateway/gateway.py

Line 151 in befdbec

server = yield from asyncio.start_server(

After investigating code it seems to work like this:
During startup, use value of bind_address when forwarding CORBA traffic.
For next connections, use get_host_name function ot get hostname. This is passed to asyncio.start_server. Then the result is queried as server.sockets[0].getsockname() and hence we have IP address of first interface.

From what I think, when asyncio.start_server gets hostname, it takes whatever IP address is first. I think, there should be passed explicit value of bind_address

@vxgmichel what do you think about my idea?