SqlServer system store does not support multiple interfaces

[amccool]

The membership table for sql server systemstore has only a single column for the address for a silo. this prevents a silo from running the interconnect on one interface and the proxyport on another interface

[veikkoeeva]

Can you share more details on what you have in mind? Do you have multihomed server with a dual NIC setup where the proxy interface is on the another NIC and the silos communicate via the other one?

[amccool]

while experiencing #1135 on a system with three interfaces I noticed that only the silo connection's IP is listed in dbo.MembershipReadAll view. Both ports have columns but there is only a single column for the IP address.

C:\ProgramData\chocolatey\lib\ConsoleZ\tools>netstat -ano | findstr 40028
TCP 172.18.4.149:11111 0.0.0.0:0 LISTENING 40028
TCP 192.168.2.123:30000 0.0.0.0:0 LISTENING 40028

DeploymentId Address Port Generation HostName Status ProxyPort RoleName InstanceName UpdateZone FaultZone SuspectingSilos SuspectingTimes StartTime IAmAliveTime ETag Version VersionETag
R5Enterprise 172.18.4.149 11111 187742168 doink 3 30000 R5Ent.OrleansSiloHostService empty silo 0 0 NULL NULL 2015-12-13 22:36:10.943 2015-12-13 22:36:15.280 0x0000000000002977 90 0x0000000000002978

so when the client tries to connect:
[2015-12-13 22:36:45.401 GMT 4 INFO 100000 OutsideRuntimeClient 127.0.0.1:0] OutsideRuntimeClient.ConstructorReset(): client Id *cli/1f5552a5
Terminating due to exception:
System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> Orleans.Runtime.SiloUnavailableException: The target silo became unavailable for message: Request *cli/1f5552a5@16288d9b->S172.18.4.149:30000:187742168TypeManagerId@S00000011 #2: global::Orleans.Runtime.ITypeManager:GetTypeCodeMap(). Target History is: S172.18.4.149:30000:187742168:TypeManagerId:@S00000011

Its trying to connect to the wrong IP/port combo

[veikkoeeva]

@amccool When you write columns, do you mean rows? A point to note is that the view is removed in newer versions and plain tables are used, though the queries return the same data. Related to #1135, do you suspect the result would be the same when SystemStoreType it set to any other supported type such as AzureTable? That is, would the fix affect only relational storage provider or some other pieces of Orleans too. Maybe @sergeybykov or @gabikliot knows?

You are experiencing a real problem, but until I get to debug deeper, I try to jus debug "on the forums". :) In other words, I like to help, but it likely goes towards the end of the week. A tangential note here the tables could use some refactoring. Without too much thinking and if this requires more modification, it might make sense to refactor at least the tables to support multiple NICs/IPs (maybe that's what is fundamentally the issue here too), see how to build the "suspecting silos" information and so on.

P.S. You can generate markdown tables with generates, one such online is here.

[amccool]

I mean columns. I have not executed against the AzureTable system store so I couldn't say the different.

This issue is not a show stopper, but it is causing confusion in our integration environment.

[shayhatsor]

@gabikliot, this is a bug, but in Orleans in general and not just in SqlServer system store. It seems that you can configure a silo with any proxy ip:port, and it will listen. But no client will be able to connect to it if the proxy ip you're using isn't the same as the silo to silo ip. It happens because this data is not communicated to the clients, only the proxy port is kept. We can either remove it as a configuration option, or close this issue and open a more general bug. IMO the config option should be removed, or at least throw an exception when people define two different ips.

[gabikliot]

Agree it is general.
We should probably open a new issue about allowing to provide a separate IP (and not just port) for client connections. And still of course support the single IP for both if only one is given. So any existing configuration should still work as is, plus the extra option of the 2nd IP should be supported.

[shayhatsor]

closed because I opened a more general bug #1812