Orleans and Cloud Service Providers

[galvesribeiro]

Although Orleans can run perfectly on-premises and on any other Cloud Service Provider (CSP), I saw people asking about having a deeper/transparent integration with those CSPs by leverage their service offers.

This issue will track a series of issues that will implement minimal providers like the ones already created for Azure but instead, targeting other major CSPs:

• Amazon Web Services (AWS) - Amazon AWS integration #2006
• Google Cloud Platform (GCP) - Google Cloud Platform (GCP) integration #2057

Feel free to make suggestions of other major CSPs to be included on this list or if you wish to contribute on those. Discussions about an specific CSP's provider implementation please keep inside its linked issue.

Lets put Orleans to run everywhere! 💃

[gabikliot]

This is great!
It is of course not new, we have been talking about it a lot in the past (e.g., #473).
Also, some work have already been done in the GCP space (#1174 - Google Protobuf, part of OrleansGoogleUtils).
This line of work will take the framework to the next level and unblock a great potential of application mobility across cloud providers.

[galvesribeiro]

@gabikliot Yep, I already saw the google part, I just started with AWS cause I have someone that want to use Orleans here but they are 100% on AWS so I just want to get them started asap.

When the time comes, I will get to google as well :)

[galvesribeiro]

Ok... As much as I want that providers to reach the main branch and support the major CSP player, I'm dropping this issue and all its related PRs and closing it.

The reason is that after a long discussion with the core team (as much as I don't agree with the decision), the decision was to move it to OrleansContrib. By doing that, it will slow down the development of those components and I have no time/resource to invest on it. The reason it will slow down is that in order to properly test the whole providers we will need the internal tests from the main repo and we can't do that in a manageable way by having it on OrleansContrib.

I feel sad about that and I still believe Orleans should reach other CSPs easily. Thanks to everyone who contributed with either comments or reviews on the related issues/PRs, but I'll drop this engagement since I have another higher priority and time consuming engagements on Orleans like .Net Core support.

I hope eveyone understand. Thanks.

[gabikliot]

I am really disappointed and frustrated by that decision as well.
You listed all the exact reasons why I believe it should be in the main repo.
We already have ZK, and Consul and SQL in the main repo. Why AWS/GCP are different?

Mostly, the biggest mistake is the lost opportunity. Almost no one ever talks about the lost opportunity, about "what could have been if I made the right decision" (an example of a lost opportunity - SF have Orleans Actors 3 years ago). But the real visionary and strategic thinking is about not loosing the opportunity. In this case - more .NET apps beyond Azure; potential huge uptake by other communities that are not aware of Orleans; ...
And the cost to have that opportunity - a bit bigger main repo? Poor internal tooling (manual braches syncs, better code sharing between repos) dictating the platform capabilities and forcing us to loose this potential opportunity?
That would not have been the decision I would have made.

[galvesribeiro]

@gabikliot I agree with you... Like I said... As much as I don't agreed with them, they explained their point. Sadly, it is not what we want.

I'm not trying to extend a debate here. I've already put my point to the core team which is the same as yours and that was their decision.

I wish I could have time/resources to do it outside so I can at least help people that asked for the AWS support for now but like I said, that is something I don't have Today unfortunately.

[jdom]

I'd like to clarify this a little bit with more context.
The reason we decided not to merge this PR yet is because we still don't have a customer who is vested in using it production and hardening it. We would need this to be a little bit more stable (proven) before we can start supporting it.
We are not dismissing this feature at all. As curators of the project, we just need to guarantee some quality of what we release.
Other similar external dependencies that not all of the committers are very familiar with -like ZooKeeper, SQL, etc- had customers already using them and vested in them before we made them part of the main repo.
We are also doing this with things that come from internal MS teams (like MSR and 343i). That's one reason why merging the multi-cluster PRs is taking so long. Some people are already using it even before all the PRs are merged. The same thing with the EventHub stream provider, and countless others.

Adding these dependencies would introduce a huge new testing area that we just don't have the bandwidth to handle in the short time frame that @galvesribeiro expects, and that's why we rely on the community to help us get there. I was just pushing back on the ask for "merging this PR fast so that @galvesribeiro can work on the next set of related PRs".
As an option to get this stabilized, I suggested publishing to OrleansContrib so that we can have nuget packages being released and people can start using the feature. In the meantime the PR could stay open on hold (or closed temporarily, I have no opinion here) until the feature is hardened. During this hardening period we will definitely point people to it and give it as much visibility as we can, as we DO want the feature to be merged.

So in short, we should not cut this feature. We should just be diligent and not throw it in and hope that it works.
@galvesribeiro said he's building this but not planning on using it. Let's find a proper vehicle to get this feature more mature.

[galvesribeiro]

IMHO, if we don't have the feature, we don't have the users. For example, the one I was trying to help is sad but will not use Orleans anymore since it doesn't play with AWS.

But again, points taken and lets not make this issue a big debate.

Let's move forward.

[jdom]

Let's keep talking in private. We can get together with them in a Skype call and work with them. If we can have them be the vested customer, awesome, that's what we need to get this feature validated and eventually merged.

[galvesribeiro]

OK. More one thing... Before anyone ask, the decision to drop it is mine, and mine only. Because I don't have time and resources to make a full test set external to Orleans main. By having it on main solution, I could leverage the current test infrastructure which would cut a lot of time and that is why I decided to start this at first place. So there is no implicit meaning here that "Orleans core team doesn't want other CSPs". If someone else can do it, please reopen the issue and move forward.

[gabikliot]

Let me suggest a compromise.

1. Definitely, we should NOT be "merging this PR fast". We should use the same high quality bar that we use for any other PR. This goes without saying.
2. My recommendation is to start with the membership provider. I will review it thoroughly, at the same level of due-diligence I did for other membership providers (Consul and ZK and Cassandra).
3. We were pretty fast and efficient in accepting ZK and Consul mbr providers and I see no reason to why we can't do it here at the same high level of quality and speed as before. I estimate that we can have the MBR provider be merged within a week at most, maybe faster depending on how fast we iterate.
4. We will have the same level of testing for AWS provider as ZK/Consul (manually run tests, no automatic since all of those require pre-installed dependent services).
5. We have roughly the same level of commitment from @galvesribeiro w.r.t future customer as we had from @PaulNorth about Consul and @shayhatsor about ZK. No one can commit for 100%, but we know @galvesribeiro and I trust he will keep supporting this code in the future.
6. MOSTLY important: there is zero risk merging this PR, since it does not change a single line in main Orleans binaries, it is 100% pluggeable external provider, thus is can not destabilize the rest of Orleans. This is VERY different from the Geo code which changes key internal components, and thus of course has to be reviewed and vetted with much more care. That is not to say that anyone suggested that the AWS MBR provider to compromise on the general high quality bar. As I said already, I promise to review it thoroughly, as I did before for other providers.
7. Once we are done with MBR, we can move to storage. Others (@shayhatsor , @ReubenBond, ... ) might help there. If not, I will review it as well.
8. I therefore see no risks for the project overall, or any concerns for the curators of the project. On the other hand, the cost of the loss opportunity is enormous in my opinion, as I wrote above.

[shayhatsor]

@galvesribeiro, I understand that you don't want a debate regarding this issue, but IMHO this is a very dangerous precedent.
I'll just share my experience, which is from both sides of the coin, as a developer and as a customer of Orleans.
I absolutely agree with you, this should be in the main repo. It can be tagged as a "beta" feature, it can also be undocumented and unsupported. This enhancement has a huge potential of opening Orleans to significant untapped markets.
When I implemented ZooKeeper membership provider, it was part of my company's evaluation process of Orleans (same as your situation). Had I got an answer like you got here, we'd probably go elsewhere. The amazing dedication of @gabikliot, @sergeybykov and others is still one of the biggest selling points of Orleans to date. I believe it tipped the scale towards building our entire SOA architecture around it.
About the stability/maturity of the provider, it takes time and it takes community support, which is what you get here. I don't see any difference between this provider and others. We review the hell out of it, hope for the best and fix as we encounter bugs.
The same stability/maturity arguments could have been made about any of the new providers, and a single company that's saying it might use Orleans shouldn't be the deciding factor whether a new provider should be added to the main repo. There aren't many people that are ready, willing and able to contribute, we should embrace them.
Where would Orleans be now had we sent everyone to OrleansContrib ?