Control of Custom Model Deployment

[wbond]

During earlier conversations, you had indicated we would be able to pick or disable a custom model. Our first model finished training (🎉) but apparently it was automatically deployed. This ended up not being a big deal since we had intentionally created a new org to prevent unintentional rollout.

Looking at the UI, I see no option to disable the custom model. Is that forthcoming? We'd like to have control over the model chosen so we can roll back a custom model if it has issues. Currently the only way we can see doing that is to revoke access to Copilot through our "custom model org", but that gets noisy to remove and add users to since it triggers emails to them.

[taywrobel]

Hey there @wbond, thanks for reaching out!

From the custom models settings page which lists the models that have been trained, you should be able to select the three dots "meatballs menu" on the active model and that should open a dialog which includes deleting the model should you need to.

Can you check and ensure that the menu and option are present for you? If not, I'll open an issue on our side since that'd be a bug.

[wbond]

Unfortunately I do not see those options.

[rgpass]

@taywrobel @wbond The "Retrain model" and "Delete model" buttons only show up when the user is no longer rate limited. The rate limit is 2 weeks, so @taywrobel's screenshot is a bit over 2 weeks and @wbond's is not.

The reasoning for hiding Retrain is that it wouldn't be possible to retrain within that time. For Delete, we didn't want users to delete a model just to realize they can't train again because of the rate limit.

It makes sense to me to show Retrain and add inactive text (Primer example).

For Delete, we could show it and have the delete modal tell the user that they won't be able to retrain until (some datetime here).

@taywrobel Do you want me to go ahead and do both of those changes?

[wbond]

There should absolutely be a way to disable the custom model without deleting it. Training will cost money, so you shouldn’t be forced to delete something you paid for to not use it right now.

[wbond]

For issues like this, our rep suggested I post here in discussions. However, it appears Joe's comment suggests that bugs go into Issues. I have another apparent bug. Where should I post that?

[rgpass]

You can drop them in the orca repo's issues. If you add the label ui and assign me, I will be on it. If it's not UI related, I can forward it along to the right person.

[wbond]

It appears I don’t have access to that repo - I’m not a GitHub employee.

[rgpass]

I see! Joe would know best how bugs are tracked, so Issues works.