Brand2

API versioning discussion

Problem reported by Webio - 11/20/2023 at 5:56 AM

Resolved

Hello,

I think we need to talk seriously about API versioning. Does SM devs have any suggestiom how this can be handled?

For sure something has changed between latest stable .NET Framework version and beta .NET Core versions. Is there any place where we can see all changes made to API endpoints?

This small changes create issues where providers created for certain API will just stop working without any notice (example: MSP Server provider which works fine on current stable version returns error for .NET API). I will investigate it and probably fix it but it makes question do I need to update MSPControl provider everytime API is being updated and create some kind of versioning for version on which this change occured?

Thanks

16 Replies

Reply to Thread

Tim Uzzanti Replied

11/20/2023 at 8:01 AM

Employee Post

API's are backwards compatible. We may add data to outputs but we do it in a way that there should be no issues. If there is an API not working, it would be a bug. Please provide whatever calls you are concerned about and we can take a look. There was a post by Chris which has been overlooked and I stumbled across it when I searched for other concerns. I've asked our team to respond to that thread and we will look at those as well.

Tim Uzzanti

CEO

SmarterTools Inc.

www.smartertools.com

Webio Replied

11/20/2023 at 9:26 AM

Does this call:

https://mail.smartertools.com/Documentation/api#/reference/SmarterMail.Web.Api.SettingsController/GetAutoResponderSettings/get

has been changed recently with wantHtml param at the end of GET?

Tim Uzzanti Replied

11/20/2023 at 9:34 AM

Employee Post

That was one of the two reported by Chris that were overlooked. It will be resolved.

Tim Uzzanti

CEO

SmarterTools Inc.

www.smartertools.com

Webio Replied

11/20/2023 at 10:20 AM

Another thing. Something which worked before (on latest stable .NET Framerowk but does not work on beta .NET Core):

https://mail.smartertools.com/Documentation/api#/reference/SmarterMail.Web.Api.SystemAdminSettingsController/SetUsersMailSettings/post

Query to:

/api/v1/settings/sysadmin/post-user-mail

With content:

{"email":"some@email.com","userMailSettings":{"isEnabled":true,"maxSize":"0","lockPassword":false,"replyToAddress":"","signature":""}}

Returns:

{
  "message": "Object reference not set to an instance of an object.",
  "success": false
}

So maybe this is not a change but some bug in API.

ALSO it would be great to have in API documentation info which params are required and which are not.

SO. Before sending this ticket I've opened dev tools and noticed that webmail is using:

https://mail.smartertools.com/Documentation/api#/reference/SmarterMail.Web.Api.DomainSettingsController/SetUsersMailSettings/post

instead of:

https://mail.smartertools.com/Documentation/api#/reference/SmarterMail.Web.Api.SystemAdminSettingsController/SetUsersMailSettings/post

Both of them looks the same but one is being posted with domain token and second one with admin token. One with domain token works and one with sysadmin token returns error mentioned above.

Webio Replied

11/21/2023 at 12:51 AM

It would be also good to have info when some method was introduced. Now I'm not sure if method SetUsersMailSettings was already present under Domain API section when I was writing latest SM provider for MSP or it was added recently. If this kind of changes will be made to API all external providers will have to be released as new ones based on build when they where working. So it will not be like it was in the past like SM VERSION.x but SM BUILD which as you can see in recent changes will enforce to release new providers much much often

Webio Replied

11/21/2023 at 3:21 AM

Another one (build 8720 - call using domain token):

URL/api/v1/settings/domain/dkim-settings

Content:

{"fields":["return-path","received","comments","keywords","bcc","resent-bcc","dkim-signature"],"keySize":"KiloSize"}

Returns:

{
  "message": "Object reference not set to an instance of an object.",
  "success": false
}

Call to (with sysadmin token):

/api/v1/settings/sysadmin/dkim-settings/DOMAINNAME

returns the same error

Employee Replied

12/5/2023 at 8:42 AM

Employee Post

Hi Webio,

We have addressed the other items in this thread. However, we have not been able to replicate the "object reference..." response with the DKIM-settings call. If you would like to troubleshoot this response, please respond to the support ticket we started.

Kind regards,

Grady Werner Replied

12/6/2023 at 7:56 PM

Employee Post

The issue with that Ron is that some of our customers with very little (or no) programming knowledge sometimes use tools like control panels from various eras. We try very hard to maintain all the legacy calls so that those tools don't break, because otherwise we have a bunch of small shops who use older cPanel or Plesk or one of those get into a broken state with no realistic way to resolve it.

As a dev, trust me, I'd LOVE it if we could wipe away those legacy apis and just focus on newer ones, but it's not really an option.

Grady Werner

SmarterTools Inc.

www.smartertools.com

echoDreamz Replied

12/6/2023 at 11:15 PM

Versioning is great... because just like Grady said, legacy apps can be broken by changes, but so can any modern updated software, publish a new SM build and 1 call signature was changed and oh no, or in our case where the mailStoreVersion property values were changed, we updated, then they changed again, and we had to update again.

Many of the products we use have versions, either supplied in the URL or as a parameter. IE /api/v1/ then they publish an update /api/v1.1 etc. Some use a header like X-API-Version etc. PayPal has different API versions, Coinbase Commerce, Braintree and many others.

ST also does not appear to publish when an API's signature has changed, the expected parameters or return structure.

Webio Replied

12/7/2023 at 12:18 PM

IMHO there should be something (with or without versioning) to be able to track changes:

API changes planned ahead
changelog
information next to methods when it was changed (what was removed or added) or introduced
which params are required and which are optional

Ron - are you believing that API changes in such product like mail server can be scheduled to yearly updates?

Webio Replied

12/11/2023 at 5:27 AM

To summarise:

Changelog for each method is needed with info when it was changed. The same provider which checked for stable .NET Framework version (build 8664) for basic functionality inside MSPControl panel after update to latest SM .NET Core RC stopped to work in some part. This can be fixed but how now I can know which API changes are valid in current API documentation? How I can know what has changed between build 8664 which is the last one known for now as stable and runned on .NET Framework and latest .NET Core RC build?

This changes are causing issues with development of valid providers for control panels which are being used by various types of companies. Some of them keep stable version some of them are installing latest ones. Now having only one documentation without any changelog this is impossible to maintain or you will have to create new provider each time something will break.

As you can see Ron changes (even micro ones like echoDreamz pointed out) are critical when it comes to managing SM from withing additional software like hosting control panels which expect to have specific API methods (which where working just fine during provider development).

I'm not talking about legacy API which was mentioned by Grady. IMHO they should just announce legacy API drop and thats it. My post is about current API and as far as I can see there is no policy for maintaining this API in easy way for developers. It would be good to have some voice in this discusson from them. I totally get it that since this API is being used by webmail GUI so if webmail GUI is being changed and updated with new functions then API must be also but the same API is being used by products which don't need this changes AND here is the place where API versioning would solve all problems.

Sébastien Riccio Replied

12/29/2023 at 11:47 PM

Hello,

We finally decided, after one year, to upgrade our production server (was running a stable 8251) to 8664, with confidence as Tim is saying in this thread that the API is backward compatible. What a mistake.

I'm spending the night chasing API calls that don't work anymore.

For example for updating a user, it used to be a POST to:

/settings/domain/user/user@domain.tld (but this now returns 404's, seems to be replaced by /settings/domain/post-user)

Impersonating was a POST to /settings/domain/impersonate-user/user@domain.tld (again a 404, it doesn't take the mailbox name in URL anymore, but only in POST data)

Looks like there are many other API changes like this and I'm good to spend hours this night to chase them.

Really we need a change log for the API, it's a nightmare to play a try / fail / guess / fix api game after a major upgrade :(

Kind regards.

Sébastien Riccio

System & Network Admin

https://swisscenter.com

Webio Replied

12/30/2023 at 1:26 AM

IMHO it's all becasue API is inseparable from current GUI. If GUI changes there is a big chance that in some point of time API will change also and thats why we need to have some API section which is dedicated to simple remote operations not related to GUI. We basically need modern legacy API in current API. This would solve all issues with versioning, method changes, param changes etc etc and will not force ST to document each API change in their methods used by web GUI.

Grady Werner Replied

1/2/2024 at 10:29 AM

Employee Post

You all make good points and discussions are ongoing over here to provide you with a solution.

We'd love nothing more to make a "master" API that never changes, but truthfully that is a major undertaking. The design task on how to generate an API that can transparently support new features, removed features, and changes is a big one. For example, Microsoft has been trying to make an API with Graph (that doesn't even do configuration stuff) and has spent years trying to make it consistent and it still keeps changing and is extremely buggy.

We would LOVE to see something that generic, but it's probably not something we're ready to tackle yet, especially while Linux dev is going on.

However, I see your pain. And at the very least we should be able to auto-generate a change log for the API between versions. I'll let you know how things go on that front.

Regarding the removed user@domain.tld calls, browsers were blocking those calls as malicious in some ad-blockers, so we had to use POST calls instead (same with the impersonation call). But the old ones should have been left and it was an error that they were removed.

Grady Werner

SmarterTools Inc.

www.smartertools.com

Webio Replied

1/2/2024 at 10:40 AM

But how about having a clone od legacy API in latest API? Those methods are very basic and don't need to be expanded (except autoresponder which now has two fields instead of one handled by legacy API). Over 15 years in shared hosting I didn't need anything more than legacy API provides. DKIM is something in bonus which can be handled using regular API calls used by webmail GUI BUT changes for DKIM can be handled when they occur and they don't break main functionality which is creating basic domain configuration and manage mailboxes. Thats all

I think there can't be no master API because of connection to normal SM GUI. If you want to provide stable solutions for real API usage (current API has become IMHO internal API to communicate with GUI instead of being API for external management) then you must divide it to internal API which can be used and external stable API for remote management.

Webio Replied

1/11/2024 at 5:44 AM

It would be good to have at least some roadmap here. Seriously I'm not interested in creating new provider for MSPControl each time some method that is being used by this provider will be changed in SM API. At least some comment about my last message from ST devs.

Calling current SM API an internal API is IMHO very accurate and for having stable running remote environment which connects to SmarterMail we need something separate of methods used by Webmail GUI.

Another thing is what exactly calling method Obsolete in API documentation means? How this affect this method functionality in upcoming versions. It will be removed?

Back to Community Threads

Reply to Thread

Enter the verification text