3

I'm creating v2 of an existing RESTful web api.

The responses are JSON lists of objects, roughly in the form:

[
  {
     name1=value1,
     name2=value2,
  },
  {
     name1=value3,
     name2=value4,
  }
]

One problem we've observed with v1 is that some clients will access fields by integer position, instead of by name. This means that if we decide to add fields to the response (which we had originally considered a compatibility-preserving change), then some of our client's code breaks, unless we add the fields at the end. Even then, other clients code breaks anyway, because they will fail in some way when they encounter an unexpected attribute name.

To counter this in v2, we are considering randomly reordering the fields in every response. This will force clients to index fields by name instead of by position.

Additionally, we are considering adding a randomly-named field to every response. This will force clients to ignore fields they don't recognize.

While this sounds somewhat barmy, it does have the advantage that we will be able to add new fields, safe in the knowledge that this isn't breaking any clients. This means we can issue compatible updates to v2.1, v2.3, etc at the same URL, and that means we will only have to maintain & support a smaller number of API versions.

The alternative is to issue compatibility-breaking v3, v4, at new URLs, which means that we will have to maintain & support many incompatible API versions, which will stretch us that little bit thinner.

Is this a bad idea, and if so, why? Are there any other similar ideas I should think about?

Update: The first couple of responses are pointing out that if I document the issue (i.e. indicate in the docs that fields may be added or reordered) then I am no longer to blame if client code breaks when I subsequently add or reorder fields. Sadly I don't think this is an appropriate option for us: Many dozens of organisations rely on the functionality of our APIs for real-world transactions with substantial financial impact. These organisations are not technically oriented - and the resulting implementations at the client end cover the whole spectrum of technical proficiency. We already did document that fields may get added or reordered in the docs for v1, and that clearly didn't work, because now we're having to issue v2 because many clients, due to lack of time or experience or ability, still wrote code that breaks when we add new fields. If I were now to add fields to the interface, it breaks a dozen different company's interfaces to us, which means they (and us) are bleeding money every minute. If I were to refuse to revert the change or fix it, saying "They should have read the docs!", then I will soon be out of the job, and rightly so. We may attempt to educate the 'failing' partners, but this is doomed to fail as the problem gets larger every month as we continue to grow. My question is, can I systemically head the whole issue off at the pass, preventing this situation from ever arising, no matter what clients try to do? If the techniques I suggest would work, why shouldn't I use them? Why isn't everyone else using them?

Improve this question
2
  • My view is if your documentation makes it clear that the order is not guarantied and new fields could be added at anytime it is the client's fault if they break on such occasions. Commented Feb 7, 2012 at 17:32
  • I understand that viewpoint, but I simply don't think it's an option for us. We have many commercial interests using our interface for transactions with real financial impact. If I were, without notice or notification, to break dozens of different companies longstanding interfaces with us and then refuse to fix it, saying "they should have read the docs!" then I'd soon be out of a job, and rightly so. The point of this question is how can I reduce the frequency of that situation ever arising. Commented Feb 8, 2012 at 15:43

2 Answers 2

Reset to default
1

If you want your media types to be "evolvable", make that point very clear in the documentation. Similarly, if the placement order of fields is not guaranteed, make that explicitly clear too. If you supply sample code for your API, make sure it does not rely on field ordering.

However, even assuming that you have to maintain different versions of your media types, you don't have to version the URI. REST gives you the ability to maintain the same version-agnostic URI but use HTTP content negotiation (via the Accept and Content-Type headers) to offer different payloads at the same URI.

Therefore any client that doesn't explicitly wish to accept your new v2/v3/etc encoding won't get it. By default, you can return the old v1 encoding with the original field ordering and all of those brittle client apps will work fine. However, new client developers will know (thanks to your documentation) to indicate via Accept that they are willing and able to see the new fields and they don't care about their order. Best of all, you can (and should) use the same URI throughout. Remember - different payloads like this are just different representations of the same underlying resource, so the URI should be the same.

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Thanks for the pointers about 'Accept' and 'Content-type' to overload the same URI, that's good to know. I have reservations about this as an answer though, which I'll clarify above in the original question... Cheers!
Um, I didn't really explain my reservations in the updated question, did I? They are: Your suggestion seems to direct me to an easier way to release more versions of the API, all sharing the same URL. I agree this is fabulous. However, my question is, can I use the techniques described to reduce the number of new API versions I have to produce and support and maintain in the first place?
Incidentally, we do supply sample code, but only in Python, and consumers are using many different languages, so mostly tend to code up their own solutions.
1

I've decided to run with the described techniques, to the max. I haven't heard any objections to them that hold any water for me. Brian's answer, about re-using the same URI for different API versions, is a solid and much-appreciated complementary idea (with upvote), but I can't award it 'best answer' because it doesn't get to the core of my original question.

Improve this answer

2 Comments

I think you thought of a simple yet effective solution. I just hope the overhead of ordering the columns isn't too great
Feedback from 5 years later: It worked well, and I do it again whenever I get the chance. Fortunately the overhead of reordering fields is small: Randomizing the order is substantially cheaper than sorting, for example, and all of this bit twiddling on the server is many many orders of magnitude faster than the database access we've done, and sending the resulting response over the internet to the client.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.