I’m currently working on the details for profile extensions and checking several use cases, regarding the stability of the spec, the update process and I’m also working on a possible (server/client) implementation to validate some ideas.
One problem I see is extensions that extend other extensions. The problem is, that this introduces a dependency between extensions, which is important for the implementation, because it could be required that such extensions are processed in the correct order.
Another problem are extensions, that introduce some kind of universal key, comparable to meta, not bound to specific objects (so they could also extend each other extension). Such extensions introduce additional problems, because of the unclear dependencies.
I tried to find use cases for the problems, but don’t know any. So do you have an example for me?
And what do you think: Should such extensions be allowed (in general/in the first step)?
If you would allow them: How and where would you define extension dependencies?
I tried to work through the different cases here and I think the bottom line is that there are some times when recipients might have to know about the relationship between different extensions—but I don’t think this issue is limited to extensions that extend one another, and I’m not sure there’s much we can do about it.
Imagine there are two extensions, A and B, where B extends A. By this I mean that B defines some additional data fields beyond those defined by extension A.
Under my proposal on Github, if B is used in a document, it gets one key under which all its data lives (with the exception of fields that are explicitly defined to be able to cross extension boundaries). So, if B is used in a document without A, the simplest way to do that would be to put fields defined by A and B together under B’s key, like so:
To support the above use, extension B is registered with a definition that simply incorporates the definition of A. That is, for every field defined by A (that B supports), extension B’s definition says that the meaning of that field when it occurs in B’s key is the same as is it’s meaning in A.
Because the document that’s using extension B provides all the data that’s needed to process B within its "key-for-extension-b", there’s no need for the document to declare that B depends on A.
The problems arise, though, when both B and A are used in the same document. If we don’t care about possibly repeating some data, which I don’t (given gzip and extension negotiation, which often obviates the need to include both), using Extensions A and B in the same document would look like this:
"key-for-a": {
"some-field-from-a": true
},
"key-for-b": {
"some-field-from-a": true, // could have a different value than in "key-for-a".
"some-field-from-b": true
}
Now, A and B can both be processed independently (the needed data for each is self-contained). But, whether they can be processed in an arbitrary order depends on their effect on the recipient’s state, and can’t be determined just from the payload.
Moreover, as I said earlier, I don’t think this uncertainty is limited to extensions that extend one another. That is, B extending A probably makes it more likely that B and A manipulate the same recipient state, but there’s no guarantee that extensions C and T, which don’t extend each other but do related things, won’t also have order-dependent side-effects.
I’m really not sure what to do about this. The simplest thing would probably be to ignore this problem and just to leave it up to the recipient to keeps its state consistent. This is inline with the idea of just passing open-ended representations, and it can probably work in almost all cases where the API server is sending the document and the client receiving it. (See this excellent article for context.) But it might be insufficient for certain cases when the client is sending a document to the API server and needs some behavioral contract. Maybe that’s a useful way to divide cases?
Anyway, I can imagine some other approaches here too, but all my thoughts are still pretty raw. I’m curious what you think @ziege, and also want to get @dgeb, @steveklabnik, and @tkellen in here if possible. I’m also gonna do some more reading and see what I can turn up.
I’m not sure what you mean here. Every key that’s introduced is still connected to the extension that introduced it, so the dependencies should be clear.
Regarding your example, I’d say, that extension B is just an extended version of extension A. One could say a new (backwards compatible) version of extension A. And if handled like a new version, it replaces the previous one, so that there is no problem with using both at the same time. (I already covered the problem of extension versioning in the document on which I’m current working - hope to finish in within the next days.)
I think it’s okay to disallow the extension of extensions, because there is no real need for it. And if so, this can be allowed later on. But this doesn’t solve all conflicts, e.g. multiple extension can require to be always processed first or last (e.g. a signature extension for signing documents).
I’m currently working on making it compatible with JSON API 1.0, because some rules defined there make it hard to implement versioning - but I think I found a way. I also have an idea to support invalid key names…
if you have ideas for how to handle those, I’m open to them!
Let’s assume we have a “signature” and “encrypt” extensions, both request to be processed first/last. One could think that it should not make a difference, in which order they are processed, because in the end you get the same result:
decryption → validation and signing → encryption
validation → decryption and encryption → signing
But that’s not true, because client and server have to process them in the same way. So the server has to tell the client not only the supported extensions but also the processing order?!
I doubt that we’d have an encryption extension in particular, since it’d make more sense to just use HTTPS. But I understand your point here, which is that there could be cases where order matters and the client can’t figure out the order just based on the extension’s definition. So we may have to design some order-advertising mechanism.
My instinct, though, is that we don’t know enough yet about the requirements around extension ordering. And, if we try to design something now that handles all kinds of complex scenarios, I’m worried that we’ll end up designing something that’s more complicated than it needs to be, which would be bad. Therefore, I think we should do two things:
We should launch a simple extension system as a “beta” (in the same way there was a testing period before 1.0 launched). That system might not have features for controlling the order of extension processing at first, but we could see what extensions people try to make and what capabilities are needed, and then build those capabilities into the design before we launch it officially.
Even in the system we launch after the beta, that system should be designed so that we can improve it over time, if needed, in a backwards compatible way. This should be pretty easy to support, though the particulars of how to do it depend on the design we settle on.
