Standard too rigid to deal with derived datasets?


#1

My applications deals a lot with statistics, analysis and derived data sets in general.
Some of this data is calculated on the fly when it is requested, some is calculated
in batch at a regular interval.

For example:

/users/123/records/?relationships=zscore
/users/123/statistics/

The data returned by these may be constantly changing (like, minute to minute),
and there is no way to refer to the returned objects via a unique id, because they
aren’t stored in such a way that they can looked up via that unique id. It is
though compulsory in the spec to include a unique id. Of course, I can include a
unique id (i.e some random number), but surely this only confuses the client
as perhaps now they think they can use this id to later refer to this resource!

Another issue I’ve run into recently, is that I want to force a recalculation
of a user’s statistics (because the statistics are not calculated on the fly, they
are precalculated and put into a DB table). I was trying to think of a creative
way to do this that was still sane, and I came up with doing a ‘PATCH /users/123/statistics/’.
Because I want to indicate to the server that I want that particular resource updated. Of course
the client does not set the data, the server calculates it, but yet I still have to pass a resource object
complete with id (according to the spec). IMHO including that data would be extremely
confusing (i.e ‘why are you sending data that is not actually part of the updated resource?’)

I’ve just been making it up as I go along, and abandoning parts of the spec that I feel
don’t make sense to my application, and including those that I feel do.

Am I doing it wrong?

Cheers


#2

Not if you’re creating a hypermedia API. (In this case clients are not allowed to construct URLs, only to follow URLs provided by the server.) But if you’re just creating a plain JSON data API then this could be a problem.

What about a a POST /users/123/statistics/refresh ? (You might find this blog helpful.)

If you do, really you’re just inventing your own standard. So potentially you’re not going to be able to leverage any tools / libraries that understand JSON API, and you’ll have to provide all of these yourself (or expect your client developers to create their own). That seems like you’d be giving yourself a lot of extra work.
I’d want to stick to the standard, and if I couldn’t, I’d be thinking about others, such as HAL, JSON-LD, Siren, UBER etc. rather than rolling my own.


#3

No, not a hypermedia API in this case.

Yes a path with refresh was my first thought but just as quickly dropped the idea because it is a verb. However, that link (and the links within it) are certainly thought provoking. It certainly gets me thinking beyond CRUD and nouns vs verbs.

You’re absolutely right. Thankfully the data structures are relatively simple, so there isn’t too much work involved. APIs were certainly not something that I spent most of my time thinking about, but the more time I do spend thinking about them, it becomes clear that CRUD and the vnd.api+json standard do not really suit my application.

For the time being I will continue on adapting a square peg to a round hole, but for v2 I will re-think CRUD and perhaps a more suitable standard :slight_smile:


#4

I have thought about the issue described by the op too. Data aggregations (sums, averages, whatever) will not have an id by themselves, which escapes the jsonapi idea. However, the statistics for a user will always be identified by the user’s id.
So one could back reference certain statistics ‘highjacking’ the user id, i.e. make the user statistics use the user’s id as its id. However, this falls short if you had time relevant data where you do data aggregations for certain periods, or also for further filter query parameters, like the provided example /users/123/records/?relationships=zscore.
I could not come up with an approached that I really liked for that problem so far.