Time series data for JSON API

1

Hey All,

I have a bunch of questions about JSON API structure but I’ve just got this question about time series data before I create those other discussions.

I am aware that both Tom Dale and Yehuda Katz have mentioned that Ember Data is not designed for time series data, but I’m wondering if that design decision still applies to JSON API?

I’m working on a PR for Kue to add a time series endpoint for jobs that have completed on a hourly basis. The structure that I have for a response at the moment is an object where the key is a timestamp floored to the nearest hour and the value is an array of Job Ids. Here is an example:

{
  "complete": {
    "1451610000000": [],
    "1451613600000": [],
    "1451617200000": [],
    "1451620800000": [],
    "1451624400000": [],
    "1451628000000": [],
    "1451631600000": [],
    "1451635200000": [],
    "1451638800000": [],
    "1451642400000": [],
    "1451646000000": [],
    "1451649600000": [],
    "1451653200000": [],
    "1451656800000": [],
    "1451660400000": [],
    "1451664000000": [],
    "1451667600000": [],
    "1451671200000": [],
    "1451674800000": [],
    "1451678400000": [],
    "1451682000000": [],
    "1451685600000": [],
    "1451689200000": [],
    "1451692800000": [
      "68",
      "69",
      "70",
      "71",
      "72",
      "73",
      "74",
      "75",
      "76",
      "77",
      "78",
      "79",
      "80",
      "81",
      "83",
      "84",
      "86",
      "88",
      "90",
      "93",
      "95",
      "97"
    ],
    "1451696400000": [],
    "1451700000000": [],
    "1451703600000": [],
    "1451707200000": [],
    "1451710800000": [],
    "1451714400000": [],
    "1451718000000": [],
    "1451721600000": [],
    "1451725200000": [],
    "1451728800000": [],
    "1451732400000": [],
    "1451736000000": [],
    "1451739600000": [],
    "1451743200000": [],
    "1451746800000": [],
    "1451750400000": [],
    "1451754000000": []
  },
  "failed": {
    "1451610000000": [],
    "1451613600000": [],
    "1451617200000": [],
    "1451620800000": [],
    "1451624400000": [],
    "1451628000000": [],
    "1451631600000": [],
    "1451635200000": [],
    "1451638800000": [],
    "1451642400000": [],
    "1451646000000": [],
    "1451649600000": [],
    "1451653200000": [],
    "1451656800000": [],
    "1451660400000": [],
    "1451664000000": [],
    "1451667600000": [],
    "1451671200000": [],
    "1451674800000": [],
    "1451678400000": [],
    "1451682000000": [],
    "1451685600000": [],
    "1451689200000": [],
    "1451692800000": [
      "82",
      "85",
      "87",
      "89",
      "91",
      "92",
      "94",
      "96",
      "98",
      "99",
      "100"
    ],
    "1451696400000": [],
    "1451700000000": [],
    "1451703600000": [],
    "1451707200000": [],
    "1451710800000": [],
    "1451714400000": [],
    "1451718000000": [],
    "1451721600000": [],
    "1451725200000": [],
    "1451728800000": [],
    "1451732400000": [],
    "1451736000000": [],
    "1451739600000": [],
    "1451743200000": [],
    "1451746800000": [],
    "1451750400000": [],
    "1451754000000": []
  }
}

Ideally I would like this to be able to be a hasMany relationship in Ember i.e. making use of JSON API references so that the client can go request the Job details when required instead of having a massive response up front.

Let me know if you want me to explain this more or if you need me to give more examples.

2

There are a number of ways you can go about this, but the key thing to note is that JSON API is designed around each response returning “primary data” under the data key, and that data must be a single resource object (basically, an instance of a model) or an array of resource objects.

So, you need to pick here what kind of resources you want to return… Probably the answer is something like a “time bucket”. So, since type names in json api are plural by convention, you’d have a response that looks something like this:

{
  "data": [{
    "type":"time-buckets",
    "id":"1451610000000",
    "relationships": {
      "successes": { 
        "data": [ {"type": "jobs", "id": "21"}, {"type": "jobs", "id": "22"} ] 
      },
      "failures": {
        "data": [ {"type": "jobs", "id": "17"}, {"type": "jobs", "id": "42"} ] 
      }
    }
  }, {
    "type":"time-buckets",
    "id":"1451613600000",
    "relationships": {
      "successes": { 
        "data": [ {"type": "jobs", "id": "23"}, {"type": "jobs", "id": "24"} ] 
      },
      "failures": {
        "data": [ {"type": "jobs", "id": "19"}, {"type": "jobs", "id": "29"} ] 
      }
    }
  }
  /* etc... */ 
  ]
}

In this model, ember data will pick up the successes and failures as “hasMany” job references from each “time-buckets” resource.

1 Like
3

I’m wondering if there’s been any movement here on the extension to support timeseries data within the scope of the JSON API project.

There’s a natural need for a formalised JSON API approach, and I’m very supportive of the concept, but at the same time I would love to see this advance so that there’s less of a mish-mash approach depending on the data types.

Thoughts?

1 Like
4

So just wondering if you were able to resolve this satisfactorily?

5

I’m curious, what about this solution does not solve the problem in a satisfactory way?