Resource Relationships Across Micro Services

[niltz]

Lets say I have two micro services:

1. Account - Stores users, organizations, and which user belongs to which organization(s)
2. Posts - Stores posts, and each post has a creator (user) and belongs to a particular organization

The Posts api is secured with oauth2 so I have access to the currently authenticated user.

• Should the server enforce that a user relationship for “creator” be included in the post? The server would still need to verify that the relationship specified matches the oauthed user.

• What is the best way to enforce that an organization relationship is specified? Any specific HTTP error code I should use?

-What is the best way to get a list of posts for an organization? I can’t do:
GET /account_service/organizations/<id>/relationships/posts
Because the Account service doesn’t have any idea about posts.

Do I use filters like:
GET /post_service/posts?filter[organizations.id]=<id>
Given that I will only be retrieving posts for a particular business, making it part of the query string seems weird.

Or do I add an organizations endpoint to the posts service like:
GET /post_service/organizations/<id>/relationships/posts which also seems weird because organizations don’t live in the posts service (and I don’t think this plays well with ember data anyway)

[maartenzwart]

Hi niltz,

I get the feeling that this is more a micro-service architecture issue than it is an (JSON:)API one. If I understand correctly, your question behind your post is how to perform joins across different micro-services, which is a common complexity for this type of architecture. My advise would be, don’t do them at all.

With micro-services, redundancy is inevitable. I think in your example the best way to go is to duplicate the data from the Account service in your Posts service. That way, the endpoint would be

GET /post-service/organizations/{organization-id}/posts

Of course the Account service is still “owner” of all the data that has to do with accounts. So changes need to take place there. Following the eventual consistency model, you can then update the duplicated data in your Post service.

Hope that helps.

Regards,
Maarten

[niltz]

Thanks for the reply Maarten,

I don’t want to do a join across microservices. What I want is for the posts in the post-service to have an organizationId for the owning organization, and I don’t know how to set up that relationship.

I guess I could make the organizationId a regular required attribute, but I was hoping I could set it up as a relationship so that we get some ember-data features.

But setting it up as a relationship is a bit confusing across microservices.

I don’t want to duplicate the organizations in the post-service becuase they could get out of sync, and that is a lot of duplicated data. Also, with eventual consistency, what happens if a user creates a new organization, and before that organization is duplicated to the post-service, the user tries to create a post under that organization…there is a race condition there. Or maybe I misunderstood your suggestion.

[niltz]

Here is a similar post:

Belongs-to in microservices

I have two models, Post and Comment. They are stored in different databases and managed by different application servers. I want to offer a way of searching Comments within a Post, so I add GET /api/comments?filter[post.id]=post12345&filter[text][match]=help. As far as I can tell, that endpoint should include post relationships within each comment’s resource object if and only if it also includes the post in included and the user asked for that: GET /api/comments?filter[post.id]=post12345&fil…

Where they are trying to deal with relationships between services as well.

Maybe it is best to just stick with a regular attribute for organizationId, and keep relationships for things within the same service?

[niltz]

It seems like there is a proposal to handle this, but it hasn’t had any updates for a while:

Issue: Relationships between resources controlled by different APIs

Perhaps for now, when there is a relationship across services/apis, I just use a plain attribute instead of a relationship

[kael-shipman]

For what it’s worth, I tend to agree with @maartenzwart here: I believe you should elevate Organization and User to first-class citizens in the Posts domain. When you do that, everything makes sense. What this implies is a sort of descending data model where you define whatever properties and relationships you can on objects in a Global namespace, then extend those in specific data domains (in your case, Accounts and Posts). See the end of the post for an example of what this might look like.

My opinions (for now) regarding your specific questions are as follows:

1. "Should the server enforce that a user relationship for “creator” be included in the post?" My opinion is no, because, as you said, this is inferred. Allowing a consumer of the API to access to a field that is neither writable nor useful as a read-only field will only make it confusing extra information. (The last JSON:API-based API I wrote did that exact thing and I ended up hating it.) This is one situation in which I think it’s acceptable that the actual API you expose deviate a bit from the formal rules. After, you’re building an API to serve people, not a data modeling philosophy.
2. "What is the best way to enforce that an organization relationship be specified?" As stated above, my opinion is that it would be perfectly valid to include organization as a required relationship. As @maartenzwart noted, data replication is a valid part of micro-service architecture. In this case, I would expect the Posts service to have an organization model that includes a users to-many relationship and an organization name. The Accounts service will likely store much more information about an organization, for example billing info, contact info, permissions, whatever. I assume you would probably keep them in sync via a message queue that does one-way updates from Accounts domain to Posts domain.
3. "What is the best way to get a list of posts for an organization?" Given the above, my opinion on this is to go with your third option, adding an endpoint like GET /post_service/organizations/<id>/realtionships/posts.

One extra comment here: If you choose not to include these objects as first-class citizens in your domain-specific models, you should keep in mind (and read up on) the “Hypermedia” part of the JSON:API spec, which defines how you would include a links object in the relationship that specifies where to actually find that relationship. That would, to a certain degree, solve the confusion about where that relationship actually resides.

Below is an abbreviated model spec using made-up pseudo-JSON:API base interfaces and written in typescript notation. This is meant to demonstrate that a) there are certain attributes and relationships that are common across all domains (Global) and b) the actual classes should be thought of as semantically the same, though with differing properties across domains:

export namespace Global {
    export interface User extends AbstractResource {
        type: "users";
        attributes: {
            name: string;
            email: string;
        }
        relationships: {
            organizations: ToManyRel<Organization>;
        }
    }

    export interface Organization extends AbstractResource {
        type: "organizations";
        attributes: {
            name: string;
            email: string;
        }
        relationships: {
            users: ToManyRel<User>
        }
    }
}

export namespace Accounts {
    export interface User extends Global.User {
        attributes: {
            approved: boolean;
            accountType: "free"|"basic"|"premium";
            // ....
        }
        relationships: {
            bankAccounts: ToManyRel<BankAccount>;
        }
    }

    export interface Organization extends Global.Organization {
        relationships: {
            primaryContact: ToOneRel<User>;
        }
    }
}

export namespace Posts {
    export interface User extends Global.User {
        relationships: {
            posts: ToManyRel<Post>;
        }
    }

    export interface Organization extends Global.Organization {
        relationships: {
            posts: ToManyRel<Post>;
        }
    }

    export interface Post extends AbstractResource {
        type: "posts";
        attributes: {
            title: string;
            content: string;
            timestamp: number;
        }
        relationships: {
            creator: ToOneRel<User>;
            editors: ToManyRel<User>;
            organization: ToOneRel<Organization>
        }
    }
}

Sorry for the long post! I’m developing my thoughts on these issues myself and it’s always helpful to write it all down :). Would love to know what you think about the ideas.