Content

tldr

The whole point of the above wall of text is only to say that expressing how to fetch data is kind of hard and there are a lot of bad solutions.[4]

Most of the GraphQL APIs I’ve seen do a similar thing as the above. But, it seems like you can get away with a lot more because GraphQL APIs have schemas. Presumably, mistakes in clients can be caught early by programming against an explicit and typed interface. The consequences of having a complex interface is minimized by tooling.

Instead of encoding our filters into a URL query string, we can encode it as part of a GraphQL query and pass it either in the URL of a GET request or as data in a POST request if our API does away with GET requests entirely.

Here’s an example from the GraphQL project website:[5]

{
  human(id: "1000") {
    name
    height
  }
}

… which looks up and returns a human with that id.

Some GraphQL APIs are creative with parameter names and include operators in them, like human(name__contains: ...) or something like that. And some REST APIs let you do this with query string filtering too, like /human?name__contains=.... It’s basically the same thing. (Except GraphQL is typed, while I think x-www-form-urlencoded encodes the boolean true and the string "true" the same way which can be a pain.)

It’s worth mentioning that GraphQL makes it natural for clients to specify the shape of the data in the selection set; what fields we want the API to return and any related records as well.

PostgREST is a really fun and interesting program. Their syntax for returning related records was apparently inspired by GraphQL. It lets us write something like /users?select=*,comments(*)&comments.thread=123 which would return a set of users and their authored comments for a specific thread. PostgREST was also designed with the awareness that nested endpoints are filters. When you think about it, even /table/<id> is just a filter. So write /table?id=eq.123 instead. This has the advantage of removing the question of how to build an item endpoint for a model with a composite primary key.

I also want to mention HTSQL. I haven’t heard anybody else talk about it and haven’t properly used it myself. So I don’t really have any insight to offer. But the design and literature seem super interesting. It looks expressive and powerful, but I’d also expect this to mean that it’s more difficult to implement and a bit harder to learn. Although, I think a goal of theirs was to create something easier to digest than SQL.

Admittedly, more expressiveness isn’t necessarily worth the complexity it may introduce. Let’s suppose we want to find posts, but only ones made since their author’s last login. Using HTSQL, we can write something like /posts{title,author{name}} ?published_at>author.last_login which looks like.

{
  "posts": [
    {
      "title": "interesting post",
      "author": {
        "name": "Spongebob"
      }
    },
    {
      "title": "why starfish are the best",
      "author": {
        "name": "Patrick Star"
      }
    }
  ]
}

This is pretty cool. I’m not saying that this here exactly is the line, but there probably is some point where queries/requests become so esoteric that giving them first-class support might not be worth it.

PostgREST has a bit on computed columns which can be used to define interesting expressions on the server that are usable by clients. Which is better than nothing and preserves the purity/simplicity of the filter and selection syntax.

PostgREST also has a whole thing for doing executing stored procedures, for when your models have special verbs that don’t fit the CRUD model. So that’s a cool thing worth mentioning. Because even if 90% of the APIs you’re trying to create on the web is just for content that fits CRUD, the other 10% is important enough that, if you ignore it, it’s likely to take as much or more time to solve than the first 90%. And that sucks.

An important fact of PostgREST and HTSQL is that they operate using introspection. Since the features are written for database concepts, any kind of snowflakeyness must exist in the database that is being reflected. And we don’t have any stupid excuses like a we didn’t copy-paste the filter handling code correctly between endpoints because they share the same code in the first place.[6] Business rules are naturally idiosyncratic and minimizing that so they don’t leak into our programming interfaces is valuable.

Here’s a picture of someone else’s cat to break up the text.

shower thoughts

This is getting too long, but I wanted to include some garbage I’ve been thinking about off and on for quite some time.

Interacting with content follows similar patterns. You usually want to a combine or compose the following:

• Fetch content. Which might involve filtering, ordering, limits, or offsets.

• Mutate something either modifying fetched content or creating new content.

• Present fetched content with some shape maybe using some kind of pagination.

Performing this interaction using GraphQL or URL query strings is about encoding this information into a fairly human readable and writable string.

At the risk of getting much deeper into this, I’ll mention MongoDB real quick. I find it interesting because you can write an expression as a JavaScript object using some special notation for operators. In the request, the structure is serialized to BSON. Being a binary format, BSON is easier for machines to parse than text formats[7] and it supports quite a few more types than JSON so applications don’t have to guess at if something is a date or a string.

{ $or: [ { qty: { $lt: 30 } }, { item: /^p/ } ] }

Structured queries like this are interesting because the format is more to do with how to arrange native types, rather than arranging a string. (Assuming you can send your structure over the wire by using a suitable serialization format.)

On the other hand, strings are universal. Some programs may not have strong notions of objects or structures. Like maybe something written in bash (or Scratch?). Or maybe some weird domain-specific languages. I’m not sure.

Part of me believes in a future where we can compose and serialize generic query structures that describe what content to gather or what mutations to perform. And then, if needed, reference them from some remote procedure call/not CRUD specific format like GraphQL.

Because, for putting content on the web, the databases we make and the engines they use important, if not the most important things. But our time is spent, over and over, on writing protocols for moving that information around and also they can’t talk to each other.

[
  [
    "title",
    "published_at",
    { "author": [ "id", "name" ] }
  ],
  [
    "interesting post",
    "Fri Nov 22 11:56:49 PST 2019",
    [ 1, "spongebob" ]
  ],
  [
    "why starfish are the best",
    "Fri Nov 22 13:12:32 PST 2019",
    [ 2, "Patrick Star" ]
  ]
]