Asko Nõmm

Musings of a Software Engineer

Creating a simple HTTP Router in Clojure

4th November, 2020~13 min read

A bit of a detour from Swift to what is probably my favorite language, Clojure. Fun fact; I was forced to learn Clojure when I joined GreenPowerMonitor back in 2018 much to my surprise given I joined for JavaScript and it turned out to be a wonderful mistake because as a result of that I learned what Lisp is and how nice it is to write.

Anyway, as I've started work on a not-so-secret project of mine and I found myself in need of a router but didn't want yet another dependency nor so much complexity, I decided to roll my own simple and to the point router.

HTTP Server

The first thing we need in a Clojure application that needs to serve the web is an HTTP server. An HTTP server essentially takes in user requests to your application under a specified IP/URL and port and then serves the user with a response that you create. Honestly, I'm not even going to attempt to roll this on my own, but I recommend either HTTP-Kit or Ring with Ring Jetty adapter. Both of them should be pretty much interchangeable for the current use case, but I went with Ring because HTTP-Kit doesn't support cookies and my secret project needs cookies.

A very basic "hello, world" example could be as simple as:

(ns app.core
  (:require [ring.adapter.jetty :as jetty])
  (:gen-class))

(defn -main [& args]
  (jetty/run-jetty (fn [request]
                     {:status 200
                      :headers {"Content-Type" "text/html"
                      :body "Hello, world"})
                   {:port 8080}))

You see all we do here is require the ring.adapter.jetty namespace, then call jetty/run-jetty (or HTTP-Kit's run-server) with an anonymous function that takes request as its parameter (this will be important) and expects the return of the function to be a response map which needs to consist of :status as the HTTP status code, :headers for the HTTP header and :body for the response content. As the second parameter to run-jetty we specify a configuration map which in our case only contains the port at which we'd like to serve our application. Feel free to change it to whatever you want.

Router

Really what a router is, is all in the name - it routes information depending on input. When a new HTTP request comes into our anonymous function above (remember I said it's important!) it comes with a ton of information, but the information we'll be needing is :request-method and :uri, information which is readily available in the request variable.

The :request-method will be either :post, :get, :put, etc. The :uri will be the path of the URL, so if the user visits your application with the URL localhost:8080/hello then the :uri will be "/hello".

Now let's start with a data structure out of which we'll build a full routing system. Let's start with this:

(def routes [{:method :get
              :path "/"
              :response (fn [args]
                          {:status 200
                           :headers {"Content-Type" "text/html"}
                           :body "Home page."})
              {:method :get
               :path "/hello"
               :response (fn [args]
                           {:status 200
                            :headers {"Content-Type" "text/html"}
                            :body "Hello page."})}
              {:method :get
               :path "/hello/:name"
               :response (fn [args]
                           {:status 200
                            :headers {"Content-Type" "text/html"}
                            :body (str "Hello, " (get args :name))})}}])

You can see that this contains three routes, one for the home page, one for the "/hello" page and for the "/hello/:name" page to demonstrate custom parameters in URLs. It also specifies with what method should we listen to and if the :method and the :path matches the request, then we return the :response.  

Each response is an anonymous function (but can of course also be a regular function) which has a parameter args (we'll talk about this further down in this post) that we use to get things like URL parameter values, as is apparent in the third route to display the person's name that is found in the URL path.

Matching

Now that we have the request that comes as the parameter to our anonymous function that we pass to run-jetty and we have our routes data structure which contains paths, we need to match the request with a path to determine which route out of all the routes do we want to use. So let's create a function called route-path-matches?, yes, with a question mark because it returns a boolean, like so:

(defn route-path-matches? [path request-path])

Since the path and the request-path are identical in their shape with path only differing in that it has custom parameter keywords for matching against any word. Confusing? Don't worry, I'll explain.

If the request-path is "/hello/john" and the path is "/hello/:name" then we want to make sure that the :name keyword can be any word, so that "/hello/john" and "/hello/:name" would be a match, as clearly if those two were compared directly they wouldn't be a match. Here regex can come to our rescue as the first thing we'll be doing is converting path to a regex string by replacing all occurrences of :keyword with a regex \w+ which means any word. We can do so like this:

(clojure.string/replace path #":(\w+)" "\\\\w+")

So this will turn a string like "/hello/:name" into a regex string "/hello/\w+". In case you are wondering why there are a whopping four backward slashes (\\\\) it's because a single backward slash is not permitted as an escape character and thus results in an error, however, four backward slashes will result in a single backward slash. I don't know the underlying reason for this design, if you do, please get in touch!

And so we can put this together to match the request-path with a complete function that looks like this:

(defn route-path-matches? [path request-path]
  (let [regex (clojure.string/replace path #":(\w+)" "\\\\w+")]
    (when (re-matches (re-pattern regex) request-path)
      true)))

So when the regex matches, we return true. In case you didn't know, if the when fails, it will return nil so there's no need for an if here. So now that we call this function, the behaviour is following:

(route-path-matches? "/hello/:name" "/hello/john") ;; returns true
(route-path-matches? "/hello/:name" "/hello") ;; returns nil

Composing parameters

Alright, so route matching works. But we also need to match that :name keyword with the actual value in the URL so that when the URL is "/hello/john" and our route path is "/hello/:name", we'd get a map like this:

{:name "john"}

Or when the url is "/hello/john/age/20" and the route path is "/hello/:name/age/:age", then we'd get a map like this instead:

{:name "john"
 :age "20"}

Let's start by creating a function called route-path-params, like so:

(defn route-path-params [path request-path])

Like in the case of route path matching we did previously, all we need for parameter composing is path and request-path, and since the two are identical in how many slashes they have (e.g. if path is "/hello/:name" and request-path is "/hello/john", then both of them have 2 forward slashes), we can simply split both of them up by the "/" character, like so:

(defn route-path-params [path request-path]
  (let [split-path (clojure.string/split path #"/")
        split-request-path (clojure.string/split request-path #"/")]))

And now we have two sets. However those sets will also contain empty strings, because the split function will split AT the character "/", meaning that there will be an item in the set as "", because there is nothing before the first "/". We can fix this by removing all the empty items from the set, like this:

(defn- route-path-params [path request-path]
  (let [split-path (vec (remove #(= "" %) (clojure.string/split path #"/")))
        split-request-path (vec (remove #(= "" %) (clojure.string/split request-path #"/")))]))

And we'll also be turning them into vectors because we'll be needing to use indexes to access specific parts and sets do not have indexes. As to why we need indexes, well, we'll be looping over the split-path vector and checking if we're dealing with a keyword because if we are, we'll be using that index to get the equivalent from split-request-path, which will contain an answer to our keyword. Genius, right? We'll be doing it with a map-indexed function built into Clojure, like this:

(map-indexed (fn [index item]
               (when (clojure.string/starts-with? item ":")
                 {(keyword (subs item 1)) (get split-request-path index)}))
             split-path)

You see map-indexed will take an anonymous function with the first parameter of that being the index of the current item and the second being the item itself. In that anonymous function we check if the item starts with the : character, because if it does, we know it's a keyword. And if it is a keyword, we'll be returning a map with that keyword and we'll be using the index to retrieve the answer to that keyword from split-request-path.

Now if the request-path would be "/hello/john/age/20" and our route path would be "/hello/:name/age/:age" then we'd get this as a result:

({:name "john"}
 {:age "20"})

Which is cool, but not what we want. What we'd like is one cohesive map instead of multiple ones in a set. We can do so by using the into function to force our set into a sorted map instead, like so:

(into (sorted-map) ...our-set)

So now putting this all together, to compose parameters, we'd use this:

(defn route-path-params [path request-path]
  (let [split-path (vec (remove #(= "" %) (clojure.string/split path #"/")))
        split-request-path (vec (remove #(= "" %) (clojure.string/split request-path #"/")))]
    (into (sorted-map) (map-indexed (fn [index item]
                                      (when (clojure.string/starts-with? item ":")
                                        {(keyword (subs item 1)) (get split-request-path index)}))
                                    split-path))))

Which would result in a map like this:

{:name "john"
 :age "20"}

Routing the request

Now that we have the foundation for our router, let's write a function that uses that foundation to route the request and return the correct response. This function needs to take in the current request and the collection of routes that we made, like so:

(defn route [request routes])

As the first order of business, let's get the :uri and :request-method from request, like so:

(defn route [request routes]
  (let [request-path (get request :uri)
        request-method (get request :request-method)]))

Now we need to turn our routes into responses that we match against request, for that we will be looping over the routes and checking if the route :path matches with the route-path-matches? function, like so:

(for [{:keys [method path response]} routes]
  (when (and (= method request-method)
             (route-path-matches? path request-path))
    (response)))

First, we destructure the first parameter of for, which will be the route itself, to get the method, path, and response from it. Then we make sure that the method from the route matches the request-method (e.g. is :get, :post, etc) and finally we use the route-path-matches? function to make sure the path matches request-path. If all of these conditions are met, we call the anonymous function we created for each route, which will return the response map.

However, remember that each of the anonymous functions we created for routes also takes in a parameter called args! This will contain information from the request such as the URL parameters, for which we made the function route-path-params previously, so with passing that information to the response we'd call the response callback like this:

(response {:params (route-path-params path request-path)})

Now the loop that we constructed will return us a set of all the routes that matched the request. And yes, there can be multiple. For example, if we have routes with paths that go like this:

And if the URL is "/hello-world", then both of these routes would match, but obviously, we can't show two different responses to the user at the same time, so we need to choose. To make both of those routes possible, we'd need to always declare the static route before the dynamic one, so "/hello-world" needs come before "/:post-name" in the routes map. And if you do that then the answer to the question of which route from the set that matched the request to show is simple: the first one.

All we'd need to do is call wrap the responses with a first function. But that makes evident another issue, which is that the first item in the responses set could easily be empty given that within the for loop we use a when, which means that any route that didn't match the request would also be in the responses set, but would show up as nil, because that's what when returns when it evaluates to false. A solution to that is also simple, simply use remove function, pass the second parameter to that as nil? and the third being the responses set.

A complete route function then would be:

(defn route [request routes]
  (let [request-path (get request :uri)
        request-method (get request :request-method)
        responses (for [{:keys [method path response]} routes]
                    (when (and (= method request-method)
                               (route-path-matches? path request-path))
                      (response {:body (parse-request-body (get request :body))
                                 :query-string (get request :query-string)
                                 :params (route-path-params path request-path)})))]
    (first (remove nil? responses))))

You can see that I added things like :body (for input you get with POST data) and :query-string also to response for good measure, because you probably will need them at one point. But now that our route function is complete, we can use it in our -main function like this:

(defn -main [& args]
  (jetty/run-jetty (fn [request]
                     (route request routes))
                   {:port 8080}))

And boom, your routes should now work.

Pretty syntax bonus

I love when things are easy to read, sometimes at the cost of the underlying solution being harder to read, which is a bitter-sweet thing and you can completely choose to ignore this bit. However, if you'd like your routes map being easier to read, so that instead of this:

(def routes [{:method :get
              :path "/"
              :response (fn [args]
                          {:status 200
                           :headers {"Content-Type" "text/html"}
                           :body "Home page."})
              {:method :get
               :path "/hello"
               :response (fn [args]
                           {:status 200
                            :headers {"Content-Type" "text/html"}
                            :body "Hello page."})}
              {:method :get
               :path "/hello/:name"
               :response (fn [args]
                           {:status 200
                            :headers {"Content-Type" "text/html"}
                            :body (str "Hello, " (get args :name))})}}])

You instead would have this:

(def routes [{:get "/" :do home}
             {:get "/hello" :do hello}
             {:get "/hello/:name" :do hello}])

Then follow along to this pretty syntax bonus chapter of the post.

Response functions

We need to create the response functions home and hello for our routes, but first, let's create a utility function called response so we wouldn't have to repeat ourself too much:

(defn response [body]
  {:status 200
   :headers {"Content-Type" "text/html"}
   :body body})

This response function will keep us from repeating the response map, so now that we want to display something to the user we can just call (response "hello").

Now, on to the response functions home and hello:

(defn home [_]
  (response "Home page."))
(defn hello [{:keys [params]}]
  (if (get params :name)
    (response (str "Hello, " (get params :name)))
    (response "Hello, World.")))

As you saw from our routes map, we're re-using the hello response function for two routes. One of them has a dynamic parameter, the other doesn't, and we use that to conditionally display a name.

Modifying the router

In order to make {:get "/path" :do response-function} possible, we need to slightly modify our route function.

Inside the for loop where we destructure the first parameter (the route itself) to get the method, path and response we no longer will do that. Instead, we will get the key of the first item in the route (the method), the value of the first item in the route (the path), and the value of the last item in the route (the response function). Our route function effectively becomes this:

(defn route [request routes]
  (let [request-path (get request :uri)
        request-method (get request :request-method)
        responses (for [route routes]
                    (let [method (key (first route))
                          path (val (first route))
                          response (val (last route))]
                      (when (and (= method request-method)
                                 (route-path-matches? path request-path))
                        (response {:body (parse-request-body (get request :body))
                                   :query-string (get request :query-string)
                                   :params (route-path-params path request-path)}))))
        response (first (remove nil? responses))]
    (if response response {})))

You may also see that I've moved the return of this function to the response variable and I'm now returning an if instead. This is because previously when there would be no route matches, you would get an error in your console saying "Received nil as response map", so you will no longer get that, because now no matter if there is a route match, the HTTP server will always get a map back.

And that's how you make a simple Router in Clojure. Pretty cool, right?