Writing a web framework in Lua and WSAPI


Learning LUA by example
12-Aug-2015 | Anton Alex Nesterov

A couple words about Lua itself.

Lua is a lightweight language created to be embeddable and portable. It is very fast and flexible. Lua’s most powerfull features are Tables, Coroutines, and Tail Recursion.

I didn’t know anything about Lua before I got a Django project which used UWSGI. Of course I’ve heard many controversial opinions about Lua from different people (gamers, experienced programmers, electronic engineers, traders, etc). The fact is that I’ve never heard a neutral opinion about Lua - it was always either positive or negative.

After a little research I found out that there is OpenResty and some other attempts to bring Lua into web development. All benchmarks I saw showed very good performance. UWSGI offered a Lua support through a plugin. The plugin documentation suggested to consider Lua as a solution for slow areas in the code instead of dealing with C. It got my interest because writing C extensions for Python isn’t something enjoyable for me (BTW there’s D).

WSAPI

So I had free time and I’ve decided to learn more about Lua. Of course my first idea was to write a simple web framework. My experience wanted it to be something similar to what I knew from Python universe.

UWSGI documentation suggested to use WSAPI. WSAPI is a WSGI-like abstract layer for Lua. WSAPI also provides its request and response objects, testing tool, and CGI interface.

To use WSAPI you need to write a function which expects a server environment and returns status, headers and a corotine which generates response body.

    function main(environ)
        local headers = { ["Content-type"] = "text/html" }
        local function content()
            coroutine.yield("<p>Hello Universe!</p>")
        end
        return 200, headers, coroutine.wrap(content)
    end

Such function can be an entry point of our web-framework where we can register callbacks and middlewares and create request-response objects.

Application Architecture.

The web framework should provide routing and middlewares at first.

Let’s imagine how it could look:

    --pseudo code
    local Web = require "framework-name"
    local app = Web()

    app:middleware(
      function(request, response)
        res:write("I'm a middleware!")
      end
    )

    app:get("/", some_get_callback)
    app:post("/:name/:id", some_post_callback)
    ...

The pseudo code assumes that the framework should be represented as a class so we can operate with instance of framework object which holds methods to register middlewares and routes. I assume that the framework method names should correspond to the http request method names (get,post,put,delete).

Lua language doesn’t provide classes, it provides prototyping instead. Thankfully, Lua community have found lots of solutions to emulate familiar class behavior. I picked the one from here.

Creating a class im my case looks like following:

    local Web = class(
        function(self, opts)
            --constructor is here
        end
    )

Adding a method:

    function Web:mymethod(..args)
        self.something = somewhere
    end

Here you have to pay attention to the colon(:) syntax. Colon in Lua is a syntax sugar that gives us an access to the ‘self’ object.

Writing web framework

At first, let’s write the constructor of the future web framework. Application object has to include attributes to keep middleware functions, and registered endpoint callbacks. I named the future framework as ‘Slash’

The Construcror:

local Slash = class(
  function(self, opts)
    self.opts = opts or {} -- options
    self.middlewares = {} -- global middlewares
    self.endpoint_middlewares = {} -- enpoint middlewares
    self.callbacks = {} -- registered callbacks
  end
)

Next, we have to write route matching logics. The route matching will go in two steps: First is to convert the path to a pattern which we can compare to regitered route callbacks. Second is to register or get a callback using this pattern.

The framework url syntax assumes two variable types: String or Number. URL pattern should provide information about request method. In the future we can use patterns to match request urls to callbacks.

Converting an url to a Lua pattern:

function Slash:toPattern(path, method)
  -- returns a string pattern for an endpoint path
  -- example: /path/:name/:id -> ^GET:/path/(%w+)/(%d+)/?$

  local params = path:split('/') -- i wrote String:split separately
  local pattern = "^"..(method or 'GET')..":" -- (..) is string concatenation

  for i,v in ipairs(params) do
    pattern=pattern.."/"
    if v==":id" then
      pattern=pattern.."([%d]+)"
    elseif v:starts(":") then
      pattern=pattern.."([%s%S]+)"
    else
      pattern=pattern..v
    end
  end

  pattern=pattern.."/?$" -- !! end slash is optional
  return pattern

end

Now we have to somehow register callbacks. Let’s write a ‘route’ method where we can use our pattern convert function. We will use the pattern as a key name for callback. We have to restrict overriding already registered callbacks.

Route method:

function Slash:route(url, method, callback)
  -- register a callback
  local pattern = self:toPattern(url , method)
  if not (self.callbacks[pattern]) then
    self.callbacks[pattern] = callback
  else
    error(
      "Callback for "..url.." already registered. Pattern: "..pattern ..
      "\n Use Slash:middleware(cb,[url]) to register a middleware on this endpoint."
    )
  end

end

Using a similar approach we will create a function to register middlewares. In this case the url pattern is used for enpoint specific middlewares.

function Slash:middleware(callback, url)
  --register a middleware
  if not url then
    table.insert(self.middlewares, callback)
  else
    local tbl = self.endpoint_middlewares[self:toPattern(url)] or {}
    table.insert(tbl, callback)
    self.endpoint_middlewares[self:toPattern(url)] = tbl
  end
end

In order to get a regitered callback we need to match a URL path to pattern:

function Slash:getCallback(url, method)
  --get registred callback
  for k,v in pairs(self.callbacks) do
    if (method..":"..url):match(k) then
        return v
    end
  end
end

To make it all work we have to write a function which will call all middlewares and registered callback when requesting some url.

function Slash:call(url, method, request, response)
  -- calls all middlewares and ends with the enpoint callback

  local endpoint = self:getCallback(url, method) --get callback (will be called at last)
  if not endpoint then
    return error("not found")
  end

  local all = {} --here we'll keep all callbacks (including middlewares)

  for i,v in ipairs(self.middlewares) do
      table.insert(all, v) -- global middlewares
  end

  for k,v in pairs(self.endpoint_middlewares) do
    if ("GET:"..url):match(k) then -- enpoint middlewares
      for i,j in ipairs(v) do table.insert(all, j) end
      break
    end
  end

  table.insert(all,endpoint)

  for i,callback in pairs(all) do
    if callback(request,response) then
      break
    end
  end

end

In sake of better user experience we can wrap the ‘route’ method so that it corresponded request method name.

function Slash:get(path, callback)
  self:route(path, 'GET', callback)
end

function Slash:post(path, callback)
  self:route(path, 'POST', callback)
end

function Slash:put(path, callback)
  self:route(path, 'PUT', callback)
end
function Slash:delete(path, callback)
  self:route(path, 'DELETE', callback)
end

Finally, we need an WSAPI entry point where we will handle the requests.


local request = require 'wsapi.request'
local response = require 'wsapi.response'

function Slash:make()
  -- returns a wsapi handler

  return function (env)

    local req = request.new(env)
    local res = response.new(200,{})

    success, err = pcall(
      function()
        local path = env.PATH_INFO
        for k,v in pairs(self.apps) do
          if string.starts(path,k) then
            local p = path:gsub(k,'')
            req.path_data = p:split('/')
            return v:call(path:gsub(k,''), req.method, req,res)
          end
        end
        req.path_data = path:split('/')
        self:call(path, req.method, req,res)
      end
    )
    if success then
      return res:finish()
    else
      res.status = 500
      return res:finish()
    end
  end

end

Now we’re ready to use it in action.

Example:

Slash = require "slash"
app = Slash()


-- add a global middleware
app:middleware(
    function(req,res)
        res:write("string from global middleware <br>")
    end
)


app:route('/', 'GET',
  function(req,res)
    res:write("Hello World!")
  end
)

app:delete('/something/:id',
    function(req,res)
        res:write("Ooops! Somting was accidentally deleted.")
    end
)

return app:make()

To see more features like templating, sessions, and sub applications visit Slash repository at GitHub. There are also some tests and UWSGI config.
Slash repository on GitHub


Thank you and be back.