How do you run your Common Lisp (web) application on your server? Nowadays most GNU/Linux distros have Systemd. I recently used it more, with a mix of applications running from source, from a binary, running locally or on my VPS. I had to bypass a few gotchas, so let’s recap’ what you need to know.

Also stay tuned: next, we’ll see how to build a standalone binary for your Common Lisp application with Deploy (so that we handle foreign libraries like libssl), how to include your Djula HTML templates as well as your static assets. This, in turns, makes it straightforward to ship your web app into an Electron desktop window.

INFO: dear readers, here's the best price I can set for my Udemy course "Learn Lisp effectively" , it will be available for 5 days! Thanks all for your support (NB: I'm working on the chapter on condition handling and new chapters are available for existing students).

Before Systemd

Let’s say we can run our app like this: we load the system definition, its dependencies, and we call the entry point.

sbcl --load my-app.asd \
     --eval '(ql:quickload :my-app)' \
     --eval '(my-app:start-app)'

Now we need to put the app on the background, we must ensure that if it fails, it is restarted, if the server is restarted, our app too, we must ensure we get to see the logs, etc.

Maybe you run your app inside of Emacs on your VPS… maybe you run your app inside tmux. This works and it is convenient to get back to the Lisp REPL, but this doesn’t ensure a restart on failure.

Systemd: daemonizing, restarting in case of crashes, handling logs

Systemd (or the service system of your distro) can help with all that.

Write a service file like this:

$ emacs -nw /etc/systemd/system/my-app.service
[Unit]
Description=Lisp app example

[Service]
WorkingDirectory=/path/to/your/app
# Next, your command, with the full path to SBCL.
# This works, locally. Be sure to see the last section.
ExecStart=/usr/bin/sbcl --load run.lisp
# or: use a path to your binary.
Type=simple
Restart=always
RestartSec=10
# Use environment variables:
Environment="SECRET=pGNqduRFkB4K9C2vijOmUDa2kPtUhArN"

# Start or restart at boot:
[Install]
WantedBy=basic.target

When you run your app on the terminal, ensure that its webserver (Hunchentoot here) stays listening correctly on the foreground (otherwise see below):

$ sbcl --load run.lisp
Hunchentoot server is started.
Listening on localhost:9003.

Now run this command to start the service:

sudo systemctl start my-app.service

to check its status use this:

systemctl status my-app.service

Systemd handles logs for you. We make our app write to stdout or stderr, Systemd writes logs:

journalctl -u my-app.service

use -f -n 30 to see live updates of logs, where -n is the number of lines you want as context.

The following tells Systemd to handle crashes and to restart the app:

Restart=always

and this makes it start the app after a reboot:

[Install]
WantedBy=basic.target

to enable it:

sudo systemctl enable my-app.service

Now keep in mind a couple things.

Make it stay on the foreground

The first gotcha is that your app must stay on the foreground.

If you run your app from source, you might have nothing to do, you’ll get a Lisp REPL, from which you can interact with your running application. Awesome.

But, if you build a binary, you might see this error when you run it with Systemd:

* ;
; compilation unit aborted
; caught 1 fatal error condition" error.

This puzzled me: I thought I had a Lisp prompt (the * ;) and that my program crashed, but no. I knew it, that’s simply Lisp quitting too early. Don’t rush and double check that your binary runs correctly.

What you must do can be found elsewhere (the Cookbook!): in your main function where you start your app, in this example with Hunchentoot, put its thread in the foreground:

;; with bordeaux-threads. Also sb-ext: join-thread, thread-name, list-all-threads.
(bt:join-thread (find-if (lambda (th)
                            (search "hunchentoot" (bt:thread-name th)))
                          (bt:all-threads)))

Let it crash: --disable-debugger

We want our app to crash so that it can be re-started automatically: you’ll want the --disable-debugger flag with SBCL, when you run your app from sources.

Relying on Quicklisp

When you run your apps locally, you most probably rely on Quicklisp being installed and being started in your init file (~/.sbclrc):

;;; The following lines added by ql:add-to-init-file:
#-quicklisp
(let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp"
                                       (user-homedir-pathname))))
  (when (probe-file quicklisp-init)
    (load quicklisp-init)))

There are 2 gotchas.

Systemd will, by default, run your app as root, so:

  • if it happened you did install Quicklisp on your production machine, you probably didn’t install it as root, so Systemd won’t find the init file that initializes Quicklisp (and so your startup scripts will fail).
    • you can use SBCL’s --userinit flag to tell the username where to find the init file.
    • you can set the Systemd user with User=xyz in the [service] section (disclaimer: untested).
  • the Quicklisp snippet will fail at (user-homedir-pathname), for a clash on usernames too, so Quicklisp won’t find its setup.lisp file. I replaced this function call with a hard path (/home/vindarel/), until I used a standalone binary.

That’s it. Now you can deploy in peace. I hope I saved you some hours. Now these issues are better google-able \o/

See you around and stay tuned.