It’s been already a little more than a year that I began my Lisp journey. I made quaterly news digests, mainly from reddit’s feed:
Time has come for a yearly overview ! What happened in the Common Lisp world ? Are there (or groundbreaking promising useful fun) projects, articles, discussions, tutorials ?
No need to say, I won’t reference everything we find in the quaterly posts, which don’t list all new projects appearing on Quicklisp (we can find these in the monthly Quicklisp releases) or Github.
I hope this overview will sharpen your interest on what is in my opinion an under-sold and still very promising language and plateform, that I happen to like more and more (and sooo more than Python ;) ).
Happy discoveries.
Table of Contents
Documentation
Common Lisp’s online documentation could be more thorough and welcoming. Fortunately, a few of us revived some projects and work on it -my favourite project being the Common Lisp Coobook. This year, we got tutorials on:
- Datastructures
- Debugging, including how to interactively debug a spacecraft,
- the Common Lisp Object System (CLOS)
- Loop, iteration, mapping
- Database access and persistence
- Error and condition handling
- Numbers and multidimensional arrays
- Scripting and building self-contained executables
- Working with types
along with many improvements on other pages, like on getting started and editor support.
Which brings me to it: the editors situation is much more open than you think:
- The editor of choice is still Emacs with Slime (or Sly),
- However, we can get started with Emacs and Lisp in 3 clicks with Portacle, a self-contained batteries-included sbcl-included portable Emacs tailored for CL,
- For Vim and NeoVim we have SLIMV, VLIME, and plugins can be written for NeoVim using cl-neovim.
- Or if we want an editor written in cl, there’s the self-contained Lem editor, which also works for Python, Go, Rust, Nim, Scheme, HTML, JSX, along with a directory mode, an experimental LSP mode, calc-mode, and more,
- Not to forget that Mac Os X users can use the Clozure Common Lisp IDE
- All editions of LispWorks (including the free) include the LW IDE
- For users of Eclipse IDE, there is the Dandelion plugin
- For popular editors, the experience is getting very good on Atom and the popular Visual Studio Code can be made to work with CL using cl-lsp.
- We have an ipython-like REPL (cl-repl),
- and for interactive notebooks, we have Jupyter kernels and yet another notebook (Darkmatter).
A very welcome improvement is the Common Lisp fundation’s website: https://common-lisp.net/ It got a massive update and is now attractive. We had http://lisp-lang.org/ (don’t miss its success stories section (did you know that pgloader was re-written from Python to CL ? :) )), but common-lisp.net was a googlers’ honey pot.
This website uses two “awesome” lists that were created or massively furnished last year:
- the Awesome-CL list, updated with hundreds of commits, which hopefully makes for a more discoverable and appealing ecosystem, and
- Awesome Lisp Companies: it was needed because Lispers didn’t know a lot of companies using CL appart from IRobot, Google’s ITA (powering Kayak, Orbitz and others), Grammatech, YCombinator, Siscog or other dead ones.
Other places to learn Common Lisp include:
- cl-exercise: a Common Lisp Learning System running on browsers
- coding schools, like Kattis
- and competitive Programming websites like CodeForces, HackerEarth, HackerRank, and CodeChef.
- lastly, Peter Norvig’s book Paradigms of Artificial Intelligence Programming is available on Github
We also regularly have new screencasts to enjoy:
- a lot being from Baggers: he does the following and he streams live nearly weekly
- little bits of Lisp: short videos to learn Lisp basics
- lots of bits of Lisp: long videos to dive deep in advanced subjects (macros, CFFI,…)
- Pushing pixels with Lisp: mostly working with OpenGL
- and more !
- Shinmera has lots of videos too, we can see him working on game engines, games, his libraries, Qt applications and more,
- the CL study group (here, an introduction to Screamer, a non-deterministic programming library)
Implementations
Time is good for Common Lisp implementations. Most date back from
decades and already proved what they can do (remember, SBCL is a
descendant of the Lisp that went to space). Hence the lack of hype,
IMO. Yet, many are in active development, and keep improving. As
/u/defunkydrummer
observed:
We are lucky to live in a time where Lisp development is still ongoing, many teams carrying the flag of open-source Lisp:
- SBCL (new release today)
- SICL (last commit 2 hours ago)
- ECL (last commit, yesterday),
- CLASP (last commit 2 days ago)
- CCL (last commit 7 days ago),
- CLISP (two weeks ago),
- CMUCL (1 month ago)
- ABCL (3 months ago)
SBCL has monthly releases. If you read the release notes, you might worry:
the amount of changes in each release is decreasing these years
but, as /u/baggers
notes:
I think the commits tell a slightly different tale though. There is always a lot of background ‘making stuff better’ work than won’t appear as the explanation would either be ultra internal and specific or would be super vague and very similar each month (for example ‘stuff is slightly faster’).
For one that would be overly specific this one might make for a good example. It’s grand work, but doesn’t surface in any specific lisp feature, stuff is just better.
Furthermore, a maintainer:
Or the developers are too lazy to describe their changes.
which isn’t a good reason ;)
We got a new release of Corman Lisp, a high performance Windows/32bit specific implementation with a built in IDE,
we have CLASP, targetting C++ through LLVM (see “Lessons Learned Implementing Common Lisp with LLVM”), built with the Cleavir compiler, part of SICL, a very new implementation of Common Lisp with fresh ideas,
we have ABCL targetting the JVM, Embedable Common Lisp, without forgetting active commercial ones, like LispWorks and AllegroCL. While I’m at it, you might want to have a look at MOCL for IOs, Android and OSx.
We got a nice talk by Clozure Common Lisp’s maintainer: this Old Lisp (this one may be the second most used implementation, particularly good for development – super fast compilation times (I heard it compiles itself in seconds), advising, watched objects, its own IDE).
Last note, a SBCL maintainer started a RISC-V port: First RISCy Steps – Porting SBCL to the RISC-V
So: welcome to this new world. It’s bigger than I thought, for sure.
Projects
I only list some projects that can be of interest to anybody. For the full stuff see the quaterly posts !
New projects
- Next browser 1.2.0 is out!: a browser exposing all its internals to CL. Be productive.
- CANDO - A Computational Chemistry programming environment integrating Common Lisp and C++ based on the Jupyter notebook
- Coalton, a dialect of ML embedded in Common Lisp (alpha)
- Voxel game engine (Minecraft) - a Minecraft engine. Allows for interactive changes.
- Emotiq - blockchain in Common Lisp
- Temperance - logic programming (in development, reached v1.0.0)
- MAGICL: Matrix Algebra proGrams In Common Lisp - Rigetti Computing (quantum computing)
- SHCL: An Unholy Union of POSIX Shell and Common Lisp (reddit)
- JSCL 0.7.0 now supports CLOS thanks to the work of vlad-km
- cl-torrents 0.9 - readline interface and 1337x.to scraper - a simple tool to search for torrents on popular trackers. My first CL app. Web and GUI interfaces in the making.
- Introducing Seed: An Interactive Software Environment in Common Lisp
- Tovero is a 3D modeling system for Common Lisp
- RMSBolt: See what your compiler is going inside of Emacs (has minimal support for Common Lisp)
- pngload: A PNG (Portable Network Graphics) image format decoder
- cl-vep: a video effects processor
- algebraic-data-library
- Petalisp: Elegant High Performance Computing
- wiki-lang-detect: Text language identification using Wikipedia data
- Dufy, a color library
- ppath, a path manipulation library
- cl-statistics.lisp
- Powerlisp: A simple tool to automate your work with dmenu/rofi
- json-mop: A metaclass for bridging CLOS and JSON objects
- clsh: a set of Lispy bindings for running and composing *nix processes
- filtered-functions - enables the use of arbitrary predicates for selecting and applying methods.
Web
- Weblocks’ new quickstart - Weblocks is an isomorphic web frameworks that allows to write interactive web apps without writing Javascript (nor writing code that transpiles to JS). It is seeing a massive update right now. Being Lisp, we can build a self-contained executable of our web app, send it to the server, run it and see it from the outside.
- three email libraries
- reddit1.0 source code (comments), then Reddit’s code runs on SBCL. See also reddit.
- Interactive Common Lisp code snippets in any web page
- arboreta-wasm - Common Lisp tooling for WebAssembly
For web libraries, see https://github.com/CodyReichert/awesome-cl#network-and-internet
GUI
- nodgui - yet another Tcl/Tk-based GUI package for Common Lisp (based on Ltk, with syntax sugar and more meta-widgets)
- IUP bindings GUI stuff (in the works)
- YstokWidgets Professional Edition
- MIDGETS - A collection of CAPI widgets and utilities
- subtext: A mostly-text-based UI bridges Common Lisp objects and runs of text. Minimal text-based user interface
- ftw: Common Lisp Win32 GUI library
- Cocoa interface code written in Lisp for use with Clozure Common Lisp
- McCLIM 0.9.7 “Imbolc” release
- Demo SBCL script using Gtk
- Demo ABCL script using Java Swing
for GUI libraries: https://github.com/CodyReichert/awesome-cl#gui
Package management
Quicklisp is the de facto package manager, but new projects come to complement it and bypass its limitations:
- the second version of Ultralisp is available - Ultralisp is an important project that fills a gap. It is a quicklisp distribution which updates every 5 minutes. It is also a Weblocks application!
- quicksys - installs systems from multiple Quicklisp distributions.
For more options, see Qlot (install and pin libraries locally, like Python’s virtualenv) and Roswell.
Deployment
- Apache Thrift gains CL support
- s2i-lisp: Common Lisp + Quicklisp OpenShift Build Image
- lisp-images: Docker images for common lisp development (with some others, see the awesome-list)
- A docker container for CL development (also lisp-devel, CI on CL Cookbook)
- Kubernetes Client Library for Common Lisp
- Heroku buildpack for Common Lisp
- cl-aws-custom-runtime - An example of using Common Lisp (SBCL) as a custom runtime on AWS lambda.
- prometheus.cl - Prometheus.io client. Grafana dashboard for SBCL and Hunchentoot metrics (memory, threads, requests per second,…).
We can also deploy apps on Digital Ocean, and no need to say that deploying a self-contained executable is easy, connecting to a remote instance too.
Music
- Music: Music framework for musical expression in Common Lisp with a focus on music theory (built from scratch, on development)
- Composing in Lisp with Csound
- Shuffletron, a Common Lisp Music Player for the terminal
see also audio and music composition software
(re)Discoveries
- lfarm - a library for distributing work across machines (on top of lparallel and usocket)
- Screamer - nondeterministic programming. Augment Common Lisp with practically all of the functionality of both Prolog and constraint logic programming languages (10 yo, Nikodemus)
- quid-pro-quo: a contract programming library in the style of Eiffel’s Design by Contract
- Cells, spreadsheet-like expressiveness for CLOS
- cl-bibtex: A compatible re-implementation of the BibTeX program in Common Lisp, with a BST-to-CL compiler
- C language syntax embedded in Common Lisp
- gendl - Generative Programming and Knowledge-based Engineering (KBE) system embedded in Common Lisp
- Cognitive Robot Abstract Machine = Common Lisp + ROS
- Esrap - a packrat parser for Common Lisp
- C-Mera, a Common Lisp source-to-source compiler to generate C/C++
- cl-bench - Common Lisp benchmarking suite
- QGAME: Quantum and Gate Measurement Emulator
Articles
- A Road to Common Lisp (hacker news comments). You should read this one.
- How the strengths of Lisp-family languages facilitate building complex and flexible bioinformatics applications
- Writing a natural language date and time parser - internals of the Common Lisp library Chronicity
- Implementing Hunchentoot custom sessions
- Overview of Documentation Generators (codex, coo, declt, staple, cldomain)
- Converter of maps from Reflex Arena to QuakeWorld. cl-yacc, 3d-matrices
- Debugging Common Lisp in Slime
- Packages in Common Lisp, a tutorial (pdf)
- How to write test fixtures for FiveAM - Dark Chestnut
- Franz and Semantic Web Co. Partner to Create a Noam Chomsky Knowledge Graph
- Compiler basics: lisp to assembly
- Marvin Minsky - Scientist - The beauty of the Lisp language
- Excavating a Common Treasure: Common Lisp
- Fun with Macros: If-Let and When-Let / Steve Losh
- Extempore - The design, implementation and application of a cyber-physical programming language, Andrew Sorensen, Thesis, 2018 (PDF)
- Uniform Structured Syntax, Metaprogramming and Run-time Compilation
- Simple expression evaluator comparison between Haskell, Rust, and Common Lisp
- Lisping at JPL
and also
- Lisp, Jazz, Aikido: Three Expressions of a Single Essence
- Why lisp - biolisp
- Fun with Macros: Gathering / Steve Losh
- Experience writing a full featured livejournal blog client in Common Lisp. Part 2: client logic
- The (Un)common Lisp approach to Operations Research (2012)
- Alien: Return of Alien Technology to Classical Planning
- Emacs + ECL on Android
- Generic, consistent and dotted access of data structures with Access - lisp-journey (reddit)
- LLVM’s garbage collection facilities and SBCL’s generational GC
- A bunch of utilities from (again) sjl: higher order functions, sequences, debugging, profiling.
- The return of cl-notebook
- Testing the SERIES package
On games:
- About Making Games in Lisp - Gamedev
- Creating a (Non-Trivial) Lisp Game in 2018 (they just launched a Crowdfunding)
- A Story of (defun games ())
- Getting Started With trivial-gamekit
Other screencasts
- Lisp, The Quantum Programmer’s Choice - Computerphile episode 2
- McCLIM + Maxima: plot manipulation
- McCLIM + Maxima: vector demo
- Comfy Lisp Programming - Project “Wikify” | Episode 2 @ 10am PST
- Common lisp and C++17 Live coding stream | TinyCDN CFFI Interop | Episode 13
- Growing a Lisp compiler - Amsterdam Lisp
- Web Development in Emacs, Common Lisp and Clojurescript - Potato (Slack-like)
Discussion
- Lisp and the remote agent - aka Lisp in a spacecraft - with an AMA of Ron Garret
- How to make (Common) Lisp popular?
- Feedback from a new LispWorks user (how is LispWorks the company going ?)
- How do you normally use a program once written ?
- Structs vs Parametric Polymorphism (an answer to the “switching from Common Lisp to Julia - thoughts ?” post) also this discussion
- How to work on a project and make sure dependencies are tracked correctly?
- Does anyone else hate LOOP?
- What does it take to understand the true power of Lisp?
- How did Lisp make your life easier ?
- Should local variables be avoided when possible when doing functional programming?
- Is ABCL an active project and does it support JRE 1.11?
- Has the Gnu Coreutils ever been implemented in Lisp? If not, would that be a worthwhile project?
- Common Lisp and Machine Learning these days
- Has anyone considered or started a project to write a CL implementation in WebAssembly?
- What do you recommend to work with SQL databases ? What’s your experience with Mito ? and sqlite only interface: cl-sqlite or cl-dbi ? and is there an ORM that generates classes from table definitions ?
Learning Lisp
- I want to try Lisp, how should I begin?
- What lisp dialect for “real world” applications?
- What do commercial Lisps offer that frees don’t?
- Which (non-Clojure) Lisp to learn first?
- Can CL implement Clojure’s keyword as function syntax?
- Why did you decide to learn Lisp?
- How do you go about starting a Common Lisp Project? A beginner looking for pointers.
- As a newbie, what I will miss if I choose Racket over Common Lisp? Or if I happen to learn both at somepoint in future, choosing Racket/Common Lisp now would make sense?
- What can other languages do that Lisp can’t ?
Common Lisp VS …
- How did the Common Lisp community survived without the equivalent of clojure.spec ?
- Is there a Lisp that is considered “excellent” about error handling ?
- Lisp Dialect survey
- the Julia challenge
- Python pitfalls ?
- How a Common Lisp programmer views users of other languages (humor)
- Miller School Researchers Help Push the Limits of Programming Languages in Biology
- Lisp vs Java (thought you guys might find this humorous)
- What other languages besides Lisp do you enjoy programming in?
Enjoy the material, and see you soon !
thanks to /u/defunkydrummer
for proofreading.
fix or improve this article on Gitlab.