Make ANSI Common Lisp available on Haiku (again)

Back in 2019 I attempted to port SBCL Common Lisp to Haiku (see this topic). Unfortunately, because of health issues and other personal problems I did not do much. I even could not say for sure when my state was sufficiently stable to make a new attempt or not (and I tried several times and failed). From the last year on I feel I can resume my work, hence this topic will hopefully be more exciting for Lisp users.

Simplistic overview of Common Lisp distributions
Common Lisp distributions are different in how they approach performance:

  • There are distributions running upon some virtual machine provided by some other technology. One example is ABCL (https://abcl.org/) (available in Haiku Depot) running upon JRE (also available in Haiku Depot). Such distributions are highly portable but are very unproductive in sense of performance.
  • There are intermediary distributions, which are natively compiled but rely on C standard library (as well as some well-established 3rd party libraries) for portability. One example is CLISP (https://clisp.sourceforge.io/). They are usually small, quite good for porting and quite good in performance. However, they lack OS specific handling.
  • There are distributions, which are kept as close to hardware and OS as possible. One example being SBCL (http://sbcl.org/). They offer the best performance but usually much harder to port.

In the past, I use ABCL and with support from both ABCL and SBCL team had success in compiling SBCL runtime and large part of its library. Today, the last version of ABCL (1.9.3) does not compile the latest version of SBCL (2.4.9). This very situation was in the past (it was ABCL 1.6.1 and SBCL 2.0.4) and it was solved with the release of ABCL 1.7.0. Unfortunately, much has changed from then, and I cannot reproduce that result (compilation of SBCL 2.0.4 with ABCL 1.7.0).

I get a suggestion from one forum member to try to port CLISP to Haiku. Although CLISP homepage states the current version 2.49 is from 2010, if we look at its git repository (https://gitlab.com/gnu-clisp/clisp) we can see it is actively developed. So, I cloned CLISP Git repository and relatively painless compiled it in Haiku, for now without FFI.

Although I still fail to build SBCL, this itself is a great achievement. I will carefully document all my actions in porting CLISP and hopefully this will result in Haiku recipe for it.

9 Likes

I know, an image is worth 100 words. So, here it is:

Building CLISP on Haiku looks very promising.

  1. For now, one dependent library, libffcall, is not available in Haiku Depot.
  2. One recoverable crash occurs during configure run, maybe because of this missing library. But the build can continue to finish.
  3. From included self tests 56 out of 59 pass.
3 Likes

Look: add appveyor smokers ¡ rurban/clisp@46e78a1 ¡ GitHub

“add utils/libffcall-1.13-20170225-funbegin.patch:
the released libffcall and this beta are broken. patch it.
error: pasting formed ‘__builtin_avcall:’, an invalid preprocessing token
FUNBEGIN(__builtin_avcall)”

This project is very interesting: ecl / ecl ¡ GitLab

https://ecl.common-lisp.dev/main.html

Do you have a link for that? Can’t find a hit for “libffcall” at repology (if you are not planning a recipe for it)?

Accordingly to DEPENDENCIES file from CLISP Git repository (https://gitlab.com/gnu-clisp/clisp/-/blob/master/DEPENDENCIES?ref_type=heads):

1 Like

Found it, it’s called “ffcall” it seems: ffcall package versions - Repology

EDIT: should be fairly straightforward according to Arch:

1 Like

Thank you @marcoapc, ECL was in my radar some time ago. Unfortunately, I did not have as much success with it as with CLISP. But of course, in absence of alternative, I will take a closer look at it.

1 Like

The clean build of CLISP on Haiku can be as following:

  1. Install dependencies (libffcall is missing for now):
  • libsigsegv_devel
  • libiconv_devel
  • libunistring_devel
  • gettext_libintl_devel
  • readline_devel
  • ncurses6_devel
  1. Clone repository:
    git clone https://gitlab.com/gnu-clisp/clisp.git
  2. Because Haiku user has root privileges, issue:
    export FORCE_UNSAFE_CONFIGURE=1
  3. Configuration:
    cd clisp && ./configure --with-included-regex --without-ffcall --prefix=/boot/home/config/non-packaged
    Where parameter meaning is:
  • --with-included-regex - to avoid crash during configure run,
  • --without-ffcall - do not use libffcall for now,
  • --prefix=/boot/home/config/non-packaged - install in non-packaged folder for now.
  1. Edit src/config.lisp:
  • Change:
    (defparameter *editor* "vi" "The name of the editor.")
    to:
    (defparameter *editor* "nano" "The name of the editor.")
    or maybe even:
    (defparameter *editor* "/boot/system/apps/Pe/Pe" "The name of the editor.")
  • Add after:
    ;; (setq *browser* :firefox)
    the following:
    (setq *browser* "/boot/system/apps/WebPositive")
  1. Building:
    make
  2. Testing:
    make check-tests # 56 / 59 tests pass
    make check-recompile # Passes. It claims to rebuild CLISP with itself. I am not sure exactly what does it mean, CLISP is built using mainly gcc.
  3. Installation:
    make install

I believe, at this stage I can start preparing the recipe for it. Even if this is not in optimal state, it is easier to modify the recipe afterwards than create it from scratch. I start from these resources:

Any suggestion is warmly welcome!

1 Like

PR up at: ffcall, new recipe by Begasus ¡ Pull Request #11248 ¡ haikuports/haikuports ¡ GitHub

Maybe useful :slight_smile:

1 Like

Thank you @Begasus. I will test CLIPS with ffcall if it performs significantly better than without. Namely, the failing 3 tests.

Update: And of course, thank you for suggestions!

1 Like

I also did a “export LDFLAGS=”-lbsd -lnetwork" as I saw some network/socket checks when configure was running, on not finding libsigsegv, this is fixed with the clone (looks like latest tag in there got fixes for it since then).

EDIT: from the failed tests I see these 2 mainly in their reports:

Form: (LET ((FILE "test-pipe")) (UNWIND-PROTECT (DOLIST (B '(NIL T)) (WITH-OPEN-STREAM (OUT (MAKE-PIPE-OUTPUT-STREAM (FORMAT NIL "/bin/cat > ~A" FILE) :EXTERNAL-FORMAT CHARSET:UTF-8 :BUFFERED B)) (LOOP FOR I BELOW 1000 DO (PRINT I OUT))) (SLEEP 1) (WITH-OPEN-FILE (IN FILE :DIRECTION :INPUT :EXTERNAL-FORMAT CHARSET:UTF-8) (LOOP FOR I BELOW 1000 DO (ASSERT (= I (READ IN)))))) (DELETE-FILE FILE)))
CORRECT: NIL
CLISP  : ERROR
OS-ERROR(-2147483643): Invalid Argument
OUT:
"[OS-ERROR]: OS-ERROR(-2147483643): Invalid Argument
"

Form: (CLEAR-OUTPUT *QUERY-IO*)
CORRECT: NIL
CLISP  : ERROR
OS-ERROR(-1): General system error
OUT:
"[OS-ERROR]: OS-ERROR(-1): General system error
"
1 Like

@alpopa thanks on the nice roadmap you mentioned, up and running now :slight_smile:
Creating a recipe shouldn’t be too hard, I’ll let you tackle it and if you need help just give a yell :smiley:

~> clisp
  i i i i i i i       ooooo    o        ooooooo   ooooo   ooooo
  I I I I I I I      8     8   8           8     8     o  8    8
  I  \ `+' /  I      8         8           8     8        8    8
   \  `-+-'  /       8         8           8      ooooo   8oooo
    `-__|__-'        8         8           8           8  8
        |            8     o   8           8     o     8  8
  ------+------       ooooo    8oooooo  ooo8ooo   ooooo   8

Welcome to GNU CLISP 2.49.93+ (2024-07-04) <http://clisp.org/>

Copyright (c) Bruno Haible, Michael Stoll 1992-1993
Copyright (c) Bruno Haible, Marcus Daniels 1994-1997
Copyright (c) Bruno Haible, Pierpaolo Bernardi, Sam Steingold 1998
Copyright (c) Bruno Haible, Sam Steingold 1999-2000
Copyright (c) Sam Steingold, Bruno Haible 2001-2018

Type :h and hit Enter for context help.
1 Like

Yes, the failed tests write a .erg file with details of that went wrong. Because the error is of type OS-ERROR, it is related to some missing / wrong linkage. Namely, the value -2147483643 seems to be some sort of error code instead of real Lisp object.

You can check the error code in Terminal (I’m no good on what it refers to, but still) :slight_smile:

~> error -2147483643
0x80000005: Invalid Argument

Yes, something like this. Lisp test expected some socket / stream / whatever to communicate with, but received an integer. This means, that Lisp object was not really created, instead an error was signaled.

Because the error type is OS-ERROR, I understand Lisp VM tries to use some OS library to create that object, which failed. Maybe the creation of object is good, but is not (correctly) passed between Lisp VM and native OS code. For such situations FFI (Foreign Function Interface) is used, and CLISP has libffcall for that.

I believe such investigation is best done by CLISP developers. I searched the contact information and for now contacted clisp-devel mailing list. Waiting for answer…

1 Like

This question is very similar to this one, it’s another language, but it’s related to C: bit rotation - Post.Byes

Maybe… I only see similar value about equals to minus infinity. Usually such values are generated by integer overflow of some register value. But they are small positive integers, which are simply codes for different errors.

Maybe some info here? (looks like it’s not the first time it’s popped up)

Yes, I understand that.

This is clear for me. But “Invalid Argument” or “Bad Value” is too generic. Is this numeric value not matching expected numeric value? Is it a number instead of object? Or an object of not expected type? Or an object of unknown type? Is it null redeference? Is it an uninitialized object? Is it something different? All these use-cases can be treated as “Invalid Argument” / “Bad Value”, but this is a symptom rather than the cause of the problem.

I let CLISP developers to analyze is possible.

1 Like