Skip to content

Cross#1728

Draft
Bike wants to merge 252 commits intomainfrom
cross
Draft

Cross#1728
Bike wants to merge 252 commits intomainfrom
cross

Conversation

@Bike
Copy link
Member

@Bike Bike commented Feb 4, 2026

Overhauls Clasp's build procedure. Instead of bootstrapping itself, another Lisp like SBCL loads a bytecode compiler (Maclina) and a specialized first class environment, and uses these to compile the Clasp library code into a bytecode FASL. This FASL can then be loaded by the Clasp C++ runtime which is mostly unchanged.

A "native" build in which Clasp itself is compiled by clasp-cleavir can be done by taking this bytecode FASL, loading that same specialized environment, and then using it to natively compile the Clasp library code. This produces a bytecode FASL with native versions of most functions through the b2b compiler; the native code is kept as attributes in the FASL.

This is mostly done but still needs a bit more work. In particular a few little things are probably still broken.

Other changes include:

  • FASOs are mostly dead (native FASLs can be used instead)
  • ext:deoptimize can be used to temporarily replace native-compiled functions with more debuggable bytecode versions
  • some first class global environment support (needs organization & cleanup)

Bike added 30 commits July 2, 2025 18:23
This helps with things like :long-float that are determined by
build parameters, but more importantly things like
:long-float/binary80 that are controlled by the C++ runtime.

Thanks yitzi for help with the script.
there are far too many of them, god damn
This should let us clear out way more of the explicit definition
of CORE, but I don't want to bother figuring out what's actually
needed, at the moment
and add the CLEAVIR-ENV nickname. The common-macros defpackage
doesn't add nicknames if the package already exists, so the first
definition of the package we give it needs to have all the nicks.
it's used in clasp-cleavir for type derivers
why did i think this had to be done at compile time? i dunno
anyway this makes cross-clasp easier
it's useless, but so it goes
makes things easier for cross-clasp
dunno why we would need it in init
It might be smarter to just not use CL, but we do actually have
some code that's read in the LLVM-SYS package, like defaults in
lambda lists defined in C++.
for debugging. Probably should hook these in to clos:g-f-methods
eventually
probably would be even better to make %make-package use and unuse
appropriately
not hooked up yet but does at least load. waow
this lets infinities be printed in primitive clasp.
No idea how this works in existing clasp - I guess it must load
the code treating with-irbuilder as a function and so on, and then
that gets overriden in the compiled code? That's pretty stupid
definitely beats keeping track of internal functions
Bike added 29 commits December 9, 2025 19:42
So that READ when compiling in an alternate environment works.
Give up if something goes wrong because who cares. Also continue
better if something's missing from the fmap. no big
With the new compile-file-pathname we were using .faso but putting
a bytecode fasl in it. oops
since it supports first class environments now, mostly. Optimization
is still technically off (it uses main env identities and inline
definitions in Cleavir, I think?) but that doesn't matter since
the FCGE definitions have the same properties as the main env's.

I can actually do a native self-build now. incredible shit. go me
I think this remove all use of #+bytecode #-bytecode, which is good
since relying on build parameters to tell us what compiler is being
used is not compatible with a b2b world.
though it doesn't seem to have a ton of practical effect, so I
should probably bark up a different tree. Oh well. I think doing
less work is nice regardless.
native compilation still takes a noticeable chunk of time, so do
it in threads if we can swing it.
avoid having MESSAGE quit. that's a separate concern
This caused numerous errors in native compilation, in particular
ruining the CLOS code walker.
constant-form-value needs fixing up for both cmp:lexenv and
cleavir environments
This way it matches *compile-file-parallel* and we don't need to
worry about values other than :parallel and :serial. We could
restore :execution in the future if we ever have other methods.
problems happened with e.g. (min (floor ...)) because lo and behold
FLOOR returns two values, conflicting with the &rest nil in the
type declaration. I think putting in VALUES is ok, though it's
slightly suboptimal in bytecode
Preliminary merge before merging cross into main, hopefully.
These were replaced with C++ code when yitzi added long-float, but
that change somehow didn't make it fully into the cross branch.

There's no point in keeping the code around commented out so
deletion it is.
no longer required as we are moving to native code in FASLs
just :native t and a default. keeping it simple
if the modules are out of order, the loader messes up finding the
literals. no good at all
instead of cross-clasp. this silences some warnings during build.
Still not an exported interface, though.
catch will be more of a journey

This could probably be used for the bytecode-skipping compiler,
presuming we even want to retain it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant