could we pre-compile user_words.fs as part of the build process?

[patricksurry]

As Sam described here, currently user-defined forth words are embedded as compact ascii text and compiled at startup:

... forth_words.fs and user_words.fs ... contain Forth code that is run at startup - the code is stored in the binary as strings and then EVALUATE is run on those strings at startup. ... There is a python program to strip the comments and replace all whitespace with a single space between words (to shrink the strings down as much as possible) ...

This is fine as long as your forth is not too big, since otherwise you waste the space of the ascii representation that's only used once.

I'm curious if this compilation could be done as part of the build process, i.e. compile a preliminary binary without the user words, and use that binary in a simulator to compile the user words in a way that could be appended to the native words.

It would presumably require temporarily moving the 'here' pointer to wherever you want the native words to end up in the final image, or that the generated code is all relocatable? Or making the compiler generate a relocation table 😅

I suppose it also gets fiddly if you're trying to target a native platform where addresses might be different than how they look in the simulator?

[SamCoVT]

It's an interesting thought. I agree that the biggest technical issue is going to be when the memory map is fiddled with. I know of one user who has Tali in low memory and the dictionary in high memory, for example. Because the words can have a mix of opcodes and data, there also isn't a good way to relocate the code once it's been compiled.

By the time a user is bringing in more than a few K of Forth source, they probably should be adding mass storage. Tali is reasonably fast at loading screens from block storage with fast block I/O routines.

[SamCoVT]

As a side thought along this same tangent, what about having a live Tali be able to create a "bootable" dump of the existing memory, including fixing up the COLD tables for ZP and UP variables with current values. You'd need to save the RAM where words were compiled as well. The more I think about it, the more complicated it gets...

[patricksurry]

Yes! I do exactly for my 'colossal cave' port which is built on a slightly non-standard Taliforth kernel. I'm slowly trying to unify it with the main branch but it is super tight for space. Your platform reorg should make things a little easier.

It does a bunch of juggling to read in 27K of compacted forth at startup from a block device, compile it over top of itself and then load in a prebuilt data blob before dumping a bootable memory image.

See https://github.com/patricksurry/advent-forth