Javy QuickJS and host function

[kakawait]

I'm pretty sure isn't a bug! I may misunderstand a concept, but I'm trying to use javy + quickjs with host function.

I've snippetted something like (which is totally a copy past of a doc snippet + UT code)

public static void main(String[] args) {
    var func = new HostFunction(
            "console2",
            "log2",
            List.of(ValueType.I32, ValueType.I32),
            List.of(),
            (Instance instance, long... args2) -> {
                var len = (int) args2[0];
                var offset = (int) args2[1];
                var message = instance.memory().readString(offset, len);
                System.out.println(message);
                return null;
            });


    ByteArrayInputStream stdin = new ByteArrayInputStream("".getBytes(UTF_8));
    var stdout = new ByteArrayOutputStream();
    var stderr = new ByteArrayOutputStream();

    var wasiOpts =
            WasiOptions.builder()
                    .withStdout(stdout)
                    .withStderr(stderr)
                    .withStdin(stdin)
                    .build();

    var wasi = WasiPreview1.builder().withOptions(wasiOpts).build();
    var quickjs =
            Instance.builder(loadModule("quickjs-provider.javy-dynamic.wasm"))
                    .withImportValues(
                            ImportValues.builder().addFunction(func).addFunction(wasi.toHostFunctions()).build())
                    .build();

    var greetingMsg = "Hello QuickJS!";

    byte[] jsCode = ("import { log2 } from 'console2'; console2.log2('test');").getBytes(UTF_8);
    var ptr =
            quickjs.export("canonical_abi_realloc")
                    .apply(
                            0, // original_ptr
                            0, // original_size
                            1, // alignment
                            jsCode.length // new size
                    )[0];

    quickjs.memory().write((int) ptr, jsCode);
    var aggregatedCodePtr = quickjs.export("compile_src").apply(ptr, jsCode.length)[0];

    var codePtr = quickjs.memory().readI32((int) aggregatedCodePtr); // 32 bit
    var codeLength = quickjs.memory().readU32((int) aggregatedCodePtr + 4);

    quickjs.export("eval_bytecode").apply(codePtr, codeLength);

    // stderr?
    System.out.println(stderr.toString(UTF_8));
}

But when running I've following error

Exception in thread "main" com.dylibso.chicory.runtime.TrapException: Trapped on unreachable instruction

Thus I'd like to know, if it's even possible to mix dynamic javy with quickjs and host function?
And yes, have you some tips for me?

[evacchi]

which version of Javy are you using? check this test case

https://github.com/dylibso/chicory/blob/main/wasi-tests/src/test/java/com/dylibso/chicory/wasi/WasiPreview1Test.java#L92-L104

IIRC we are currently testing against javy v1.4.0

chicory/wasm-corpus/Dockerfile

Line 35 in 54cbf87

RUN curl -L -O https://github.com/bytecodealliance/javy/releases/download/v1.4.0/javy-x86_64-linux-v1.4.0.gz && \

[kakawait]

which version of Javy are you using? check this test case

https://github.com/dylibso/chicory/blob/main/wasi-tests/src/test/java/com/dylibso/chicory/wasi/WasiPreview1Test.java#L92-L104

IIRC we are currently testing against javy v1.4.0

chicory/wasm-corpus/Dockerfile

Line 35 in 54cbf87
RUN curl -L -O https://github.com/bytecodealliance/javy/releases/download/v1.4.0/javy-x86_64-linux-v1.4.0.gz && \

I'm re-using the pre-compiled https://github.com/dylibso/chicory/blob/main/wasm-corpus/src/main/resources/compiled/quickjs-provider.javy-dynamic.wasm

So I don't know which version of javy it was

Are you recommending me to recompile it with 1.4.0?

[andreaTP]

Hi @kakawait and thanks for your interest in the project!

it's even possible to mix dynamic javy with quickjs and host function?

I'm not aware of a way to make imported functions available to the host script in Javy, but I haven't looked much into this detail, this issue seems relevant to your question.

As far as I remember @bhelx always said that only the "command pattern" was supported, but this statement might not be up-to-date anymore.

Trapped on unreachable instruction

I know there is space for improvement here, this is what usually happen when the module crashes (e.g. throws an exception).

have you some tips for me?

I think that the best starting point would be this closed issue.

Given the current context, I believe this issue should be opened in the Javy repo, at the same time, I'm extremely curious about the result, so, please keep me in cc. if you make any progress on this aspect. I'll be happy to assist when possible.

[bhelx]

@kakawait let me see if i can understand the issue. It looks like javy now requires you to interact with the canonical abi, which i know about, but don't have a lot of experience with. What makes you think import { log2 } from 'console2'; is mapped to the wasm import interface?. I don't recall them supporting host functions. We shared a codebase a while ago and we had to implement our own, but that was a couple years ago. I can't seem to find in the code where they support this FFI like interface.

In the meantime, i wanted to point out that if you want to use quickjs and chicory, you could look into Extism:

• https://github.com/extism/chicory-sdk
• https://github.com/extism/js-pdk

Our JS-PDK was at one point the same codebase as Javy. we support host functions and have a much simpler ABI that you generally don't need to think about. I'm sort-of biased here though, so I understand if you disagree and prefer to put things together yourself.

[kakawait]

Firstly, I really appreciate your reactivity and help! Thank you.
And keep in mind I'm really new on anything related to wasm/wasi or even javy! So don't be afraid if I could write / ask something stupid 😂

I'll check deeply every linked issue that @andreaTP shared with me above. In addition, I've also found this promising issue bytecodealliance/javy#878 that I've commented.

Regarding Extism, to be honest is on my radar, I've just started by chicory + javy more for learning path in able to understand, or at least try to understand, low level concept before starting Extism.
I don't yet time to look more deeply on Extism but what I've appreciate using directly javy (through Chicory) is a capacity call Javy api compile_src and thus be able to compile on fly js source. I've to check if something possible with Extism

[bhelx]

Regarding Extism, to be honest is on my radar, I've just started by chicory + javy more for learning path in able to understand, or at least try to understand, low level concept before starting Extism.

Learning is a great goal! And maybe you'll find some better ways yourself. I thought i'd bring it up in case you thought you had to do it that way. There are a few different paths you can take.

I don't yet time to look more deeply on Extism but what I've appreciate using directly javy (through Chicory) is a capacity call Javy api compile_src and thus be able to compile on fly js source. I've to check if something possible with Extism

Like javy, extism "wizens" the javascript code into js bytecode at "compile time". We don't offer the ability to execute qjs bytecode directly, though I could add that for you. But, you can dynamically execute javascript code using eval. Here is how you can do with extism: https://extism.org/blog/sandboxing-llm-generated-code . The wasm sandbox effectively makes eval safe.

note: extism should also support just storing the func ref in a global and as long as the module is alive, the code is compiled.

As for speed, you probably don't want to directly execute qjs bytecode most of the time (though there are a few reasons you might). eval is essentially doing the same thing as what your code is doing above. it compiles the raw js into bytecode then executes it in one step.

[kakawait]

@bhelx brillant I've never thought of using eval and your Extism blog post is a great resource for what I would like to achieve.

To be honest, I'm evaluating wasm/javy to have a safe (enough) sandbox env to execute external and arbitrary js code but with host function, thus the parallel with LLM result is really close to my final scenario, except in my use case is human that generate js code.

But, if I've capability to compile the incoming js code (because in UX there is a clear delay between when submitting code and when it will effectively executed), you're recommanded me to compile it rather than evaluating using eval?

Even if I think I'll use Extism at the end of the journey, "for science" I'll try to succeed (or not) doing it with plain old chicory + javy and if something valuable I'll share you my feedback (maybe blog post or doc PR)

PS: what do you mean by qjs exactly?
PS2: Why chicory? (If you asked yourself) because my host is JVM and trying to replace Nashorn/Rhino...

[bhelx]

To be honest, I'm evaluating wasm/javy to have a safe (enough) sandbox env to execute external and arbitrary js code but with host function, thus the parallel with LLM result is really close to my final scenario, except in my use case is human that generate js code.

yes, i think either using javy directly or extism would both be good options for this. both are safe. but your solution is only as good as your host functions and what you do with the output. you have to treat all calls and returns from the plugin as untrusted input.

But, if I've capability to compile the incoming js code (because in UX there is a clear delay between when submitting code and when it will effectively executed), you're recommanded me to compile it rather than evaluating using eval?

eval will both compile and execute. you could do those steps separately, but that wouldn't necessarily make it faster on it's own. Also, consider that your bottleneck is actually passing data back and forth across the wasm boundary. you want to minimize that. compute speed is not the primary concern unless you are doing some image processing or something. it is fast otherwise. Another thing to note, if you keep the module alive you don't necessarily need to "recompile" the js code everytime. but the security of that really depends on whether or not you want your wasm instance to be mutli-tenant, or just owned by 1 tenant. if you use the same instance for different users then memory may leak b/w the two. i generally recommend creating at minimum 1 instance per user.

Here is some more information about performance https://dylibso.com/blog/how-does-extism-work/ . Also specific to extism, but is generally applicable.

Even if I think I'll use Extism at the end of the journey, "for science" I'll try to succeed (or not) doing it with plain old chicory + javy and if something valuable I'll share you my feedback (maybe blog post or doc PR)

That would be awesome! would be happy to help any way i can.

PS: what do you mean by qjs exactly?
sorry i was shortening quickjs. it's a common shortening but not that obvious.

PS2: Why chicory? (If you asked yourself) because my host is JVM and trying to replace Nashorn/Rhino...

are you asking why we created chicory? i think this part of the readme covers it https://github.com/dylibso/chicory?tab=readme-ov-file#why

for a deeper dive i recommend this talk https://www.youtube.com/watch?v=00LYdZS0YlI

[andreaTP]

Thanks for the details @kakawait , I'm looking forward to hearing how this exploration progresses!

PS2: Why chicory? (If you asked yourself) because my host is JVM and trying to replace Nashorn/Rhino...

I did something similar here: java-operator-sdk/kubernetes-glue-operator#169
feel free to take inspiration if it works for you.

Fun fact: in a curious turn of events we even started exploring using Chicory as a WebAssembly runtime for Rhino: rhino/rhino-wasm#1 that's up for the grabs if anyone is interested!

[kakawait]

Whoo too much good reading and sample.

I need to digest everything now!

Thank you so much.

Regarding Rhino wasm seems to be promising but still an early project.

I've also seen that you've worked on making chicory jsr-223 compliant which is a bit close to rhino wasm project but different 🤣

My goal will be firstly to achieve with chicory and javy then same with Extism (I like the Extism' idea to be wasm runtime abstraction) then blogging it or something similar

@bhelx good feedback regarding performance, but I'll keep it simple because in my use case safety and sandboxing is way more important (top prio) than pure performance.

Should I close the issue for now? (At least for le I've enough information to start doing things)

[andreaTP]

Should I close the issue for now? (At least for le I've enough information to start doing things)

Sounds good, let's close this one, but feel free to keep us updated (here or on Zulip 😃 ).
Good luck with your exploration @kakawait !

[andreaTP]

@kakawait I'm about to get back to this, and I had the opportunity to speak with the Javy author at WASM I/O.

According to him, the starting point would be to build a custom plugin:
https://github.com/bytecodealliance/javy/blob/main/crates/test-plugin/src/lib.rs

[kakawait]

Great thank you. I've a bit delayed my PoC due to other emergency but still on my backlog

[andreaTP]

Last update on this thread, I'll keep iterating on it but this little example is working:

https://github.com/andreaTP/chicory-js-poc