unknown handle index 10

[kesmit13]

I'm using componentize-py on some new Python packages that still attempt to import parts of the standard library that don't exist (_ssl in this case). I'm running into a traceback that isn't terribly helpful though. It appears as though the file indexes created don't match the ones at runtime so instead of seeing the Python error get printed, it just crashes. I'm sure it's something I'm doing incorrectly when creating the components, but I'm not sure what that is. Can anyone help with this?

    0: 0x49a8ba7 - <unknown>!<wasm function 99>
    1: 0x479e985 - libc.so!streams_method_output_stream_check_write
    2: 0x4799a59 - libc.so!write
    3:  0x129f3 - <unknown>!<wasm function 367>
    4: 0x304ff49 - libpython3.14.so!_Py_write
    5: 0x314272e - libpython3.14.so!_io_FileIO_write
    6: 0x2e1d9cf - libpython3.14.so!method_vectorcall_FASTCALL_KEYWORDS_METHOD
    7: 0x2e0ed31 - libpython3.14.so!PyObject_VectorcallMethod
    8: 0x3150b20 - libpython3.14.so!_textiowrapper_writeflush
    9: 0x3152233 - libpython3.14.so!_io_TextIOWrapper_write
   10: 0x2e1d7da - libpython3.14.so!method_vectorcall_O
   11: 0x2e0db7d - libpython3.14.so!PyObject_Vectorcall
   12: 0x2f9a7a8 - libpython3.14.so!_PyEval_EvalFrameDefault
   13: 0x2f9b463 - libpython3.14.so!_PyEval_Vector
   14: 0x2e0df1d - libpython3.14.so!_PyFunction_Vectorcall
   15: 0x2e106ae - libpython3.14.so!method_vectorcall
   16: 0x2e0d9fb - libpython3.14.so!_PyVectorcall_Call
   17: 0x2e0dca3 - libpython3.14.so!_PyObject_Call
   18: 0x2e0dd48 - libpython3.14.so!PyObject_Call
   19: 0x2f8d367 - libpython3.14.so!_PyEval_EvalFrameDefault
   20: 0x2f9b463 - libpython3.14.so!_PyEval_Vector
   21: 0x2e0df1d - libpython3.14.so!_PyFunction_Vectorcall
   22: 0x2e105f2 - libpython3.14.so!method_vectorcall
   23: 0x2e0d9fb - libpython3.14.so!_PyVectorcall_Call
   24: 0x2e0dca3 - libpython3.14.so!_PyObject_Call
   25: 0x2e0dd48 - libpython3.14.so!PyObject_Call
   26:  0x122bd - <unknown>!<wasm function 233>
   27: 0x2c22286 - libcomponentize_py_runtime_sync.so!<pyo3::instance::Py<pyo3::types::any::PyAny>>::call_method1::<&pyo3::instance::Py<pyo3::types::string::PyString>, pyo3::instance::Bound<pyo3::types::tuple::PyTuple>>
   28: 0x2c1b7cf - libcomponentize_py_runtime_sync.so!wit_dylib_export_call
   29: 0x4997a13 - <unknown>!singlestore:udf/function-handler#initialize
note: using the `WASMTIME_BACKTRACE_DETAILS=1` environment variable may show more debugging information: unknown handle index 10

[kesmit13]

I attempted to investigate using Claude Code. This is the answer it came up with. This is far deeper into WASM than I'm familiar with, so I can't say with any certainty that it is correct.

---

libc.so directly imports wasi:io/streams@0.2.0 methods including [method]output-stream.check-write. This is not going through the p1 adapter — the libc in this componentize-py build uses native WASI p2 streams directly.

So libc.so's write() function calls [method]output-stream.check-write with a resource handle (index 10) that it got from somewhere. Since the main module's memory is snapshotted, libc has the handle value 10 baked into its data from pre-initialization. But when we instantiate, the VM's HandleTable is empty — handle 10 doesn't exist.

This is the fundamental problem: the component's p2 libc stores resource handle indices in its snapshotted memory, but those handles only existed in the VM during componentize-py's pre-init run.

The issue is in the component itself — componentize-py's snapshot captures stale p2 resource handles in libc.so's memory. This is a componentize-py bug/limitation: it doesn't account for the fact that WASI p2 resource handles won't survive re-instantiation.

To summarize my findings:

1. Our WASI setup is correct — WasiCtxBuilder, add_to_linker_sync,
   SinkOutputStream/MemoryOutputPipe for stderr — this is all standard.
2. We're not missing any parameter. The problem is not in our server.
3. The root cause: componentize-py 0.21.0 builds components where libc.so uses native WASI p2 stream operations (not the p1 fd_write adapter). During pre-initialization, libc.so calls get-stderr/get-stdout/get-stdin, gets p2 resource handle indices (like 10), and stores them in memory. The snapshot captures that memory. On fresh instantiation, those handle indices reference non-existent entries in the VM's HandleTable.
4. This is a componentize-py issue, not a wasmtime-wasi configuration issue on our side. The p1 adapter path would re-initialize correctly (its globals reset to 0), but libc.so bypasses the adapter entirely with direct p2 stream calls.

[dicej]

Thanks for reporting this, @kesmit13.

My guess is that this is related to Python code running during pre-init which is opening a file and then expecting it to still be open at runtime. The way componentize-py works is that runs the top level of the Python code you give it as a pre-init step, then it takes a snapshot of the state of the program at that point and packages it into the output component. That means the top level code can import other modules, read from and write to the parts of the host filesystem it has access to, etc. However, if it (or a module that it imports) opens a file during pre-init and then tried to read from or write to it at runtime (e.g. after restoring the snapshot), the file won't be there anymore and the host will not know what that file handle refers to.

One possible workaround would be to delay running any code which opens files (and expects to keep them open) until runtime. If that's happening in a library you're importing, you might need to delay the import. However, that also means you'll need to make sure the imported module is available on the filesystem at runtime, not just at build time, and that the path is in PYTHONPATH.

[dicej]

Related: #23

[kesmit13]

What's strange is that in this case, it's trying to write stderr. As far as I can tell, this is caused by Python trying to write an error to stderr because the import couldn't be found. I could be wrong about that because I'm having trouble tracing down what's actually happening in Python at the time of this crash.

[dicej]

If you can provide a way to reproduce it, I'll be happy to try debugging it. Writing to stderr should work fine, both during and after pre-init, per

componentize-py/runtime/src/lib.rs

Lines 1183 to 1199 in 89af297

	// This tells the WASI Preview 1 component adapter to reset its state.
	// In particular, we want it to forget about any open handles and
	// re-request the stdio handles at runtime since we'll be running under
	// a brand new host.
	#[link(wasm_import_module = "wasi_snapshot_preview1")]
	unsafe extern "C" {
	#[link_name = "reset_adapter_state"]
	fn reset_adapter_state();
	}
	// This tells wasi-libc to reset its preopen state, forcing
	// re-initialization at runtime.
	#[link(wasm_import_module = "env")]
	unsafe extern "C" {
	#[link_name = "__wasilibc_reset_preopens"]
	fn wasilibc_reset_preopens();
	}

[kesmit13]

You can access the WASM file here: https://tinyurl.com/3rxyhxfd . It has 4 exports, but fails in the initialize export.

[dicej]

Thanks, but in order to debug it I was planning to build the component from source with CPython built in debug mode, and possibly add logging to the CPython internals if necessary. Do you have a way to repro from .py file(s)?

[kesmit13]

I don't have anything in a public repo at the moment. Let me see if I can reproduce in a smaller test case...

[kesmit13]

I created a branch of the singlestoredb-python repo that contains what you need to reproduce: https://github.com/singlestore-labs/singlestoredb-python/tree/wasm-compat. If you save the Python code below as simple_funcs.py, and clone and pip install the singlestoredb branch to a virtual env, you should get the WASM file that I was having trouble with:

build.sh

componentize-py" \
    -d "$BRANCH_ROOT/wit" \
    -w external-udf \
    componentize \
    -p "$VENV_SITE_PACKAGES" \
    -o "py_handler_simple.wasm" \
    simple_funcs

simple_funcs.py

from singlestoredb.functions import udf
from singlestoredb.functions.ext.wasm.udf_handler import FunctionHandler  # noqa: F401

@udf
def greet(name: str) -> str:
    return f'Hello, {name}!'

[kesmit13]

Here's a little script that does everything described above (except for saving the Python code as simple_funcs.py.

git clone -b wasm-compat --single-branch --depth 1 git@github.com:singlestore-labs/singlestoredb-python.git
uv venv && uv pip install ./singlestoredb-python
componentize-py \
    -d singlestoredb-python/wit \
    -w external-udf \
    componentize \
    -p . \
    -p .venv/lib/python3.13/site-packages \
    -o py_handler_simple.wasm \
    simple_funcs

[dicej]

Thanks, I'll take a look later today.

[dicej]

Ok, it ended up being pretty simple to fix, but will require a patch to wasi-libc.

The code I mentioned in #207 (comment) is supposed to reset wasi-libcs stdio state, but the wasilibc_reset_preopens function only affects WASIp1 state, not p2 (or p3) state.

With earlier versions of wasi-libc, only the p1 state mattered, since p2 support was handled by the wasi_snapshot_preview1 adapter. However, more recent wasi-libc versions have native p2 (and some p3 support), meaning the adapter isn't used for stdio or file I/O anymore.

I'll post a patch to wasi-libc shortly and link to it here.

[kesmit13]

Cool. Thanks!

[dicej]

Here's the fix: WebAssembly/wasi-libc#776

[kesmit13]

I'm guessing this is going to take a while to cycle into the standard componentize-py tooling...