wasi: add cat/sort/tail/touch integration coverage and related fixes by DePasqualeOrg · Pull Request #11712 · uutils/coreutils

DePasqualeOrg · 2026-04-08T10:57:39Z

This draft tracks the combined WASI work that is being divided into smaller PRs. It is not intended to merge in its current form.

Planned sequence

A cat PR will enable its WASI integration coverage and document the small set of unsupported test conditions.
A touch PR will add WASI timestamp support and focused integration coverage.
A cp PR will add WASI symlink and timestamp behavior with focused tests.
A tail PR will enable its WASI coverage and document follow-mode and platform limitations.
The sort work will be divided into independently buildable review stages before submission because the current combined diff remains too large.

oech3 · 2026-04-08T11:01:30Z

Would you split PR (at least for symlink support)?

DePasqualeOrg · 2026-04-08T11:17:19Z

I separated the symlink changes out into #11713.

DePasqualeOrg · 2026-04-08T12:44:28Z

I added a new commit to address an issue that would cause CI failures.

The WASI CI uses stable Rust, but std::os::wasi::fs requires the unstable feature(wasi_ext) gate. This commit replaces all unstable APIs with stable libc equivalents:

uucore/fs.rs: FileInformation now stores libc::stat instead of std::fs::Metadata on WASI, using libc::fstat/libc::stat/libc::lstat for dev/ino/nlink access
cp.rs: std::os::wasi::fs::symlink_path replaced with libc::symlink
uucore/lib.rs: #![cfg_attr(..., feature(wasi_ext))] removed entirely
is_enotsup_error() updated to use libc::EOPNOTSUPP on WASI instead of a hardcoded value

DePasqualeOrg · 2026-04-08T12:56:38Z

I resolved the linter error in CI.

github-actions · 2026-04-08T12:57:09Z

GNU testsuite comparison:

GNU test failed: tests/tail/tail-n0f. tests/tail/tail-n0f is passing on 'main'. Maybe you have to rebase?
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/expand/bounded-memory is now passing!
Congrats! The gnu test tests/printf/printf-surprise is now passing!

github-actions · 2026-04-08T13:20:40Z

GNU testsuite comparison:

Skip an intermittent issue tests/tail/symlink (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/cp/link-heap is now being skipped but was previously passing.
Note: The gnu test tests/pr/bounded-memory is now being skipped but was previously passing.
Congrats! The gnu test tests/expand/bounded-memory is now passing!
Congrats! The gnu test tests/printf/printf-surprise is now passing!

github-actions · 2026-04-08T13:39:19Z

GNU testsuite comparison:

Skip an intermittent issue tests/cut/bounded-memory (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/expand/bounded-memory is now passing!
Congrats! The gnu test tests/printf/printf-surprise is now passing!

oech3 · 2026-04-08T15:24:15Z

Is it able to split PR per utility? Diff is still too big. How about sort?

DePasqualeOrg · 2026-04-08T19:49:41Z

The changes here are interdependent, and splitting out sort would result in PRs of ~560 and ~110 lines each, which would need to be sequenced properly and rebased on upstream changes. They should also probably wait for #11717, in which I've enabled integration tests. If/when that PR is merged, I can enable more tests for these tools in this PR. Keeping them in one PR would reduce the cognitive load for me, since I'm now keeping track of three related PRs.

DePasqualeOrg · 2026-04-12T23:43:28Z

I've added WASI integration coverage for four more tools, plus the fixes needed to make those tests pass.

New integration tests

Added test_cat, test_sort, test_tail, test_touch to wasi.yml. Every skip carries a specific reason; those reasons are categorized in docs/src/wasi-test-gaps.md.

Fixes

touch: filetime's WASI backend is stubbed out (FileTime::from_last_{access,modification}_time panics; set_file_times returns "Wasm not implemented"). Swapped for rustix::fs::utimensat and Metadata::{accessed,modified} → FileTime::from_system_time. 31 tests fixed.

sort: /tmp isn't a WASI preopen by default, so the hardcoded fallback broke external sort. Added uucore::fs::wasi_default_tmp_dir (gated to cfg(target_os = "wasi")) which returns /tmp if visible, else the current directory. --tmp-dir and TMPDIR still override. 4 tests fixed.

cp: replaced CString + unsafe libc::symlink with rustix::fs::symlink (matches the ln pattern). Implemented --preserve=timestamps on WASI via rustix::fs::utimensat (mirroring touch) and reverted the Preserve::Yes { required: false } downgrade so cp -a no longer silently drops timestamps.

uucore: FileInformation's WASI arm hand-rolled fstat/stat/lstat via libc + CString. Collapsed into the unix arm – rustix::fs::Stat is a typedef to libc::stat on WASI, and rustix::fs::{stat, lstat, fstat} work there without any cfg gate. Removed ~35 lines of unsafe code.

cat: stub stays – when stdout is inherited from a host file descriptor, wasmtime reports its fstat as all-zero so dev/inode comparison can never match. Comment updated to reflect that accurately.

Deferred

test_cp is not added here. ~41 of its failures still cluster around WASI symlink handling that needs follow-up: under wasmtime, symlinks with absolute targets (for example bar -> /foo) fail in ways that do not match POSIX, while equivalent relative symlinks work. The same limitation also affects readlink, realpath, and other symlink-heavy paths. Separately, some remaining failures are test-harness expectation issues because the WASI guest sees the per-test tempdir as /, not as the host absolute temp path. I documented both gaps in docs/src/wasi-test-gaps.md for follow-up.

Verification

Integration tests pass on macOS host and on Linux in an Ubuntu 24.04 Docker container (1592 passed / 0 failed / 176 ignored).

github-actions · 2026-04-12T23:54:59Z

GNU testsuite comparison:

Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/dd/no-allocate is now passing!
Congrats! The gnu test tests/printf/printf-surprise is now passing!

github-actions · 2026-04-13T00:01:20Z

GNU testsuite comparison:

Skip an intermittent issue tests/cut/bounded-memory (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/tail-n0f (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tty/tty-eof (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/dd/no-allocate is now passing!
Congrats! The gnu test tests/printf/printf-surprise is now passing!

github-actions · 2026-04-13T01:24:26Z

GNU testsuite comparison:

Skip an intermittent issue tests/tty/tty-eof (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/dd/no-allocate is now passing!
Congrats! The gnu test tests/printf/printf-surprise is now passing!

github-actions · 2026-04-13T02:05:50Z

GNU testsuite comparison:

Skip an intermittent issue tests/cut/bounded-memory (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tty/tty-eof (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/date/date-locale-hour (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/basenc/bounded-memory is now being skipped but was previously passing.
Congrats! The gnu test tests/dd/no-allocate is now passing!
Congrats! The gnu test tests/printf/printf-surprise is now passing!

sylvestre · 2026-04-20T20:17:18Z

-        } else {
-            filetime::set_file_times(dest, atime, mtime)?;
+        #[cfg(target_os = "wasi")]
+        {


please move that into a new function

sylvestre · 2026-04-20T20:18:35Z

i am sorry but it can't be merged this way, it needs to be split into several PRs

also, it adds way too many wasi_no_threads everywhere

github-actions · 2026-04-21T09:17:40Z

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/retry (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/tail-n0f (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/tail/symlink (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/misc/write-errors was skipped on 'main' but is now failing.

DePasqualeOrg · 2026-04-21T09:57:33Z

Before I split this up, I moved the sync/threaded variants into separate files, which reduces the number of cfg attributes in uu_sort significantly. If that's the direction you'd like to take, I'll start splitting this into separate PRs.

sylvestre · 2026-08-07T22:33:26Z

could you please keep your comments shorter on this PR ? it is hard to read otherwise.
thanks

sylvestre · 2026-08-07T22:33:26Z

could you please keep your comments shorter on this PR ? it is hard to read otherwise.
thanks

Add #[cfg(target_os = "wasi")] blocks alongside existing unix and windows platform code. No changes to existing platform behavior. Enables compilation to wasm32-wasip1 and wasm32-wasip1-threads targets for running in WASI-compatible runtimes like WasmKit and Wasmer.

On wasm32-wasip1 (no atomics), sort crashes because ext_sort, merge, check, and rayon all spawn threads unconditionally. Add synchronous code paths gated on cfg(all(target_os = "wasi", not(target_feature = "atomics"))) so sort works on both wasip1 (sync) and wasip1-threads (threaded). Key changes: - Extract read_to_chunk() from chunks::read() for shared use - Add synchronous ext_sort with chunked sort-write-merge flow - Add SyncFileMerger for threadless merge operations - Add synchronous check for order verification - Gate rayon par_sort with sequential fallback

eduardomourar · 2026-08-08T01:16:53Z

+# The gap this closes: integration tests are host-built and many are gated
+# with #[cfg(not(target_vendor = "apple"))] / #[cfg(target_os = "linux")],
+# so macOS silently excludes them.


This has been improved a bit with the merge of #13666, where we have enabled more tests that can run in Unix-based systems (including macOS).

eduardomourar · 2026-08-08T01:27:05Z

I believe I missed this PR because it was in draft. There are few changes here that are overlapping with PR #13625 I am currently working on. Also, the spin-off PRs I created might simplify the implementation for this PR.

eduardomourar · 2026-08-08T02:02:07Z

@sylvestre, let me know if you agree on the following proposal, please?

Already covered by my PRs (please drop from this PR):

cp changes → fix(wasi): cp attribute preservation and symlink handling #13685 (under review, mergeable)
touch changes → fix(wasi): touch timestamp handling #13686 (open, same approach)
uucore/fs.rs wasi_default_tmp_dir → fix(mktemp): add wasi platform support for temporary directory handling #13786 (mktemp WASI support)

Unique value in this PR that I'd love to see land:

The entire sort refactoring (parallel.rs, check/ module split, ext_sort/sync.rs, merge/sync.rs) is architecturally cleaner and more complete than my simpler fix(wasi): sort ordering check and external-merge separator #13688. I'm happy to close mine in favor of your sort work.
The build.rs with wasi_no_threads cfg alias is a great ergonomic win.
The wasi-test-gaps.md documentation additions are valuable.
The Docker test script is a nice developer experience addition.

Suggested path forward:

Wait for PR 13685 (cp) and PR 13686 (touch) to merge
Rebase this PR dropping cp/touch/uucore-fs changes
Focus this PR on the sort refactoring + test annotations + docs
I'll close my PR 13688 once your sort changes are ready
The tail stubs and cat comment improvement are fine to keep — no conflict.

DePasqualeOrg · 2026-08-08T07:44:48Z

could you please keep your comments shorter on this PR ? it is hard to read otherwise. thanks

@sylvestre, I don't know why you keep posting that. We already discussed this. Just ignore this draft PR.

As I stated multiple times before, this is in draft because it's not meant for review. It's my combined Wasm fixes from which smaller PRs can be split out.

@eduardomourar, I had Codex review your PRs, comparing them with the changes I already have here, and it found multiple issues. For that reason, I'll submit my own PRs based on this work that I submitted already four months ago, and @sylvestre can decide which he wants to work with.

eduardomourar · 2026-08-08T09:46:16Z

I don't think I follow your reasoning. You have this PR in draft (that part I understand). You will be creating multiple PRs out of this one. Then, I say that I have already started doing that for you (without me realizing it, of course). If you have found any issues, I am happy to make the necessary changes to accommodate your approach. It is very similar so they should be small tweaks that can done without any real impact.

sylvestre · 2026-08-08T09:54:25Z

@sylvestre, I don't know why you keep posting that. We already discussed this. Just ignore this draft PR.

because i am tired of seeing 3 pages comments written by LLM

DePasqualeOrg · 2026-08-08T10:28:10Z

@sylvestre, there are no 3-page comments here.

Just ignore this draft PR and any other draft PRs, which represent work in progress, until they're marked as ready for review, at which point comments will be concise as per your preference.

DePasqualeOrg force-pushed the wasi-support branch 2 times, most recently from 9bced35 to aa75a81 Compare April 8, 2026 11:08

DePasqualeOrg changed the title ~~wasi: atomics-aware threading, synchronous sort pipeline, and symlink support~~ wasi: atomics-aware threading and synchronous sort pipeline Apr 8, 2026

DePasqualeOrg force-pushed the wasi-support branch from 6b396f5 to 562cefd Compare April 8, 2026 13:31

DePasqualeOrg force-pushed the wasi-support branch from 562cefd to e6f63b1 Compare April 12, 2026 23:42

DePasqualeOrg changed the title ~~wasi: atomics-aware threading and synchronous sort pipeline~~ wasi: add cat/sort/tail/touch integration coverage and related fixes Apr 12, 2026

DePasqualeOrg force-pushed the wasi-support branch from 9645693 to 7c34168 Compare April 12, 2026 23:54

sylvestre reviewed Apr 20, 2026

View reviewed changes

DePasqualeOrg force-pushed the wasi-support branch from de79c07 to eaa010b Compare April 21, 2026 09:09

DePasqualeOrg force-pushed the wasi-support branch 2 times, most recently from 5aa26b0 to f96a56c Compare April 21, 2026 09:41

DePasqualeOrg force-pushed the wasi-support branch from f96a56c to 260501c Compare May 9, 2026 14:26

DePasqualeOrg force-pushed the wasi-support branch from e7a99ab to f2a1656 Compare May 28, 2026 10:06

DePasqualeOrg force-pushed the wasi-support branch 3 times, most recently from b887346 to ae780ef Compare August 4, 2026 13:06

DePasqualeOrg added 16 commits August 8, 2026 00:37

sort: add wasi_no_threads cfg alias to reduce predicate verbosity

98cd261

Replace unstable wasi_ext with stable libc calls

990ceff

Fix clippy warnings for wasm32-wasip1 target

43dd815

Add integration tests for cat/sort/tail/touch + underlying fixes

bc62e5b

Annotate more WASI-incompatible tests caught on Linux

8a51183

Clarify WASI symlink and guest-path test gaps

9544216

Fix cspell errors in CI

5dfb20b

tail: ignore test_gnu_args_f under wasi_runner

cc9539f

tail: ignore test_follow_inotify_only_regular under wasi_runner

2dfc7ca

sort: isolate WASI sync code into sibling modules

64b3e83

util: add run-wasi-tests-docker.sh for local Linux verification

89730f3

cp: extract WASI timestamp logic into set_timestamps function

5f9a830

tests/sort: ignore test_consistent_sorting_with_i18n_collate on WASI

102641e

tests/touch: ignore upstream symlink tests on WASI

7c4fd27

DePasqualeOrg force-pushed the wasi-support branch from ae780ef to 7c4fd27 Compare August 7, 2026 23:03

eduardomourar reviewed Aug 8, 2026

View reviewed changes

util: improve Docker WASI integration helper

7f0b7cf

Uh oh!

Uh oh!

Conversation

DePasqualeOrg commented Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Planned sequence

Uh oh!

oech3 commented Apr 8, 2026

Uh oh!

DePasqualeOrg commented Apr 8, 2026

Uh oh!

DePasqualeOrg commented Apr 8, 2026

Uh oh!

DePasqualeOrg commented Apr 8, 2026

Uh oh!

github-actions Bot commented Apr 8, 2026

Uh oh!

github-actions Bot commented Apr 8, 2026

Uh oh!

github-actions Bot commented Apr 8, 2026

Uh oh!

oech3 commented Apr 8, 2026

Uh oh!

DePasqualeOrg commented Apr 8, 2026

Uh oh!

DePasqualeOrg commented Apr 12, 2026

New integration tests

Fixes

Deferred

Verification

Uh oh!

github-actions Bot commented Apr 12, 2026

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

github-actions Bot commented Apr 13, 2026

Uh oh!

sylvestre Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

DePasqualeOrg Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

sylvestre commented Apr 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DePasqualeOrg commented Apr 21, 2026

Uh oh!

sylvestre commented Aug 7, 2026

Uh oh!

sylvestre commented Aug 7, 2026

Uh oh!

eduardomourar Aug 8, 2026

Choose a reason for hiding this comment

Uh oh!

eduardomourar commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eduardomourar commented Aug 8, 2026

Already covered by my PRs (please drop from this PR):

Unique value in this PR that I'd love to see land:

Suggested path forward:

Uh oh!

DePasqualeOrg commented Aug 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eduardomourar commented Aug 8, 2026

Uh oh!

sylvestre commented Aug 8, 2026

Uh oh!

DePasqualeOrg commented Aug 8, 2026

Uh oh!

Reviewers

Assignees

Labels

DePasqualeOrg commented Apr 8, 2026 •

edited

Loading

sylvestre commented Apr 20, 2026 •

edited

Loading

github-actions Bot commented Apr 21, 2026 •

edited

Loading

eduardomourar commented Aug 8, 2026 •

edited

Loading

DePasqualeOrg commented Aug 8, 2026 •

edited

Loading