Skip to content

fs/mnemofs: Add mnemofs version 1 support.#18884

Open
resyfer wants to merge 1 commit into
apache:masterfrom
resyfer:mnemofs_v2
Open

fs/mnemofs: Add mnemofs version 1 support.#18884
resyfer wants to merge 1 commit into
apache:masterfrom
resyfer:mnemofs_v2

Conversation

@resyfer
Copy link
Copy Markdown
Contributor

@resyfer resyfer commented May 15, 2026

Split mnemofs into allocation, directory, file, CTZ, and read/write modules, and update direntry traversal and file handling. Add superblock format version 1 support, reject newer on-flash versions, and preserve the mounted version when rewriting metadata.

Update the NAND simulator drivers for the new mnemofs behavior by fixing spare writes, exposing the erase state, allowing raw reads, and documenting the background-task startup flow.

Summary

The existing mnemofs implementation has a logic flaw which requires a huge amount of memory for the flush operation (which happens rarely, but happens). The fix to this required changes to the core of the file system design, hence the big changes.

Impact

Upon this review completing, mnemofs's structure should be free of any logic flaws, and hence should be good to be designated for the structure of version 1. Testing should only require changes to logic, not the structure.

Testing

This has been tested using the nand simulator. An example log of creating and writing to a file and reading its contents (/hi/a/b/c.txt) along with the info/error logs (some parts have been truncated to be readable):

nsh> nand
binfs_stat: Entry
nand_devscan: Retrieving bad block information. nblocks=1024
nand_checkblock: Page 0 block 4 marker=00
nand_devscan: Good blocks: 0 - 4
nand_devscan: Block 4 is bad
...
nand_devscan: Block 456 is bad
nand_checkblock: Page 0 block 500 marker=00
nand_devscan: Good blocks: 457 - 500
nand_devscan: Block 500 is bad
nand_devscan: Good blocks: 501 - 1024
Driver running!
nsh> ls /dev
binfs_stat: Entry
/dev:
 console
 ...
 nand
 ...
 zero
nsh> mount -t mnemofs -o autoformat /dev/nand /hi
binfs_stat: Entry
find_blockdriver: pathname="/dev/nand"
find_blockdriver: /dev/nand is a MTD
mnemofs_bind: sb allocated: 0x7f2a5f7b40e0
nand_wrapper: [UPPER 1 | ioctl] Command: 1537, Arg : 139819967267056
nand_wrapper: [UPPER 1 | ioctl] Done
nand_wrapper: [UPPER 2 | ioctl] Command: 1546, Arg : 139819967267104
nand_wrapper: [UPPER 2 | ioctl] Done
mnemofs_bind: Mount option set: autoformat
nand_wrapper: [UPPER 3 | isbad] Block: 0
nand_wrapper: [UPPER 3 | isbad] Done 0
nand_wrapper: [UPPER 4 | isbad] Block: 0
nand_wrapper: [UPPER 4 | isbad] Done 0
nand_wrapper: [UPPER 5 | bread] Startblock: 0, N Pages: 1, Buffer: 0x7f2a5f7b4198
nand_bread: startpage: 0 npages: 1
nand_readpage: block=0 page=0 data=0x7f2a5f7b4198
nand_wrapper: [UPPER 5 | bread] Done
nand_wrapper: [UPPER 6 | isbad] Block: 1
nand_wrapper: [UPPER 6 | isbad] Done 0
nand_wrapper: [UPPER 7 | isbad] Block: 0
nand_wrapper: [UPPER 7 | isbad] Done 0
nand_wrapper: [UPPER 8 | erase] Startblock: 0, N Blocks: 1
nand_erase: startblock: 00000000 nblocks: 1
nand_wrapper: [UPPER 8 | erase] Done
nand_wrapper: [UPPER 9 | isbad] Block: 1
nand_wrapper: [UPPER 9 | isbad] Done 0
nand_wrapper: [UPPER 10 | erase] Startblock: 1, N Blocks: 1
...
nand_erase: startblock: 0000001b nblocks: 1
nand_wrapper: [UPPER 58 | erase] Done
nand_wrapper: [UPPER 59 | isbad] Block: 28
nand_checkblock: Page 0 block 28 marker=00
nand_wrapper: [UPPER 59 | isbad] Done 1
nand_wrapper: [UPPER 60 | isbad] Block: 29
nand_wrapper: [UPPER 60 | isbad] Done 0
nand_wrapper: [UPPER 61 | erase] Startblock: 29, N Blocks: 1
nand_erase: startblock: 0000001d nblocks: 1
nand_wrapper: [UPPER 61 | erase] Done
...
nand_readpage: block=1 page=13 data=0x7f2a5f7b4198
nand_wrapper: [UPPER 35085 | bread] Done
nand_wrapper: [UPPER 35086 | isbad] Block: 1
nand_wrapper: [UPPER 35086 | isbad] Done 0
nand_wrapper: [UPPER 35087 | bread] Startblock: 30, N Pages: 1, Buffer: 0x7f2a5f7b4198
nand_bread: startpage: 30 npages: 1
nand_readpage: block=1 page=14 data=0x7f2a5f7b4198
nand_wrapper: [UPPER 35087 | bread] Done
nand_wrapper: [UPPER 35088 | isbad] Block: 1
nand_wrapper: [UPPER 35088 | isbad] Done 0
nand_wrapper: [UPPER 35089 | bread] Startblock: 31, N Pages: 1, Buffer: 0x7f2a5f7b4198
nand_bread: startpage: 31 npages: 1
nand_readpage: block=1 page=15 data=0x7f2a5f7b4198
nand_wrapper: [UPPER 35089 | bread] Done
nand_wrapper: [UPPER 35090 | isbad] Block: 610
nand_wrapper: [UPPER 35090 | isbad] Done 0
nsh> ls
binfs_stat: Entry
/:
 bin/
 data/
 dev/
 etc/
 hi/
 proc/
 tmp/
nsh> cd hi
binfs_stat: Entry
nsh> mkdir a
binfs_stat: Entry
nand_wrapper: [UPPER 35091 | isbad] Block: 1
nand_wrapper: [UPPER 35091 | isbad] Done 0
nand_wrapper: [UPPER 35092 | bread] Startblock: 17, N Pages: 1, Buffer: 0x7faff4db4198
...
nand_bwrite: startpage: 17 npages: 1
nand_wrapper: [UPPER 35125 | bwrite] Done
nsh> ls
binfs_stat: Entry
/hi:
nand_wrapper: [UPPER 35126 | isbad] Block: 1
nand_wrapper: [UPPER 35126 | isbad] Done 0
nand_wrapper: [UPPER 35127 | bread] Startblock: 17, N Pages: 1, Buffer: 0x7faff4db4198
...
nand_bread: startpage: 31 npages: 1
nand_readpage: block=1 page=15 data=0x7faff4db4198
nand_wrapper: [UPPER 35155 | bread] Done
 a/
nand_wrapper: [UPPER 35156 | isbad] Block: 1
nand_wrapper: [UPPER 35156 | isbad] Done 0
nand_wrapper: [UPPER 35157 | bread] Startblock: 17, N Pages: 1, Buffer: 0x7faff4db4198
...
nand_bread: startpage: 31 npages: 1
nand_readpage: block=1 page=15 data=0x7faff4db4198
nand_wrapper: [UPPER 35185 | bread] Done
nsh> cd a
binfs_stat: Entry
nand_wrapper: [UPPER 35186 | isbad] Block: 1
nand_wrapper: [UPPER 35186 | isbad] Done 0
nand_wrapper: [UPPER 35187 | bread] Startblock: 17, N Pages: 1, Buffer: 0x7faff4db4198
...
nand_bread: startpage: 31 npages: 1
nand_readpage: block=1 page=15 data=0x7faff4db4198
nand_wrapper: [UPPER 35245 | bread] Done
nsh> mkdir b
binfs_stat: Entry
nand_wrapper: [UPPER 35246 | isbad] Block: 1
nand_wrapper: [UPPER 35246 | isbad] Done 0
nand_wrapper: [UPPER 35247 | bread] Startblock: 17, N Pages: 1, Buffer: 0x7faff4db4198
...
nand_bread: startpage: 12177 npages: 1
nand_readpage: block=761 page=1 data=0x7faff4db4198
nand_wrapper: [UPPER 35309 | bread] Done
mfs_dir_mkdir: created dir a/b
nand_wrapper: [UPPER 35310 | bwrite] Startblock: 12177, N Pages: 1, Buffer: 0x7faff4db4198
nand_bwrite: startpage: 12177 npages: 1
nand_wrapper: [UPPER 35310 | bwrite] Done
nsh> ls
binfs_stat: Entry
nand_wrapper: [UPPER 35311 | isbad] Block: 1
nand_wrapper: [UPPER 35311 | isbad] Done 0
nand_wrapper: [UPPER 35312 | bread] Startblock: 17, N Pages: 1, Buffer: 0x7faff4db4198
...
nand_bread: startpage: 30 npages: 1
nand_readpage: block=1 page=14 data=0x7faff4db4198
nand_wrapper: [UPPER 35338 | bread] Done
nand_wrapper: [UPPER 35339 | isbad] Block: 1
nand_wrapper: [UPPER 35339 | isbad] Done 0
nand_wrapper: [UPPER 35340 | bread] Startblock: 31, N Pages: 1, Buffer: 0x7faff4db4198
nand_bread: startpage: 31 npages: 1
nand_readpage: block=1 page=15 data=0x7faff4db4198
nand_wrapper: [UPPER 35340 | bread] Done
/hi/a:
nand_wrapper: [UPPER 35341 | isbad] Block: 1
nand_wrapper: [UPPER 35341 | isbad] Done 0
nand_wrapper: [UPPER 35342 | bread] Startblock: 17, N Pages: 1, Buffer: 0x7faff4db4198
...
nand_wrapper: [UPPER 35398 | bread] Done
nand_wrapper: [UPPER 35399 | isbad] Block: 761
nand_wrapper: [UPPER 35399 | isbad] Done 0
nand_wrapper: [UPPER 35400 | bread] Startblock: 12191, N Pages: 1, Buffer: 0x7faff4db4198
nand_bread: startpage: 12191 npages: 1
nand_readpage: block=761 page=15 data=0x7faff4db4198
nand_wrapper: [UPPER 35400 | bread] Done
 b/
nand_wrapper: [UPPER 35401 | isbad] Block: 761
nand_wrapper: [UPPER 35401 | isbad] Done 0
nand_wrapper: [UPPER 35402 | bread] Startblock: 12177, N Pages: 1, Buffer: 0x7faff4db4198
...
nand_bread: startpage: 12190 npages: 1
nand_readpage: block=761 page=14 data=0x7faff4db4198
nand_wrapper: [UPPER 35428 | bread] Done
nand_wrapper: [UPPER 35429 | isbad] Block: 761
nand_wrapper: [UPPER 35429 | isbad] Done 0
nand_wrapper: [UPPER 35430 | bread] Startblock: 12191, N Pages: 1, Buffer: 0x7faff4db4198
nand_bread: startpage: 12191 npages: 1
nand_readpage: block=761 page=15 data=0x7faff4db4198
nand_wrapper: [UPPER 35430 | bread] Done
nsh> cd b
binfs_stat: Entry
nand_wrapper: [UPPER 35431 | isbad] Block: 1
nand_wrapper: [UPPER 35431 | isbad] Done 0
...
nand_bread: startpage: 12190 npages: 1
nand_readpage: block=761 page=14 data=0x7faff4db4198
nand_wrapper: [UPPER 35578 | bread] Done
nand_wrapper: [UPPER 35579 | isbad] Block: 761
nand_wrapper: [UPPER 35579 | isbad] Done 0
nand_wrapper: [UPPER 35580 | bread] Startblock: 12191, N Pages: 1, Buffer: 0x7faff4db4198
nand_bread: startpage: 12191 npages: 1
nand_readpage: block=761 page=15 data=0x7faff4db4198
nand_wrapper: [UPPER 35580 | bread] Done
nsh> pwd
binfs_stat: Entry
/hi/a/b
nsh> echo "hello world from mnemofs" > c.txt
binfs_stat: Entry
nand_wrapper: [UPPER 35581 | isbad] Block: 1
nand_wrapper: [UPPER 35581 | isbad] Done 0
nand_wrapper: [UPPER 35582 | bread] Startblock: 17, N Pages: 1, Buffer: 0x7faff4db4198
nand_bread: startpage: 17 npages: 1
nand_readpage: block=1 page=1 data=0x7faff4db4198
nand_wrapper: [UPPER 35582 | bread] Done
nand_wrapper: [UPPER 35583 | isbad] Block: 1
nand_wrapper: [UPPER 35583 | isbad] Done 0
nand_wrapper: [UPPER 35584 | bread] Startblock: 18, N Pages: 1, Buffer: 0x7faff4db4198
...
nand_bread: startpage: 12195 npages: 1
nand_readpage: block=762 page=3 data=0x7faff4db4198
nand_wrapper: [UPPER 35872 | bread] Done
nand_wrapper: [UPPER 35873 | bwrite] Startblock: 12195, N Pages: 1, Buffer: 0x7faff4db4198
nand_bwrite: startpage: 12195 npages: 1
nand_wrapper: [UPPER 35873 | bwrite] Done
nsh> cat c.txt
binfs_stat: Entry
nand_wrapper: [UPPER 35874 | isbad] Block: 1
nand_wrapper: [UPPER 35874 | isbad] Done 0
nand_wrapper: [UPPER 35875 | bread] Startblock: 17, N Pages: 1, Buffer: 0x7faff4db4198
nand_bread: startpage: 17 npages: 1
nand_readpage: block=1 page=1 data=0x7faff4db4198
nand_wrapper: [UPPER 35875 | bread] Done
nand_wrapper: [UPPER 35876 | isbad] Block: 1
nand_wrapper: [UPPER 35876 | isbad] Done 0
nand_wrapper: [UPPER 35877 | bread] Startblock: 18, N Pages: 1, Buffer: 0x7faff4db4198
nand_bread: startpage: 18 npages: 1
nand_readpage: block=1 page=2 data=0x7faff4db4198
...
mfs_dir_create_file: created file a/b/c.txt
...
nand_bread: startpage: 12207 npages: 1
nand_readpage: block=762 page=15 data=0x7faff4db4198
nand_wrapper: [UPPER 35963 | bread] Done
nand_wrapper: [UPPER 35964 | isbad] Block: 763
nand_wrapper: [UPPER 35964 | isbad] Done 0
nand_wrapper: [UPPER 35965 | bread] Startblock: 12210, N Pages: 1, Buffer: 0x7faff4db4198
nand_bread: startpage: 12210 npages: 1
nand_readpage: block=763 page=2 data=0x7faff4db4198
nand_wrapper: [UPPER 35965 | bread] Done
hello world from mnemofs

@resyfer
Copy link
Copy Markdown
Contributor Author

resyfer commented May 15, 2026 •
edited
Loading

This changes the core design and dependencies of the structures of the file system. Since a file system, especially a tightly knit one like this, can't really work with some structures stuck in past and some being new, there was no hope for incremental changes even if I tried to find any.

This is an essential rewrite of mnemofs, and I think it would be better reviewed as addition than a change of existing code. Upon discussions with @acassis, I've decided to put this PR as one big PR instead of smaller non-functional ones.

@github-actions github-actions Bot added Area: Drivers Drivers issues Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces. labels May 15, 2026
@acassis
Copy link
Copy Markdown
Contributor

acassis commented May 15, 2026

@resyfer please fix this issue:

/home/runner/work/nuttx/nuttx/Documentation/applications/testing/nand_sim/index.rst:156:Title underline too short.

linguini1
linguini1 previously approved these changes May 15, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@linguini1 linguini1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably no one knows better about this than you!

Out of curiosity, has this ever been tried on real NAND flash yet? I would really love to use this FS.

@acassis
Copy link
Copy Markdown
Contributor

acassis commented May 16, 2026

Probably no one knows better about this than you!

Out of curiosity, has this ever been tried on real NAND flash yet? I would really love to use this FS.

He only tested on simulator, I will test here using W25N01GVZEIG NAND chip.

@linguini1
Copy link
Copy Markdown
Contributor

linguini1 commented May 16, 2026

Let me know how it goes! This would be great for a lot of devices I have planned.

@resyfer
fs/mnemofs: Add mnemofs version 1 support.
136868a 
Split mnemofs into allocation, directory, file, CTZ, and read/write
modules, and update direntry traversal and file handling. Add superblock
format version 1 support, reject newer on-flash versions, and preserve
the mounted version when rewriting metadata.

Update the NAND simulator drivers for the new mnemofs behavior by fixing
spare writes, exposing the erase state, allowing raw reads, and documenting
the background-task startup flow.

Signed-off-by: Saurav Pal <resyfer.dev@gmail.com>
@resyfer
Copy link
Copy Markdown
Contributor Author

resyfer commented May 17, 2026

I will have a NAND flash with me in a couple of days, and I can test it out on it. Should it better that we keep this PR open till the actual hardware is confirmed?

@linguini1
Copy link
Copy Markdown
Contributor

linguini1 commented May 17, 2026

To the best of my knowledge, wasn't the old version only tested on a simulator? If so, this PR is fine to merge before the hardware tests since it's was never guaranteed to work on real hardware. If you find some issues then you can upstream patches which will hopefully be smaller

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@linguini1 linguini1 linguini1 left review comments

@pkarashchenko pkarashchenko Awaiting requested review from pkarashchenko pkarashchenko is a code owner

@jerpelea jerpelea Awaiting requested review from jerpelea jerpelea is a code owner

At least 2 approving reviews are required to merge this pull request.

Assignees

No one assigned

Labels

Area: Drivers Drivers issues Size: XL The size of the change in this PR is very large. Consider breaking down the PR into smaller pieces.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@resyfer @acassis @linguini1