gh-152433: Windows: modernize and refactor fileutils to allow build for Windows UWP by thexai · Pull Request #153700 · python/cpython

thexai · 2026-07-14T13:45:16Z

Windows: modernize and refactor fileutils to allow build for Windows UWP

This is probably the last relevant commit missing to allow compile all Python core for UWP.

GetFileInformationByHandle is not supported in UWP, then is replaced with more modern GetFileInformationByHandleEx calls.

~~Also code refactored to avoid some duplicated code.~~

Note that Xbox (all models) runs only on Windows 11 based OS. Then is guaranteed that "fast stat" code (based on GetFileInformationByName) is supported:

cpython/Modules/posixmodule.c

Line 2299 in 050a84b

if (_Py_GetFileInformationByName(path, FileStatBasicByNameInfo,

"slow stat" code path is not much relevant for UWP but also works if replaced CreateFileW with CreateFile2 (in other PR).

Issue: Windows: improve compatibility to build for UWP / Xbox #152433

thexai · 2026-07-17T14:24:19Z

Updated with more simplified and less invasive changes.

chris-eibl · 2026-07-19T10:00:05Z

This breaks CI, e.g.

test_1686475 (test.test_os.test_os.StatAttributeTests.test_1686475) ... Windows fatal exception: access violation

Can you reproduce this on a local Windows installation and fix?

Regarding tests I have another question: are you able to run the tests on XBOX in general? I'd be curious in how many of them fail ...

…ows UWP

thexai · 2026-07-19T11:42:00Z

This breaks CI, e.g.

test_1686475 (test.test_os.test_os.StatAttributeTests.test_1686475) ... Windows fatal exception: access violation

Can you reproduce this on a local Windows installation and fix?

Reproduced and now should be fixed in latest PR update. 🙂

It's important to note that some tests fail because they expect specific responses to unusual situations that may not have a single logical outcome. For example:

If a file is inaccessible (due to permissions), the parent directory is read, and if it's no longer available, the original error is not restored (instead of return access violation error, return file no exist error).

    # bpo-46785: the implementation of os.stat() falls back to reading
    # the parent directory if CreateFileW() fails with a permission
    # error. If reading the parent directory fails because the file or
    # directory are subsequently unlinked, or because the volume or
    # share are no longer available, then the original permission error
    # should not be restored.

Also on UWP, due security restrictions (every App runs isolated) is not possible to access to some information in the file system like st_dev (VolumeSerialNumber) or st_ino.

chris-eibl

LGTM, but I'd appreciate a second pair of eyes @python/windows-team.

zooba

Only left two comments, but the guts of it is, I think we can switch to FileStandardInfo for everyone, and that should simplify this change down to basically not needing the separate UWP path at all.

thexai · 2026-07-21T07:57:11Z

Updated PR with one simplification:

UWP not support GetFileInformationByHandle but the struct used to pass info BY_HANDLE_FILE_INFORMATION is still defined and can be used in some places e.g here:

cpython/Modules/posixmodule.c

Lines 2027 to 2043 in 202c4c8

    
           find_data_to_file_info(WIN32_FIND_DATAW *pFileData, 
        
                                  BY_HANDLE_FILE_INFORMATION *info, 
        
                                  ULONG *reparse_tag) 
        
           { 
        
               memset(info, 0, sizeof(*info)); 
        
               info->dwFileAttributes = pFileData->dwFileAttributes; 
        
               info->ftCreationTime   = pFileData->ftCreationTime; 
        
               info->ftLastAccessTime = pFileData->ftLastAccessTime; 
        
               info->ftLastWriteTime  = pFileData->ftLastWriteTime; 
        
               info->nFileSizeHigh    = pFileData->nFileSizeHigh; 
        
               info->nFileSizeLow     = pFileData->nFileSizeLow; 
        
           /*  info->nNumberOfLinks   = 1; */ 
        
               if (pFileData->dwFileAttributes & FILE_ATTRIBUTE_REPARSE_POINT) 
        
                   *reparse_tag = pFileData->dwReserved0; 
        
               else 
        
                   *reparse_tag = 0; 
        
           }

This allows remove part of UWP only code and some ifdef.

thexai · 2026-07-21T09:41:42Z

Simplification 2:

Merges _Py_attribute_data_to_stat and _Py_attribute_data_to_stat_UWP in one.

Use same code for UWP and Desktop.

thexai · 2026-07-21T11:23:03Z

I think I've finished this!

Minimal changes and the same common code for UWP and Desktop.

Tests should pass...

zooba

Apologies for lots of suggestions, but I'm excited about this cleanup! We're so close to really streamlining this code, so it'd be great to go the whole way.

My main target is to fully remove the BY_HANDLE_FILE_INFORMATION usage entirely, and also to minimise the logic in the to_stat function (more overwriting, less checking). The comments I've left should lead you there, but feel free to ignore them if you see another way.

zooba · 2026-07-21T15:15:33Z

+    if (info) {
+        result->st_size = (((__int64)info->nFileSizeHigh) << 32) + info->nFileSizeLow;
+        result->st_nlink = info->nNumberOfLinks;
+        if (!id_info)
+            result->st_dev = info->dwVolumeSerialNumber;
+    }
+    if (standard_info && !info) {
+        result->st_size = standard_info->EndOfFile.QuadPart;
+        result->st_nlink = standard_info->NumberOfLinks;
+    }


I think we can simplify even further by just letting later checks overwrite earlier ones, so if both info and standard_info are provided, the latter will overwrite.

e.g. (don't apply this directly):

Suggested change

if (info) {

result->st_size = (((__int64)info->nFileSizeHigh) << 32) + info->nFileSizeLow;

result->st_nlink = info->nNumberOfLinks;

if (!id_info)

result->st_dev = info->dwVolumeSerialNumber;

}

if (standard_info && !info) {

result->st_size = standard_info->EndOfFile.QuadPart;

result->st_nlink = standard_info->NumberOfLinks;

}

if (info) {

result->st_size = (((__int64)info->nFileSizeHigh) << 32) + info->nFileSizeLow;

result->st_nlink = info->nNumberOfLinks;

result->st_dev = info->dwVolumeSerialNumber;

// add the st_ino = from info assignment

}

if (standard_info) {

result->st_size = standard_info->EndOfFile.QuadPart;

result->st_nlink = standard_info->NumberOfLinks;

}

zooba · 2026-07-21T15:16:39Z

        result->st_ino = file_id.st_ino;
        result->st_ino_high = file_id.st_ino_high;


Put these under the st_ino != 0 && st_ino_high != 0 check from (original) line 1134

zooba · 2026-07-21T15:17:17Z

           index is likely to be zero anyway. */
-        result->st_ino = (((uint64_t)info->nFileIndexHigh) << 32) + info->nFileIndexLow;
+        if (info)
+            result->st_ino = (((uint64_t)info->nFileIndexHigh) << 32) + info->nFileIndexLow;


Put this up under the if (info) block at the start of the function (that is, unconditionally [apart from info] assign the file index, and we'll overwrite with id_info later if it's non-zero)

zooba · 2026-07-21T15:17:54Z

+    const DWORD fileAttributes = basic_info ? basic_info->FileAttributes : info->dwFileAttributes;
+
+    result->st_file_attributes = fileAttributes;
+    result->st_mode = attributes_to_mode(fileAttributes);


Put st_file_attributes under each original if block, and st_mode can stay here and reference st_file_attributes

zooba · 2026-07-21T15:18:59Z

+    result->st_dev = 1;
+    result->st_ino = basic_info->CreationTime.QuadPart;


These can go right at the start and be our defaults. But they should really both be 0, which is a supported result for both fields (and indicates they can't be used for comparisons).

zooba · 2026-07-21T15:21:24Z


    find_data_to_file_info(dataW, &file_info, &reparse_tag);
-    _Py_attribute_data_to_stat(&file_info, reparse_tag, NULL, NULL, &entry->win32_lstat);
+    _Py_attribute_data_to_stat(&file_info, NULL, reparse_tag, NULL, NULL, &entry->win32_lstat);


This is the only place where file_info is still used, and I'm pretty sure we can make find_data_to_file_info fill out a different type (presumably FILE_STANDARD_INFO), which would then let us entirely remove the old info argument from _Py_attribute_data_to_stat

I'd thought of that too: there's no reason to still use old BY_HANDLE_FILE_INFORMATION struct just for this.

But it'll take me a while because I won't be at a PC tomorrow 🙂

That's fine, we're not in any hurry right now. Just glad to have you working on it

bedevere-app Bot mentioned this pull request Jul 14, 2026

Windows: improve compatibility to build for UWP / Xbox #152433

Open

bedevere-app Bot added the awaiting review label Jul 14, 2026

thexai force-pushed the uwp-fileutils branch from a26c789 to 866215a Compare July 17, 2026 14:22

thexai force-pushed the uwp-fileutils branch 3 times, most recently from 57f8125 to f0f837b Compare July 19, 2026 08:33

pythongh-152433: Windows: modernize fileutils to allow build for Wind…

190b3f2

…ows UWP

thexai force-pushed the uwp-fileutils branch from f0f837b to 190b3f2 Compare July 19, 2026 11:25

chris-eibl added the sprint label Jul 19, 2026

github-project-automation Bot added this to Sprint Jul 19, 2026

github-project-automation Bot moved this to Todo in Sprint Jul 19, 2026

chris-eibl approved these changes Jul 19, 2026

View reviewed changes

bedevere-app Bot removed the awaiting review label Jul 19, 2026

github-project-automation Bot moved this from Todo to In Progress in Sprint Jul 19, 2026

bedevere-app Bot added the awaiting merge label Jul 19, 2026

zooba reviewed Jul 20, 2026

View reviewed changes

Comment thread Modules/posixmodule.c

Comment thread Modules/posixmodule.c Outdated

Simplification 1

5dabc7c

thexai added 2 commits July 21, 2026 10:03

Full remove of _Py_find_data_to_stat

43d4fc2

Simplification 2

003af98

thexai added 4 commits July 21, 2026 11:45

Fix typo

a0f6bb7

Simplification 3: Remove GetFileInformationByHandle on Desktop

9a694a3

Use same code for UWP and Desktop.

Remove unnecessary/accidental changes

b81a7d1

Preserve old code format

24a0f47

zooba reviewed Jul 21, 2026

View reviewed changes

		result->st_ino = file_id.st_ino;
		result->st_ino_high = file_id.st_ino_high;

		result->st_dev = 1;
		result->st_ino = basic_info->CreationTime.QuadPart;

Uh oh!

Uh oh!

Conversation

thexai commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thexai commented Jul 17, 2026

Uh oh!

chris-eibl commented Jul 19, 2026

Uh oh!

thexai commented Jul 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chris-eibl left a comment

Choose a reason for hiding this comment

Uh oh!

zooba left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

thexai commented Jul 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thexai commented Jul 21, 2026

Uh oh!

thexai commented Jul 21, 2026

Uh oh!

zooba left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

thexai commented Jul 14, 2026 •

edited

Loading

thexai commented Jul 19, 2026 •

edited

Loading

thexai commented Jul 21, 2026 •

edited

Loading