Skip to content

testsuite: only use follow_symlinks if supported#7754

Open
RayyanAnsari wants to merge 1 commit into
borgbackup:masterfrom
RayyanAnsari:utime-win
Open

testsuite: only use follow_symlinks if supported#7754
RayyanAnsari wants to merge 1 commit into
borgbackup:masterfrom
RayyanAnsari:utime-win

Conversation

@RayyanAnsari

@RayyanAnsari RayyanAnsari commented Aug 1, 2023

Copy link
Copy Markdown
Contributor

On Windows, follow_symlinks is not supported by every os function yet, using False can raise a NotImplementedError. Check os.supports_follow_symlinks to decide whether to use True or False.

(this should fix the utime tests that were being skipped before)

On Windows, follow_symlinks is not supported by every os function yet, using False can raise a NotImplementedError.
Check os.supports_follow_symlinks to decide whether to use True or False.
@RayyanAnsari

Copy link
Copy Markdown
Contributor Author

follow_symlinks is also used in other places. perhaps I should change them?

@codecov-commenter

codecov-commenter commented Aug 1, 2023

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.68%. Comparing base (1c8da8f) to head (32813e5).
⚠️ Report is 2240 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7754      +/-   ##
==========================================
- Coverage   83.70%   83.68%   -0.02%     
==========================================
  Files          66       66              
  Lines       11906    11906              
  Branches     2158     2158              
==========================================
- Hits         9966     9964       -2     
- Misses       1366     1367       +1     
- Partials      574      575       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Comment on lines 149 to +151
try:
os.utime(filepath, (1000, 2000), follow_symlinks=False)
new_stats = os.stat(filepath, follow_symlinks=False)
os.utime(filepath, (1000, 2000), follow_symlinks=False if os.utime in os.supports_follow_symlinks else True)
new_stats = os.stat(filepath, follow_symlinks=False if os.stat in os.supports_follow_symlinks else True)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, guess that needs to be done differently:
The code right above either creates a symlink (with a non-existing target as it seems) or a file.
So we can't just arbitrarily switch follow_symlinks to True/False below:

  • the target does not exist
  • also, line 144 seems to imply we want to know about utime behaviour on symlinks

So guess this needs double-checking:

  • what tests do use this and what exactly do we need to check here?
  • the utime call needs to match the fs object it is used with

@ThomasWaldmann ThomasWaldmann Aug 2, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also: never use a condition to decide between True or False. Above, you could've just used follow_symlinks=os.stat not in os.supports_follow_symlinks.

Reads a bit strange, but guess that's because the default (and always supported) behaviour is with True and the special (and sometimes unsupported) behaviour is with False.

@ThomasWaldmann

Copy link
Copy Markdown
Member

This needs to be improved / continued by a windows-based developer.

@ThomasWaldmann

Copy link
Copy Markdown
Member

commenting to trigger re-indexing of this PR, so it gets visible again on the github web UI open PR list.

@ThomasWaldmann

Copy link
Copy Markdown
Member

Review done by Claude Fable 5:

Overview: The goal of this PR is legitimate — on Windows, os.utime is not in os.supports_follow_symlinks (in CPython's os.py, only os.stat gets the MS_WINDOWS entry), and borg's own win32 restore path (Archive.restore_attrs, the else: # win32 branch) restores mtimes with a plain follow-the-link os.utime. So reporting utime as supported for regular files on Windows is consistent with what borg actually does, and would un-skip meaningful tests.

Findings:

  1. The PR's premise is partially obsolete; it needs a rebase and a rescoped description. Current master already catches NotImplementedError in is_utime_fully_supported(), so the crash described here no longer happens — the probe cleanly returns False on Windows and the tests skip. The only remaining effect of this PR is flipping that result to True on Windows without symlink privileges. The function has also moved and gained @functools.lru_cache, so the patch should be refreshed against current master, and the description reframed as "enable utime tests on Windows" rather than "avoid NotImplementedError".

  2. The fallback is only correct by accident, and it doesn't achieve its goal on symlink-enabled Windows. When symlinks are supported (Windows Developer Mode / admin), the probe target is a dangling symlink, so the fallback utime(..., follow_symlinks=True) hits FileNotFoundError and the function still returns False — the utime tests stay skipped exactly where this PR intended to enable them. Conversely, the True result in the no-symlinks branch is only safe because are_symlinks_supported() being False guarantees the test suite can't create symlinks either; that cross-function invariant is invisible at this line. Prefer an explicit branch (see suggestion below) over relying on the dangling-symlink failure.

  3. The os.stat fallback is dead code. os.stat is in os.supports_follow_symlinks on every platform CPython supports (HAVE_LSTAT/HAVE_FSTATAT/MS_WINDOWS), so False if os.stat in os.supports_follow_symlinks else True always evaluates to False. The stat line should stay as plain follow_symlinks=False.

  4. Awkward conditional expression. False if X in Y else True is just X not in Y — if the fallback pattern is kept, follow_symlinks=os.utime not in os.supports_follow_symlinks says the same thing more directly.

Suggested shape (explicit instead of per-call fallback):

@functools.lru_cache
def is_utime_fully_supported():
    with unopened_tempfile() as filepath:
        if are_symlinks_supported():
            if os.utime not in os.supports_follow_symlinks:
                # symlinks can exist, but their timestamps can't be set -> not fully supported
                return False
            os.symlink("something", filepath)
            try:
                os.utime(filepath, (1000, 2000), follow_symlinks=False)
                new_stats = os.stat(filepath, follow_symlinks=False)
                ...
        else:
            # no symlinks possible in test data, probing a regular file is enough;
            # follow_symlinks=False may be unsupported (win32) and is irrelevant here
            open(filepath, "w").close()
            try:
                os.utime(filepath, (1000, 2000))
                new_stats = os.stat(filepath, follow_symlinks=False)
                ...

This keeps the semantics honest: "fully supported" still means False on symlink-capable platforms that can't set symlink timestamps, while enabling the tests on Windows CI without symlink privileges.

(Going further — returning True on symlink-capable Windows too — would make _assert_dirs_equal compare symlink mtimes that the win32 restore path never sets, so that would need a separate capability check and is better left out of this change.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants