Skip to content

Add "system" option to override-allocator directive - #160100

Merged
rust-bors[bot] merged 11 commits into
rust-lang:mainfrom
fs-rachel:fix-jemalloc-directive
Jul 31, 2026
Merged

Add "system" option to override-allocator directive#160100
rust-bors[bot] merged 11 commits into
rust-lang:mainfrom
fs-rachel:fix-jemalloc-directive

Conversation

@fs-rachel

@fs-rachel fs-rachel commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

View all comments

This allows a per-target override-allocator to revert to the system allocator even if there is a global rule specifying a different allocator.

Remap jemalloc = false to override-allocator = "system" instead of mapping it to "override-allocator is unset"; this restores the correct precedence behaviour of this directive.

This also allows Config::override_allocator() to always resolve the directive to a definite value, rather than needing it to be wrapped in an Option, which simplifies some other code.

Finally, print a more specific warning when jemalloc is set, telling users the appropriate setting to use for override-allocator depending on whether jemalloc is set to true or false.

Fixes #160084

@rustbot

rustbot commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jul 28, 2026
@rustbot

rustbot commented Jul 28, 2026

Copy link
Copy Markdown
Collaborator

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

  • Owners of files modified in this PR: bootstrap
  • bootstrap expanded to 6 candidates
  • Random selection from Mark-Simulacrum, clubby789, jieyouxu

@fs-rachel

Copy link
Copy Markdown
Contributor Author

In case it is useful for review purposes, I've uploaded a version of this patch with some extra tracing code to a different branch:

https://github.com/fs-rachel/rust/tree/fix-jemalloc-directive-with-tracing

@fs-rachel

Copy link
Copy Markdown
Contributor Author

Small procedural question about @Kobzol's suggestion of renaming OverrideAllocator -> Allocator and friends: As this directive is still extremely new, how should I handle this in the change tracker? Is it preferred to edit the existing message (as in the current version of the patch), or leave that as is and add a new entry about the renaming?

@Kobzol

Kobzol commented Jul 28, 2026

Copy link
Copy Markdown
Member

Good point, I was also thinking about that. Even though it's new, I think we should just create a new change tracker entry.. some people might already renamed the option (the jemalloc option is quite popular, I'd say), and if we changed the entry from under them, it would be confusing.

It's not great that we do a breaking change so soon, but I think that it also doesn't warrant creating yet another way to configure the allocator.

We could also just keep the override-allocator name to not break backwards compatibility. And then the the "unoverride" method would be override-allocator = false, or something along those lines.

@Kobzol

Kobzol commented Jul 28, 2026

Copy link
Copy Markdown
Member

r? @Kobzol

@rustbot rustbot assigned Kobzol and unassigned jieyouxu Jul 28, 2026
@fs-rachel

Copy link
Copy Markdown
Contributor Author

Good point, I was also thinking about that. Even though it's new, I think we should just create a new change tracker entry.. some people might already renamed the option (the jemalloc option is quite popular, I'd say), and if we changed the entry from under them, it would be confusing.

Okay, I'll re-upload with that change in a minute

It's not great that we do a breaking change so soon, but I think that it also doesn't warrant creating yet another way to configure the allocator.

We could also just keep the override-allocator name to not break backwards compatibility. And then the the "unoverride" method would be override-allocator = false, or something along those lines.

IMHO allocator = "system" communicates its intent just a little clearer than override-allocator = false does

@Kobzol

Kobzol commented Jul 28, 2026

Copy link
Copy Markdown
Member

Yeah, it does :) And even if we changed the default allocator on some targets in the future, "system" vs "jemalloc" still seems like reasonable options, we could maybe add "default" in the future, but I don't think that would be needed, the default will likely always be the system one.

@fs-rachel
fs-rachel force-pushed the fix-jemalloc-directive branch from 849982d to ed7c99e Compare July 28, 2026 19:29
@Zoxc

Zoxc commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

I'm not sure about dropping the override part. We're taking potentially unsafe actions when overriding the C/C++ allocator with this option, while allocator = "system" is a safe non-action. Maybe it should also be in the build section given that it's not limited to Rust code?

@Kobzol

Kobzol commented Jul 28, 2026

Copy link
Copy Markdown
Member

I don't think that overriding is a load-bearing word in tems of unsafety, you can probably cause a whole lot of issues or unsafety by (mis)configuring other bootstrap options, which don't talk about overriding a specific thing.

Good point about the build section though. I think that the allocator option originally started as being Rust only, but eventually it got extended also to C/C++ (mainly to LLVM), because of issues caused by using different allocators for Rust and C/C++.

Since we are already renaming the option and breaking people who updated bootstrap since last week (hopefully not too many of them), moving the allocator option into the build section from the rust section sounds reasonable to me. CC @Mark-Simulacrum @jieyouxu if you have any thoughts on this.

By the way, all uses of override-allocator=jemalloc under src/ci will have to be replaced with the new name.

@jieyouxu

Copy link
Copy Markdown
Member

Moving into the build section seems reasonable

@fs-rachel
fs-rachel force-pushed the fix-jemalloc-directive branch from ed7c99e to ff6dcf4 Compare July 29, 2026 10:48
@rustbot

rustbot commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

rustc-dev-guide is developed in its own repository. If possible, consider making this change to rust-lang/rustc-dev-guide instead.

cc @BoxyUwU, @tshepang

This PR modifies bootstrap.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

Warning

If you are changing how CI LLVM is built or linked, make sure to bump
src/bootstrap/download-ci-llvm-stamp.

cc @jieyouxu

@rustbot rustbot added A-CI Area: Our Github Actions CI A-meta Area: Issues & PRs about the rust-lang/rust repository itself A-rustc-dev-guide Area: rustc-dev-guide A-testsuite Area: The testsuite used to check the correctness of rustc T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Jul 29, 2026
@fs-rachel
fs-rachel force-pushed the fix-jemalloc-directive branch from ff6dcf4 to 76bb5f0 Compare July 29, 2026 10:52
@fs-rachel

Copy link
Copy Markdown
Contributor Author

Tidied things up and moved rust.allocator -> build.allocator; ready for re-review

@fs-rachel

Copy link
Copy Markdown
Contributor Author

And a note to myself: Once this is merged I need to send a separate PR to rustc-dev-guide to update things there

@Kobzol Kobzol left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, left two comments.

@bors try @rust-timer queue

View changes since this review

if let Some(allocator) = builder.config.override_allocator(target) {
let allocator = builder.config.allocator(target);
if allocator != Allocator::System {
features.push(allocator.feature_name().to_string());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that a better design (rather than panicking) would be for feature_name() to return Option<&'static str> (or just Option<String>, doesn't really matter), and then we'd use if let Some(feature_name) = allocator.feature_name() { ... }.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -406,7 +402,6 @@ pub fn check_incompatible_options_for_ci_rustc(
err!(current_rust_config.llvm_tools, llvm_tools, "rust");
err!(current_rust_config.llvm_bitcode_linker, llvm_bitcode_linker, "rust");
err!(current_rust_config.jemalloc, jemalloc, "rust");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should still check the allocator from the build section.

@fs-rachel fs-rachel Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had a look at what this is actually doing, and I think it should also check the target.<host>.jemalloc and target.<host>.allocator settings, so I'll add those as well

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh and there's one glaring issue I've noticed: If there are incompatible settings in the target.<host> section, the warning message incorrectly states the issue is in the build section

Honestly this function should probably be cleaned up and moved elsewhere, but I'll at least fix that one while I'm here.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tbh it's a sanity check, the function doesn't have to be perfect. I don't think we ever dealt with target options in these sanity checks, and there are also other options that could affect the build. It's a best-effort check.

@rust-bors

This comment has been minimized.

rust-bors Bot pushed a commit that referenced this pull request Jul 29, 2026
Add "system" option to `override-allocator` directive
@fs-rachel
fs-rachel force-pushed the fix-jemalloc-directive branch from b0cb8f9 to 92c4436 Compare July 30, 2026 13:04
@rustbot

rustbot commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

@fs-rachel
fs-rachel force-pushed the fix-jemalloc-directive branch from 92c4436 to 0ca1049 Compare July 30, 2026 13:40

@Kobzol Kobzol left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still some mentions of override-allocator under src/ci (you can grep for it). Could you please replace those? Thank you!

View changes since this review

Comment thread src/bootstrap/src/utils/change_tracker.rs
@fs-rachel

Copy link
Copy Markdown
Contributor Author

There are still some mentions of override-allocator under src/ci (you can grep for it). Could you please replace those? Thank you!

I did a find-and-replace before rebasing, and I don't see any new instances when I search for override-allocator now

We don't sort those by PR number, it is actually load-bearing that the latest update is the last in the array 😅 So please move this one to the end (not sure if it was done manually or if it's some rebase fallout).

That was a merge conflict during rebasing, and I just didn't realize that it had to be ordered like that. Fixed.

@Kobzol

Kobzol commented Jul 31, 2026

Copy link
Copy Markdown
Member

Oh, maybe I rebased wrongly or my IDE showed stale result. Nevermind, it indeed looks like there aren't any mentions left. Thank you!

@bors r+ rollup=iffy

@rust-bors

rust-bors Bot commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

📌 Commit 9dd4861 has been approved by Kobzol

It is now in the queue for this repository.

@rust-bors rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 31, 2026
JonathanBrouwer added a commit to JonathanBrouwer/rust that referenced this pull request Jul 31, 2026
…r=Kobzol

Add "system" option to `override-allocator` directive

This allows a per-target `override-allocator` to revert to the system allocator even if there is a global rule specifying a different allocator.

Remap `jemalloc = false` to `override-allocator = "system"` instead of mapping it to "override-allocator is unset"; this restores the correct precedence behaviour of this directive.

This also allows `Config::override_allocator()` to always resolve the directive to a definite value, rather than needing it to be wrapped in an `Option`, which simplifies some other code.

Finally, print a more specific warning when `jemalloc` is set, telling users the appropriate setting to use for `override-allocator` depending on whether `jemalloc` is set to true or false.

Fixes rust-lang#160084
JonathanBrouwer added a commit to JonathanBrouwer/rust that referenced this pull request Jul 31, 2026
…r=Kobzol

Add "system" option to `override-allocator` directive

This allows a per-target `override-allocator` to revert to the system allocator even if there is a global rule specifying a different allocator.

Remap `jemalloc = false` to `override-allocator = "system"` instead of mapping it to "override-allocator is unset"; this restores the correct precedence behaviour of this directive.

This also allows `Config::override_allocator()` to always resolve the directive to a definite value, rather than needing it to be wrapped in an `Option`, which simplifies some other code.

Finally, print a more specific warning when `jemalloc` is set, telling users the appropriate setting to use for `override-allocator` depending on whether `jemalloc` is set to true or false.

Fixes rust-lang#160084
rust-bors Bot pushed a commit that referenced this pull request Jul 31, 2026
…uwer

Rollup of 21 pull requests

Successful merges:

 - #160100 (Add "system" option to `override-allocator` directive)
 - #159675 (rustc: Support `--jobs` options for limiting parallelism in various parts of the compiler)
 - #159999 (Fix invalidation of stdlib in bootstrap when using non-LLVM codegen backends)
 - #160233 (Bubble bad path error while parsing field to avoid unecessary second error)
 - #160272 (rustc_metadata: Move native library search code to `rustc_codegen_ssa`)
 - #154202 (rustfmt: Format `cfg_select!`)
 - #158835 (rustc_passes: lint unused `#[path]` attributes on inline modules)
 - #159520 (Suggest `Vec<T>` instead of `[T]`)
 - #160034 (Move "macro only" check for `#[allow_internal_unsafe/unstable]` to attribute parser)
 - #160066 (rustc_middle: lint attribute cleanups)
 - #160085 (Remove various superfluous lint attributes)
 - #160113 (Coalesce `rustc_on_unimplemented` attributes and lint malformed filters)
 - #160119 (fix query cycle in `coroutine_hidden_types` for the next solver)
 - #160147 (tests: Remove `-Zthreads` options from tests in `ui/parallel-rustc`)
 - #160157 (Remove outdated comments from `va_list.rs`)
 - #160159 (More accurately check for interior mutability in `invalid_reference_casting` lint)
 - #160208 (rustdoc: Fix crash when trying to list attributes on an opaque type)
 - #160244 (Rename splat to rustc_splat in error messages)
 - #160246 (dont fire `unused_mut` on `&pin mut self`)
 - #160247 (Configure backport nominations for rustfmt)
 - #160274 (renovate: update lock files weekly)
rust-bors Bot pushed a commit that referenced this pull request Jul 31, 2026
…uwer

Rollup of 22 pull requests

Successful merges:

 - #160100 (Add "system" option to `override-allocator` directive)
 - #160220 (Refactor: shrink region ext traits)
 - #159675 (rustc: Support `--jobs` options for limiting parallelism in various parts of the compiler)
 - #159999 (Fix invalidation of stdlib in bootstrap when using non-LLVM codegen backends)
 - #160233 (Bubble bad path error while parsing field to avoid unecessary second error)
 - #160272 (rustc_metadata: Move native library search code to `rustc_codegen_ssa`)
 - #154202 (rustfmt: Format `cfg_select!`)
 - #159520 (Suggest `Vec<T>` instead of `[T]`)
 - #159710 (Add rustdoc/cargo PGO profiles to reproducible artifacts)
 - #160034 (Move "macro only" check for `#[allow_internal_unsafe/unstable]` to attribute parser)
 - #160066 (rustc_middle: lint attribute cleanups)
 - #160085 (Remove various superfluous lint attributes)
 - #160113 (Coalesce `rustc_on_unimplemented` attributes and lint malformed filters)
 - #160119 (fix query cycle in `coroutine_hidden_types` for the next solver)
 - #160147 (tests: Remove `-Zthreads` options from tests in `ui/parallel-rustc`)
 - #160157 (Remove outdated comments from `va_list.rs`)
 - #160159 (More accurately check for interior mutability in `invalid_reference_casting` lint)
 - #160208 (rustdoc: Fix crash when trying to list attributes on an opaque type)
 - #160244 (Rename splat to rustc_splat in error messages)
 - #160246 (dont fire `unused_mut` on `&pin mut self`)
 - #160247 (Configure backport nominations for rustfmt)
 - #160274 (renovate: update lock files weekly)
@rust-bors
rust-bors Bot merged commit 124f594 into rust-lang:main Jul 31, 2026
13 checks passed
rust-timer added a commit that referenced this pull request Jul 31, 2026
Rollup merge of #160100 - fs-rachel:fix-jemalloc-directive, r=Kobzol

Add "system" option to `override-allocator` directive

This allows a per-target `override-allocator` to revert to the system allocator even if there is a global rule specifying a different allocator.

Remap `jemalloc = false` to `override-allocator = "system"` instead of mapping it to "override-allocator is unset"; this restores the correct precedence behaviour of this directive.

This also allows `Config::override_allocator()` to always resolve the directive to a definite value, rather than needing it to be wrapped in an `Option`, which simplifies some other code.

Finally, print a more specific warning when `jemalloc` is set, telling users the appropriate setting to use for `override-allocator` depending on whether `jemalloc` is set to true or false.

Fixes #160084
@rustbot rustbot added this to the 1.99.0 milestone Jul 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-CI Area: Our Github Actions CI A-meta Area: Issues & PRs about the rust-lang/rust repository itself A-rustc-dev-guide Area: rustc-dev-guide A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

jemalloc / override-allocator bootstrap option cannot be un-set for a specific target if set globally

6 participants