[WIP] feat: migrate pgrx extensions to crane build system #1974
+568
−492
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
yvan-sraka
force-pushed
the
feat/use-crane
branch
2 times, most recently
from
906fbe3 to
8f0dffb
Compare
yvan-sraka
force-pushed
the
feat/use-crane
branch
from
8f0dffb to
becd92c
Compare
yvan-sraka
force-pushed
the
feat/use-crane
branch
3 times, most recently
from
d95ecff to
38ead9e
Compare
These overlay packages will be removed once the extensions are updated to use the new `buildPgrxExtension` function.
The previously used version was failing to build:
/build/source/src/common/get_check_data.c: In function 'pgr_SPI_getText':
/build/source/src/common/get_check_data.c:307:28: error: passing argument 1 of 'DatumGetCString' makes integer from pointer without a cast [-Wint-conversion]
307 | return DatumGetCString(SPI_getvalue(*tuple, *tupdesc, info.colNumber));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| |
| char *
In file included from /build/source/include/c_common/postgres_connection.h:29,
from /build/source/src/common/get_check_data.c:27:
/nix/store/nn6vpjqlsxxmynwcrsib9agb3hpr5cqj-postgresql-17.4/include/server/postgres.h:335:23: note: expected 'Datum' {aka 'long unsigned int'} but argument is of type 'char *'
335 | DatumGetCString(Datum X)
| ~~~~~~^
make[2]: *** [src/common/CMakeFiles/common.dir/build.make:149: src/common/CMakeFiles/common.dir/get_check_data.c.o] Error 1
This was discussed upstream here: https://postgr.es/m/0a997455-5aba-4cf2-a354-d26d8bcbfae6%40technowledgy.de Also applied by nixpkgs here: NixOS/nixpkgs#342026
We cannot use nodejs.libv8 as it is too new for plv8 3.1. We will be able to use it with plv8 3.2.
dbmate 2.27 is now using a varchar for the version column in the schema_migrations table. See amacneil/dbmate#641
Updated postgis.nix to use nixpkgs sfcgal instead of our custom package, eliminating the need to maintain a separate sfcgal build definition.
pg_net requires an older version of curl (pinned from nixpkgs commit a76c4553d7e741e17f289224eda135423de0491d) to resolve build compatibility issues.
Updated cargo-pgrx buildPgrxExtension with rustc wrapper to handle empty postmaster_stub.rs files that cause duplicate symbol errors. (first two filenames are `` and `/build/tmp.*/postmaster_stubs/nix/store/*/bin/postmaster_stub.rs`) pg_jsonschema cargo checks fails on specific versions (0.2.0, 0.3.1, 0.3.3).
Remove deprecated functions that were removed in newer pgrouting versions (_pgr_betweennesscentrality and _pgr_contractionhierarchies).
Improves reproducibility and avoids evaluation-time network access by using a proper flake input instead of fetchTarball for the older nixpkgs needed by pg_net and plv8.
…64-darwin Version 0.19.5 has a dangling pointer issue in src/core.c:177 that causes compilation to fail on aarch64-darwin with newer clang versions. This adds -Wno-error=dangling-assignment to allow compilation to proceed. The flag is restricted to aarch64-darwin only since GCC on Linux doesn't recognize this warning option.
Add -headerpad_max_install_names linker flag to prevent 'install_name_tool: changing install names or rpaths can't be redone' errors on macOS. This allocates sufficient header space for install_name_tool to modify the shared library during post-install processing.
… extensions
Test assertions were inconsistent: expecting {version}.so and {extension_name}-{version}.so
patterns while actual libraries use {lib_name}-{version}.so (e.g. pg_partman_bgw-5.3.1.so,
timescaledb-loader-2.16.1.so). Updated PostgresExtensionTest to accept Optional[str] lib_name
parameter and use consistent {lib_name}-{version}.so assertions throughout.
Added missing timescaledb-loader.so symlink creation and updated the switch script to
maintain both symlinks when switching versions.
Fixes 'Expected timescaledb version 2.16.1, but found timescaledb-loader-2.16.1.so' and
'Expected pg_partman version 5.3.1, but found pg_partman_bgw-5.3.1.so' test failures.
Use text instead of character varying since varchar without length constraint is equivalent to text in PostgreSQL and provides better semantic meaning.
Use overlay instead of passing full nixpkgs-oldstable to avoid exposing entire old nixpkgs attribute set. Now provides specific packages (curl_8_4, v8_oldstable) through overlay.
Keep revision support that was accidentally simplified during nixpkgs update. Restores conditional logic to support building orioledb from specific git revision hashes instead of only tags.
These files were moved to subdirectories and the old .nix files should be removed since they were moved to new directory structure with default.nix files.
Remove nixpkgs-go124 dependency since current nixpkgs has Go 1.25.4 which is sufficient for packer. Remove nix-fast-build input as unused and add follows directives to resolve duplicate dependencies.
Run flake-linter to eliminate duplicate dependencies, and remove unused cargo-pgrx / rustc versions hashes.
Move pgrouting.nix to pgrouting/default.nix and groonga package to pgroonga/groonga.nix. Update imports and fix relative paths for versions.json and patch files.
Replace inline bash script generation with writeShellScriptBin for rustc wrapper that filters empty postmaster_stub.rs arguments. Apply wrapper only for pgrx < 0.12 since issue was fixed upstream in pgrx#1435 and pgrx#1441.
Move nixpkgs-oldstable import to a let binding to avoid importing it three times...
Remove supabase-groonga from global flake packages and import it locally in pgroonga extension instead. Other components access groonga via pgroonga.passthru.groonga since groonga is only used by pgroonga.
samrose
force-pushed
the
update-nixpkgs
branch
from
95dd61a to
288b31d
Compare
yvan-sraka
force-pushed
the
feat/use-crane
branch
3 times, most recently
from
655ece0 to
4665a3f
Compare
This change adds crane support to the existing pgrx extension builder, enabling better incremental build performance and caching. The unified buildPgrxExtension.nix accepts an optional craneLib parameter and uses crane's two-phase build pattern (buildDepsOnly + buildPackage) when available, falling back to rustPlatform otherwise. The key insight was that crane's modifiedSrc derivations created platform-specific builds during evaluation, causing IFD cross-compilation failures. The solution removes unnecessary source modifications and relies on extensions' existing postPatch mechanisms to handle external Cargo.lock files. This prevents IFD issues while maintaining crane's incremental build benefits. For wrappers extension, the preConfigure script was enhanced to handle both legacy table-style and modern inline Cargo.toml dependency formats, and conditionally skips git URL modifications when using cargoVendorDir to prevent networking issues in sandboxed builds. pg_jsonschema and wrappers now use crane successfully, while pg_graphql remains on rustPlatform due to git dependency versioning complexities that can be addressed separately.
yvan-sraka
force-pushed
the
feat/use-crane
branch
from
4665a3f to
14906f4
Compare
yvan-sraka
force-pushed
the
update-nixpkgs
branch
2 times, most recently
from
4f0a68d to
328f4db
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change adds crane support to the existing
pgrxextension builder, enabling better incremental build performance and caching forpg_jsonschema,pg_graphql, andwrappersextensions. Crane separates dependency builds from main crate builds, allowing dependencies to be cached independently.The unified
buildPgrxExtension.nixnow accepts an optionalcraneLibparameter. When provided, it uses crane's two-phase build (buildDepsOnly+buildPackage). Otherwise, it falls back torustPlatform.buildRustPackage. Both paths share the same build and install logic, avoiding code duplication.Two compatibility issues between crane and
rustPlatformare handled. First, crane requiresCargo.lockat the source root whilerustPlatformaccepts externallockFilepaths. The builder copies external lock files into the source directory when using crane. Second, crane'svendorCargoDepsusesbuiltins.fetchGitwhich only fetches the default branch, failing when git dependencies reference non-default branches. The builder usesrustPlatform.importCargoLockwithallowBuiltinFetchGitinstead, which searches all refs.Extensions opt into crane builds by passing
useCrane = truetomkPgrxExtension. All existing build parameters remain compatible with both backends.