Add UV/WSL2 build support for pip-based workflows #2588
+1,008
−38
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add support for HSIM_NO_SUBMODULE_UPDATE and HSIM_SKIP_CMAKE_BUILD environment variables to allow more flexible build workflows. HSIM_NO_SUBMODULE_UPDATE=1: Skip git submodule update step, useful when submodules have local patches or custom commits. HSIM_SKIP_CMAKE_BUILD=1: Reuse existing build artifacts when packaging, avoiding full rebuild when creating wheels from already-built binaries. This is particularly useful for: - UV-based development workflows without conda - Creating wheels from manually-configured CMake builds - Preserving local modifications to submodules Related to UV/pip installation support for WSL2 and Linux environments.
Add HSIM_DISABLE_CUDA_DEVICE environment variable to bypass CUDA device enumeration in EGL context initialization. This is essential for WSL2 environments where CUDA device detection can fail or cause issues with D3D12-based GPU rendering. When HSIM_DISABLE_CUDA_DEVICE is set to any value, the EGL context will skip setCudaDevice() and use the default device, allowing proper initialization with Mesa D3D12 backend on WSL2. Usage: HSIM_DISABLE_CUDA_DEVICE=1 python your_script.py This enables GPU-accelerated rendering on WSL2 with NVIDIA drivers via the D3D12 backend (GALLIUM_DRIVER=d3d12).
Add documentation for the <cstdint> include patches required in the OpenEXR submodule to compile with modern C++ compilers. The patches cannot be committed directly since they're in a submodule, but this document provides instructions for applying them manually.
Add detailed documentation for building Habitat-Sim wheels for UV/pip workflows on WSL2 with GPU support. Includes: - Complete build instructions from scratch - Environment variable reference - WSL2 GPU setup and usage - Integration with TBP Monty project - Known limitations and compatibility notes
- Add uv.lock for reproducible Python dependencies - Ignore build-headless/ directory in .gitignore 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
|
Hi @killerapp! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at [email protected]. Thanks! |
Changes: - Renamed package to 'habitat-sim-uv-wheels-experimental' for PyPI - Added PyPI-compatible metadata (license, URLs, description) - Configured setup.cfg to avoid license-file metadata issues - Added CLAUDE.local.md documenting TBP Monty integration Published wheel: https://pypi.org/project/habitat-sim-uv-wheels-experimental/ This allows pip/UV users to install habitat-sim without conda: pip install habitat-sim-uv-wheels-experimental Built for TBP Monty (Thousand Brains Project) sensorimotor learning. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
… set - Change version to 0.2.2a1 (alpha release) - Fix setup.py to copy compiled .so files to wheel when skipping CMake build - Add GitHub Actions workflow for PyPI publishing - Wheel now includes habitat_sim_bindings.cpython-38-x86_64-linux-gnu.so (442MB) This fixes the missing bindings issue where wheels built with HSIM_SKIP_CMAKE_BUILD=1 only contained Python source files. The compiled C++ extension is now properly included in the package. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
- Create automated patch script to fix cstdint includes in OpenEXR - Update workflow to apply patches after checkout - Fixes compilation errors with modern C++ compilers (GCC 13+)
- Cache entire build directory with compiled dependencies - Cache key based on source file hashes for automatic invalidation - Excludes wheel build artifacts (bdist, lib dirs) - Reduces build time from ~15min to ~2min on cache hits
The full CMake build happens in a previous step with python setup.py. The wheel build step should always reuse those artifacts with HSIM_SKIP_CMAKE_BUILD=1 to avoid CMake version compatibility issues with uv's isolated build environment.
- Add custom bdist_wheel class to generate manylinux_2_31_x86_64 tag - PyPI requires manylinux tags for Linux wheels, not generic linux_x86_64 - Compatible with Ubuntu 20.04+ (glibc 2.31+)
- Copy .so files to source tree before building wheel - Use setup.py bdist_wheel instead of uv build (avoids isolated env) - Add _ext directory to MANIFEST.in - Fixes 168KB wheel missing 442MB compiled bindings
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds support for building Habitat-Sim using UV/pip-based workflows on WSL2, eliminating the conda dependency. It includes build environment variables, WSL2 GPU compatibility fixes, and comprehensive documentation.
Key Changes
Build Environment Variables (
setup.py)HSIM_NO_SUBMODULE_UPDATE=1: Skip git submodule updates during build (useful for pre-patched environments)HSIM_SKIP_CMAKE_BUILD=1: Reuse existing CMake build artifacts (accelerates iterative builds)WSL2 GPU Compatibility (
src/esp/gfx/WindowlessContext.cpp)HSIM_DISABLE_CUDA_DEVICE=1: Bypass CUDA device enumeration on WSL2Comprehensive Documentation
UV_WSL2_BUILD.md: Complete guide for building with UV on WSL2OPENEXR_PATCHES.md: Required<cstdint>header fixes for OpenEXR submoduleUV Workflow Support
.gitignore: Addedbuild-headless/directoryuv.lock: Python dependency lock file for reproducible buildsWhy This PR?
Problem: Building Habitat-Sim traditionally requires conda, which can be heavyweight and incompatible with modern Python workflows using UV or pip.
Solution: This PR enables building wheels directly with UV/pip on WSL2 by:
Use Cases
Test Plan
HSIM_DISABLE_CUDA_DEVICE=1allows rendering on WSL2HSIM_NO_SUBMODULE_UPDATE=1skips submodule operationsHSIM_SKIP_CMAKE_BUILD=1reuses existing builds🤖 Generated with Claude Code