Skip to content

Conversation

@neo-jesse
Copy link
Collaborator

@neo-jesse neo-jesse commented Nov 12, 2025

Migrate Python dependency management from Pipenv to UV

Summary

Migrates api and shared-data projects from pipenv to uv for faster dependency resolution and installation. All other projects continue using pipenv.

Key Changes for Api and shared-data

  • Migrated api and shared-data from Pipfile to pyproject.toml with uv.lock
  • Created scripts/python-uv.mk for UV projects; scripts/python.mk remains for pipenv projects
  • Updated GitHub Actions setup action to detect project type (UV vs pipenv) and install dependencies accordingly
  • Fixed dependencies: added opentrons-hardware[FLEX] to api dev deps, numpy to shared-data main deps
  • Fixed mypy errors (redundant casts, unused ignores, untyped decorators)
  • Migrated JSON schema validation from jsonschema.RefResolver to referencing library
  • Refactored GitHub Actions Python setup into modular actions (install-uv, cache-uv, setup)
  • Fixed Windows PATH issues for uv in Make commands
  • Added install-dev-deps: 'true' to workflows that run lint/test for UV projects

Changes for Pipenv Projects

  • Updated all pipenv Makefiles to use pipenv sync --dev to ensure dev dependencies are installed
  • Fixed workflows to properly install dev dependencies for lint/test jobs

Benefits

  • Faster dependency resolution and installation for api and shared-data
  • Standardized pyproject.toml configuration for migrated projects
  • Improved CI reliability with proper caching and dev dependency handling
  • Gradual migration path - other projects can be migrated incrementally

@codecov
Copy link

codecov bot commented Nov 12, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 61.42%. Comparing base (6ee83fc) to head (245675d).
⚠️ Report is 2 commits behind head on edge.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             edge   #20135      +/-   ##
==========================================
+ Coverage   56.06%   61.42%   +5.35%     
==========================================
  Files        3663     3284     -379     
  Lines      308848   257692   -51156     
  Branches    45926    22164   -23762     
==========================================
- Hits       173168   158277   -14891     
+ Misses     135435    99175   -36260     
+ Partials      245      240       -5     
Flag Coverage Δ
app 46.26% <ø> (+<0.01%) ⬆️
components ?
hardware ?
labware-library ?
opentrons-ai-client ?
protocol-designer 19.47% <ø> (ø)
react-api-client ?
shared-data ?
step-generation 5.67% <ø> (-0.01%) ⬇️
system-server ?
update-server ?
usb-bridge ?

Flags with carried forward coverage won't be shown. Click here to find out more.
see 1467 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@neo-jesse neo-jesse marked this pull request as ready for review November 13, 2025 17:40
@neo-jesse neo-jesse requested review from a team as code owners November 13, 2025 17:40
@neo-jesse neo-jesse requested review from ncdiehl11 and removed request for a team November 13, 2025 17:40
Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's try and do all we can to avoid changes that aren't necessary for the uv cutover. that's stuff like

  • Altering the pipenv_opts definition requiring changes to all the makefiles to push --dev up
  • Whatever is causing the typechecking to change and requiring some extra casts
  • Whatever made us reformat ot3controller.py

There's also some other little stuff:

  • is there ever a time we would not want to install dev dependencies? can we just remove that argument to the action, or invert it to no-install-dev-deps and specify it more rarely?
  • let's rely on python depspecs for the linux only dependencies
  • some code duplication around path specification i think we can pull
  • let's just double check that the versions of all the dependencies, including dev dependencies, are the same in the new uv and the old pipenv dependency specs. that will let us change fewer things at one time

Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very much getting there! A couple little nits to pick; the only big things left are some github actions stuff, the thing with the mypy configuration in api, and a weirdness with the dependencies in api.

@sfoster1 sfoster1 changed the title feat(uv, ci): Api and shared-data uv migration chore: python project migration to uv Dec 9, 2025
@neo-jesse neo-jesse force-pushed the api-and-shared-uv-migration branch from ab14763 to 433422a Compare December 9, 2025 20:25
@neo-jesse neo-jesse requested a review from a team as a code owner December 9, 2025 22:59
@neo-jesse neo-jesse changed the base branch from edge to bump-to-scarthgap December 10, 2025 14:45
@neo-jesse neo-jesse force-pushed the api-and-shared-uv-migration branch 4 times, most recently from 433422a to ff23648 Compare December 10, 2025 19:03
skowalski08 and others added 28 commits December 18, 2025 10:24
# Overview

Added playwright testing for moving Lids (TC and Universal) around the
deck manually or with the gripper.
Addresses Phase 1: RQA-4958 & Phase 2: RQA-4951

## Test Plan and Hands on Testing

E2E tests

## Changelog

Added tests to check Move Labware on Flex Protocol
Added functions that allow you to easily add move labware with or
without gripper

## Review requests

- Are selectors strong enough? (I did use codegen)
- There are two notes on test_pd_move_labware, search "###NOTE" and you
will see my comments

## Risk assessment

- Selectors likely fragile
…lCount prop (#20387)

# Overview

Removes remaining mentions of `maxPoolCount` since we can get this
property from the hopper's set stored labware

## Test Plan and Hands on Testing

ran unit tests and all passed!

## Risk assessment

low
### Skip deployment for fork PRs (Dependabot)

Problem:
Dependabot PRs (from forks) fail on deploy jobs because they can't
access secrets/OIDC tokens for AWS credentials.

Solution:
Added conditions to skip deployment jobs for fork PRs while still
running tests and builds.

Changes:
Updated deploy job conditions in 5 workflows to check if PR is from a
fork:
components-test-build-deploy.yaml
ll-test-build-deploy.yaml
pd-test-build-deploy.yaml
docs-build.yaml
docs-build-deploy.yaml

Result:
Tests and builds run on all PRs (including Dependabot)
Deployment only runs for same-repo PRs and push events
Fork PRs (Dependabot) skip deployment instead of failing

Condition added:
`(github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository)`
…for previewCaptureImage (#20379)

# Overview

Create bindings for `previewCaptureImage` and implement in the app

## Test Plan and Hands on Testing

- console logged that post request was being sent with the correct
settings:
<img width="1497" height="589" alt="Screenshot 2025-12-15 at 12 33
18 PM"
src="https://github.com/user-attachments/assets/849db6da-9b84-4fba-a280-371c90278f3f"
/>

- updated unit tests

## Review requests


## Risk assessment

medium
* docs: update python version in DEV_SETUP
… substeps (#20391)

## Overview

In our UI for showing substeps, Thermocycler profiles are peculiar in that we don't actually get data from `ThermocyclerProfileSubstepItem`. Instead, we get it from the step form. This means the properties of `ThermocyclerProfileSubstepItem` are unused by anything.

The properties of `ThermocyclerProfileSubstepItem` would need to update in #20390. Since they're unused, it seems easier to just delete them instead of updating them.

This goes towards EXEC-2009.
* refactor(components,protocol-designer): refactor Flex component
* fix(step-generation): fix white screen from def undefined
* fix(app): fix CSS Modules migration bugs buttons
# Overview

Tracks flex stacker step creation, labware loaded into the stacker with
liquids, and how many stackers are loaded per protocol.

## Test Plan and Hands on Testing

- Added log statements to liquids step to test
- Will need to smoke test stacker commands once step-generation is
complete.

Liquids log:
<img width="803" height="256" alt="Screenshot 2025-12-16 at 1 56 43 PM"
src="https://github.com/user-attachments/assets/3a2df9a5-fffb-4a14-a142-dc5b466239f1"
/>

## Risk assessment

low

closes EXEC-1546
* fix(components): fix Link component text wrap
…s is crashing (#20401)

# Overview

`<SecondStepsMoveLiquidTools>` is failing somehow because it can't find
the requested liquid class:

https://opentrons-76.sentry.io/issues/6987511353?project=4509363266387968

The liquid class we want comes from `formData.liquidClass`, so:

- There could be an invalid liquid class name in the form somehow,
- Or it's undefined,
- Or the `NONE_LIQUID_CLASS_NAME` -> `WATER_LIQUID_CLASS_NAME` fallback
is broken?

This PR tries to catch the error with a more informative error message
to help us figure out what's going on.

## Test Plan and Hands on Testing

Smoke-tested locally with a Transfer step.
Run CI tests.

## Risk assessment

Low.
# Overview

Copy fixes to protocol setup buttons.
finishes EXEC-1434

## Test Plan and Hands on Testing

smoke tested:
<img width="910" height="272" alt="Screenshot 2025-12-16 at 12 12 43 PM"
src="https://github.com/user-attachments/assets/72edfc19-5e5f-4c86-99a3-ed67f96d56b0"
/>
<img width="890" height="581" alt="Screenshot 2025-12-16 at 12 13 11 PM"
src="https://github.com/user-attachments/assets/440aa2de-1973-4253-8972-b7560c77a10e"
/>


## Changelog

- removed navigate to liquids text


## Risk assessment

low
* chore(app-shell, app-shell-odd) update electron version to v39
…ng them into a third mixed liquid (#20359)

# Overview

When multiple liquids are loaded into the same well, our current
behavior is to layer them on top of each other. This creates an unclear
UI because the shared well is always highlighted.

This PR creates a third liquid by combining the liquids in one well into
a command seperated list. The total volume is displayed.

## Test Plan and Hands on Testing
- wrote unit tests.
- smoke tested with protocol in ticket RQA-4799

## Changelog


https://github.com/user-attachments/assets/da157f7f-6f60-49cd-be24-d34dfafef780
Smoke tested with two mixed liquids, one being a mix of 3 and one being
a mix of 2
<img width="734" height="490" alt="Screenshot 2025-12-17 at 12 35 33 PM"
src="https://github.com/user-attachments/assets/8c5a31a0-a630-4329-b446-2ccb00733168"
/>


## Review requests

- I think combining liquid volume into total volume creates a confusing
experience for the user

## Risk assessment

- high changes the logic to parse through liquid loads to display in
deck setup
- this creates a different behavior than in PD - which replaces the
previous liquid if you load another one into the same well
* chore update node version
…state fns (#20388)

# Overview

Needed to update the labware state for the empty and store flex stacker
state functions 😄 small fixes to both

also introduces the `SYSTEM_LOCATION` labware location when you empty
something from the hopper and filters out that location in the dropdown
options for moveLiquid and moveLabware

## Test Plan and Hands on Testing

test that you can empty a labware on the flex stacker and the labware
disappears on the hopper
test that you can also store a labware and it moves to the hopper
location > NOTE this one does super work right now since you can't add a
labware to the shuttle location properly? Should be fixed by Nick's work
though that he is currently working on

## Changelog

some fixes with updating robot state in the 2 state updates


## Risk assessment

low
…bwareRenders` in `DeckSetupDetails` (#20410)

This PR adds a special-cased labware render for the top-most labware
group of a hopper, along with its potential highlight and labware
controls components.

known issues for followup:
- shuttle state is not properly updated with moveLabware step
…lor for web (#20405)

* fix(components, app): fix pipette container chip and add backgroundColor for we
…ow keeping track of ongoing profiles (#20358)

## Overview

This is another step towards EXEC-2009.

On the road to getting step-generation to support concurrent
Thermocycler profiles, this lets it represent the fact that a profile is
currently in progress (i.e., a profile has been started and has not yet
been awaited).

## Changelog

`ThermocyclerModuleState` is expanded:

* The `blockTargetTemp` property is generalized. It's now
`currentBlockActivity`.
  
  Formerly, `blockTargetTemp` could only represent "the block is targeting
a temperature" and "the block is deactivated." It can now additionally
represent "the block is running a Thermocycler profile."
  
  Nothing sets the state to "the block is running a Thermocycler profile"
yet, nor does anything look for it. We're just updating the type to
allow it.

* Not only do we store the fact that a profile is running, but also the
steps of the profile, and the Protocol Engine task ID that it was
started with. In subsequent PRs, this information will help us when we
process `waitForTask` commands and need to update the Thermocycler state
to end the profile.

* A new property, `numProfilesStarted`, counts the number of times a
profile has been started on the Thermocycler. This is going to be used
to generate unique Python variable names, like `task_1 =
thermocycler.start_execute_profile(...)`, `task_2 =
thermocycler_start_execute_profile(...)`, etc.
# Overview

Enables view labware hover text for labware in the hopper at the end of
the run

Also updated css style to ensure you can scroll through the list of
labware. Prior to this change, the labware list got cut off.

## Test Plan and Hands on Testing

- smoke tested labware viewing at start of run and end of run


https://github.com/user-attachments/assets/6dcdd48f-da70-44b4-8d4e-43096e60015c


## Risk assessment

low

Closes EXEC-2140

---------

Co-authored-by: Jethary Alcid <[email protected]>
…e for move labware (#20412)

This PR wires up stacker shuttle state updates for labware stack moves
to and from the flex stacker shuttle.

Closes EXEC-2145
* fix(app): fix the footer height for RobotUpdateProgressModal
Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do it!!!!

@neo-jesse neo-jesse merged commit 80298ff into edge Dec 18, 2025
122 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.