Now that uv’s MSRV is Rust 1.94, use clearer `Option` and `Result`
predicate helpers where it makes sense.
This replaces `map(...).unwrap_or(...)` probes with APIs such as
`is_some_and`, `is_none_or`, and `is_ok_and`, preserving existing
ownership and `None`/`Err` behavior.
## Summary
While centralising the whole environment, I noticed that this code was
unnecessarily complicated so I deleted it.
Using `remove_dir` instead of `junction::delete` also fixes a bug of
sorts if we were to actually encounter a regular directory symlink on
windows.
`junction::get_target` has this weird quirk where it will produce
non-verbatim long paths. This means that prefix comparisons will work
assuming the prefix isn't a verbatim path (so actually this is a bug
disguised as a feature).
To avoid the opposite issue in this case, it's necessary to verbatim
both paths.
## Test Plan
Existing coverage, some augmentations, and a targeted test for a
verbatim_path idiom.
## Summary
This PR runs [hawk](https://github.com/astral-sh/hawk) over the uv
codebase to:
1. Remove dead code (i.e., code marked as `pub` that isn't called by any
other crates).
2. Reduce visibility of symbols (i.e., to make code non-`pub` if only
called internally, etc.).
I skimmed through the changes and added exceptions where they felt
warranted (to `hawk.toml`). I've included the `hawk.toml` file in the
prior commit, but am excluding it from being committed, since it's not
at a point that we have automated tooling.
## Summary
Previously installing a windows python on linux failed, because the
windows python install does not have a `python` binary and it does not
use a `_sysconfigdata_*` file.
## Test Plan
Manually tested by running `uv python install
cpython-3.12.3-windows-x86_64-none`
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
This is a Codex-driven PR to remove as many `pub` interfaces as we can,
and then fixing the resulting Clippy violations. (I then read the diff
and reverted some changes.)
We use `pub` to expose members across crates within our workspace, even
though we don't intend each of those `pub` members to be usable API --
we are largely our only caller. (I've explicitly retained pieces that we
added to our API based on prior requests from other users.) As a result,
we can remove >1,000 unused LOC.
## Summary
We need to normalize any relative managed-Python install roots before
checking whether the active environment’s interpreter is uv-managed, so
that `sync --active` reuses the environment.
Closes https://github.com/astral-sh/uv/issues/16631.
<!--
Thank you for contributing to uv! To help us out with reviewing, please
consider the following:
- Does this pull request include a summary of the change? (See below.)
- Does this pull request include a descriptive title?
- Does this pull request include references to any relevant issues?
-->
## Summary
<!-- What's the purpose of the change? What does it do, and why? -->
bug fix, in find version 3.1 should not match 3.10
## Test Plan
<!-- How was it tested? -->
---------
Co-authored-by: konstin <konstin@mailbox.org>
Co-authored-by: Zanie Blue <contact@zanie.dev>
Includes a few things...
- Drops preview warnings for use of `uv python upgrade` and `uv python
install --upgrade`
- Adds `--resolve-links` to `uv python find`, which I needed in test
cases to retain existing snapshots
- Fixes issues in our "Using environment ..." messages on Windows which
were incorrect
- Refactors `from_executable` for the `PythonMinorVersionLink` type
(https://github.com/astral-sh/uv/pull/17842/commits/28b2ed2525327d94fdf5372a29bbbc476d74680f)
to use the type system to prevent incorrect construction (for above)
- Removes special casing where we only upgrade links if they already
exist, which existed so preview wasn't needed on every invocation
- Fixes a bug with `PythonMinorVersionLink::exists` which returned
`true` even if the link pointed to the wrong Python installation leading
to discovery failures
## Summary
This PR replaces `bitflags` in favour of `enumflags2` (which we already
transitively depended on) so that `PreviewFeatures` can be replaced with
`PreviewFeature` which is an enum. This clarifies intent in cases where
we only care about one specific `PreviewFeature`.
To avoid a bunch of boilerplate changes, the `Preview` wrapper has been
kept and creation now involves a `&[PreviewFeature]` in all cases. The
alternative was to have everything which initialises a `Preview` use
`BitFlags` directly and possibly to remove `Preview` entirely but this
keeps things simpler and limits the changes throughout the rest of the
codebase solely to changes which deal with the name changes (ALL_CAPS to
PascalCase) and the impact on `--show-settings` which I don't believe we
care about stability of output for?
The changes to `--show-settings` could be avoided with some custom
`Debug` implementation but that seems excessive.
This PR will impact #16452. But the changes were inspired by trying to
remove the need for that particular PR to add more runtime type
checking.
## Test Plan
Existing tests were adjusted (I also fixed some missing cases). The test
for panicking in cases which are now prevented through the use of type
changes has been dropped. All the rest of the tests were ran, snapshot
changes reviewed and applied.
When a process is running and another calls `uv cache clean` or `uv
cache prune` we currently deadlock - sometimes until the CI timeout
(https://github.com/astral-sh/setup-uv/issues/588). To avoid this, we
add a default 5 min timeout waiting for a lock. 5 min balances allowing
in-progress builds to finish, especially with larger native
dependencies, while also giving timely errors for deadlocks on (remote)
systems.
Commit 1 is a refactoring.
This branch also fixes a problem with the logging where acquired and
released resources currently mismatch:
```
DEBUG Acquired lock for `https://github.com/tqdm/tqdm`
DEBUG Using existing Git source `https://github.com/tqdm/tqdm`
DEBUG Released lock at `C:\Users\Konsti\AppData\Local\uv\cache\git-v0\locks\16bb813afef8edd2`
```
`.rsrc` is the idiomatic way of storing metadata and non-code resources
in PE
binaries. This should make the resulting binaries more robust as they
are no longer
dependent on the exact location of a certain magic number.
Addresses: #15022
## Test Plan
Existing integration test for `uv-trampoline-builder` + addition to
ensure robustness
to code signing.
---------
Co-authored-by: samypr100 <3933065+samypr100@users.noreply.github.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
Turns out if the minor versions matched we returned false from
`is_upgrade_of` instead of continuing to compare prerelease versions.
Closes#15955.
Note: test cases were initially generated by Claude - I tried making
them shorter.
## Test plan
```
❯ cargo run -- -v python upgrade 3.14
[...]
DEBUG Inspecting existing executable at `/Users/zsol/.local/bin/python3.14`
DEBUG Replacing existing executable for `cpython-3.14.0rc2-macos-aarch64-none` at `/Users/zsol/.local/bin/python3.14` with executable for `cpython-3.14.0rc3-macos-aarch64-none` since it is an upgrade
DEBUG Updated executable at `/Users/zsol/.local/bin/python3.14` to cpython-3.14.0rc3-macos-aarch64-none
Installed Python 3.14.0rc3 in 5.04s
+ cpython-3.14.0rc3-macos-aarch64-none (python3.14)
[...]
❯ uvx python3.14 -V
Python 3.14.0rc3
```
The initial motivation for this change was that we were using both the
`windows`, the `window_sys` and the `windows_core` crate in various
places. These crates have slightly unconventional versioning scheme
where there is a large workspace with the same version in general, but
only some crates get breaking releases when a new breaking release
happens, the others stay on the previous breaking version. The `windows`
crate is a shim for all three of them, with a single version. This
simplifies handling the versions.
Using `windows` over `windows_sys` has the advantage of a higher level
error interface, we now get a `Result` for all windows API calls instead
of C-style int-returns and get-last-error calls. This makes the
uv-keyring crate more resilient.
We keep using the `windows_registry` crate, which provides a higher
level interface to windows registry access.
Allows pinning the Python build version via environment variables, e.g.,
`UV_PYTHON_CPYTHON_BUILD=...`. Each variable is implementation specific,
because they use different versioning schemes.
Updates the Python download metadata to include a `build` string, so we
can filter downloads by the pin. Writes the build version to a file in
the managed install, e.g., `cpython-3.10.18-macos-aarch64-none/BUILD`,
so we can filter installed versions by the pin.
Some important follow-up here:
- Include the build version in not found errors (when pinned)
- Automatically use a remote list of Python downloads to satisfy build
versions not present in the latest embedded download metadata
Some less important follow-ups to consider:
- Allow using ranges for build version pins
## Summary
This is causing some cyclic dependencies issues for me, because these
can be used in virtually _any_ crate (like `uv-install-wheel`), which
then means that all of `uv-configuration` becomes a dependency, etc. I
think this should be a leaf crate so that we can safely depend on it
anywhere.
We should not unnecessarily leak memory. Instead, we follow the general
patterns and use `Cow` for strings that can be from either a static or a
dynamic source.
## Summary
Make the use of `Self` consistent. Mostly done by running `cargo clippy
--fix -- -A clippy::all -W clippy::use_self`.
## Test Plan
<!-- How was it tested? -->
No need.
I think this would give us better hygiene than a global flag. It makes
it easier for users to opt-in to overlapping features, such as Python
upgrades and Python bin installations and to disable warnings for
preview mode without opting in to a bunch of other features. In general,
I want to reduce the burden for putting something under preview.
The `--preview` and `--no-preview` flags are retained as global
overrides. A new `--preview-features` option is added which accepts
comma separated features or can be passed multiple times, e.g.,
`--preview-features add-bounds,pylock`. There's a `UV_PREVIEW_FEATURES`
environment variable for that option (I'm not sure if we should overload
`UV_PREVIEW`, but could be convinced).
Motivated by some code duplication highlighted in
https://github.com/astral-sh/uv/pull/14201, I noticed we weren't taking
advantage of the existing implementation for casting to a str here.
Unfortunately, we do need a special case for CPython still.
> NOTE: The PRs that were merged into this feature branch have all been
independently reviewed. But it's also useful to see all of the changes
in their final form. I've added comments to significant changes
throughout the PR to aid discussion.
This PR introduces transparent Python version upgrades to uv, allowing
for a smoother experience when upgrading to new patch versions.
Previously, upgrading Python patch versions required manual updates to
each virtual environment. Now, virtual environments can transparently
upgrade to newer patch versions.
Due to significant changes in how uv installs and executes managed
Python executables, this functionality is initially available behind a
`--preview` flag. Once an installation has been made upgradeable through
`--preview`, subsequent operations (like `uv venv -p 3.10` or patch
upgrades) will work without requiring the flag again. This is
accomplished by checking for the existence of a minor version symlink
directory (or junction on Windows).
### Features
* New `uv python upgrade` command to upgrade installed Python versions
to the latest available patch release:
```
# Upgrade specific minor version
uv python upgrade 3.12 --preview
# Upgrade all installed minor versions
uv python upgrade --preview
```
* Transparent upgrades also occur when installing newer patch versions:
```
uv python install 3.10.8 --preview
# Automatically upgrades existing 3.10 environments
uv python install 3.10.18
```
* Support for transparently upgradeable Python `bin` installations via
`--preview` flag
```
uv python install 3.13 --preview
# Automatically upgrades the `bin` installation if there is a newer patch version available
uv python upgrade 3.13 --preview
```
* Virtual environments can still be tied to a patch version if desired
(ignoring patch upgrades):
```
uv venv -p 3.10.8
```
### Implementation
Transparent upgrades are implemented using:
* Minor version symlink directories (Unix) or junctions (Windows)
* On Windows, trampolines simulate paths with junctions
* Symlink directory naming follows Python build standalone format: e.g.,
`cpython-3.10-macos-aarch64-none`
* Upgrades are scoped to the minor version key (as represented in the
naming format: implementation-minor version+variant-os-arch-libc)
* If the context does not provide a patch version request and the
interpreter is from a managed CPython installation, the `Interpreter`
used by `uv python run` will use the full symlink directory executable
path when available, enabling transparently upgradeable environments
created with the `venv` module (`uv run python -m venv`)
New types:
* `PythonMinorVersionLink`: in a sense, the core type for this PR, this
is a representation of a minor version symlink directory (or junction on
Windows) that points to the highest installed managed CPython patch
version for a minor version key.
* `PythonInstallationMinorVersionKey`: provides a view into a
`PythonInstallationKey` that excludes the patch and prerelease. This is
used for grouping installations by minor version key (e.g., to find the
highest available patch installation for that minor version key) and for
minor version directory naming.
### Compatibility
* Supports virtual environments created with:
* `uv venv`
* `uv run python -m venv` (using managed Python that was installed or
upgraded with `--preview`)
* Virtual environments created within these environments
* Existing virtual environments from before these changes continue to
work but aren't transparently upgradeable without being recreated
* Supports both standard Python (`python3.10`) and freethreaded Python
(`python3.10t`)
* Support for transparently upgrades is currently only available for
managed CPython installations
Closes#7287Closes#7325Closes#7892Closes#9031Closes#12977
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Previously you could get a confusing error message like this:
```
$ docker run ghcr.io/astral-sh/uv run python
error: Could not read ELF interpreter from any of the following paths: /bin/sh, /usr/bin/env, /bin/dash, /bin/ls
```
Now the error message is better:
```
error: Failed to discover managed Python installations
Caused by: Failed to determine the libc used on the current platform
Caused by: Failed to find any common binaries to determine libc from: /bin/sh, /usr/bin/env, /bin/dash, /bin/ls
```
See https://github.com/astral-sh/uv/issues/8635.
---------
Co-authored-by: konsti <konstin@mailbox.org>
Co-authored-by: Zanie Blue <contact@zanie.dev>
Rustfmt introduces a lot of formatting changes in the 2024 edition. To
not break everything all at once, we split out the set of formatting
changes compatible with both the 2021 and 2024 edition by first
formatting with the 2024 style, and then again with the currently used
2021 style.
Notable changes are the formatting of derive macro attributes and lines
with overly long strings and adding trailing semicolons after statements
consistently.
## Summary
This adds GraalPy download metadata so that `uv python install graalpy`
works. See https://github.com/astral-sh/uv/issues/13114
## Test Plan
The existing integration test was changed to test this functionality.
## Summary
In preview mode on windows, register und un-register the managed python build standalone installations in the Windows registry following PEP 514.
We write the values defined in the PEP plus the download URL and hash. We add an entry when installing a version, remove an entry when uninstalling and removing all values when uninstalling with `--all`. We update entries only by overwriting existing values, there is no "syncing" involved.
Since they are not official builds, pbs gets a prefix. `py -V:Astral/CPython3.13.1` works, `py -3.13` doesn't.
```
$ py --list-paths
-V:3.12 * C:\Users\Konsti\AppData\Local\Programs\Python\Python312\python.exe
-V:3.11.9 C:\Users\Konsti\.pyenv\pyenv-win\versions\3.11.9\python.exe
-V:3.11 C:\Users\micro\AppData\Local\Programs\Python\Python311\python.exe
-V:3.8 C:\Users\micro\AppData\Local\Programs\Python\Python38\python.exe
-V:Astral/CPython3.13.1 C:\Users\Konsti\AppData\Roaming\uv\data\python\cpython-3.13.1-windows-x86_64-none\python.exe
```
Registry errors are reported but not fatal, except for operations on the company key since it's not bound to any specific python interpreter.
On uninstallation, we prune registry entries that have no matching Python installation (i.e. broken entries).
The code uses the official `windows_registry` crate of the `winreg` crate.
Best reviewed commit-by-commit.
## Test Plan
We're reusing an existing system check to test different (un)installation scenarios.
## Summary
For example, `cargo run python install
cpython-3.12.8-linux-x86_64_v3-gnu` (on macOS) shouldn't attempt to
patch the dylib. At present, it leads to this warning:
```
warning: Failed to patch the install name of the dynamic library for /Users/crmarsh/.local/share/uv/python/cpython-3.12.8-linux-x86_64_v3-gnu/bin/python3.12. This may cause issues when building Python native extensions.
Underlying error: Failed to update the install name of the Python dynamic library located at `/Users/crmarsh/.local/share/uv/python/cpython-3.12.8-linux-x86_64_v3-gnu/lib/libpython3.12.dylib`
```
## Summary
Fixes#10598
## Test Plan
Looking for input here @zanieb. How/where would you include tests for
this?
More broadly: do we want a failure to perform the rename to be a hard
error? Or should it start out as a warning?
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
## Summary
This PR reimplements
[`sysconfigpatcher`](https://github.com/bluss/sysconfigpatcher) in Rust
and applies it to our Python installations at install-time, ensuring
that the `sysconfig` data is more likely to be correct.
For now, we only rewrite prefixes (i.e., any path that starts with
`/install` gets rewritten to the correct absolute path for the current
machine).
Unlike `sysconfigpatcher`, this PR does not yet do any of the following:
- Patch `pkginfo` files.
- Change `clang` references to `cc`.
A few things that we should do as follow-ups, in my opinion:
1. Rewrite
[`AR`](https://github.com/bluss/sysconfigpatcher/blob/c1ebf8ab9274dcde255484d93ce0f1fd1f76a248/src/sysconfigpatcher.py#L61).
2. Remove `-isysroot`, which we already do for newer builds.
## Summary
This PR adds `--install-dir` argument for the following commands:
- `uv python install`
- `uv python uninstall`
The `UV_PYTHON_INSTALL_DIR` env variable can be used to set it
(previously it was also used internally).
Any more commands we would want to add this to?
## Test Plan
For now just manual test (works on my machine hehe)
```
❯ ./target/debug/uv python install --install-dir /tmp/pythons 3.8.12
Searching for Python versions matching: Python 3.8.12
Installed Python 3.8.12 in 4.31s
+ cpython-3.8.12-linux-x86_64-gnu
❯ /tmp/pythons/cpython-3.8.12-linux-x86_64-gnu/bin/python --help
usage: /tmp/pythons/cpython-3.8.12-linux-x86_64-gnu/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ...
```
Open to add some tests after the initial feedback.
---------
Co-authored-by: Zanie Blue <contact@zanie.dev>
Zsolt Dollenstein