"cargo install" apparently ignores "Cargo.lock" as opposed to "cargo build"

I've encountered a strange issue today.
Background: The percent-encoding crate recently made a new major release. Knowing that stuff could break I held off on updating the crate because I haven't yet had time to check what changed.
Problem
When I compile my binary crate using cargo build (with and without --release) everything builds fine. However, if I compile my binary using cargo install --path . -f, compilation fails with an error related to percent-encoding (the breaking change warranting the major version bump I assume..). In the output I see that it compiled percent-encoding v2.0.0 instead of percent-encoding v1.0.1 as specified in Cargo.lock. (Note: The Cargo.toml doesn't specify a specific version.)
The crate is the following, if you want to try at home:
https://github.com/df5602/bingers
Steps

$ cargo clean
$ cargo build

...
Compiling percent-encoding v1.0.1
...
Finished dev [unoptimized + debuginfo] target(s) in 48.32s

$ cargo build --release

...
Compiling percent-encoding v1.0.1
...
Finished release [optimized] target(s) in 1m 25s

$ cargo install --path . -f

  Installing bingers v0.1.0 (/path/to/bingers)
    Updating crates.io index
   Compiling percent-encoding v2.0.0
   Compiling bingers v0.1.0 (/path/to/bingers)
error[E0432]: unresolved import `percent_encoding::QUERY_ENCODE_SET`
  --> src/tvmaze_api.rs:10:45
   |
10 | use percent_encoding::{utf8_percent_encode, QUERY_ENCODE_SET};
   |                                             ^^^^^^^^^^^^^^^^ no `QUERY_ENCODE_SET` in the root

error: aborting due to previous error

I can work around it by specifying an explicit version in Cargo.toml, but the observed behaviour was surprising to me...
Notes
Output of cargo version:
cargo 1.36.0 (c4fcfb725 2019-05-15)
OS: Mac

This has been changed in 1.37 (beta).  Starting in that version the --locked flag can be used to force cargo to use the Cargo.lock file.
The intent is that cargo install will behave the same whether it is from a registry or a local path.
We discussed this a bit (https://github.com/rust-lang/cargo/pull/6840#pullrequestreview-226227411 is most relevant).  Ideally Cargo would issue a message with a hint to try --locked, but I'm not so confident that would work well. I think since it can result in any error, it wouldn't be able to know if --locked would actually fix it, so I think it would be misleading too often.

I'm using cargo 1.36 (stable) which shouldn't show that behaviour yet, right? (Running cargo install with --locked worked, though...)

Reading up on the linked issue: if I understand correctly, going forward, the behaviour will be different between cargo build and cargo install regarding dependency resolution (at least for binary crates)? If yes, I would consider that as very surprising! (So, what's the Cargo.lock for?

I know, I should probably fix my Cargo.toml and specify proper versions instead of using "*". But until now, it never really mattered for binary crates. Updating dependencies was always a manual process, so that you can deliberately audit the changes if you wish to do so (for me it was basically running cargo outdated, check the listed crates and look for the changelog, and eventually running cargo update -p xyz).
If that behaviour changes, I would expect at least a warning à la "Warning: Ignoring lock file!" and probably a PSA in the Rust release notes.

1. Running `cargo install` with `--locked` worked, though...

Ah, yea, in that sense there is no change in behavior.  I mis-remembered how it used to work.  The present stable behavior for install --path is that Cargo.lock is ignored by default, and --locked can be used to honor it.  That hasn't changed.  (What did change is installing from a registry, which didn't support lock files at all.)
It's difficult to balance the defaults here.  I think it would be confusing if cargo used Cargo.lock differently based on the source.  And for now we don't want to default to using Cargo.lock for installing from a registry.  I think that's up for debate on changing it, but it was the conservative choice we made for now.  If we ever decide to change it, we would probably want to add a flag to override it, and that adds more complexity.

Ah ok, so I guess it was the first time I ran into this because I deliberately held off on updating a dependency (due to breaking changes) which happens quite rarely...
It's good to know that this is the default behaviour, but I think this should be a lot better documented, because it's highly unintuitive and surprising:
Until now, my mental model of cargo install --path was basically "run cargo build --release and copy the binary somewhere". But apparently that's wrong, it should be "compile the binary slightly differently than cargo build --release and copy the binary somewhere".
I think it's the wrong default for binary crates because it means that you're deploying different binaries than you're testing! (In my opinion it would be ok, if cargo build would ignore lockfiles by default and you could explicitly opt-in to using lockfiles. But currently the default is otherwise.)

I think this should be a lot better documented

There's a fairly lengthy paragraph on https://doc.rust-lang.org/nightly/cargo/commands/cargo-install.html that explains how cargo install handles Cargo.lock (this is new for 1.37, so it hasn't hit stable, yet).  If there's any part that could be clarified let us know, we're always eager to improve the documentation.

I assume this will also be part of the cargo install --help page? I think the behaviour is described very clearly in the updated help page.
Given that the described behaviour is apparently expected behaviour, I think we can close this issue?

https://doc.rust-lang.org/nightly/cargo/commands/cargo-install.html has some misleading documentation under "Manifest Options" for --frozen / --locked:

The --frozen flag also prevents Cargo from attempting to access the network to determine if it is out-of-date.

This is untrue for cargo install.  In fact, I don't think this can ever succeed for install unless the crate has no dependencies.  Maybe not even then.

These may be used in environments where you want to assert that the Cargo.lock file is up-to-date (such as a CI build) or want to avoid network access.

This is inaccurate for cargo install.  If anything, you'd use these flags to use an "out-of-date" lock file - in the sense that all the documentation suggesting cargo install xyz will be using different versions.

If the lock file is missing, or it needs to be updated ...

This is technically accurate, but the way it implies it might not need to be updated is kinda confusing, as which crates are used will always be updated without these flags - for the install build at least - regardless of if they "need to be" or not.
If this documentation is automatically shared with other commands, it might at least be worth breifly mentioning cargo install being a special case.  Maybe something like:

Unlike cargo build, cargo install will ignore Cargo.lock files completely without this flag."

?

Ideally Cargo would issue a message with a hint to try --locked, but I'm not so confident that would work well.

Instead of trying to guess when an error can be fixed by --locked, it might be useful to instead warn if Cargo.lock is present, and specifies different versions than are being used, since that's just begging for version confusion and building everything twice needlessly.  --locked is one possible fix, but cargo update is another - since presumably most docs suggest cargo install xyz rather than cargo install xyz --locked?

(I also suspect I landed on the "Manifest Options" section by searching the cargo install docs page for "--frozen", which caused me to skip over the earlier/better explaination of locking behavior entirely.)

I assume this will also be part of the cargo install --help page?

Not for now, unfortunately we don't share the man pages with the help text.  The long-term intent is to make it available in some fashion.

If this documentation is automatically shared

Yes, it is shared.  We can probably make it special-cased for install, but that would take a bit of restructuring.

In that case I take back what I said earlier: I don't think I'd have the idea to look-up a description of a command online, if said command also has an extensive man page, so I still consider it undocumented behaviour...
I also agree with @MaulingMonkey: If this is the desired behaviour, then it should at least generate warnings, something along those lines (modulo wordsmithing):
Warning: Ignoring `Cargo.lock` file!
Detecting version mismatch:
    | Specified in `Cargo.lock` | Used in build
------------------------------------------------
foo |                    v1.0.1 |        v1.0.3

... possibly followed by a few suggestions on how to deal with the situation (i.e. use --locked, run cargo update, etc.)
(Although I don't know the inner workings of cargo, so I can't judge how much work that would be...)

It appears that cargo install --locked doesn't honour the lockfile for a git source (with a virtual manifest, in case that's relevant).
> cargo install -Z install-upgrade alacritty --git https://github.com/thunderseethe/alacritty --rev 853619f3dd8cb3a9de25beb1ac5e8ab6358d6986 --locked
    Updating git repository `https://github.com/thunderseethe/alacritty`
  Installing alacritty v0.3.3 (https://github.com/thunderseethe/alacritty?rev=853619f3dd8cb3a9de25beb1ac5e8ab6358d6986#853619f3)
    Updating crates.io index
error: failed to compile `alacritty v0.3.3 (https://github.com/thunderseethe/alacritty?rev=853619f3dd8cb3a9de25beb1ac5e8ab6358d6986#853619f3)`, intermediate artifacts can be found at `/tmp/cargo-installHN73vr`

Caused by:
  multiple packages link to native library `freetype`, but a native library can be linked only once

package `freetype-sys v0.7.1`
    ... which is depended on by `freetype-rs v0.19.1`
    ... which is depended on by `font v0.1.0 (https://github.com/thunderseethe/alacritty?rev=853619f3dd8cb3a9de25beb1ac5e8ab6358d6986#853619f3)`
    ... which is depended on by `alacritty_terminal v0.3.3 (https://github.com/thunderseethe/alacritty?rev=853619f3dd8cb3a9de25beb1ac5e8ab6358d6986#853619f3)`
    ... which is depended on by `alacritty v0.3.3 (https://github.com/thunderseethe/alacritty?rev=853619f3dd8cb3a9de25beb1ac5e8ab6358d6986#853619f3)`
links to native library `freetype`

package `servo-freetype-sys v4.0.3`
    ... which is depended on by `freetype v0.4.1`
    ... which is depended on by `harfbuzz-sys v0.3.2`
    ... which is depended on by `harfbuzz_rs v1.0.0`
    ... which is depended on by `font v0.1.0 (https://github.com/thunderseethe/alacritty?rev=853619f3dd8cb3a9de25beb1ac5e8ab6358d6986#853619f3)`
    ... which is depended on by `alacritty_terminal v0.3.3 (https://github.com/thunderseethe/alacritty?rev=853619f3dd8cb3a9de25beb1ac5e8ab6358d6986#853619f3)`
    ... which is depended on by `alacritty v0.3.3 (https://github.com/thunderseethe/alacritty?rev=853619f3dd8cb3a9de25beb1ac5e8ab6358d6986#853619f3)`
also links to native library `freetype`

If I then install from the checkout produced it succeeds fine
> cargo install -Z install-upgrade alacritty --path ~/.cargo/git/checkouts/alacritty-abdebd0a1ed9f82f/853619f/alacritty --locked
  Installing alacritty v0.3.3 (/home/wim/.cargo/git/checkouts/alacritty-abdebd0a1ed9f82f/853619f/alacritty)
    Updating crates.io index
   Compiling autocfg v0.1.6
[...]

I've hit this in rust-analyzer a couple of times, and I would appreciate the warning. How does this interact with publish-lockfile will adding publish-lockfile = true to Cargo.toml make Cargo to use lockfile when installing? Or is the publish-lockfile feature effectively a no-op?

How does this interact with publish-lockfile?

publish-lockfile has been removed and is a no-op now.  Cargo now automatically publishes the lock file if there are any binaries in the package.  cargo install always ignores the Cargo.lock unless you specify --locked.

Let me see if I'm understanding this correctly, because lockfiles have been a pet-peeve of mine in other ecosystems (looking at you NPM).

cargo build will respect the Cargo.lock file installing only dependencies that are known to work, since those are the versions everyone is building with
cargo install will ignore the Cargo.lock file, installing the latest matching versions with respect to the Cargo.toml file, presumably because there are security or other important patching an installing user may want

This seems wrong. As far as I'm concerned cargo build is to make as cargo install is to make install (as things are typically implemented). If we have a lockfile, both operations should respect it. How else will I know that people's builds (and installs) will work!?
As the author of Rust software, I want to use cargo update to try to update my dependencies, but I might hit problems, and they'll need to be resolved before I publish my updated lockfile. Of course, an ideal release of software is tested in some sense!
On the other hand I may find myself using only dependencies who follow "semantic versioning" effectively, and thus may decide to offer users automatic updates for dependencies. In this case I'd need to lock non-conforming dependencies explicitly in the Cargo.toml moving forward.
Now, since we do allow published crates to be installed remotely, and some of these crates will become stale, it makes sense to allow new users of software correctly following semantic versioning to automatically get the updates. But this should be an opt-in process, since not all software follows semantic versioning (or is generally capable of handling automatic updates to dependencies).
I might be missing a lot of details for why this next idea won't work, but in an automatically updating setup (like one following semver), I think I'd expect to not have a Cargo.lock file at all.

I personally would be fine with switching the default cargo install behavior, as I feel reliable builds are more important than possibly getting "fixed" dependencies.  But there has been resistance to this in the past.  I think I have mitigated some of the concern by issuing warnings on yanked dependencies.  But presuming few use --locked, we don't really know what the impact of that will be.  The warning isn't very helpful, since most users will have no idea what it means or control over updating those dependencies, so I imagine it has a risk to be confusing at the least.
There's also a bit of risk that this would change current behavior, which some projects may be relying upon in some way.  I'm not sure how things like dh-cargo would be impacted, or what they would prefer.

It is surprising, indeed, and a security concern too: consider binary projects/apps that may just provide source tarballs or a repo somewhere, specially small ones within the Linux ecosystem.
End-users will download a binary package and then run cargo install. They may or may not know about Rust/Cargo. Now:

If they are lucky, the binary works as intended.
If they are not so lucky, it doesn't.
And if they are really unlucky, arbitrary nasty code just got executed.

And all this happens even if they have verified the checksums/signatures of their downloads or their checked out signed Git tags!
I would say:

At least both build and install should behave the same in all respects (except actually installing, of course). I believe this is what users of traditional build systems expect.
Both commands should default to failure if Cargo.lock needs to be modified for any reason. For binaries, they should also fail if it has to be created. This makes the behavior more consistent by default (i.e. whether Cargo.lock is there does not change behavior of build), makes the act of updating the dependencies explicit, whether via update or a shorthand, and ensures source releases work as intended.

(This is another example of "I don't want anything to change, but I also want new changes".  It's surprising how often this obviously (when expressed in these simple terms) irreconcilable conflict arises :P )

As a "simple user" I find this behavior completely unexpected and therefore breaks the most important rule of all rules when it comes to software development: https://en.wikipedia.org/wiki/Principle_of_least_astonishment
I have no idea if you have the power to change this, but I would change it even if it breaks backwards compatibility. AFAIU cargo install will strictly work more often than currently by changing the default behavior to enable --locked. Are there any cases where cargo install would not work if it used --locked?

@NickeZ I think the "case" that won't work is automatic (potential security related) updates for downstream installers. However, as I've mentioned above, I believe that should be handled differently.  This behavior surprised me too, and having some discussion on it in a now closed issue doesn't bother us.

Our team got hit by this today and we found this behavior to be very surprising. Consistency is the key here, defaulting to use lock file for something and not for others are very inconsistent / surprising / confusing.

If I'm not wrong, this behavior broke wasm-pack installation and CI pipelines for multiple people - more info - https://github.com/rustwasm/wasm-pack/issues/818#issuecomment-602274678

Ran into this as well this morning, when I switched up Docker build hosts and lost my old copy of the rust base image (I know, I know -- shame on me for not pinning the tag).  The dichotomy seems surprising to me, and I'm not sure how I feel about the argument for "cargo install will pick up the security patches" - are people regularly cargo install-ing their upstream binaries over and over again? I rarely do one-off re-installs to get updates.  Either the package ecosystem provides a mass-method (apt upgrade and friends) or I use something else with a high churn-rate (rebuilding Docker images nightly/weekly/etc.)
While a hint about the absence of --locked in the presence of Cargo.lock might be lost on some users (arguably not those who are cargo install-ing their software), it would definitely cut down on the problem I'm seeing: Docker builds breaking inexplicably.
My problem started when a tried to compile something with Diesel + UUID - cargo build (honoring the lock file) was happily installing 0.7.4 of uuid, and (on my local dev machine, outside of Docker) everything was fine.
When I went to build the Docker images to crank out a quick demo environment for a colleague, I ran into a whole bunch of errors about uuid::UUID not implementing the trait bounds needed for Diesel ORM bits (Insertable and co.).  Google turned up a (seemingly) far-fetched root cause: multiple versions of UUID.  I blew away my local compiled cache and did a new cargo build locally - just the one version, v0.7.4.  When I looked at the failing Docker build output, however, I saw that cargo (via an unlocked install) was indeed installing both v0.7.4 (my dep) and v0.8.1 (no idea where that came from).
I did some more digging, found a thing called cargo tree that shows a dependencies graph, but still couldn't trace the source of the new dep.  On a whim, I decided to change the Docker image to use cargo build instead of cargo install and it worked.
That, in turn, lead to another jaunt through the googles looking for why cargo install --path . and cargo build do different things.
(Apologies for the long-winded narrative, but this has eaten about 2h out of my day for a task with an expectation of "30s to kick off; go get a cup of tea check back later" levels of required focus).
What I personally would have liked to have seen:

A warning at the top of the log, at the beginning of cargo build's execution, saying something like "Hey there, I see you have a Cargo.lock file (at ./Cargo.lock), but are not running with --locked.  This might break some things [and here's why]."
The same warning, in the event that the installation fails.

Having the warning at both ends of the log means that (a) I see it when it breaks and there's tons of compiler errors between start-of-exec and the final exit, and (b) I see it if I run cargo install manually, before I get too far into downloading and compiling the several hundred crates that make up my project.

A warning at the top of the log, at the beginning of cargo build's execution, saying something like

Did you mean cargo install's execution?

Indeed I did. Oops!
Corrected.

are people regularly cargo install-ing their upstream binaries over and over again?

It's not exactly about regularly installing, it's more about regularly publishing.  Let's say there is a security update in a popular library. There could be thousands of binary projects relying on it directly or indirectly.  If Cargo used the original Cargo.lock, all of those binary projects would need to publish an update, otherwise all users will get the old version.
I realize it is awkward and confusing, but switching the default has its own drawbacks.  It's not really clear which is best.
Implementing the hint is on my todo list, will hopefully get to it at some point.

If Cargo used the original Cargo.lock, all of those binary projects would need to publish an update, otherwise all users will get the old version.

There are several problems with that approach:

End users that already installed their tools won't get the update (what @jhunt meant, I think), because virtually all end users are used to check if there is a new release/tag/announcement, and if not, stop right there.
End users that installed in different days may have an app that differs in behavior from someone else's build, while their --version claims it is the same. Even if they built the tool in the exact same environment. Bugs may not be reproducible, reports will be harder to parse since end users may not tell you exactly what versions were used for every dependency, etc.
Even if somehow end users get used to this approach (which not common outside Rust), projects with pre-built binaries need to re-publish anyway, so you get two different approaches to updates.

In summary: we cannot rely on cargo install to be the way to get bug fixes and security patches to end users.
However, in my opinion, the bigger problem is what I mentioned a few replies above: every time you run install, you are trusting an unknown number of maintainers, effectively downloading and running arbitrary code.

are people regularly cargo install-ing their upstream binaries over and over again?

It's not exactly about regularly installing, it's more about regularly publishing. Let's say there is a security update in a popular library. There could be thousands of binary projects relying on it directly or indirectly. If Cargo used the original Cargo.lock, all of those binary projects would need to publish an update, otherwise all users will get the old version.
I realize it is awkward and confusing, but switching the default has its own drawbacks. It's not really clear which is best.
Implementing the hint is on my todo list, will hopefully get to it at some point.

Everyone is now teaching cargo install --locked because not doing so breaks the builds. Why would I put anything else in my readmes and dockerfiles? The default leads to broken builds and support tickets.

This probably mirrors the experience of others, but cargo-udeps had cargo install broken 3 times within a period of 10 days in late February/early March:

Feb 29: https://github.com/est31/cargo-udeps/issues/53 caused by https://github.com/TeXitoi/structopt/issues/354
Mar 5: https://github.com/est31/cargo-udeps/issues/54
Mar 7: https://github.com/rust-lang/cargo/issues/7974

After the second breakage, I changed the README to suggest usage of the --locked param: https://github.com/est31/cargo-udeps/commit/bcd755d8f7c912270b3ec32cfab37c31a890aac0
Of course, @ehuss 's concerns are valid too, but the state of minor updates potentially breaking everything for tons of users isn't great either.
What about crates.io maintaining a lockfile automatically? It could update the lockfile say daily and if there were any changes, run a basic cargo check on the tool in a docker container to make sure it builds at least and only update the lockfile on success. Of course this (short lived) auto-lockfile could not be part of the package as it'd invalidate all the hashes but instead be offered separately. It could be the default option that's a compromise between safety and tested behaviour. Old lockfiles as well as build logs wouldn't have to be kept/archived for a long period of time, as if you want to pin things, you should still use --locked which would still work as it works now. Only the default would change from using an untested set of dependencies to one which has had basic verifications.

run a basic cargo check on the tool in a docker container to make sure it builds at least and only update the lockfile on success

Failing to build is not the major issue. Even if you ran tests and they had perfect coverage, the approach would still be non-deterministic and unsafe.

run a basic cargo check on the tool in a docker container to make sure it builds at least and only update the lockfile on success

Failing to build is not the major issue. Even if you ran tests and they had perfect coverage, the approach would still be non-deterministic and unsafe.

Exactly, failing the build is the good case this is equivalent to UB causing a segfault in C applications. It is immediately evident that there is something wrong. The real problem here is when your application being built and used with unknown dependencies, which is more similar to UB causing wrong data being read/written silently. I wish we had deterministic builds by default, but until then I will suggest everyone to write --locked everywhere.

If Cargo used the original Cargo.lock, all of those binary projects would need to publish an update, otherwise all users will get the old version.

This won't help anybody who already has the crate installed.
Consider this exaple:
Somebody installed crate foo 1.0.0, this pulled dependency bar 1.0.1. On the next day bar 1.0.2 has been released with important security fix.
Now people who installed foo on it's release have executable with security issue and other people cannot install foo at all because dependency xyz has broke the build.
That's issue that crate maintainers should take care of by using tools like https://github.com/RustSec/cargo-audit to detect the issue and publish fixed realease afterwards.

Just spent a few hours debugging why I can't reproduce my builds just to find this :(

This behavior broke cargo-spellcheck installation, because hunspell-sys depends on a yanked patch version of bindgen.

Another downside of ignoring Cargo.lock by default:
Yesterday, one of my dependencies accidentally released a breaking change without bumping its major version.  This broke cargo install for my binary crate (which is already annoying, because I would prefer that my users get the same dependencies as me by default).  I fixed this by publishing a new version of my crate that works with the new version of the dependency.
But if the upstream dependency maintainers later decide to yank the incorrectly-released version, that will break cargo install for my binary crate a second time (because it now builds only with the yanked version, which is not available when ignoring Cargo.lock), and I'll need to publish yet another emergency fix in order to keep my crate from having broken installs by default.

Just adding two more pieces of anecdata: this recently broke builds of two Rust CLIs that I depend on. I found it pretty surprising as an end user that checking Cargo.lock into version control is encouraged for "leaf" projects (e.g., CLIs) but isn't actually respected by default by cargo install!
One thing that makes this particularly fraught: Cargo seems to think that rc versions are SemVer compatible with each other, meaning that 1.0.0-rc.2 can be silently selected over 1.0.0-rc.1 unless the maintainer specifically pins the rc with =. As a result, a decent chunk of the Rust i18n ecosystem had a broken dependency tree for over a week.

Just adding two more pieces of anecdata: this recently broke builds of two Rust CLIs that I depend on. I found it pretty surprising as an end user that checking Cargo.lock into version control is encouraged for "leaf" projects (e.g., CLIs) but isn't actually respected by default by cargo install!
One thing that makes this particularly fraught: Cargo seems to think that rc versions are SemVer compatible with each other, meaning that 1.0.0-rc.2 can be silently selected over 1.0.0-rc.1 unless the maintainer specifically pins the rc with =. As a result, a decent chunk of the Rust i18n ecosystem had a broken dependency tree for over a week.

FWIW, making install be --locked by default would have avoided much of the fallout from https://github.com/rust-lang/cargo/issues/9101.

FWIW, making install be --locked by default would have avoided much of the fallout from https://github.com/rust-lang/cargo/issues/9101.

I ran into this today. The issue wasn't the build breaking, but cargo install unexpectedly downloading and building newer versions of dependencies when it should have been using cached build artifacts. I was very confused by the default behavior of cargo install.
I think it would be better if cargo install defaulted to using the lockfile, and a new flag was added that enabled the current behavior, something like cargo install --update-dependencies.

I wish the change suggested by @casey had been part of the 2021 edition.

--locked not being the default resulted is cargo-expand and bat builds breaking yesterday:

sharkdp/bat#1991
dtolnay/cargo-expand#122

I was astonished (as @NickeZ put it) that bat's lockfile was ignored.
Then again, a semver violation was noticed because of this. However, I would rather prefer users opting into being guinea pigs than forcing it. (e.g. by using --locked by default and --unlocked by choice)

I wonder if the solution here could be to both stop publishing Cargo.lock by default (instead, make this opt-in), and then always enable --locked implicitly in the presence of a lockfile.

@ehuss Hi! Is there a final decision on the desired behaviour/where to go from now on? :)

stop publishing Cargo.lock by default (instead, make this opt-in)

That would just be a footgun, i.e. allow breaking by default.

Frankly, as nice as it is to super easily install software from the registry, I wouldn't be offended at all by needing a new sub-command and workflow to accomplish this.
For example:
cargo fetch utc
cd utc
cargo install

I've chosen to reuse cargo-fetch here (without much previous experience with it) because it feels fitting. But something like cargo checkout could also make sense.
This mirrors the good ol' clone and make workflows we all know and love.

Cargo should definitely never secretly download different dependencies than the ones specified by the project. This is definitely a bug. Some projects vet dependencies and others need to pin specific versions because of breakage. Regardless of the reason, if a project specifies the versions of dependencies the project should be built with, Cargo should respect that.

I commented on this over a year ago, but IMO the problem isn't ignoring the lockfile itself: it's that Cargo treats certain releases as semver-compatible when it shouldn't. In particular, Cargo ignores semver for pre-releases: it treats X.Y.Z.pre.2 as a compatible upgrade for X.Y.Z.pre.1, when prereleases are explicitly allowed to break compatibility.
TL;DR: It's not Cargo's fault that cargo install fails when a package violates semver, but it is Cargo's fault when Cargo itself ignores semver. I think a good fix here would be to change the dependency resolution behavior to treat prereleases as incompatible, potentially add a flag allowing users to opt-into resolution with pre-releases, and retain the current default behavior.

it is Cargo's fault when Cargo itself ignores semver.

Cargo does not ignore semver. Matchers are not currently in the semver spec (though there is a pull request to add them that myself and the other semver maintainers have not taken the time to merge for Reasons), and Cargo implements these the same as everyone else does. Put ^1.0.0-rc.1 into https://semver.npmjs.com/, and see that it also includes rcs. The reason that this is implemented this way is that = for prereleases is not actually useful in the way that people release software and test those pre-releases.
(And I think the pre-release behavior is a red herring; as I mentioned previously, git deps are another area where this can happen quite a bit, and even outside of these more unusual constraints, anyone who releases incompatible versions that are considered compatible by the rules will cause this to happen. Most people are just good about not doing this most of the time.)

I recognize that I'm standards lawyering now, but doesn't this section of the spec imply that ^1.0.0-rc.1 ~= 1.0.0-rc.2 is incorrect?
Quote:

A pre-release version indicates that the version is unstable and might not satisfy the intended compatibility requirements as denoted by its associated normal version. Examples: 1.0.0-alpha, 1.0.0-alpha.1, 1.0.0-0.3.7, 1.0.0-x.7.z.92, 1.0.0-x-y-z.–.

Unless "might not" here is not intended to be advisory w/r/t dependency resolution. I agree that = isn't ideal for prereleases, but the current behavior is also not ideal.

And I think the pre-release behavior is a red herring

It's definitely a subcase 🙂 -- Cargo can't prevent people from making semver errors, but there are quite a few linked issues above where "matching prereleases" have caused breakage with otherwise well-behaved dependencies. IMO those are cases worth addressing, at least if my reading of the semver spec is correct (which it might not be!)

I recognize that I'm standards lawyering now,

Understood, no problems. This is a common thing that comes up when people talk about semver, you're not alone :)
The key issue is that the spec says "here's what version numbers should mean in relation to each other" and then the matchers have their own ways of matching versions together that don't necessarily mean the same things as what the spec says. They're separate things. Most of the text in the 11 points of the spec were written first, and then various matchers were developed afterwards by implementors. ~ happened, and then ^ was a refinement of that, based on how those 11 points actually played out in practice. And even then, it's not that these things being different is even a problem; you want pre-release versions to possibly break everything, because that's the point of a pre-release, but you also want your pre-releases to be tested, and so you want the ability for someone testing a pre-release to be able to automatically upgrade to the newer pre-releases when possible, to try them out.
All of this works because the lockfile (both in Cargo and in Bundler, and to some extent, npm) allow you to only upgrade when you actively choose to upgrade, which is the normal flow for updating dependencies. That cargo install doesn't respect the lockfile by default means that it has a different flow, which leads to this friction. That's why I believe that it should respect the lockfile, rather than trying to change the semantics of ^.

It's extremely annoying when there is chains of dependencies each doing a pre-release, and depending on the pre-release of the following one. Because then, basically any delay in API breaking upgrades along that chain causes breakages for users at the end, unless they have a checked in lockfile. The rust-windowing crates used to do this a few years ago and it was extremely annoying for me as downstream. The only remedy is using = but the entire chain has to do that.
Maybe this is a documentation issue, but it's one of those cases that are easy to get wrong and easy to cause lots of downstream damage, so one can argue that it's more than that.

@djc you are right, the last discussion was, while still in the general "how to avoid breakage for users" setting, offtopic.
My proposal is upthread, in the second paragraph of this comment, it got only one response as far as I can tell. The basic idea is to have some mechanism of crates.io running a CI for crates that updates their lockfiles. Not the ones in the .crate file, as that would invalidate the hash, but a separately maintained one that cargo can download.
The advantage is that non-breaking security updates would still arrive in the ecosystem, while minimizing breakage.
Building each binary crate each day would be a massive and mostly wasteful effort, but maybe the build/check frequency could change with a crate's popularity. A crate that gets downloaded less than one time per week does not need daily updates, and I'd wager this is most crates. On the other hand, my cargo-udeps crate gets hundreds of downloads per day (probably most of it non-caching CI usage), so giving it more regular updates than some crate that gets 1 download per week makes sense. There are crates with even more daily downloads.
Designing a CI system is a complicated major effort, but Rust already runs CI on crates regularly through crater, as well as through docs.rs.
There would also be the need for some kind of notification mechanism if there is a build failure. Basically replacing user reports with automated ones, but at least then it doesn't create such a bad situation for all users.
The feature is a large design space and probably deserves its own RFC. E.g. one could think about running the test suite, or not running it, or running it in a custom way, or using custom operating systems like Windows, or maybe withholding updates of special crates as they are known to be broken but are being worked at and one wants to enable updates while this occurs...

Might it make sense to add a boolean ignore-lockfile field to [install] in the Cargo configuration file that defaults to true but that users can change to false? And that could potentially have its default flipped at an edition boundary?
Note that this wouldn't help publishers at all, but it would at least allow users to choose their desired behavior so that adoption can happen piecemeal.

Not sure if this experience report is useful to anyone, but as someone not heavily involved in the Rust ecosystem and just trying to install a useful tool via cargo install --git ..., this was very puzzling to debug.

This came up in passing in Office Hours and a case come up that seemed intriguing.  For commands centered on the cargo workflow or for casual programs, the current default makes sense.  For programs more generally used, like ripgrep, they have other release channels in addition to crates.io that they have to do a new release for anyways when new dependencies come out that affect users.

For commands centered on the cargo workflow or for casual programs, the current default makes sense.

What do you mean by this? The current default will cause compile errors and other issues when new versions of dependencies are brought in. Is this a good idea for non-advanced users?

Such issues are rather rare.

Pick one:

[ ] Behavior that causes issues sometimes (rarely)
[ ] Behavior that doesn't cause issues

🤔

Don't just ignore the arguments mentioned upthread for why a --locked default can cause issues. That's not the way to have a proper discussion.

For me, the surprising behaviour is not really when I install from crates.io. It is when I run cargo build or cargo run on a local repository, it isn't the same as when I run cargo install --path .. It just feels counter intuitive.

Sorry, that was not clear.  Maybe better as "the existing trade offs apply for the cargo install side".  More so my point was that the problem I was introducing is neutral for cargo plugins and casual releases but applies for all other tools.

I ran into this today. The issue wasn't the build breaking, but cargo install unexpectedly downloading and building newer versions of dependencies when it should have been using cached build artifacts. I was very confused by the default behavior of cargo install.
I think it would be better if cargo install defaulted to using the lockfile, and a new flag was added that enabled the current behavior, something like cargo install --update-dependencies.

I ran into this again today, when a user reported that just install was failing due to an update to a dependency which introduced a build failure. I was able to fix it for the latest version by pinning the dependency to the last working version. However, this issue persists in 95 old versions of just, and backporting the fix to all of them is infeasible, so they will continue to be broken indefinitely.
There are definitely tradeoffs here, and they've been well enumerated in this thread. I don't mean to comment again without new information, but, given the amount of issues that have been caused by not respecting the lockfile by default, I feel like the tradeoffs are strongly in favor of respecting the lockfile by default.
If you search GitHub for "cargo install --locked", you'll find 6.2k results, and if you search for "cargo install --locked" path:*.md, restricting the search to markdown files, which are probably documentation, you'll find 2k results. This suggests that crate authors document cargo install --locked as the preferred way to install their crates, as well as extensively using cargo install --locked in installation scripts. This negates much of the benefits of not respecting the lockfile by default, since it seems to be very common practice to opt-out of the default behavior.
What needs to happen to drive this issue forward? I don't want to trivialize the issue, but I feel like it would be a net positive to simply change the default back to the pre cargo 1.37 behavior.
I think that the main benefit to not respecting lockfiles is for downstream packagers, e.g., linux distros and package managers like homebrew, and that those downstream packagers are in a good position to evaluate the tradeoffs appropriately, and opt-in to not respecting the lockfile. Especially since those downstream packagers often have continuous integration pipelines that check for build failures, and that they are likely to be able to report the issue to the correct project, i.e., the broken dependency, and not the parent crate, which is where issues are currently usually reported.

@casey Searching the markdown usage was brilliant, and very telling. Thank you for your well-formed comment.

If you search GitHub for "cargo install --locked", you'll find 6.2k results, and if you search for "cargo install --locked" path:*.md, restricting the search to markdown files, which are probably documentation, you'll find 2k results. This suggests that crate authors document cargo install --locked as the preferred way to install their crates, as well as extensively using cargo install --locked in installation scripts. This negates much of the benefits of not respecting the lockfile by default, since it seems to be very common practice to opt-out of the default behavior.

It suggests that some crate authors do that.  But to actually put that number into perspective we'd need some sort of baseline. Is this 5% of crate authors, 50%, 95%? We have no way of knowing.

I just noticed this blog post from August 2023 where the guidance on how to deal with Cargo.lock changed.
May be relevant
https://blog.rust-lang.org/2023/08/29/committing-lockfiles.html

If you search GitHub for "cargo install --locked", you'll find 6.2k results, and if you search for "cargo install --locked" path:*.md, restricting the search to markdown files, which are probably documentation, you'll find 2k results. This suggests that crate authors document cargo install --locked as the preferred way to install their crates, as well as extensively using cargo install --locked in installation scripts. This negates much of the benefits of not respecting the lockfile by default, since it seems to be very common practice to opt-out of the default behavior.

It suggests that some crate authors do that. But to actually put that number into perspective we'd need some sort of baseline. Is this 5% of crate authors, 50%, 95%? We have no way of knowing.

FWIW, all "cargo install" in markdown files is 52.2k ("cargo install --locked" is 2k): https://github.com/search?q=path%3A*.md+"cargo+install"&type=code
Also note that these search results miss the cases where there are other flags (or spaces) between "cargo" and "install", and "install" and "--locked". (e.g., my CLI projects recommends "cargo +stable install ... --locked")

I opened a PR proposing to change cargo install to respect lockfiles by default. Copying my comment there:

Rendered
Currently, cargo install does not respect lockfiles by default, which causes breakages when dependencies release breaking but semver-compatible versions. This RFC proposes to change cargo install to respect lockfiles by default.
There's been a lot of discussion on https://github.com/rust-lang/cargo/issues/7169 without much progress, so I thought it would be productive to open an RFC proposing changing the default behavior of cargo install to respect lockfiles by default, to provide a concrete proposal that can be discussed.
Even if this RFC is ultimately rejected, it will provide some forward progress on the issue, since we can then focus on an alternative to changing the default behavior.

I think the last point is key. If the RFC is rejected, we can then all think hard about a solution to the issue aside from changing the default behavior, perhaps introducing a --unlocked flag, and making one of --locked and --unlocked mandatory.

At the very least, Cargo and crates-io could provide more of a hint to users to use --locked when the package has a lockfile.

FWIW, I do think using locked packages should be the default behavior.
As a maintainer of some small binary packages, I  know the versions of dependencies used work exactly as expected, at least (in admittedly limited) the cases I've tried / intended for them to be used.
Regradless of the fact the "real semver" should "never break" - let's be real, it is very possible, and it is a pretty bad experience if now my package can't be installed because of a dependency. If I am not mistaken, in Node.JS, you can use a shrinkwrap (similar to lockfile) to force pinned dependencies, to ensure users effectively have "reproducible builds" - including bugs if any.
Furthermore, somewhat based on Hyrums Law , maybe my program unintentionally relies on a bug in a dependency (at least viewed as a bug by the dependencie's author) - and if they decide to fix it in a patch version then it could break my application.
There's also the extreme case of "protestware", e.g. author intentionally introduces a breaking change in patch version. Now sure, this would eventually get caught by cargo mods (I am not entirely sure whom), but if I want to protect my users, I would want my lockfile to be respected.

One important aspect of this conversation is the knock-on effects of this kind of change. While it's true that this will lead to fewer failed builds due to accidental semver violations, it will also make it more likely for semver violations to not be caught, and thus to proliferate. In other words, what were trading off is reduced user friction in the face of semver violations (build more likely to succeed out of the box) against more outdated dependencies and lower awareness of semver violations (both instances of them and the general concept).
I think any decision needs to start with an acknowledgement of this tradeoff, and then an explicit rationale for choose one set of benefits + costs over those of the other options.
Personally, I think this is a case where the occasional user frustration is unfortunate, but better than the alternative. I'd be sad to see the semver violation learning opportunity disappear, and I'd be sad to see users end up (unknowingly) with older dependencies than are available at install time.

Detecting semver violations really shouldn't be up to your users.

I'd be sad to see the semver violation learning opportunity disappear

I'd argue the opportunity is still there - but violations would be discovered by maintainer of a package when they do a dependency update, rather than the end user.
As an example, in a node project I had used rolling-rate-limiter , and locked the version. As a maintainer, I can use npm (Node's cargo , kind of) to view outdated dependencies, and if I want, update them and release a new patch version of my project. This would automatically update semver compatible dependencies  due to my explicit action .
In doing so, I discovered a breaking change in their patch version (see: https://github.com/peterkhayes/rolling-rate-limiter/issues/64) , raised the issue, and the maintainer acknowledged the error, and I think they did learn something.

At the very least, Cargo and crates-io could provide more of a hint to users to use --locked when the package has a lockfile.

Packages published with bins always have a lockfile.
We do suggest --locked when a dependency is found that isn't compatible with the toolchain you are using.  I think it would be a good quality of life improvement to also suggest --locked if there is a compilation error.  Even better if we can limit the suggestion to non-local dependencies.  I also suspect we should print the project's homepage URL on error to help connect users with where to go for support.

I still stand by my idea from four years ago of an online service that maintains a lockfile (as the default behaviour, obviously there should be overrides for both latest lockfile and --locked): https://github.com/rust-lang/cargo/issues/7169#issuecomment-614343884
It doesn't need to immediately update the crates.io maintained Cargo.lock at every single update, but if it makes an attempt, it only updates the crates.io maintained Cargo.lock if some basic checks succeeded, like cargo check. This is not a full replacement of running the testsuite or manual verification, but it's way better than what we have now where builds are breaking. Don't let perfect be the enemy of the good :).
If after an attempt to build, the build fails, it maybe can show a warning to everyone running cargo install that the last Cargo.lock update on 2023-12-03 failed so it still uses the version from 2023-11-15, with a link to the build failure log and the request to maybe contact maintainers about it.
We already run crates.io wide compilations in multiple settings like rustdoc and crater. Admittedly, unlike those two earlier methods it would build at a higher cadence, but for costs we only care about the average cadence of updates. crates that see regular downloads and recent releases can get treated differently than those with few downloads, security fixes differently from normal ones.
If the default cadence was 6 weeks, and the top 10% by downloads get 2 weeks, and the top 5% get 1 day, we'd still have an average close to 6 weeks, so roughly the same as the cost we already spend on crater, if we built every crate, but we'd only run this on the minority of binary crates.
What would also be useful would be ways for crate authors to manually re-trigger the update process, say after some fix was released, and another way to revert to the previous Cargo.lock if there was some breakage.
We can also fizzle out updates google play style to exponentially increasing numbers of users, resembling a sigmoid curve, that way if there is some other breakage, it's not immediately broken for everyone.

[a] way to revert to the previous Cargo.lock if there was some breakage.

one can make a version of this without crates.io maintained Cargo.lock: right now, binary authros are fully dependent on the dependency maintainers [sic] to release a fixed version of a crate. What if binary authors could issue (automatically expiring) version pins for a dependency of their binary crate? You can already now do something like that by depending on the dependency directly in your Cargo.toml in a way that it is pinned, but it's a bit unwieldy and requires an entire release.

@est31 Having a lockfile that automatically gets updated is just the worst of both worlds. The reason lockfiles are great because it ensures the output binary behaves exactly the same, not just that the output compiles. If you updated them automatically you'd just run into more esoteric issues that are essentially the same.
If I'm uploading a binary to crates.io, I want people to run exactly what I released. I do not want any dependencies updated, even if those are "security" updates. That should be up to me.

If I'm uploading a binary to crates.io, I want people to run exactly what I released. I do not want any dependencies updated, even if those are "security" updates. That should be up to me.

You are aware, I hope, that there are already a lot of people who have a low opinion of distribution channels that aren't Linux distro packages because of how much responsibility they place on the upstream maintainer to respond promptly to security advisories.
(i.e. "Look at how badly Docker failed on ensuring a steady supply of security updates! Flatpak and statically-linked languages like Go and rust are evil and should not be allowed!")
...and Cargo.lock doesn't have the dodge Flatpak does, that the things which are most likely to need security updates are in the runtime, not the package.
I don't think there's a simple solution to reconcile the two conflicting needs here.
Not to mention that we don't know how places that focus on giving the downstream the tools to protect themselves (eg. lib.rs) would react to packages with stale lockfiles if something like this doesn't include a mechanism for allowing Crates.io to overrule Cargo.lock pins for packages when the maintainer is going through a bad bit and isn't in a state of mind to push an update that would have just gone through without a Cargo.lock.
We could easily wind up with packages getting tarred with a bad reputation because they don't meet some externally imposed standard for how quickly security fixes must be rolled out... heck, that was one of the things that sparked a lot of fear in the earlier drafts of the E.U.'s new rules for safer software infrastructure.

You are aware, I hope, that there are already a lot of people who have a low opinion of distribution channels that aren't Linux distro packages because of how much responsibility they place on the upstream maintainer to respond promptly to security advisories.

Yes, that includes myself. But this just makes it worse, not better. If you're asking me, then this does nothing (or close to nothing) for security but has significant impact on reliability. You can't just cargo update some random abandoned project and pretend that suddenly makes it secure and up-to-date.
And it's not like cargo has a good way of updating binaries installed through it anyway. So the people most likely to be using outdated dependencies, even if upstream has already pushed a new lockfile, are the most vulnerable. Not the ones just installing it.

We could easily wind up with packages getting tarred with a bad reputation because they don't meet some externally imposed standard for how quickly security fixes must be rolled out... heck, imposing unreasonable demands for prompt security response on hobby projects was one of the things that sparked a lot of fear in the earlier drafts of the E.U.'s new rules for safer software infrastructure.

Updating the lockfile changes almost nothing about this. Security issues are introduced in binaries regularly, not just libraries. And it's also possible that a breaking change is required to fix them. If people don't update their software, it's not safe to use, that's just the nature of the beast.

You can't just cargo update some random abandoned project and pretend that suddenly makes it secure and up-to-date.

As a maintainer of some projects I don't yet feel are ready to push to crates.io, I find that GitHub Dependabot is quite good at offering me dependency security fix PRs that pass all my GitHub Actions runs while only doing the kind of semver bumping that cargo install does implicitly.
(So long as the dependency in question actually offers a minor version update for Dependabot to generate a security-update PR from, of course.)

Updating the lockfile changes almost nothing about this. Security issues are introduced in binaries regularly, not just libraries. And it's also possible that a breaking change is required to fix them. If people don't update their software, it's not safe to use, that's just the nature of the beast.

 There's no such thing as an incremental improvement." argument to me.
...especially in projects like mine, where I'm generally cautious enough about how I design my things that any vulnerabilities are far more likely to be in my dependencies than in how I'm gluing them together.
In the real world, people install unmaintained software. It's just human psychology that "I need this solved now" is a more significant psychological input than "there might be a hidden flaw in this which could bite me later".

At the very least, Cargo and crates-io could provide more of a hint to users to use --locked when the package has a lockfile.

Packages published with bins always have a lockfile.
We do suggest --locked when a dependency is found that isn't compatible with the toolchain you are using. I think it would be a good quality of life improvement to also suggest --locked if there is a compilation error. Even better if we can limit the suggestion to non-local dependencies. I also suspect we should print the project's homepage URL on error to help connect users with where to go for support.

It is nice that cargo does include a hint to try using --locked when install fails, but I was suggesting that this be a warning upfront.  And in the case of crates-io, it suggests to do cargo add or add the package to the manifest, which is of course a current issue. So it would be nice if when that was fixed, in addition to suggesting to install the crate, it could hint that the package has a lock file that can be used via --locked.

Since concrete instances of issues stemming from not respecting lockfiles are well understood and documented, it would be useful to collect information on concrete instances of the benefits of respecting lockfiles by default.
It would be very helpful to know:

When has respecting lockfiles by default prevented binary crates from being compiled and installed with bugs, when those bugs would have been prevented by using the latest semver-compatible versions of dependencies?

When has respecting lockfiles by default prevented binary crates from being compiled and installed with security vulnerabilities, when those bugs would have been prevented by using the latest semver-compatible versions of dependencies? I mention security vulnerabilities separately from bugs, since avoiding security vulnerabilities is often called out specifically as one of the benefits of not respecting lockfiles by default.

For either of these to be evidence that not respecting lockfiles is beneficial, it isn't enough for a crate a dependency to have a bug or security vulnerability, and the semver-compatible version that is selected by cargo install to not have the bug or security vulnerability. Crates often use only a subset of the features of dependencies, a bug or security vulnerability might exist in a dependency that cannot be encountered when using the compiled binary. So, it must be possible to encounter the bug when using the dependent.

Other things which would be useful to know:

How many crates on crates.io build with cargo install --locked but not with `cargo install?
Which crates have tests where one of cargo test and cargo update && cargo test passes and the other fails? This is a pretty strong signal of respecting lockfiles being good or bad. If cargo test passes but cargo update && cargo test fails, then new dependencies are introduced which break the crate. If cargo install succeeds in that case, then that's doubly bad, because cargo install may have installed a binary with buggy behavior that was introduced by cargo update.
Over all crates on crates.io, which dependencies are selected by cargo install --locked, and how do they differ from the dependencies in the lockfile? Inspecting each diff by hand would be impractical, but it would be easy to compile a database with statistics of the different versions that have been selected.

There's also the extreme case of "protestware", e.g. author intentionally introduces a breaking change in patch version. Now sure, this would eventually get caught by cargo mods (I am not entirely sure whom), but if I want to protect my users, I would want my lockfile to be respected.

Commenting on this point a bit more with the whole xz backdoor thing - if my binary package relies on a lib that got social engineered into someone being able to introduce a backdoor (and publish it as patch version), the current default cargo behavior would end up "upgrading" to the backdoored file if one of my users did cargo install without --locked.
I think this is another good case for locked to be the default, if the author of a project has a lockfile in place.

I still think studies need to be done to calculate how often the current behaviour pushes out security updates faster than the dev updates their lock files vs. how often  it prevents an exploit from being rolled out.
The xz backdoor thing is certainly dramatic, but we don't want to succumb to what I'll call argumentum ad dramatum for now. Reasoning like that leads to people being so scared of dying in an airplane that they discount how you're much more likely to die in the car on the way to the airport.

Como encuentro una dirección

I saw that there was a related RFC but it seems this issue is still somewhat active. I ran into significant problems with this trying to use cargo install from a docker container build step against an older Rust version.
It is my belief that the current behavior is broken.  Having done some small spelunking it appeared that it was only chosen back in 1.39 because there was a desire not to change away from the previous behavior.  At this point it's clearly correct to invert the default and provide an --unlocked option, but since this is controversial I want to propose an alternative:
cargo install should default to the --locked behavior when specifying a specific version with the --version argument.  This gives users just running it from command line to install software access to both the security fixes and semver breakages they know and love, while giving people trying to install an older version for any reason some slim hope of a compile that succeeds.
I want to briefly mention that the problem of building an older version is compounded by MSRV which is not respected by cargo when choosing the specific dependency to build against.  That's a separate issue.  I only mention it because the specific task of building anything against an older Rust version is a major pain point with the tooling right now, and making this small change to cargo install will go a long way towards helping.

Here we can see our own infra team being surprised by npm not using a lockfile when installing binaries, and this is causing issues with CI failures right now. The same arguments would also apply to cargo.

Ah, I misunderstood then, thanks.


Scan pod if unknown owner

Relates to #13
Previous output:
❯ kubectl dds -n gha-rbourne
could not find resource manager for type Runner for pod gha-rbourne-12345-67890
NAMESPACE	TYPE	NAME	STATUS
❯

New Output:
could not find resource manager for type Runner for pod gha-rbourne-12345-67890
NAMESPACE                       TYPE    NAME                                            STATUS
gha-rbourne-hackathon-2023      pod     gha-rbourne-12345-67890     mounted

At this time I believe it is more worthwhile outputting low level pods when the owner is unknown understanding this can be verbose for clusters with large numbers of pods with docker host mounts, however the existing error does not make it clear the pod in question was not scanned.

I agree we should probably output something even if the type isn't known. Can you also add output if the user passes --verbose to always show the output (even if unmounted)?


Openssl not sending protocol_version alert

Description:
OpenSSL does not respond with protocol version alert for a client message with invalid version. But I observed that it prints ssl3_get_record:wrong version number in the terminal. It would be nice to respond with a protocol_version alert as well.
Environment:
Ubuntu 18.04
Openssl 1.1.1j
How reproducible:
Steps to Reproduce:
openssl s_server -key /path/to/key.pem -cert /path/to/cert.pem -accept 44330 -tls1_2 -www -no_ticket
echo -n -e "\x15\x83\x01\x00\x02\x02\x50" | nc <IP_ADDRESS> 44330
Actual results:
Openssl doesn't respond to client. The capture file is attached.
Expected results:
Send protocol version alert to client.
Attachments:
openssl capture.zip

This behaviour is deliberate. Basically if what we've received doesn't even look like TLS then we don't send an alert back:
https://github.com/openssl/openssl/blob/97b59744f2ad91adf606e6f77e355e97413e7b2c/ssl/record/ssl3_record.c#L353-L355

Thanks for your reply. It's very helpful.

@mattcaswell The message sent by the client is an alert message (type \x15) with invalid version \x83\x01 so it is a TLS message.
As per RFC8446, the version field is deprecated and should be ignored. Therefore, since the client sends an alert message while the server is expecting client hello, the server should send an unexpected_message alert.

legacy_record_version:  MUST be set to 0x0303 for all records
generated by a TLS 1.3 implementation other than an initial
ClientHello (i.e., one not generated after a HelloRetryRequest),
where it MAY also be 0x0301 for compatibility purposes.  This
field is deprecated and MUST be ignored for all purposes.

Given the major version does not match, the received data can be a pure garbage. I do not think we are going to change the behavior of the SSL server here.
Also please note that you run the server with the -tls1_2 option so RFC 8446 does not apply.

Ok. This was the original discussion about protocol version alert and it's a bit confusing to discuss another issue here. I'll submit a separate issue for TLS1.3.

This has been answered and #15246 duplicates the relevant information for TLS 1.3.  Closing this.


Unexpected decompression error

Hi,
I am faced with an error "Error: premature eof" when trying to decompress a piece of data. I've reproduced this behavior in a test where I try to decompress data using two libraries (liblzma vs lzma-rs). As you can see liblzma produces an error but output has valid decompressed data.
Here it is:
use anyhow::Result;
use liblzma::{
    read::XzDecoder,
    stream::{Filters, Stream},
};
use lzma_rs::decompress::raw::{LzmaDecoder, LzmaParams, LzmaProperties};

fn main() -> Result<()> {
    let input = [
        0_u8, 0, 129, 51, 7, 174, 15, 216, 24, 245, 200, 8, 96, 118, 244, 82, 234, 143, 39, 176,
        35, 215, 18, 32, 29, 41, 69, 26, 118, 10, 168, 24, 71, 93, 49, 151, 59, 85, 71, 218, 240,
        123, 248, 157, 185, 27, 109, 183, 16, 141, 167, 153, 15, 117, 188, 209, 92, 35, 218, 111,
        35, 118, 194, 138, 166, 217, 72, 91, 48, 231, 0, 210, 232, 191, 11, 15, 25, 191, 59, 65,
        86, 86, 188, 62, 76, 254, 10, 58, 169, 117, 77, 10, 238, 182, 202, 120, 14, 50, 254, 171,
        95, 244, 58, 127, 33, 14, 177, 215, 250, 78, 142, 47, 252, 101, 36, 184, 124, 95, 139, 4,
        117, 161, 40, 174, 68, 135, 147, 230, 162, 21, 149, 205, 64, 98, 183, 58, 171, 32, 159,
        223, 17, 252, 144, 232, 40, 190, 116, 0, 95, 219, 147, 77, 67, 214, 212, 60, 173, 95, 201,
        161, 153, 218, 183, 107, 225, 29, 39, 153, 206, 71, 61, 52, 53, 71, 231, 204, 58, 53, 251,
        88, 236, 7, 126, 147, 86, 134, 166, 235, 157, 241, 36, 99, 171, 17, 66, 150, 115, 28, 161,
        207, 53, 76, 38, 201, 19, 176, 185, 243, 30, 34, 229, 191, 199, 106, 105, 33, 209, 170, 8,
        163, 55, 173, 204, 44, 79, 96, 170, 69, 97, 185, 137, 127, 147, 243, 89, 97, 43, 4, 151,
        54, 116, 159, 95, 21, 201, 242, 125, 67, 69, 2, 131, 146, 229, 219, 5, 178, 14, 73, 66, 82,
        126, 246, 173, 8, 222, 36, 164, 20, 199, 159, 227, 132, 71, 35, 217, 203, 64, 156, 9, 145,
        177, 43, 168, 236, 165, 234, 35, 70, 194, 106, 208, 77, 122, 160, 250, 198, 108, 134, 216,
        253, 22, 168, 120, 156, 63, 205, 230, 170, 172, 231, 11, 148, 134, 32, 216, 121, 182, 86,
        96, 86, 151, 110, 92, 87, 17, 96,
    ];

    let props = [93_u8, 0, 0, 16, 0];

    // The `liblzma` part
    let mut filters = Filters::new();
    filters.lzma1_properties(&props)?;

    let stream = Stream::new_raw_decoder(&filters)?;
    let mut output1 = Vec::new();
    let mut decoder = XzDecoder::new_stream(&input[..], stream);
    let res = std::io::copy(&mut decoder, &mut output1);

    // Error: premature eof
    assert!(res.is_err());

    // The `lzma-rs` part
    let d = props[0] as u32;
    let (pb, d) = (d / 45, d % 45);
    let (lp, lc) = (d / 9, d % 9);
    let dict_size = u32::from_le_bytes([props[1], props[2], props[3], props[4]]);

    let props = LzmaProperties { lc, lp, pb };
    let params = LzmaParams::new(props, dict_size, None);
    let mut decoder = LzmaDecoder::new(params, None)?;

    let mut input = std::io::Cursor::new(input);
    let mut output2 = Vec::new();
    decoder.decompress(&mut input, &mut output2)?;

    // Outputs are the same
    assert_eq!(output1, output2);

    Ok(())
}

Hello @shamilsan
Thank you for the report.
As I checked, test cases that are expected to fail with liblzma (original xz-utils) do not fail with lzma-rs. In other words, there is a problem with original xz-utils, or lzma-rs provides functionality that is incompatible with original xz-utils. liblzma is a crate that provides rust-binding for original xz-utils, so there are no plans to directly fix this issue.


About reward model dataset or reward model

Hi, great work!
I see you will release training code. How about reward model dataset or reward model?

加一

Thank you for your interest. We have released the fine-tuning code, fine-tuning prompts, and the checkpoint of reward models

刚刚评论就发现已经released wow~ ⊙o⊙


Grab workaround for pytest-dev/execnet#195.

Originally, I just pushed this commit to main before I realized it causes failures on Python 3.8 when something attempts to construct a requirement, but creates the wrong syntax:
  File "/home/runner/work/setuptools/setuptools/.tox/py/lib/python3.8/site-packages/packaging/requirements.py", line 37, in __init__
    raise InvalidRequirement(str(e)) from e
packaging.requirements.InvalidRequirement: Expected end or semicolon (after URL and whitespace)
    execnet@ git+https://github.com/jaraco/execnet@bugfix/195-encodingwarning; extra == "testing"
             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

The error message is misleading. The problem is that PEP 508 specifies url_req as
url_req       = name wsp* extras? wsp* urlspec wsp+ quoted_marker?

Where the urlspec needs to be separated from the quoted marker (the semicolon) with whitespace.

I can see that importlib_metadata fixed this issue in 4.8.3, so there must be some other code that's generating the invalid URL.

The problem exists in build when it relies on the old, unfixed importlib.metadata found in Python 3.8 (fix was added to CPython 3.9 in https://github.com/python/cpython/issues/90263).

Although tests are passing, I don't believe PyPI would accept this package. Probably the overrides need to be in tox instead.


Lint/OrderedMagicComments reports at line is not a magic comment

Expected behavior
Given the following code:
# frozen_string_literal: true

x = { encoding: Encoding::SJIS }
puts x

I expect RuboCop to throw no error.
Because line 3 is not a magic comment.
Actual behavior
RuboCop throws an error:
test.rb:3:1: W: Lint/OrderedMagicComments: The encoding magic comment should precede all other magic comments.

Steps to reproduce the problem
Create a file with the following contents:
# frozen_string_literal: true

x = { encoding: Encoding::SJIS }
puts x

This is extremely important! Providing us with a reliable way to reproduce
a problem will expedite its solution.
RuboCop version
$ rubocop -V
0.58.1 (using Parser <IP_ADDRESS>, running on ruby 2.5.1 x86_64-darwin17)

Thanks for the interesting feedback. There was a false positive to misunderstanding the Hash notation as an encoding magic comment.
I confirmed that and I opened a PR #6117. Thanks!


Improve tunneling and add podman support

I would recommend merging this after #8. This is a rebase of #7.

After we land https://github.com/Pernosco/on-prem/pull/5 this one needs to be checked by the customers using the existing feature before I merge it.

@khuey In theory nothing should change. I tried my best to make sure that the defaults/behavior of binding stayed exactly the same.
If they still have socat installed, no harm done either.

Merge conflict fixed.


[BUG]: share screen not working

Hello, I love what you did on the agora virtual classroom. It's so simple and clean ASF but I noticed an issue during when you click on screen-sharing, the app doesn't do anything as regards to that but it will still show sharing in progress. I guess that this is a bug or something?
Originally posted by @Tolzy in https://github.com/netless-io/flat/issues/926#issuecomment-1169876747

Thanks for the feedback, we will fix this problem in the next version.


Downgrade version to 2.18.0 for ser/de of new parameters of RequestRequest

Description
Downgrade version to 2.18.0 for serialization/deserialization of new parameters of RequestRequest added in PR #16292 since it has been back ported into that version.
Related Issues
Resolves #[Issue number to be closed when this PR is merged]

Check List

[ ] Functionality includes testing.
[ ] API changes companion pull request created, if applicable.
[ ] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@owaiskazi19 - this change should be back ported into 2.18 if possible, or at least into 2.19.

@owaiskazi19 - this change should be back ported into 2.18 if possible, or at least into 2.19.

Added a backport label and the PR is created. @dbwiddis do we have to backport this to 2.18 branch as well?

@owaiskazi19 - this change should be back ported into 2.18 if possible, or at least into 2.19.

Version.CURRENT in 2.18 is precisely Version.V_2_18_0.
https://github.com/opensearch-project/OpenSearch/blob/d27ee4eb710db4d5d0776065d79408d91fd4f315/libs/core/src/main/java/org/opensearch/Version.java#L145
So no need to backport into 2.18, just putting in 2.x should work for future versions.

Although if we end up with a 2.18.1 it would need to be on 2.18 branch, so.... in that edge case, wouldn't hurt.

Now that you've closed it...heh

It'd probably be fine being 2.18.1 as current anyway, though.  So I'd just leave things as is.

I can open it again. 2.18 branch still has the TODO

Based on the earlier backward compatibility failure, that could cause problems on a mixed cluster of 2.18.0 and 2.18.1 if this was not merged into 2.18.

I was just thinking that about BWC during 2.18.0 to 2.18.1, so yeah, we should backport.


Create calculator app

Problem statement
create a calculator application with graphical user interface

I want to try it, Can I be assigned to this?

Yes @Xlgd


Remove profile resolution relative URL workarounds

https://github.com/usnistgov/oscal-content/issues/60 makes the itemURL replacement in SSPResolver unneeded.
The replacement in ProfileResolver is still needed, but is now cleaned up a bit.
Closes #505

@tuckerzp Can you take a look at these new merge conflicts?


Get-DbatoolsVersion to show active version

I often see the question "which version do you use?" and "what does Get-Module -Name dbatools -ListAvailable say?"
I think it would be good to have a command, that just returns the actual loaded version of dbatools.
And if the command fails, we know that the version is too old...

Asking what version is majority of cases no longer matters because we are not making massive breaking changes as we did prior to 1.0.
Get-InstalledModule dbatools is another command to get the newest version installed on a given machine.
If someone would like to write a Gist that has code to perform the needed check we can create a shortlink to easily share with users.

To get the version that is running should be as simple as (get-module dbatools).version. If there is something more that you want, not sure that it would take much.

Yes, (Get-Module dbatools).Version is much better than Get-Module -Name dbatools -ListAvailable. I opened this issue after reported a not working command on slack. I told him to update dbatools and he just ran Update-Module dbatools and tried again - failing as well. So he missed restarting powershell or reloading the module. And even Get-Module -Name dbatools -ListAvailable would have reported the new version. That's why I thought that running a command would be best. Loading the latest installed version if not already loaded and showing the version.
And then I thought about myself. On my lab, I have a version of dbatools installed from PSGallery for reference and tests of bugs. But I also have a git repo to test fixes or new features. The new command could not only show the version but also, if loaded from a git repo, show the branch that was active at the time of loading. But that might be a total edge case and only relevant for me.
So summing up: A new command Get-DbatoolsVersion might not be as useful as I thought when opening this issue. I will think about it one or two more days and the probably closing this.


ENHANCEMENT OF THE STAKEHOLDER SECTION AND MAKE IT ATTRACTIVE AND INTERACTIVE

### ENHANCEMENT OF THE STAKEHOLDER SECTION AND MAKE IT ATTRACTIVE AND INTERACTIVE
Is your feature request related to a problem? Please describe.
The stakeholder section currently lacks visual appeal and interactivity, which makes it feel disconnected from the rest of the website. It doesn’t engage users effectively, and I find it frustrating when stakeholders’ information is presented in a static, non-interactive format.
Describe the solution you'd like
I want to enhance the stakeholder section by introducing interactive elements, such as hover effects on each stakeholder card, animations for images and text, and an organized, visually appealing layout. The section should highlight each stakeholder with respective logos, descriptions, and links, with smooth transitions and animations to improve engagement. Custom scrollbars and iconography should also be used to ensure the section is cohesive and modern.
Additional context
Incorporating interactive icons, images, and hover transitions for each stakeholder will create a more engaging experience. The section should remain responsive across devices, ensuring ease of navigation while maintaining a professional and attractive appearance.

AS A EXPERIENCED AND WELL DONE FRONTEND DEVELOPER I CAN RESOLVE THIS ISSUE EFFECTIVELY HENCE I AM REQUESTING YOU TO PLEASE ASSINGN THIS TASK TO ME . ~Thankyou @Harshdev098


Streaming issue in 0.109.6

Thanks form a different part of the GTA- hope you're staying safe.
Post update to 0.109.6 of Home Assitant, and aiohttp 3.6.2
Recurring error is
raise ValueError('Unsupported body type %r' % type(body))
ValueError: Unsupported body type <class 'coroutine'>
2020-05-08 13:50:37 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/aiohttp/web_response.py", line 567, in body
self._body = body = payload.PAYLOAD_REGISTRY.get(body)
File "/usr/local/lib/python3.7/dist-packages/aiohttp/payload.py", line 106, in get
raise LookupError()
aiohttp.payload.LookupError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/aiohttp/web_protocol.py", line 418, in start
resp = await task
File "/usr/local/lib/python3.7/dist-packages/aiohttp/web_app.py", line 458, in _handle
resp = await handler(request)
File "/usr/local/lib/python3.7/dist-packages/aiohttp/web_middlewares.py", line 119, in impl
return await handler(request)
File "/usr/local/lib/python3.7/dist-packages/homeassistant/components/http/real_ip.py", line 39, in real_ip_middleware
return await handler(request)
File "/usr/local/lib/python3.7/dist-packages/homeassistant/components/http/ban.py", line 73, in ban_middleware
return await handler(request)
File "/usr/local/lib/python3.7/dist-packages/homeassistant/components/http/auth.py", line 127, in auth_middleware
return await handler(request)
File "/usr/local/lib/python3.7/dist-packages/homeassistant/components/http/view.py", line 125, in handle
result = await result
File "/usr/local/lib/python3.7/dist-packages/homeassistant/components/camera/init.py", line 490, in get
return await self.handle(request, camera)
File "/usr/local/lib/python3.7/dist-packages/homeassistant/components/camera/init.py", line 510, in handle
return web.Response(body=image, content_type=camera.content_type)
File "/usr/local/lib/python3.7/dist-packages/aiohttp/web_response.py", line 545, in init
self.body = body
File "/usr/local/lib/python3.7/dist-packages/aiohttp/web_response.py", line 569, in body
raise ValueError('Unsupported body type %r' % type(body))
ValueError: Unsupported body type <class 'coroutine'>
/usr/lib/python3.7/asyncio/base_events.py:1756: RuntimeWarning: coroutine 'KunaCamera.camera_image' was never awaited
handle = self._ready.popleft()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

Can confirm, I have the same issue. It seems to be polling SOME data (streaming/recording status) but no video display.

I have the same issue. Since version  0.109.6 not streaming and also the switch sensor does not work.
The recordings are still being saved locally.

Just tried the version 0.110.0 beta and it's still messed up

I did try version 0.110 too. Same result.


fix(opengb): add dedicated error for neon projects exceeded

Changes

#847  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @MasterPtato and the rest of your teammates on  Graphite


Never get unity logs

The code:
.withUnityLogger(message =>
{
log.debug(Unity: ${message});
console.log(Unity: ${message});
})
never gets hit.  When unity does have a problem the error is always:
{"unityLog":""}
Lately for no apparent reason i'm getting the error that says unity crashed and the log file it says it created is always empty.

The problem can be simulated by using the erroneous: .withBuildOptions({ strict: true }) - should be strictMode.
In unity_invoker.js if I modify the function toArgv with:
    else if (key === 'logFile') {
        var crashPath = path.join(os.tmpdir(), 'unity_crash.abcompiler.log');
        argv.push('c:\\temp\\unity.log');
    }

then I get a log.  by default the logFile property is set to true and so nothing gets pushed after it but according to https://docs.unity3d.com/Manual/CommandLineArguments.html -logFile needs a filename after it.

Hello @RobinHSanner, thanks for the report!
What version of Unity do you use? I suspect 2017.x since this used to work before.
Yes, -logFile without argument seems counter-intuitive, but this is how you were specifying to Unity to not use a log file and send back the log in real time to stdout, like any other "normal" program. Then we we were watching stdout and writing our own log file in case of problem.
I will check if the behaviour has effectively changed in Unity 2017, try to reproduce and fix the problem (implementing a different behaviour based on the version of Unity) and I'll keep you informed.

Hello again,
I can't reproduce with Unity 5.x, nor with Unity 2017.x.
Also, what's your OS? Compatibility with Windows is mostly untested, and I wouldn't be surprised if this specific point about logfile/stdout has a different behaviour on Windows.

Unity 2017.3.0f3

Okay, must be a Windows-specific problem.
(@fklingler @CrazyTales Want to fix Windows-ish things?)

So that's a problem on Windows indeed: https://answers.unity.com/questions/19566/command-line-feedback.html
One of our teammate on Win had the same problem as you.
Unless someone proposes a workaround it'll stay a wontfix, better wait a proper fix from Unity.


How to fix 'Failing to build model' error?

I am using a bunch of HTML pages saved without any extension (i.e. as plain text) in my positive and negative  directories of train_data folder. Now when I run buildModel command to train a new model based on my provided dataset, a new temporary file "smile_input.arff' is prepared but it fails to generate features, model and final pageclassifier.yml file.  I am running into this issue. Can someone help me debug this?

Moreover, I am using docker image vidanyu/ache:master

To add more to this, when I test the same command for provided sample_training_data in /config directory, I run into this issue there as well. But after deleting some of the sample files, if I run this command again, it is able to build the model for me.
This means it works for few samples, but at same time does not for few, and I cannot identify what difference in sample files is causing this.


Modular routing

Enable dynamic .use mounting for more modular applications. For example:
var mount = app.route('/').get(function (req) {
  console.log(req.params.route); // Works!
});

app.use('/:route', mount);

Hi guys I'm trying use this but not work for me I'm using express 4
I have this

var router = express.Router();
router.get('/',profile);

var profile = function (req, res) {
    console.log(req.params.username);
}

app.use('/:username', userRouter);

but req.params.username is undefined

@juliocanares please open a new issue. Otherwise, what I can say for now is that you need to enable the mergeParams option on your userRouter for it to work (see docs at http://expressjs.com/4x/api.html#router)


Implement Auto-update

Allows the app to read release information from GitHub and update existing application without needing to re-install. This should be explicitly enabled by the user and should be switched-off by default.
User flow:

If the app has auto-update enabled, it checks for update on app start or using a button within the auto-update section of the settings pop-up.
Once an update is available, it shows a pop-up with an option for the user to download it.
If the user agrees, the update happens in the background and the progress is shown in the top bar.
Once the update is downloaded, the user is shown a pop-up with the option of either installing it then or on app restart.
Either ways, once the app is updated, the user is shown the proof of the update in the top-bar via the new version number. (This assumes the app always shows the version number on the top bar).

This has been moved to [Autographa-Live].(https://github.com/friendsofagape/autographa-live/issues/3)


Implement usage of URLPattern

There's a fair bit of url/path matching, it would be good to update these to URLPatterns.
https://developer.mozilla.org/en-US/docs/Web/API/URLPattern
https://web.dev/urlpattern/

I'm working on this one...


chore: export FunctionRegion enum

What kind of change does this PR introduce?
Export the region type, so it can be used in other libraries.
What is the current behavior?
Please link any relevant issues here.
What is the new behavior?
Feel free to include screenshots if it includes visual changes.
Additional context
Add any other context or screenshots.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 92.884%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
218

Relevant Lines:
230

💛  - Coveralls