No way to install/use a specific package version?

July 2020 update
This issue is coming up on 5 years old at this point, and has gotten quite lengthy. While the issue may still be relevant, not all of the comments in this thread will be. It has been suggested that you may find it more valuable to skip to the bottom of the thread in order to get a read on the current state of things, rather than working through it from start to finish.
Here's the original post:

It provides . . . side-by-side installation of multiple versions of a package . . .
(From the Nix page)

I can't seem to find any way to install a specific package version, though - it seems like when installing a package you just get whatever is current in the channel you've subscribed to. If there's no way to request a specific package version, how is it possible to have "side-by-side installation of multiple versions"?
(Note: I understand that some packages occasionally have more than one version available in the current state of the channel and can be accessed via eg. package-name_1.2.3, but this seems to be the rare exception and still doesn't allow specifying a specific version, just a choice from a few available versions.)

Potential use cases:
Solving dependency/Cabal hell:
I found Nix through Haskell, where a lot of people offer it as a solution to "Cabal hell". However, if I can only install one version of a package via Nix, I can't solve the situation where I am working on two different projects with two different and incompatible dependency requirements.
Testing against multiple versions of dependencies:
Say I'm writing a library, for which I originally depended on foo-1.0.0. A year later I'm making updates and I want to use expand the upper bound of that dependency so that users aren't restricted to an old version of the dependency. I install foo-1.5.0, make a bunch of updates, verify that it works, but then want to double-check that everything still works with foo-1.0.0, because if the new changes don't work with it I'll have to change the lower bound of the dependency, potentially causing problems for users. However, not being able to specify to Nix that I want foo-1.0.0, I can't test against foo-1.0.0 and can only state the dependency as foo == 1.5.0, instead of the more permissible (and usable) foo >= 1.0.0 && foo <= 1.5.0.
Sharing build/development environments:

Nix makes it trivial to set up and share build environments for your projects
(From the Nix page)

I've seen people say that the ability to share build/development environments across the team solves the "Well it works on my machine" problem. However, if I send the environment (via default.nix/shell.nix etc.) to another developer to get them up to speed with the project, they may end up installing different versions of packages than me, since the packages channel may have progressed since I last installed/updated my packages. Now we're not on the same page any more and those subtle (and painful) differences and bugs can creep back in.
Deploying to servers/CI/etc:
In a similar vein to sharing development environments, it would be great to be able to use Nix to standardize the environment across all our develop/build/test/deploy phases/environments. However, since the channel could change at any point between these phases and we're not able to specify specific package versions, this isn't possible. For example, take the case where the project is deployed across multiple servers. Being in the cloud, they're ephemeral, so when one breaks for some reason it's destroyed and replaced with a new, healthy version. Except that the new version has to be brought up so speed and installs the project via Nix. Since the channel has been updated since the other servers were last updated, the new server is now running in a different environment and introduces potentially problematic inconsistencies to the system.

Now, these are all hypothetical - I don't actually have experience in any of these use cases but if my understanding is correct I can see them potentially cropping up. However, I can find no mention anywhere (despite very exhaustive searching) of anyone concerned about locking down package versions or installing specific versions when using Nix. If I'm the only one, perhaps I'm missing something fundamental here and barking up the wrong tree?
I just can't see how Nix can deliver the benefits everyone says it does without giving users a way to specify specific package versions - is there really no way to do this, or am I misunderstanding the whole system?

Single-version policy
We keep multiple versions in nixpkgs only when there's a good reason to. Nix is able to handle any number of versions/configurations, but on the other hand it's much more convenient when all (or most) use just a single one. It leads to better sharing of the effort in many respects: simplified maintenance, testing, sharing of the binaries, etc. It's what most distros do. (Only gentoo diverges from the big ones I know, and they pay a price for it.)
When we do create more variants, we just name them (attribute paths), e.g. gcc48 and gcc49 or ffmpeg and ffmpeg-full.
Only now, we build on the order of 40 thousand packages on the build farm (counting platform variants, etc.), but the number of active contributors is only in dozens. http://hydra.nixos.org/jobset/nixpkgs/trunk
Identical environments
If you're concerned about identical environments, you need to also have the same nixpkgs version, that's for sure ;-) To be certain, you can just copy those by nix-copy-closure. Note that version numbers of direct dependencies don't contain all the information at all.

Installing specific versions works just fine, AFAIC:
$ nix-env -qaP boost
boost155  boost-1.55.0
boost156  boost-1.56.0
boost157  boost-1.57.0
boost     boost-1.58.0

$ nix-env -p /tmp/foo -i boost-1.56.0
installing ‘boost-1.56.0’
these paths will be fetched (0.00 MiB download, 109.64 MiB unpacked):
  /nix/store/65fwmbxkbar04m8qqig1a7vymx0xqpg8-boost-1.56.0
  /nix/store/awb45xm2izxjj0sksrxi785shg6lxvbj-boost-1.56.0-dev
  /nix/store/c4cd7qx21v9pq4qrmj0ysz58hz2l0sxs-boost-1.56.0-lib
[...]
building path(s) ‘/nix/store/n2bwn9a87jxc6a52i2p21y4kd414ba72-user-environment’
created 3 symlinks in user environment

And here is the same thing for a Haskell package:
$ nix-env -f "<nixpkgs>" -qaP -A haskellPackages Cabal
haskellPackages.Cabal_1_18_1_6  Cabal-<IP_ADDRESS>
haskellPackages.Cabal_1_22_4_0  Cabal-<IP_ADDRESS>
haskellPackages.Cabal_1_23_0_0  Cabal-<IP_ADDRESS>

[simons@latitude:~]$ nix-env -f "<nixpkgs>" -p /tmp/foo -iA haskellPackages.Cabal_1_18_1_6
installing ‘Cabal-<IP_ADDRESS>’
these paths will be fetched (4.27 MiB download, 64.34 MiB unpacked):
  /nix/store/48h9yz58zbjd5rp17s3lyjavrd92l8lq-Cabal-<IP_ADDRESS>
fetching path ‘/nix/store/48h9yz58zbjd5rp17s3lyjavrd92l8lq-Cabal-<IP_ADDRESS>’...

*** Downloading ‘http://cache.nixos.org/nar/1agwkai3m6jc58zc8i616qlhzvqfbn7lg1dpqxcjjq3hwyq5inzf.nar.xz’ (signed by ‘cache.nixos.org-1’) to ‘/nix/store/48h9yz58zbjd5rp17s3lyjavrd92l8lq-Cabal-<IP_ADDRESS>’...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4369k  100 4369k    0     0  1153k      0  0:00:03  0:00:03 --:--:-- 1153k

building path(s) ‘/nix/store/mhbn0csr0lxnsbvf416ns82qlncnh9vw-user-environment’
created 70 symlinks in user environment

So I'm not sure what problem you see?

First of all, thank you both for your responses!

@peti Right, but as @vcunat said, "We keep multiple versions in nixpkgs only when there's a good reason to."
$ nix-env -f "<nixpkgs>" -qaP -A haskellPackages aeson
haskellPackages.aeson  aeson-<IP_ADDRESS>

So what happens if I have to fix a bug in an old project that was using <IP_ADDRESS>? I clone the project from Github, go to set it up with Nix, and now it doesn't compile because Nix wants to use <IP_ADDRESS> and doesn't give me any other options. My choices seem to be either 1) Be forced to upgrade the project to <IP_ADDRESS> just to fix a minor bug or 2) Somehow install the package from source myself and make Nix use that instead of the version in nixpkgs. Neither option makes using Nix seem like a better choice than eg. Stack or NPM or Bundler etc, since they'd handle this version requirement seamlessly.

@vcunat Thanks for the detailed explanation, that helps me understand the bigger picture better. I hadn't been aware of nix-copy-closure - that looks like a really great tool and opens up a lot of possibilities!

I guess I'm coming at this from a specific perspective - I'm coming from front-end web development (and now trying to use Haskell/GHCJS), and all the tools I've used for package management for these projects (Bower, NPM, Bundler, etc.) operate on the basis of specifying version ranges for dependencies and locking them down to versions that you know will work. This then necessitates the ability to install any arbitrary version of a package since multiple projects (or even multiple dependencies of a project) can end up specifying different versions from each other. However, the benefit is that any time you install the project anywhere you know that you'll have a working set of dependencies and thus a working project. This is also nice when it comes to deployment, because you can have confidence that if the CI build passes the code will work on the server, and you can have continuous deployment with confidence (as long as package maintainers follow semver). As I wrote above, you're also able to pull down old projects and do work on them despite depending on old packages since you're able to install the old packages without problems.
I see that there are some workarounds for some of these things, such as using a previous revision of nixpkgs, but nothing that seems easy and seamless like these other package managers. I just can't see how to use Nix if you can't (reasonably) use it to install specific versions of packages, since that's a use case I've seen again and again. For example, just today I found a bug in a package I needed to use - normally I would have tried the previous release while waiting for the maintainer to fix the package or I would have fixed it and used my fork while waiting for the PR to go through, but with Nix I was out of luck. I've also often needed to install a newer version of a package than my distro made available, which with Ubuntu came down to just simply adding a PPA, but with Nix I'm tied down to whatever's in the channel.
I do love the idea of Nix (and NixOS!), and I really want it to be something that can work for me. Perhaps I'm just missing the point or philosophy of Nix? Given that I seem to be the only person with these concerns maybe there's a reason they're a moot point and the approach I'm trying to take to deal with them is unnecessary with Nix?

So what happens if I have to fix a bug in an old project that was using <IP_ADDRESS>?

In that particular case, Nix won't help you and you are better off using a cabal-install sandbox and/or a stack build. Hackage contains ~60,000 package versions, the vast majority of which is broken and/or obsolete. IMHO, it makes no sense to put all that stuff into the Nixpkgs database, because the vast majority of it will never be used by anyone. Furthermore, adding that number of build expressions to our distribution would impact a lot of people in terms of performance and memory consumption of tools like nix-env and nix-build and Hydra. It's just not worthwhile for us.
You always have the option of adding missing versions of certain packages to your local database by means of an override as described in http://nixos.org/nixpkgs/manual/#how-to-create-nix-builds-for-your-own-private-haskell-packages. You can register aeson-<IP_ADDRESS> in your copy of Nixpkgs without needing to change the Nixpkgs git repository at all.
If you feel that there are certain older packages that Nixpkgs should contain, then you can also add them to the list at https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/configuration-hackage2nix.yaml#L35, and they'll show up in our package database within the next 2-3 days.
Anyhow, if you want a tool that can easily build any version of any package from Hackage, then cabal-install is your best choice. Nix isn't trying to accomplish that.

Sorry, I didn't mean to close the issue. I just hit the wrong button.

Sorry, I didn't mean to close the issue. I just hit the wrong button.

Thanks for clarifying :)
I've been doing a lot more research in light of what you said and I think I'm starting to understand how the pieces fit together. In the interest of brevity I won't explain all the changes in my understanding but there is still one thing I'm struggling to understand: Nix expressions have no control over the set of packages that they receive.
(tl;dr section at the bottom)
From the perspective of having pure functions and having composability/overridability etc. I see the benefit here. However, from the perspective of a web developer where getting our projects up and running needs to able to be automated across any environment (developer machines, CI, production servers, etc.) this seems like a problem, since the project is no longer "self-contained" in terms of containing all the configuration/commands/etc. needed to bootstrap itself.
For example, let's say I create a project using the nixos-15.09 channel. If any machine (eg. a co-worker's machine, or the server, etc.) is using a different channel, the project will likely fail to work on that machine because the set of packages it receives are all at different versions from those of the channel the package was created with. To make matters worse, there would be no indication anywhere in the project as to what that original channel was, so there's no way to know which is the right channel to use without trial-and-error.
Of course, that information could be documented somewhere, but having to follow specific steps outlined in the readme hurts the "reliable and reproducible" aspect of using Nix (and detracts from the benefits of having a default.nix in the first place). When writing a project with end-users in mind the ability to choose an arbitrary channel may be beneficial, but when the intended use of the project is to be easily deployed via automation it seems detrimental.
As an example of a counterpoint to Nix in this regard, Stack allows an LTS Haskell version (similar to a Nix channel if I understand correctly) to be specified in the project's stack.yaml file. The concerns and considerations of each of these approaches is beyond my capacity to understand, so I won't pretend to know what is "best" (if there even is such a thing as a global "best"), but this difference does make Nix seem lacking in regard to being "reliable and reproducible".

TL;DR
Essentially, as I understand it, in order for Nix to enable a project to be reproducible in a reliable and automatable way it would need to be able to specify what channel it needs to get its packages from. This way it 1) Does not need to be (potentially incorrectly) specified by the user and 2) Is consistent across each invocation.
Is there anything I'm missing here that Nix already has that addresses this issue? I know that a default could be specified in the arguments to the Nix expression. That approach would be good in that it still allows the argument to specified manually if necessary, but it still falls short in that the default argument could only reference a channel if it was already registered with the host system.

@mayhewluke yup that's the general idea. In our organization we have our own branch of nixpkgs that we occasionally update and run a binary cache for. This way employees can easily install dependencies on the local network and know that the builds will work, with the added benefit that it's super fast! Alternatively if you're working in a less centralized environment you could do this trick where you pin the nixpkgs version in the shell.nix itself:
https://garbas.si/2015/reproducible-development-environments.html
nix is flexible enough to cater to these different workflows. I think eventually we'll be able to target stackage from nixpkgs as well if you're into that sort of thing.

Nix expressions have no control over the set of packages that they receive.

But they do. Things that are no parameter to your functions cannot vary, i.e. https://github.com/peti/ghc-library-id-bug/blob/master/default.nix#L5.

Stack allows an LTS Haskell version to be specified in the project's stack.yaml file.

Sure, but note that this doesn't mean that the version of all packages in that release are fixed. LTS releases can and do change: https://github.com/fpco/lts-haskell/commits/master/lts-3.2.yaml.

Ah, fantastic! @jb55 that blog post and @peti your code in ghc-library-id-bug answer that one last missing link for me. I now see how it is possible to 1) Pin the revision of nixpkgs while still allowing overrides and 2) Similarly "pin" to the head of a specific channel via fetchTarball. And because Nix is brilliant it won't fetch these things over and over unless the inputs to them have changed, so performance isn't really a concern. Excellent!
@peti thanks a ton for your patience in helping me understand all this. I now have the answers to every question/concern I've raised in this thread. I'll post a summary here of my questions/the answers once I've had the chance to write it up. Hopefully this will help anyone else who may have similar questions and might also help Nix maintainers to be able to save time on answering questions :) Thanks again!

I'll post a summary here of my questions/the answers once I've had the chance to write it up. Hopefully this will help anyone else who may have similar questions and might also help Nix maintainers to be able to save time on answering questions :)

Here it is!

That seems like an awful lot of boilerplate just to pin a shell.nix to a Nix channel - and won't it cause nix-shell to fail to launch if you try to use it offline?

I find one often has to create a fork/branch of nixpkgs for a particular dev project anyway, e.g. because of a need to update something in nixpkgs and use it now without waiting to push to official master or waiting on channels. I think that simple step also "magically" solves all of the problems you mention.

That just moves the problem, doesn't it? Instead of needing some way to point a shell.nix to a channel, now you need some way to point it to your fork.

But you have lots of ways to do that: $NIX_PATH, various command-line parameters, etc.

BTW, recently nix got support to easily specify exactly the commit hash that you want to use: http://nixos.org/nix/manual/#sec-common-env

Don't you want to be able to specify a packages source in a config file, to avoid requiring each dev to set up their own environment variables or command-line parameters? I don't understand how I'm failing to communicate the motivation here.
The current situation is awkward because there exists a multiplicity of nixpkgss, and it could be solved by unifying them under a single root. Suppose nix-channel would set up an expression like this:
with rec {
  channelPath = name: file:
    "${builtins.getEnv "HOME"}/.nix-defexpr/channels/${name}/pkgs/top-level/${file}";
  channel = name: file: import (channelPath name file) { config = config; };
};
pkgs = rec {
  nixos-15-09 = channel "nixos-15.09" "all-packages.nix";
  nixos-unstable = channel "nixos-unstable" "default.nix";
  default = nixos-unstable; # or whatever the user picks as their default
}

(I've done something like this in my own config to write expressions that use multiple channels.)
Then anything that wants to be able to select (by name) which channel it uses, no command-line parameters or env vars necessary.

I personally want my dev projects to use versions independent of each other and independent of channels or the version my system uses, so that nix-channel --update won't break them, but obviously different people want different things.
Still, it would seem nice if nix-channel got better in this, e.g. the way you propose. Currently nix-env does respond in a similar way, but it seems a bit ad-hoc. Different commands react in different ways with respect to channel selection.

I don't completely understand why this issue is closed.
All I see are workarounds.
To summarize what I have read here (correct me if I am wrong)
@mayhewluke described some real drawbacks of how derivations are currently implemented:

Derivations cannot use a specific version of a package.
Derivations are limited to a very small subset of real versions for dependencies.

@peti brought up some concerns as reasons for the current behavior:

We don't want Nix to even try to build every version of every package ever.

and some workarounds:

Packages (hopefully) have a name for every version you need.
Extra packages are made with the naming convention name_1_2_3
You can specify a specific channel for nix to get packages from, and hopefully that means you can find a channel that has the version you need.

I still have some serious issues with this.

There is still no way to get a package version that isn't in a nix channel.

this may be OK for a user, but it's not OK for development.

The whole thing is just messy. Instead of one package name with several versions, we now have several package names with one version each.
It isn't obvious how to write a derivation that depends on the version you need.
You can't write a derivation that depends on a version that isn't in a channel.

My proposed solution
Here are my thoughts about how Nix should deal with package versions. Please tell me your thoughts. How hard would this really be to implement? Can we make a branch of nixpkgs, and make these changes?
Let a derivation specify a range of versions for each dependency.

This does not mean Nix has to build all available versions!
This does not mean Nix has to make every version available!

If the version I want is not defined, I can just write a new expression that provides it for me.

Different versions are provided by different derivations.
As far as I know, this is already true. The only difference would be that the names are always the same. If you tell nix to install a package, it should just grab the latest version that channel has. If you want a specific version, specify (that's the part you can't do currently).
TL;DR The versions are there! Use them!
Most, if not all derivations specify a version string, and append it to the name.
Let's just stop appending the version to the name, and treat it like the discrete data point it is.

@thomastjeffery there are projects that try to automate the conversion of other package manager indexes to Nix expressions. It still requires someone to commit in the new expressions in nixpkgs. Although the I think the haskage integration is fully automated (not sure). But the basic idea is that version numbers don't make sense in Nix because it's meant to be content addressed. There needs to be a translation from the version specification to a content addressed expression in nixpkgs to get what you want. For now most of this is done manually. Also every previous iteration of a channel is still accessible.

Oh and some packages do support version as a parameter. Then you need to write a package override and call that package with your desired version specifier. But this parameterisation of the version is not standardised and may not even work if different versions build differently!

For a clearer example, I just spent a few hours learning (#28248) that I need to add thirteen package overrides, and use the master channel, just to build yi.
This is a real problem.

there are projects that try to automate the conversion of other package manager indexes to Nix expressions

I understand that. Until I found this issue, I didn't know you could search haskellPackages directly. Until then, I was searching the generated hackage.nix
To be clear, I don't advocate providing more versions. We should, however, be clear about what versions are available without this mess of package names. It's confusing for relatively new users like myself, and makes the system more complex.
I just want to be able to specify more clearly, and to work with a cleaner, more functional codebase. After all, Nix is the pure functional package manager.

the basic idea is that version numbers don't make sense in Nix because it's meant to be content addressed.

What do you mean by that? There are packages, and those packages have versions. Sure, Nix addresses the store with hashes, but versions are still important.

If you give nix a version specification, nix doesn't know what hash this corresponds to, and where to get it. Something needs to bridge that gap. That's why versions would have to be a parameter in nix that gets mapped to a hash somewhere, however because it's meant to be pure, we cannot call into other package indexes to dynamically fetch the hash. That's why the translation from version to hash has to happen at "compile time". If we can standardise an infrastructure for doing this and then writing adapters to this infrastructure for every package index that exists, then it would be alot simpler!

I've been debating with myself of whether to pin the nix-shell to a commit hash of nixpkgs or to leave it to the OS using <nixpkgs> variable. My OS itself still pins the <nixpkgs>. The advantage of this is that every nix-shell uses the same set of nixpkg expressions, reducing the need to hold many sets of nixpkgs, and also reducing the "build" time that occurs every time you run nix-shell. However this means the dev environments aren't a fully specified closed expression. If I pin each shell.nix to a particular hash of nixpkgs, what happens when GC runs after I update my OS pinned nixpkgs? Will it wipe out all the dependencies of shell.nix forcing it to rebuild/redownload stuff when I next run nix-shell?

@CMCDragonkai not pinning hashes has bitten me too many times when building old projects. now I always pin. pin2win. as long as you have a nix-build generated symlink lying around it shouldn't get gc'd.

So what I want is to have a package's default.nix provide one explicit version. This is practically current behavior, I just don't like the current naming scheme.
The real change I want, is for a derivation to be able to specify a range of compatible versions, or even just one specific version for any given dependency.

nix doesn't know what hash this corresponds to, nor where to get it.

Nix currently knows what hash a package name corresponds to, because it is constrained to the one package that is defined by the channel it is using.

What I want is to let a channel have several derivations for a package, each with its own version.

This is current behavior.

I then want my derivation to depend on a specific package name and version.

Since versions are currently put into some package names, this is current behavior, though it would be cleaner to just use separate names.

I then want my derivation to pattern match a range of versions.

This is not current behavior.
The current channel will provide a package with the greatest version number matched by the range provided that it has defined.

What am I missing?

The main thing I want to accomplish is to distinguish between the versions of packages, and the versions of derivations.
A nix-channel is a tree of .nix files that define how to get packages. The .nix derivations themselves can be, and are updated independent of package versions. You can generally consider a nix-channel to correspond with a version of those derivations.
...but a nix-channel should not explicitly mean package version. You should be able to have multiple package versions defined by a channel.
One of the great strengths of Nix is the ability to use multiple versions of packages in the same system without renaming that package's files! So why are we renaming the packages themselves?

I then want my derivation to pattern match a range of versions.

This is not current behavior.
The current channel will provide a package with the greatest version number matched by the range provided that it has defined.

What am I missing?

I might be misunderstanding or missing the point here (sorry), but what's the point of a range if it always returns the greatest version number?
If you mean like the greatest compatible version in the version range (whatever compatible means), I might be wrong, but from my understanding, nix doesn't really do any 'dependency management' like this where to attempts to choose a correct version. Instead, everything is declaratively specified/mapped, if that makes any sense.

without renaming that package's files!

What do you mean package's files?

I might be misunderstanding or missing the point here (sorry), but what's the point of a range if it always returns the greatest version number?

If I am writing a derivation, I am probably not explicitly choosing a channel to build it against. Since different channels will provide different versions, I just want whatever is both compatible with my derivation, and available in that channel. Hence, pattern matching.

What do you mean package's files?

In Debian, ArchLinux, etc, libraries are put in the same standard path. If libstdc has more than one version installed, there is a naming conflict. This is worked around by having libstdc.so.1.
In Nix, every derivation is installed into its own folder in /nix/store, so there is never a naming conflict. You can have several versions of a package installed to the same path and name, since it really isn't the same path.
Since we don't use version numbers in derivations, we now have several packages providing the same library with a different version, but instead of having the same name, they are renamed package-name_1_2_9.
We have moved file name conflicts to package name conflicts. We aren't taking advantage of the functional nature of Nix. Avoiding name conflicts is a core feature. We brag about it on the front page! But it isn't real.

My biggest issue with renaming packages is that the normal-named package is usually older.
For example, haskellPackages.sdl2-ttf is version 1.0.0, but haskellPackages.sdl2-ttf_2_0_1 is version 2.0.1. Version 1.0.0 has literally none of the high-level API sdl2-ttf is designed to provide, and is therefore practically useless, but it is the default version.
Rather than spending hours learning that, I could have just started using the latest version of the library, like I thought I was. Derivations that wanted 1.0.0 could have said so.

If I am writing a derivation, I am probably not explicitly choosing a channel to build it against. Since different channels will provide different versions, I just want whatever is both compatible with my derivation, and available in that channel. Hence, pattern matching.

Right, but then the build becomes dependent on the channel, not on <nixpkgs>? Or..? (Also I'm assuming you mean writing a nix expression, I don't think it's common to write derivations by hand?)

Since we don't use version numbers in derivations, we now have several packages providing the same library with a different version, but instead of having the same name, they are renamed package-name_1_2_9.

I don't understand what's being renamed if the package's files are the same name? I think you mean the attributes? But if so, what's the problem with attributes being renamed though? Isn't that avoiding name conflicts by specifying the version number in the attribute, how else would you do it?

My biggest issue with renaming packages is that the normal-named package is usually older.

Oh right missed your latest comment! Yep I agree, this is usually the case in my experience though, the behaviour that you want. But if in your experience it's the opposite, then yes it should probably ideally be changed. I guess it might be dependent/related to the packages themselves and what sort of infrastructure they use, what the most used/popular versions of them currently are, and maybe even the maintainers of the package(s). But so, is there a problem apart from that though?

I don't understand what's being renamed if the package's files are the same name?

Right now, I can type nix-shell -p haskellPackages.sdl2-ttf, and have an environment with sdl2-ttf version 1.0.0, which is outdated, and unusable.
I expected to get the latest package, since I didn't explicitly provide a version, but instead I got an old version, and had to learn that the hard way.

then yes it should probably ideally be changed.

What you are saying is that sdl2-ttf should be changed to be version 2.0.1. But how did we get here in the first place? We can't specify versions for dependencies, so all the packages that already existed that depended on version 1.0.0 made 1.0.0 the default version, even though it's practically a different library from 2.0.1. The reason sdl2-ttf_2_0_1 exists is that we can't specify a version for dependencies. If we could, we would never have gotten into this mess.

I guess it might be dependent/related to the packages themselves and what sort of infrastructure they use

haskellPackages are automagically generated from hackage.

But so, is there a problem apart from that though?

At the beginning of this thread, @mayhewluke described several.
The most interesting to me was that he wanted to use an older version on purpose. Since that version has no expression, he can't, but the larger problem is that he couldn't even try.
Right now, expressions have a version, but that version can't be referenced by buildDependencies. If they can, it's an exception to the rule.

You can always use older expressions. Use the pinning method.

Yes. That is an available and usable workaround.
The problem is that we have a mess, and we are working around it instead of fixing it.

I don't consider that a workaround, that's one of the great things of content addressed package managers.

That's not my point. What I am trying to say is that pinning is a great feature that happens to be used as a workaround for this specific problem. Having a great feature doesn't diminish the problem.

@jb55 How do you deal with the situation when your default.nix and shell.nix won't match and cannot match because the "building" of something is separate from the development environment of the same thing. For example some tools are only needed during dev, and not during build. And in other cases, I only need a shell.nix and not a default.nix because the thing I'm developing isn't intended to be a nix deployable package?

How do you deal with the situation when your default.nix and shell.nix won't match and cannot match because the "building" of something is separate from the development environment of the same thing.

I'm not sure I follow. What do you mean "won't match"?

As in they are not the same file due to bringing in different dependencies, or having different environment variables.

So what you are saying is that you don't want to require a version for expressions?
That sounds fine with me. I just want to use versions where they already exist. More specifically, I want version strings to pattern-match derivations that specify a version, so I can get a derivation that specifies a version without giving that derivation a unique name.

What you are saying is that sdl2-ttf should be changed to be version 2.0.1. But how did we get here in the first place? We can't specify versions for dependencies, so all the packages that already existed that depended on version 1.0.0 made 1.0.0 the default version, even though it's practically a different library from 2.0.1. The reason sdl2-ttf_2_0_1 exists is that we can't specify a version for dependencies. If we could, we would never have gotten into this mess.

We got here because when updating sdl2-ttf, someone made a new sdl2-ttf_2_0_1 attribute instead of just using the existing sdl2-ttf attribute. They should have made a new sdl2-ttf_1_0_0 attribute instead for all packages that were still using version 1.0.0.
So in a way, specifying the version number in the attribute is a way of specifying a version for dependencies. However, if I understand what you want correctly now, I think this distinction between the package name and the version (instead of having them together) could help in giving a guarantee that you are getting the latest available version of a package in a channel.
As in, attributes cannot do that because you have no guarantee that the default (i.e. version-less) attribute is really the latest available version (e.g. sdl2-ttf was really only 1.0.0, because imo someone didn't do it right). And names, as they are currently, cannot do that because they always have the version in them, so you are always forced to specify a version (or are you.. maybe there's like some 'glob' syntax or something??). So if we split the name into the 'package name' and the version, then we can just specify a package name, and then nix can look at all the available nix expressions for a particular package name and simply pick the one with the biggest version number. And that comparison that nix would do between the versions would give us this guarantee that we got the latest available version.
Is something like that what you are looking for?

The most interesting to me was that he wanted to use an older version on purpose. Since that version has no expression, he can't, but the larger problem is that he couldn't even try.

Unless I'm missing something, he could though, he just didn't know about pinning.
I think the bigger issue with that though is that I don't really think 'channels' are all that useful for development environments. They're probably OK for users, but you're right channels are too limiting because you're always stuck on what's on your channel. In fact, iirc there was some talk about removing the entire concept of channels from nix, but I'm not sure if it was for the same/similar reasons (and can't find it atm unfortunately).

@vyp It sounds like we are on the same page regarding package names.

Unless I'm missing something, he could though, he just didn't know about pinning.

If I understand correctly, pinning allows a .nix to specify what channel it gets, not what dependency versions it gets. Package versions are still just whatever the channel provides. That may solve the "reproducible" issue, but not the "specific dependency version".
If you are dependent on the channel for reproducible builds, what happens when the channel updates a package?
Channels are effectively nix versions. Using the 17.09 channel is akin to using Debian 9, and using the master channel is akin to using Debian sid/unstable. Neither guarantees a specific package version unless you specify a specific channel revision. If I understand correctly, that is what you mean by "pinning". There are several obvious issues with that:

What's "pinning"? Is this ever explained to a new user? Where?

(Yes, I understand what it is now, but new users probably will not, meaning it doesn't exist to them.)

Your packages never get bug-fix updates
Your packages never get security updates
Broken package expressions never get fixed.
If you need to update one package, you must update them all.

This can be seen as either consistency, or limitation. It depends on the use case.

I think the bigger issue with that though is that I don't really think 'channels' are all that useful for development environments. They're probably OK for users, but you're right channels are too limiting because you're always stuck on what's on your channel.

That's the heart of the issue. The problem is not the limitation of channels, per se, but the fact that we are trying to shove them into a use-case they aren't designed for.
Like I said earlier, channels are like OS versions.

They provide the latest version of a package that they want to provide.
They may provide multiple versions of a package, to fill dependencies.

This introduces a naming conflict that we are working around instead of fixing.

They do not provide older package versions that aren't used by other packages.

That's reasonable, since users don't want them, but developers do.
How about we give developers a clear way to do that?

I don't think channels are the solution here.

No pinning isn't about which channels you get. Pinning pins to a content
addressed commit hash of nixpkgs or any nixpkgs even your own fork.
Channels is orthogonal and probably a mistake and should be removed.
On 27 Sep 2017 06:24, "Thomas Jeffery"<EMAIL_ADDRESS>wrote:

@vyp https://github.com/vyp It sounds like we are on the same page
regarding package names.
Unless I'm missing something, he could though, he just didn't know about
pinning.
If I understand correctly, pinning allows a .nix to specify what channel
it gets, not what dependency versions it gets. Package versions are
still just whatever the channel provides. That may solve the "reproducible"
issue, but not the "specific dependency version".
If you are dependent on the channel for reproducible builds, what happens
when the channel updates a package?
Channels are effectively nix versions. Using the 17.09 channel is akin to
using Debian 9, and using the master channel is akin to using Debian
sid/unstable. Neither guarantees a specific package version unless you
specify a specific channel revision. If I understand correctly, that is
what you mean by "pinning". There are several obvious issues with that:

What's "pinning"? Is this ever explained to a new user? Where?

(Yes, I understand what it is now, but new users probably will
not, meaning it doesn't exist to them.)

Your packages never get bug-fix updates
Your packages never get security updates
Broken package expressions never get fixed.
If you need to update one package, you must update them all.

This can be seen as either consistency, or limitation. It depends
on the use case.

I think the bigger issue with that though is that I don't really think
'channels' are all that useful for development environments. They're
probably OK for users, but you're right channels are too limiting because
you're always stuck on what's on your channel.
That's the heart of the issue. The problem is not the limitation of
channels, per se, but the fact that we are trying to shove them into a
use-case they aren't designed for.
Like I said earlier, channels are like OS versions.

They provide the latest version of a package that they want to
provide.
They may provide multiple versions of a package, to fill
dependencies.

This introduces a naming conflict that we are working around
instead of fixing.

They do not provide older package versions that aren't used by
other packages.

That's reasonable, since users don't want them, but developers do.
How about we give developers a clear way to do that?

I don't think channels are the solution here.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/issues/9682#issuecomment-332324361, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAnHHdLZ5YWX2XKQ5Ohtf3GeJY_wItiNks5smV2LgaJpZM4F4oZ9
.

My mistake.
From my understanding, though, channels simply represent nixpkgs git branches. I suppose pinning, then, is more like git tags. Thanks for the explanation.

Channels is orthogonal and probably a mistake and should be removed.

That's an interesting point, but since, as you said, channels are orthogonal, it would probably be best to have that discussion elsewhere.
Still, that doesn't change the limitations I mentioned, or the current situation with package names sometimes containing versions.

I think pinning is more than just git tags. Git tags is like versions (one can change a git tag to point to something else after it is first published, and there's no computational relationship between the git tag and the data/code that it points to). Pinning is making use of a content addressed hash, the result is a cryptographically guaranteed snapshot of the software you're expecting.

That may solve the "reproducible" issue, but not the "specific dependency version".

But wouldn't reproducibility involve specific dependency versions though? Otherwise in my opinion it's not reproducible.

Neither guarantees a specific package version unless you specify a specific channel revision. If I understand correctly, that is what you mean by "pinning".

What's "pinning"? [...]

Yep, this is mentioned in a link earlier in this thread: https://github.com/NixOS/nixpkgs/issues/9682#issuecomment-139831114
I'm not sure you can pin channels though because if a channel gets updated, it's sha256 would also change. However, I do know that you can do use the -I command line option to specify a nix expression search path for nix commands:
nix-shell -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-17.03.tar.gz

And thus retrieve a channel like that.

[...] Is this ever explained to a new user? Where?

Don't think so, but it definitely should! See https://github.com/NixOS/nixpkgs/issues/27994 for this.

Your packages never get bug-fix updates
Your packages never get security updates
Broken package expressions never get fixed.

Well yeah, if they would then it wouldn't be the same stuff, so not reproducible.

If you need to update one package, you must update them all.

Not sure what exactly you mean here?

This introduces a naming conflict [...]

Actually I'm not sure what naming conflict you're talking about still? In my previous comment I described a use case for separating the package name from the version, so that we can just specify a package name and have a guarantee we'll get latest version available for it in nixpkgs. But I don't think that had anything to do with naming conflicts?

They do not provide older package versions that aren't used by other packages.

That's reasonable, since users don't want them, but developers do.
How about we give developers a clear way to do that?

I don't see how separating the package name and version makes it any different for providing older versions of packages, than just using attributes like we are now?

From my understanding, though, channels simply represent nixpkgs git branches.

I mean conceptually, pretty much, I think! In fact, there's https://github.com/NixOS/nixpkgs-channels which has branches named after channels which track the nixpkgs revisions that the respective channels are using.

So for actually separating out package names and versions though, it might be because of my limited nix expression language knowledge, but I have no idea how you would implement this though, because attributes pretty much always used to refer to packages, in configs, package expressions, modules, everywhere I think. Maybe you could write your own tool or nix function or something that parses out the name and the version for all attributes that begin with the same characters, and then returns the one with the biggest version number.
If however you want to make more substantial changes than writing your own function/tool to deal with this (again, I wouldn't know what it would look like), and you feel strongly enough about this issue, I'd suggest first opening a new issue for this, or sending an email to nix-devel, and then eventually going through the RFC process to do it.

If you need to update one package, you must update them all.

Just wanted to point out that this isn't true - you can take some packages from a pinned revision and some from a channel, or anywhere else. You can see an example of using multiple package sets at the bottom of CMCDragonkai's post. So in practice if you only need to pin one package you can pin just that one package. That package will indeed not get updated (and it will have its own, old, copy of its dependencies), but that's the point of pinning.

So now I completely understand channels. Sorry it took so long. Maybe I need to read more, maybe documentation is not up to par. I'll let you come to your own conclusion on that front.

If you need to update one package, you must update them all.

Not sure what exactly you mean here?

I meant that if you are dependent on channels for dependency versions (which is not pinning like I thought), then you would not be able to select specific package versions, so as soon as you update the channel, you end up updating all dependencies. I was confused, and conflated pinning packages (reality) with pinning channels (nonsense).

I suppose pinning packages is a good way to work, but I don't necessarily see it as the only way. Sure, it's more reliable, but some features can still be seen as limitations in certain context.

You can't pick a range of usable dependency versions

That means you won't ever have a dependency that hasn't been tested, so that's a feature.
That means the user must install the specific version that you tested for your package, so that's a limitation.

This is probably the strong case for channels, since they foster cohesive dependency versions.

Pinning is a great workflow, and content-addressed packages are a core feature of Nix. It's not an easy workflow to jump into, though. We can make it easier.
If I want a package that is a certain version, I need to find out what content-address to get it from.
Right now, that involves scouring Github for the name of a package that corresponds to the version I want. Let alone the fact that there are multiple nonstandard ways to do this, I shouldn't be doing it in the first place. I'd rather add the version string to my name search.
The version string is there, but it's hard to find. It's either ignored, appended to the package name with underscores instead of dots, or something else entirely. Because the version string isn't explicitly used by Nix, it isn't required, and there is no standard way to use or provide it.
All I want is for a standard way for packages to provide version strings, and a standard way to reference them. That way I can search for version strings the same way I search for package names.

That doesn't mean versions are required. If I am looking for a version, it stands to reason that that version exists in the first place. If it did not, I wouldn't be looking for it.
That means dependency lists can specify compatible versions.

Think of it as "loose pinning". Instead of pinning a hash, you are pinning a range of hashes. For convenience, you reference the hashes using a single, list, or range of version strings. If a version string references no hashes, you know immediately. If a version string references several hashes (an expression was updated, but the version did not change), you grab the newest hash.

Now you get the version you wanted right away.
You found the hash that you want, and now you can go right to pinning.
There are no more extra package names. Forget about naming conflicts; we fixed those with content addressing.

You know, as a user I really want this feature, as a developer I can leave with out it (solutions posted upper).
For example I want to install pidgin (solving it right now). What I get - Pidgin 2.13.0 has segfaulted and attempted to dump a core file.. Nice, ok no problem. And the first thing I want to do - try older version, cause in op world it is quite common to have bugs in newer versions. Why I want to do this - fast determine (or just get piece of the info) is it problem in my system or in package. Cause I know that there is good chance that old version will work. And I want it simple without bothering about any of nix config or getting too deep. And there not so much versions needed for example just 2.10.0, 2.0.0, 1.7.0, If they are tested and stable. And what I want - just to get working software right now. I can use old version and maybe invest some of my time with fixing new one, and of course I will switch to new one if it will be fixed. (The cool thing could be - if such problem finds out new version moves to unstable and last working one to stable channels.)
Its all about comfort of each day using and entry level threshold.

@novoxudonoser Flakes should solve many of the concerns raised in this thread.

Finding what older versions are available
I've found that pinning the revision hash solves the problem of how to use an older package version, but leaves the question of how to find the revision that contains the version I want.
If the current version of a package is broken I want to use the previous one, but I found no way other than to going through nixpkgs' git history to find what revision had the package version I wanted.
To solve that I wrote a tool to search all versions of a package that were ever available in a channel, what revision they can be found in, and what command to use to install them https://lazamar.co.uk/nix-versions/

Such tools can be useful, but if a package is broken in nixpkgs... we typically don't want to leave it that way.  Moreover, if there's good motivation, we do keep multiple versions in nixpkgs (say, llvm has many).
BTW, readers of this thread might be often interested in flakes: https://github.com/NixOS/rfcs/pull/49

My impression after reading through this thread, learning about both Flakes and the search tool from @lazamar above, is that we can only use specific revisions of a nixpkgs channel to provide dependencies, but there's still no standard for specifying an individual package version. So if I need to use packages A, B, and C all pinned to specific versions I have to pull them from three different nixpkgs revisions? It seems like flakes solve reproducibility in providing a lock file but don't let me say "actually I need to use rand#0.7.2". This is a fundamental feature in other package managers like cargo and even the accursed pip, and I feel like I must be misunderstanding that it's not present in nix. Anything I'm missing?

Anything I'm missing?

From what I understand is that often arbitrary versions of packages just don't always work perfectly with each other. That's why I think flakes doesn't even try to tackle that problem. Instead we just try to get a convenient tool for reproducing the same software as defined on other machines and rely on nixpkgs being a curated set of libraries whose versions are "known to work"

rely on nixpkgs being a curated set of libraries whose versions are "known to work" with each other.

This makes sense to me. I guess it just feels odd that, for example, if I want to mostly use the 20.03 channel but pin Firefox to version 73, I have to pick myself a version of nixpkgs that had it. There are likely a bunch of revisions that include it. Do I pick the first or last commit that had it? I'm choosing a point in history for the whole build of Firefox. One side effect being that this prevents my Firefox 73 from using new patch versions of its dependencies (if it uses eg. ~1.2.0). If it did, that would break exact binary consistency, but controlling package versions with semver ranges is more powerful for the user AFAICT and gives you more insight into what you're actually installing.

The 1-Step Answer to the Original Issue
Can @mayhewluke or one of the nix maintainers add a version of answer this to the top? Because I just wasted 2.5 hours reading and attempting to understand this whole thread, when all I needed was @lazamar's criminally underrated comment.

Step 0: Find the Version you Want
Find (almost) all versions of a package using @lazamar's absolutely amazing online tool
For example:
I wanted ruby 2.5 so I searched for ruby and found:

Name
Version
Hash

ruby
2.5.5
67912138ea79c9690418e3c6fc524c4b06a396ad

Step 1: Install that version using it's Hash
(I only know this because because CLICKING the hash on @lazamar's site explains it!)
To install that ruby 2.5.5 version globally I ran
nix-env --install ruby -f https://github.com/NixOS/nixpkgs-channels/archive/67912138ea79c9690418e3c6fc524c4b06a396ad.tar.gz

To use that version of ruby in a nix-shell run
nix-shell -p ruby -I nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/67912138ea79c9690418e3c6fc524c4b06a396ad.tar.gz

To require/use that version of ruby in a nix-script run
let
     pkgs = import (builtins.fetchGit {
         # Descriptive name to make the store path easier to identify
         name = "my-old-revision";
         url = "https://github.com/nixos/nixpkgs-channels/";
         ref = "refs/heads/nixpkgs-unstable";
         rev = "67912138ea79c9690418e3c6fc524c4b06a396ad";
     }) {};

     myPkg = pkgs.ruby
in
...

Why is this search functionality not built into Nix?
(probably should open up a new issue for this^ question)
Because Nix hasn't created a standard way for packages to notate a version within their name, @lazamar's method is still somewhat of a workaround. As @thomastjeffery points out

If I want a package that is a certain version, I need to find out what content-address to get it from.

Right now, that involves scouring Github for the name of a package that corresponds to the version I want. Let alone the fact that there are multiple nonstandard ways to do this, I shouldn't be doing it in the first place. I'd rather add the version string to my name search.

The version string is there, but it's hard to find. It's either ignored, appended to the package name with underscores instead of dots, or something else entirely. Because the version string isn't explicitly used by Nix, it isn't required, and there is no standard way to use or provide it.

If the comment above was the answer, why is this thread filled with discussions of:

Flakes
sandboxes
copy-closure
$NIX_PATH
nix-build
overrides
branches of nixpkgs
channels
pinning hashes
and questions about the literal nature of Nix philosophy?

I really don't know

@jeff-hykin I've been uninvolved in this thread for almost 5 years now (for some reason your post was the first I got a notification for :confused:), and there's apparently been a lot of discussion since I last checked in. As a result, I'm too out of touch with the issue to be qualified to make a call as to what solution should be suggested (if any).
I have, however, updated the original post with a suggestion to check out the latest comments, so others don't have to slog through these ~60 comments like you did. Does that sound like a reasonable compromise?

Looks good to me, thank you for doing that! @mayhewluke

You're welcome @jeff-hykin! Happy to help.

Thanks for the synopsis, @jeff-hykin!

Why is this thread filled with discussions of...

This thread (as I saw it) was originally about a usability issue that exists in Nix.
Some people quickly pointed out elegant (from a certain perspective) workarounds to that issue, like pinning.
Some even pointed out that the Nix philosophy (as they understood it) is at its core incompatible with version numbering.
At this point, there should probably be a new issue created and discussion made about how to integrate the reality of software versioning with Nix's UI, implementation, or the Nix philosophy itself.
As I see it, this issue is a usability one, so any workarounds ought to (at the very least) be obvious and well-documented to the casual Nix user.
As it is, the simple act of installing a specific version of a package confronts the casual user with a need to understand the design and implementation of Nix, and choose a confusing (no matter how elegant) solution. Users should never be led to a GitHub issue in order to know how to use their package manager.
I don't think I'm qualified to lead this discussion, but seeing as I was contributing to this discussion nearly 3 years ago, people are still stumbling into this thread, and I haven't really seen a solution...
Here is the new issue: #93327

Thanks for the explanation and kicking off the new issue @thomastjeffery, I appreciate it!

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/nix-shell-p-python37-pkgs-pytest-fails/22533/4

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/best-practice-for-pinning-version-of-individual-packages/6194/7


Hello.js

Just a thought, but I think integrating Hello.js would be super cool.
I can only imagine such an advanced seed would be used for advanced projects with users and stuff, and Hello.js makes using social logins really easy. The rest would be up to the backend tech of course, but a well integrated, fully client side solution for logging with facebook, google etc in your seed would be awesome :)

Hi @Sn3b, definitely a great library and very useful! I'm open to things like that on branches but likely not on master branch since authentication is the type of thing that can be done so many different ways and is left to many opinions. However if you'd like to implement that on a fork and post back on this issue, I'll create a branch and I could merge it there for future use by anyone wanting to use that specific auth lib with the seed. 👍


Reset "nb_failed_login" counter in database after password modification

Expected behaviour
Our kanboard doesn't have access to internet. The captcha doesn't work. So if a user is blocked to the login page, we use administrator to "reset" is password. However the nb_failed_login counter is not reset so the user can't login.
Currently, we edit the database with an external editor to set the nb_failed_login to 0.
Actual behaviour
It should be great to reset the counter when an administrator change a password.
Steps to reproduce
1.WIth the User1, enter a bad password 3 times
2.Modify the password of User1, with the administrator
3.Try to login with User1 and its new password
Configuration

Application version: 1.0.27
PHP version: 7.0.5
PHP SAPI: cgi-fcgi
OS version: Linux 3.10.0-327.el7.x86_64
Database driver: sqlite
Database version: <IP_ADDRESS>
Browser: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:44.0) Gecko/20100101 Firefox/44.0

This now implemented in the development branch. Thanks.


Proposal to always produce a single main audio track by default

Proposal to always produce a single main audio track by default
The transcode-video tool has always produced two main audio output tracks by default when the input audio track is in surround format. One output track is in Dolby Digital AC-3 format, the surround version, and one output track is in AAC format, the stereo version.
Why two output tracks? I blame Apple for this. :)
When videos first went on sale in the iTunes Store, they were all standard definition video and stereo audio in MP4 format. The audio was encoded in a single AAC format track.
While these videos were popular, customers wanted higher quality. So, Apple increased video resolutions to 720p and then 1080p. But, likely due to backwards compatibility issues with their older hardware and software, Apple didn't replace the stereo AAC track. They just added a second track in AC-3 format with 5.1 channels.
This whacky and inefficient arrangement of two main audio tracks became a pseudo standard for output with tools like HandBrake.
Which is why transcode-video has mimicked this lunacy from the beginning, even for MKV output. Although, for MKV, the AC-3 surround track is first followed the AAC stereo track. And the convert-video tool has the unenviable task of flipping these track orders around when it switches between formats.
But I think it's time for this madness to end. To reduce output size and improve performance. if no other reason.
It seems an AAC stereo track is no longer required by most of Apple's software. It's 2017 now and I'm told that Apple TV, QuickTime Player and the iOS TV app can mix down AC-3 5.1 audio into stereo output. Imagine that.
(Although, I would really appreciate some of you double checking me on these behaviors since I don't have an Apple TV or use Apple's playback software.)
So, here's my proposal...
First, modify transcode-video to change the default audio output "width" for the main track from double to surround. This is the textbook definition of a one-line change. That's how easy it is.
Second, modify convert-video to no longer assume that audio tracks need to be swapped when moving from MKV to MP4 format, and vice versa. By default, it should leave those tracks alone. Instead, add a new --swap option to explicitly get the old behavior. Here's how it would be described in the built-in help:
    --swap          swap order of first two audio tracks

Unlike transcode-video, this modification to convert-video would be substantial. But that could be a good thing because it's an excuse to re-examine whether I can support conversion of subtitles.
Who (besides me) is comfortable with this change to the default behavior? Conversely, who really doesn't like the idea?
Let me know what you think. I won't be doing this right away, but I figured we should start discussing it now.
Thanks.

This is what I use as my default when transcoding video.
The change would be more than welcomed for my workflow.

@ericcardenas Excellent! I have long suspected there are others like us. :)
Just out of curiosity, do you play your videos on Apple devices with their software? Or do you use something like Plex on Roku or Chromecast?

I'm not sure I like the idea. While all of Apple's stuff might do a good job of downmixing on the fly, I'm not convinced that's true elsewhere. I find that the on-the-fly downmixing that my kit does is often extremely quiet, but the aac track Handbrake produces is at a decent volume
Then again, I already use my own builds of Handbrake and a patched version of your tools to get the defaults I want, so I doubt it would affect me too much anyway

@samhutchins So you don't like the idea but think it won't really affect you? Bonus points for being an honest man! :)
You make a good point about AC-3 volume levels. This is actually a "feature" of the Dolby Digital format. Go figure. But I find volume levels and normalization to be hideous on most Blu-rays and DVDs anyway so I'm always fiddling with that.

@donmelton
I mostly use Plex and Chromcast to view transcoded stuff.
I find Apples players to fiddley unless I'm watching stuff from the iTunes Store.

@ericcardenas Well put! This is exactly why I use Plex.

@donmelton well, it affects me in that I'll have to update my 'different defaults' patch, and that might be a pain to maintain for convert-video in the future. But hey, that's my problem not yours :-P.
I'll do some testing of on-the-fly downmixing  tomorrow and see if I warm to the idea

@samhutchins Thanks!

@donmelton so long as iOS (particularly phones) handle AC3 properly I'm all for it. I can remux a file after the kiddo goes to bed tonight to confirm.

I'm an iOS user and Apple TV user consistently. I'll adjust scripts as necessary to keep it compatible. If the majority are for this change, that's cool. I'll need to do some testing to confirm no issues with AC3 playback on iOS myself.
Also, you mention that the new TV App supports this. Does this also count for system-wide support? Non-American users have no access to this app.

@rodbrown1988 Videos was renamed TV in iOS 10 on all devices. I believe that's what's Don means above.

@arikalish @RodBrown1988 Yes, I mean the old "Videos" app which was mysteriously renamed "TV."
Thanks for testing!

@rodbrown1988 huh. Good to know! Sorry for assuming otherwise.

@RodBrown1988 Wow. I figured Apple had changed the name everywhere. My mistake. God willing the plumbing is the same. :) Thanks for testing!

@donmelton My experimce with this is sort of two fold. Ses, in theory, many apple hardware devices may be able to down,ix, but if you for example use Infuse, that won't down,ix on the fly. Also, I don,t know how they downkix but I know that you have thought about it.
For me, this boils down to a question of:

how much space and speed would be gained
is "video_transcoding" mainly for apple devices? If not, then leave the default as is

@JMoVS No, Infuse.app can definitely mixdown just about any surround format to stereo. That's one way it used to differentiate itself from the native player. The way it works is essentially identical to how VLC.app and mpv work on your desktop machine.
There is a small but noticeable performance improvement and size reduction leaving out the duplicate stereo channel. And the performance improvement is different depending on the default or selected AAC encoder.
My video_transcoding gem is designed to be a cross-platform front-end for HandBrakeCLI and not strictly for Apple device output. Hell, I never use Apple player software myself. :)

@donmelton I am all for the change to transcode-video- I use kodi
throughout the house and have my defaults set up to always add the
unmodified surround track (regardless of surround encoding format). File
container is always MKV.
However, I use the convert-video gem to handle the conversion / preparation
of any video I want to put on my iPad when traveling, in which case I
appreciate that it creates a 2 channel AAC version of the audio
automatically. This is particularly helpful since I will keep the original
audio track, including DTS, in the "main" MKV. I don't convert every video
for use on the iPad, it is more of an on-demand approach, which is why the
speedy convert-video approach that doesn't re-encode video is a great
solution.
Most importantly though, thank you very much for all of the effort and time
put into this, it has been a huge boon to my family, and a bigger time
savings for myself. It is bigly appreciated! :-)
On Fri, Jan 6, 2017 at 8:22 PM, Don Melton<EMAIL_ADDRESS>wrote:

@JMoVS https://github.com/JMoVS No, Infuse.app can definitely mixdown
just about any surround format to stereo. That's one way it used to
differentiate itself from the native player. The way it works is
essentially identical to how VLC.app and mpv work on your desktop machine.
There is a small but noticeable performance improvement and size reduction
leaving out the duplicate stereo channel. And the performance improvement
is different depending on the default or selected AAC encoder.
My video_transcoding gem is designed to be a cross-platform front-end for
HandBrakeCLI and not strictly for Apple device output. Hell, I never use
Apple player software myself. :)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/donmelton/video_transcoding/issues/113#issuecomment-271051676,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHAaMCbL9Yyab2gIt70RZDuDrZBa0jgcks5rPujDgaJpZM4LdMQC
.

@klogg416 You are very welcome, Kyle! Or, I should say, bigly welcome. :)
Thanks so much for your feedback. This is exactly why I ask all these annoying questions, to get great scenarios like this. And yours is exactly what has me worried about making this change.

@donmelton Just finished testing an AC3-only file on an iPhone 7 w/ iOS 10.2. Works just fine. For those wanting to test, the process I used was a bit complicated:

Pick an existing m4v conversion that has a AAC and AC3 audio.
Run convert-video to get an mkv file.
Load the file into MKVToolNix and deselect the AAC track.
Remux to a new mkv file.
Load the mkv into Handbrake, select the first few chapters and the Apple 240p30 preset.
Change audio to AC3 passthru and remove AAC track.
Let Handbrake do its conversion.
Import resulting file into iTunes library.
Home Share video to iPhone.

If you're starting with an mkv then steps 1 and 2 aren't needed.
I originally tried using convert-video at step 5, but it automatically transcodes and re-adds the AAC track when converting back to m4v. It might be worth adjusting that behavior as well.

@arikalish Thanks for testing so quickly! Just to confirm, that was playback in TV.app?
Also, those steps are quite useful.

@donmelton Yep. TV.app on iOS 10.2

Thinking only of convert-video, I wonder if it makes the most sense to have a decision tree and a couple of optional switches:
1: if AAC or AC3, leave alone.2: if DTS, FLAC, or other incompatible format then convert to AC3 for surround (>2), or AAC for stereo sources (=<2 channels)
Options:- "convert-video --audio stereo-only" replaces to existing audio track with a 2 channel AAC version.- "convert-video --audio stereo-add" effectively mimics current behaviour by downmixing a stereo AAC track and setting it as the first / default source, while leaving the already embedded audio tracks.
Food for thought.
On Fri, Jan 6, 2017 at 9:36 PM -0500, "Don Melton"<EMAIL_ADDRESS>wrote:
@klogg416 You are very welcome, Kyle! Or, I should say, bigly welcome. :)
Thanks so much for your feedback. This is exactly why I ask all these annoying questions, to get great scenarios like this. And yours is exactly what has me worried about making this change.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

@klogg416 Interesting ideas! Thanks.

@arikalish BTW, how did the downmixed surround audio sound on your iPhone? I assume it was 640 Kbps AC-3?

@donmelton honestly, I really only checked that it worked at all. I used the built in speaker and the dialog was clear and loud.

I'll take a look at the source file again tomorrow but it would have been transcode-video's default downmix from a bluray.

@arikalish As long as it didn't sound awful. :) Now all we need is for someone to try your steps on the Apple TV.
And the downmix default would have been 640 Kbps as you guessed.

Probably worth having someone try with the older Video.app on iOS is well

@arikalish Good point! Yes, maybe @RodBrown1988 can do that since we don't have access.

I just tested it on Videos.app on iOS 10.2 (actually 10.2.1 beta 2) on an iPhone 7 Plus. It works correctly. This leads me to believe it was probably added as a capability to AVFoundation/AVKit under the hood with iOS 10.
There is a significant difference in the sound mixdown depending on the input through my AirPods. Pure AC-3 5.1 Surround sounds a little more echoey but also far clearer. Perhaps iOS has a tuned 5.1 surround mixer? It's a different sound, and I'd wager the AAC 2 channel is more correct via Dolby Pro Logic, but the 5.1 sounds quite a bit better nonetheless to my ear.
For the record, my process to quickly rip out the 2.0 Channel was to use Handbrake directly and transcode an existing mp4, with a single Audio channel as an 5.1 AC-3 passthrough of the existing second channel. Transferred via iTunes to the iPhone.

@donmelton just confirming that the source audio was 640kbps. Also, here's a sample file to try playing:
https://www.dropbox.com/s/6hrelhcjb1giwd9/Robots (1080p HD)_AC3.m4v?dl=0
I'll take that down in a couple of days.
@RodBrown1988 I was hoping to avoid the video transcode effort altogether, but since I already had the stripped out file I kept going. Going from the original m4v to the smaller one directly through Handbrake makes more sense. Good call.

Just tested on Apple TV 4 running tvOS 10.1. Both work correctly, no issues. Will test an Apple TV 3 in the other room soon, but considering they put out 5.1 surround natively via HDMI, I couldn't see why it won't work correctly.

@RodBrown1988 Thanks for testing! And now on Apple TV, too. Bonus! :)
If your theory about AVFoundation/AVKit is correct, then I'll bet it works on Apple TV as well since all the devices share the same code base.
I have a similar feeling about the difference between AAC Dolby Pro Logic and Dolby Digital AC-3. The Pro Logic stereo mix seems more... natural? But the AC-3 sounds better. That could simply be due to the much higher bitrate in the AC-3. I don't know.
But either way, it's probably a wash as far as quality is concerned.
I like the shortcut steps you took to create it. Nice optimization there. :)

Audio fails to play on an Apple TV 3 running the latest software updates. So for me, this is a no-go as we use an Apple TV 3 half the time.

@arikalish I won't tell anyone where I got the file. :) Thanks!

@RodBrown1988 D'oh! That's a shame. So the Apple TV 3 is the older, larger hardware, right?

@donmelton older, smaller hardware. Apple TV 4 got larger and got the A8 chip included (wow doesn't that seem archaic now, seeing I hold an A10 in my hand!)

@RodBrown1988 Oh, that's right. Maybe the old single-core A5 chip in the Apple TV 3 is just too underpowered for mixing down or even decoding the AC-3? Bummer.

@RodBrown1988 @arikalish @ericcardenas @samhutchins @JMoVS Crap. I just checked HandBrake's new 1.0+ preset list and most of them still create two audio tracks, AAC and AC-3 in MP4 containers. Even when targeting Android and some other devices. So, the HandBrake team is definitely not moving toward the future. Maybe they have good reason I don't know about?

My only thought that may support using stereo is that the mixing down on each platform may vary dramatically. At least with a second track you avoid problems with:

Legacy platforms that don't support mixing down.
Modern platforms that each have a different mixing down that ends up with inconsistent and possibly lower quality output.

@RodBrown1988 You are probably right about both points. And it's that second quality issue that really worries me.
Obviously the legacy issue would affect you and several others with older hardware directly. But I always think, what would Steve do? :)
But I hate compromising on quality. I don't have hard evidence that's the case but I am very suspicious, as you are.
So... right now I'm leaning toward not making this change. At least, not anytime soon.

@donmelton I was inprecise - yes, infuse can mixdown, but only in the paid version. ;-) (at least that's how it was in the past)

@donmelton I've done a bit of experimenting this morning, and my feeling hasn't changed. On-the-fly downmixing works on the stuff I use, but it's often quiet. Overall I agree with @RodBrown1988; I'm not a fan of the inconsistency, and I'm quite happy to take the hit on encode time and space to get a quality stereo track as well as a quality surround track.
If you do make the change though I'll just patch it back locally so it's not of enormous consequence to me personally, but I do think it's a slightly worse experience for those who use your defaults

@JMoVS Ah, yes, it's the Pro version that can downmix audio. I bought that from the beginning so I forgot. My apologies!

@samhutchins Thanks for the testing and feedback!
Yeah, I'm starting to agree with you and @RodBrown1988 about this. Most users are probably willing to take the hit on time and space to get that high-quality AAC track, especially macOS users.
So it's likely I won't make this change. At least, not anytime soon.

@Eltito-Houston Well, the good news is that even if I don't make this change (and it seems likely I won't), you can get the same benefit by using --audio-width main=surround on the command line. Or, like me, using the new --prefer-ac3 option.

@donmelton  --prefer-ac3 is awesome! :D

@JMoVS Thank you! Glad you like it. What's really awesome is how easy it was to implement. :)

Another argument in favor of leaving the 2 channel audio in there: https://www.reddit.com/r/PleX/comments/5mekra/volume_of_downmixed_dts_and_ac3_audio_tracks_very/
Looks like a bug in Plex, but if they can't get it right I'd be wary.

@arikalish Interesting. I've never seen that problem but I'll keep an eye on it. Thanks.

Hi Don,
I've also done some testing and I'm also in favour of leaving the 2-channel audio track in place. Like others, I have a mix of h/w and s/w at home:  Plex, Infuse Pro v5 or direct play via My Computer (aka iTunes) via Apple TV 4 (and there's no TV app yet for non-US). I found no issues with your defaults, 5.1 AC3-only, 2-channel stereo only, AC3-passthrough/no stereo. Ditto for iPhone 7 and iPad Pro 9.7 and Plex on a Roku 3.
However, anyone using an Apple TV 3 will have a problem with AC3 passthrough. I have an ATV3 connected to the TV so pre-transcoded/down sampled stereo is helpful as there is no sound when playing an 5.1 AC3-passthrough via the My Computer (via remote iTunes). As people said, it's probably due to the A6 and older iOS software running on the kit.
For new users of transcode-video, I think defaulting to the pair of audio tracks is a safe option ... and it's easier for advanced users to add flags to change the default encodings. Perhaps changing the defaults at some point in the future is still a good idea when ATV3 (or any older clients) are justifiably EOL.

One final question about subtitles in convert-video ... what's the proposal? Would it be options to burn-in .srt files when going from mkv to mp4/m4v?
Thanks,
Joe

@rhapsodians Thanks for the testing and feedback! Yeah, I think the consensus is certainly building to not change the default anytime soon. Maybe revisit this in 2018? And your point about new users is well taken.
As for convert-video, burning in subtitles would require transcoding the video and that's really the domain of the transcode-video tool if you want to do that.
However, it appears newer versions of FFmpeg are capable of translating between SubRip format subtitles in MKV files and 3GPP Timed Text subtitles in MP4 files.
There's a good chance I can even get this to work for Advanced SubStation Alpha titles. Maybe even (foolishly) copy DVD-style image-based subtitles between the two formats. I'm in the midst of it now but I'll know soon. And, likely, I'll write a proposal on GitHub with a description of the new behavior to get feedback from all of you.

I think the defaults for audio actually give a good fail-safe for all the reasons discussed (even though there are better ways!). I think 2018 would be a good time to deprecate ATV3 (and older device) compatibility for default settings as things will have moved on enough by then.
And thanks for explaining the idea for convert-video. Totally misunderstood the idea - my bad! I now understand why it's a challenge (and a very good idea too!). I keep all the formats in mp4 formats mainly as a legacy from playing via iTunes/Video.app until I switched to a beefy Plex Server + NAS for home and there are good choices on iOS. Having a means to store one, correct SubRip file per movie/tv show with the options to chose mkv and/or mp4 could be a good addition.

As per Twitter, it's what I do with my scripts. Seems sensible to me.
Only issue I've had is when I was visiting family who had an Apple TV 3: I ended up having to use ffmpeg -map, as the ATV refused to decode the AC3 track.

@chrispoole643 The Apple TV 3 and earlier situation is exactly why I'm going to put this proposal on hold.

Folks, I'll probably leave this issue open for awhile longer to gather more comments, but it's clear to me now based on the feedback I've received so far that I won't proceed with the proposed change to the default behavior. At least, not anytime soon.

Of course, in time that situation may change. Eventually older models of Apple TV and other devices really will be obsolete. And then we can consider moving on. But probably not during 2017.
Anyway, I want to thank you all for that feedback! I take it all very seriously. And I'm very glad you spent the time to consider this with me. You're the best!

OK, I'm closing this for now. But I'll revisit the issue later in year or next year. Thanks again for all your feedback! And feel free to continue commenting here if you want.

OK, folks, it's been over a year since we visited this proposal. I'm re-opening it now for reconsideration.
To be clear, all I'm talking about is changing the default behavior, not the ability to get two main audio tracks, i.e. at least one in stereo AAC format, when you need them. That would still be available by doing this:
transcode-video --audio-width main=double "/path/to/Movie.mkv"

So all your legacy devices would still be supported.
Have any of you upgraded in the last year? Have the capabilities of your various playback software improved?
Please let me know. Thanks.

My playback hardware is the same as last time, but my transcode-video command is so damn long at this point anyway, I don’t mind adding a little more.
However, I suspect most people use the defaults, and I still think AC3 decoding is not as ubiquitous as you’ve suggested; for example these new default aren’t playable on my Roku 2. And as ever, I don’t trust on-the-fly downmixing. Those are my thoughts.

@samhutchins Thanks for the feedback! I was hoping you would repeat here what you put in email.

At this stage I'm capable and would personally use the new defaults. Reduced file size by not including another track is always a win. I'm also curious about running back and re-transcoding to get these and other improvements.
Personally I'd be in favour of the change with some clear documentation about how to get legacy support, but I leave this to the majority to get a feel for how many people it affects.

@RodBrown1988 Thanks for the feedback! Yes, there would be clear (and likely repeated) documentation about how to get the legacy behavior.

Hi Don,
Interesting idea but I think we need to look at the assumptions and consequences.
By just having AC3 5.1, we’re assuming that all sound will be either played through a 5.1 (or even 3.1) system OR the playback h/w and client s/w will have the ability to downmix on the fly without a playback performance hit.
So thinking about setups - I have everything coming via a Zotac i5 fanless Plex server on wired Ethernet and the main playback is on a non-4K Apple TV Plex client via a Denon receiver. So for both h/w and s/w, no issues with the change.
However, my other setups include a new Roku Express+ with Plex on WiFi being played on a 32-inch TV (small room) via the internal TV speakers. Also have an older Apple TV 3 attached to the same TV for rentals, occasional iTunes purchases - streamed and local and My Computer (locally stored mp4s). For Plex, the server side will easily transcode/downsample and I assume no issues with sound/playback via the TV. Not so sure about the ATV3 downsampling to stereo just yet.
The final setup is taking mp4s (and/or mkvs) on an iPad Pro. Here in the UK, we now have the TV app so in the past, I simply copied the content to the iPad and used Video.app, now TV.app to play them. If it’s mkv, it’s either VLC.app or Infuse.app.
So the questions I have would be:   If the default is accepted to be AC3 5.1, does the change impact non-5.1 playback both inside the home and on-the-road? Are there differing requirements depending on the server-side and playback-side devices?
I’d love to some form of h/w and s/w playback table gathering the testing from all the transcoders here so we collectively can see the impact of these stereo edge-cases. Maybe this could be used to collate the assumptions and then help make an informed decision. If the ultimate is AC3 only, then at least the community could refer to the table and assess if they needed to add in extra flags to continue with both audio tracks to maintain compatibility or simply have flexibility.
And as ever, thanks for all the hard work and sharing it with us!!
Cheers,
Joe

For me, it’s all about doing it right, once.
I have a decent Sonos 5.1 setup and a big 4K tv that I use most of the time, however there are times when I’ll be watching my Plex remotely, or I’ve downloaded a couple of movies onto a laptop, iPad or even iPhone.
For the 5.1, this change won’t make any difference but it’s the latter that would niggle at the back of my mind, as ‘something’ else is down-mixing the audio into stereo whether that’s the Plex box, VLC etc. I’d far rather be able to tell these apps to pick the nicely crafted stereo track already there and know that’s already as good as it gets.
How much additional space does a stereo audio track take, and how much additional time does it take to process and mux this second track?
Another thought would be, if you’re using transcode-video in the first place, then you probably already care about enough about quality to want the best of both worlds too.
Either way, I’ll just expand my current command line to include both tracks as normal.
Dave.

On 13 Feb 2018, at 21:45, Don Melton<EMAIL_ADDRESS>wrote:
OK, folks, it's been over a year since we visited this proposal. I'm re-opening it now for reconsideration.
To be clear, all I'm talking about is changing the default behavior, not the ability to get two main audio tracks, i.e. at least one in stereo AAC format, when you need them. That would still be available by doing this:
transcode-video --audio-width main=double "/path/to/Movie.mkv"
So all your legacy devices would still be supported.
Have any of you upgraded in the last year? Have the capabilities of your various playback software improved?
Please let me know. Thanks.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/donmelton/video_transcoding/issues/113#issuecomment-365414989, or mute the thread https://github.com/notifications/unsubscribe-auth/AAYJDUJtvP7VXYP5kEBxbko2I_bYDjL8ks5tUgJ-gaJpZM4LdMQC.

Thanks for the additional info Don. If I were to collapse down my concerns (similar to Dave above), it would be:   Assuming we have only have one 5.1 AC-3 in the mkv/mp4, then when would it be better to also add a dedicated stereo track?
Many of us will add the appropriate flags as required as we’ve followed your lead for many years but I’m thinking of people new to transcoding in this instance.
There will be lots of various h/w and s/w combinations where downsampling will be needed. But do we know of specific combinations where we can identify issues/workarounds or where the defaults definitely don’t work? If we collectively could help test this, then updating the help describing these situations, where adding the -- audio-width main=double flag is the recommendation, would be useful.
Example:   For illustrative purposes only (with no testing done yet):
Default usage (mkv or mp4, AC-3 only audio)

iOS 11.x playback using Video.app/TV.app, VLC.app, Infuse.app - stereo downsampling = automatic, no need for a dedicated stereo track
Plex Server (running on macOS or Windows x86) + Plex client playback on AppleTV, Roku, Amazon Fire, TV Plex client etc. Audio output:  internal TV speakers or stereo L-R speakers only = auto server-side downsampling, no need for a dedicated stereo track
Kodi server on macOS or Windows x86 + DLNA client playback from any device, audio output to internal TV speakers or stereo L-R speakers only = server-side downsampling, no need for a dedicated stereo track
DLNA on Intel Atom (e.g. Intel Stick) or home NAS streaming to TV speakers via built-in DNLA client. May have issues due to the speed of the server CPU, recommendation to have a dedicated, pre-transcoded stereo track to avoid playback buffering or picture-sound sync issues.
iTunes Media Sharing (mp4 + AC3) from iTunes Mac or PC + playback to My Computer on AppleTV 2/3 with playback to TV speakers or attached L-R stereo speakers ... recommendation to have a dedicated, pre-transcoded stereo track as the AppleTV 3 does not downsample. No audio will be played.
Etc.

Hope this is useful.
Joe

I include both the stereo and surround tracks in my encodes as only 2 of the 8 TV's that connect to my Plex server support surround sound. And I've found Plex to be slightly more reliable and faster when it can just passthrough what's being played rather than transcoding it.
But that doesn't really matter. This isn't a discussion about the pros and cons of including two tracks in our transcodes. All you asked about was a change to the default behaviour. And I doubt many of us here are just doing transcode-video "file.mkv". So I say feel free to change it, and we can just add --audio-width main=double from then on, it's no big deal :)
It might be worth posting a straw poll on Twitter though to see what the most popular choice is, then just go with the winner.

@damorrison Thanks for the feedback! When you are using Plex, I assume that you have to keep changing the default audio track depending on whether you're watching locally or remotely? And is remembering to select the correct track on your iPad or iPhone problematic? Or do you transcode to MP4 format?
@rhapsodians Thanks for even more feedback! To answer your first question, it's useful to add a dedicated stereo AAC track when your output pipeline (server and/or device and/or player software) cannot handle AC-3 surround audio directly or via dynamic transcoding. When this project started, that probably happened far more than half the time. These days much less so.
Also, remember that when this project started, the default bitrate for AC-3 surround tracks was 384 Kbps (a foolish attempt by me to emulate iTunes Store content). Dynamic transcoding of that to AAC stereo was perceptible. Now with the default being 640 Kbps for AC-3 surround, there's far less perceptible loss from dynamic transcoding. And essentially none from live mixdown to stereo output by player software such as mpv or VLC.
@rhapsodians You (and others) make a good point about new users to transcode-video! What are their expectations? I honestly don't know but I suspect that new users may have more capable hardware and software than we all did in the old days. Still, a change to the default could leave some of them without playable audio. Then again, that still happens now because some users don't even realize there are two audio tracks and that they may need to select between them.

There will be lots of various h/w and s/w combinations where downsampling will be needed. But do we know of specific combinations where we can identify issues/workarounds or where the defaults definitely don’t work? If we collectively could help test this, then updating the help describing these situations, where adding the --audio-width main=double flag is the recommendation, would be useful.

@rhapsodians Wow! Joe, this is a great question and idea. And your example list is a great start! I agree that we need to start compiling this information. If not to answer this question, then to think to the future when E-AC-3 (Dolby Digital Plus) might become the default surround format.
I should probably consider setting up a Wiki (if not on GitHub then somewhere else) so we can all contribute this kind of data and analysis. Now you've really given me something to think about, Joe! :)
@elliotclowes Thanks for the feedback! And permission to change the default. Seriously! (Yeah, I don't use the defaults myself anymore either. :) And I hadn't thought about doing a poll. I may try that.
BTW, folks, I just love this project because of all the users, like yourselves, who care to provide feedback, opinions and ideas. All of you are what makes this worth doing for me. Again, thank you so much.

Don, glad the feedback is useful and trigged some ideas ... we’re all on your side Gramps and here to help because you completely nailed a problem we all had/have: getting the best out of digitising our physical BR/DVD media. Everyone wins :-) And I remember to good ol’ days too and before I discovered you chatting about this stuff with Rene many years ago. 384kbps, the old scripts and hours of ripping/transcoding were cool at the time :-)
I think you’re right to point out that times have changed (hw/sw) and the value of having some form of  hw/sw recommendations matrix would be informative. As my own next big transition will be into HDR, x265 and 4K (along with the associated audio upgrades), I know I’ll learn a lot from the process again. I’m sure rallying the transcodistas out there to help build up this knowledge would be easy. Just let us know and I suspect you’ll get a lot of help :-)
But back to the original question - I’d make the change to AC3 only as the default so long as the addition of the stereo channel was explained in the ReadMe. And from there, grab yourself a huge glass of wine and enjoy!
And if you’re ever in London, any bottle of vino on me ... within reason of course as I know about you retired people!! ;-)

@rhapsodians It's a deal on the bottle of wine, sir! :)
You are very welcome for the tools. And you're right, there's much research to be done with the move to HDR, HEVC, 4K and E-AC-3. I only wish all of those technologies were accessible at the speeds we're used to with AVC and 1080p. :( But it will get there. Eventually.
I agree that any change in the default behavior will have to be loudly documented in the README and elsewhere. Much like what I did when I changed the default output format from MP4 to MKV.
But, it's entirely likely that I won't make this change this year either. Right now I'm leaning towards tabling this discussion for another year. That would give us time to work together to build up a knowledge base for new and/or less experienced users.

@samhutchins @RodBrown1988 @rhapsodians @damorrison @elliotclowes and everyone else who responded to the original proposal, I'm closing this for now. Of course, I'll probably revisit the issue. Maybe suggesting the default width of the main audio track be stereo instead of double. (That would probably make Sam happy. :) But who knows.
Anyway, thanks again for all your feedback! And feel free to continue commenting here if you want.

@samhutchins OK, your feature has been released in version 0.23.0. Thanks again!
And keep on noodling! :)


pythonPath goes wrong when there are multiple folders under a workspace

Issue Type
Bug
Description
I have a workspace which is a website, containing a folder for vue.js and another for django. In the settings.json of the workspace, the folder of django goes after the the folder of vue.js. The pythonPath of the folder of django is set to the path of a virtualenv. Everything is okay if I just open the folder of django. However, if I open the folders in the workspace, the modules only installed for the virtualenv cannot be found, and my pythonPath is missing from the list of the suggestions for python interpreters.
After looking into the source code of this extension I've got the reason. In src/client/interpreter/locators/index.ts, all the folders in a workspace would share the same object of PythonInterpreterLocatorService, or at least the same serviceContainer. Thus in getLocators, the locators returned for every folder are the same batch of objects. What makes it worse is that the locator for handling the user defined pythonPath, i.e. the CurrentPathService, is a cacheable one, so it will not carry out the search several times for different settings of each folder, but simply store the result for the first folder and make no further searches.
As for my case, I didn't set the pythonPath for my folder for vue.js, which is the first folder in my workspace. As a result, the search for the interpreter based on my settings in the second folder can never come into effect. As a workaround, I changed the order of the two folders in my workspace and now it works as expected. However, if there were several folders in a workspace using different pythonPath, such a workaround would not exist.
VS Code Info
VS Code version: Code 1.20.0 (c63189deaa8e620f650cc28792b8f5f3363f2c5b, 2018-02-07T17:10:15.949Z)
OS version: Linux x64 4.13.0-32-generic
Reproduces only with extensions

Reading your description it seems that the issue originates from an extension and not VS Code itself. Thereby I will close this issue and kindly ask you to report this issue against the extension repository.
Most extensions are on GitHub as well so it should be easy to file it against the extension. If you open an extension on the market place, click on the Support link to find out more.

Thanks again for your feedback!