[BUG] setuptools 62.1.0 breaks project that requires fortran sources (Nonexistent include directory)

setuptools version
62.1.0
Python version
3.10
OS
Linux, macOS
Additional environment information
I am using the version supplied by conda-forge
Description
We have a project that requires compiling fortran source files, and everything has worked fine with setuptools until version 62.1.0 (62.0.0, and all other lower versions, work!). The problem exists using macos and linux with github actions, as well as on my own macos machine. I re-emphasize, that this is a problem ONLY with 62.1.0.
I will admit that I have no idea how to debug this problem: in the build process, all of the source and module files compile, but at the very end when we try to compile the python wrapper, it complains about a non-existent include directory, which prevents the source file from finding a pre-compiled module.
INFO: gfortran:f90: src/PythonWrapper.f95
      f951: Warning: Nonexistent include directory ‘build/temp.linux-x86_64-3.8’ [-Wmissing-include-dirs]
      src/PythonWrapper.f95:2:12:

          2 |         use shtools, only: PlmBar
            |            1
      Fatal Error: Cannot open module file ‘shtools.mod’ for reading at (1): No such file or directory
      compilation terminated.

Expected behavior
compile with no errors, as it has been doing for the past several years...
How to Reproduce
Try to install the pyshtools project: https://github.com/SHTOOLS/SHTOOLS using pip install -v -e .
Output
Please see the github actions output for this run:
https://github.com/SHTOOLS/SHTOOLS/runs/5987783398?check_suite_focus=true

Hi @MarkWieczorek, I was trying to reproduce this problem, but I am having other issues with the dev environment that does not seem to be related with the issue you are pointing out. This is my attempt/output so far:
> docker run --rm -it python:3.10-bullseye /bin/bash

apt update
apt install -y g++ gfortran libfftw3-dev libblas-dev liblapack-dev
cd /tmp
git clone https://github.com/SHTOOLS/SHTOOLS.git
cd SHTOOLS
python -m pip install -U setuptools pip numpy
python -m pip install -v -e .

  Running command python setup.py egg_info
  /usr/local/lib/python3.10/site-packages/setuptools/dist.py:516: UserWarning: Normalizing 'v4.9.1' to '4.9.1'
    warnings.warn(tmpl.format(**locals()))
  *** pypandoc is not installed. PYPI long_description will not be formatted correctly. ***
  INSTALLING SHTOOLS v4.9.1
  INFO: get_default_fcompiler: matching types: '['arm', 'gnu95', 'intel', 'lahey', 'pg', 'nv', 'absoft', 'nag', 'vast', 'compaq', 'intele', 'intelem', 'gnu', 'g95', 'pathf95', 'nagfor', 'fujitsu']'
  INFO: customize ArmFlangCompiler
  WARN: Could not locate executable armflang
  INFO: customize Gnu95FCompiler
  INFO: Found executable /usr/bin/gfortran
  searching SHTOOLS in: build/temp.linux-x86_64-3.10
  INFO: fftw_info:
  INFO: customize UnixCCompiler
  INFO:   fftw3 not found
  INFO:   libraries rfftw,fftw not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
  INFO:   fftw2 not found
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/SHTOOLS/setup.py", line 243, in <module>
      setup(**metadata)
    File "/usr/local/lib/python3.10/site-packages/numpy/distutils/core.py", line 135, in setup
      config = configuration()
    File "/tmp/SHTOOLS/setup.py", line 194, in configuration
      fftw_info = get_info('fftw', notfound_action=2)
    File "/usr/local/lib/python3.10/site-packages/numpy/distutils/system_info.py", line 585, in get_info
      return cl().get_info(notfound_action)
    File "/usr/local/lib/python3.10/site-packages/numpy/distutils/system_info.py", line 851, in get_info
      raise self.notfounderror(self.notfounderror.__doc__)
  numpy.distutils.system_info.FFTWNotFoundError:
      FFTW (http://www.fftw.org/) libraries not found.
      Directories to search for the libraries can be specified in the
      numpy/distutils/site.cfg file (section [fftw]) or by setting
      the FFTW environment variable.
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

I got the apt install -y g++ gfortran libfftw3-dev libblas-dev liblapack-dev from the github main. Is there any other requirement?
Can you share with us a complete reproducer (building upon the example I posted above)?

Unfortunately, I don't know what the above problem is (I don't have a linux box to tinker on...). Obviously, the fftw3 library isn't being found. We had these kind of problems in the past, and then just decided to switch to conda. Perhaps you could try installing the dependencies using conda with the following environment?
channels:
    - conda-forge
dependencies:
    - fftw>=3.3.8
    - liblapack>=3.8
    - numpy
    - scipy>=0.14.0
    - matplotlib-base>=3.3
    - astropy>=4.0
    - xarray
    - requests
    - pooch>=1.1
    - tqdm
    - cartopy>=0.18.0
    - gmt>=6.1.1
    - pygmt>=0.3.0
    - palettable>=3.3
    - jupyter
    - pip
    - ducc0>=0.15

Perhaps you could try installing the dependencies using conda with the following environment?

Any chance you can share with us a reproducer using a conda environment for that then?

You could try this
conda create -n t620 python=3.10 setuptools=62.0.0
conda env update -n t620 -f environment.yml
conda activate t620
pip install -v -e .

conda create -n t621 python=3.10 setuptools=62.1.0
conda env update -n t621 -f environment.yml
conda activate t621
pip install -v -e .

Thank you very much @MarkWieczorek!
When I tried to replicate the problem locally on my machine, I noticed that the build script is looking for build/temp.linux-x86_64-3.10 while the actual directory is temp.linux-x86_64-cpython-310.
This seems to be a direct consequence of the fact that v62.1.0 updated the bundled version of distutils to use a more specific tag for the build directory in order to avoid other issues previously reported.
In theory this update should not be a problem if you are relying completely on setuptools, but it seems that SHTOOLS uses functions/classes from numpy.distutils which might be incompatible. (Mixing and matching different implementations/patches of distutils is very unlikely to work).
Indeed when I try to run SETUPTOOLS_USE_DISTUTILS=stdlib pip install -v -e . I observe a completely different error:
    INFO: gcc: src/_SHTOOLSmodule.c
    src/_SHTOOLSmodule.c:20:10: fatal error: fortranobject.h: No such file or directory
       20 | #include "fortranobject.h"
          |          ^~~~~~~~~~~~~~~~~
    compilation terminated.

which seems to be caused by an incorrect setup of the environment and not directly related to the caused described here.
Please note that users relying on the behaviour of the stdlib's distutils (or any other version of distutils distributed by a 3rd-party) may be required to set SETUPTOOLS_USE_DISTUTILS=stdlib or even pin an older version of setuptools as suggested in the changelog.
Could you try running it again in your machine with SETUPTOOLS_USE_DISTUTILS=stdlib, just to see if it works? (Unfortunately my machine does not seem to have all the correct dependencies and might be missing libraries/headers)

Build scripts may need to be updated to the new path names if they are using the internal pathname. build/temp.linux-x86_64-3.10 clashes when you are using an alternative interpreter, like pypy, so the actual build tag is now used, temp.linux-x86_64-cpython-310; otherwise rebuilding for pypy can cause hard errors or two sets of binaries in the wheels. Hard coded build paths (like in https://github.com/scikit-hep/awkward-1.0/pull/1407) have to be loosened. In that case, it was a hard coded grep for the build directory in the setup.py.

This hardcoded assumption about the build directory is now wrong:
https://github.com/SHTOOLS/SHTOOLS/blob/8d83c42d1313d5624c4db8c2e57300c5d819834e/setup.py#L177

Do you need me to test anything? If I understand the comments above, this is an issue with the last release.

The project needs to implement the changes pointed out by Henry.
The latest release contain changes that improve the build system and avoid clashes between different Python implementations. Normally, there would be no problem, however, the setup script is relying on the existence of a particular folder.

@MarkWieczorek, I think I will close this issue for now.
The proper fix for the problem seems to be the one pointed out by Henry in https://github.com/pypa/setuptools/issues/3263#issuecomment-1096800735 and https://github.com/pypa/setuptools/issues/3263#issuecomment-1096802732.
Please let us know if that works for you.
Reverting to the previous behaviour is not really an option since it would cause other problems to appear for other packages.

@abravalheri I strongly disagree with closing this issue. The suggestions that you note above are not feasible to implement for a project that makes use of setuptools. (How is one to even do this, given that there is no documentation?) Furthermore, this bug breaks many things besides just compiling code on my own computer:

It breaks all github action workflows, because they install the latest version of setuptools. This was resolved by pinning setuptools to 62.0.0.
It breaks multibuild that builds python wheels for the same reason. Again, I need to manually pin setuptools to the non broken version.

Given that the bug is in 62.1.0 and not 62.0.0, it seems to me that the best course of action is to simply revert whatever broke in 62.1.0. For info, 62.1.0 was release 11 days ago. 62.0.0 was released 17 days ago. Are there really any packages that can only run on 62.1.0 and NOT 62.0.0?

Hi @MarkWieczorek, please note the following:

whatever broke in 62.1.0: you are assuming here that setuptools 62.1.0 is broken, which is not exactly the case. Setuptools change an internal implementation detail which was not part of the public API.

The previous version had an undesirable behaviour that was fixed. Note that this undesirable behaviour was both conceptual and pragmatic, the string used for the build directory was not specific enough and ended up shared between build for different Python implementations which broke some packages using cibuildwheel. I don't think it is reasonable to revert to this incorrect behaviour.
Can you give a try to the fix proposed by Henry? That seems to be definitive the solution for this problem.

Sorry if I appeared annoyed. I am just not that familiar with how setuptools does things, and I wasn't expecting a minor release to introduce breaking changes :)
You mentioned that the problem is with this line in my setup.py:
build_lib_dir = "{dirname}.{platform}-{version[0]}.{version[1]}"

Above, it is noted that it should look like something like this: temp.linux-x86_64-cpython-310, but its not clear how to do that. I presume that I am not supposed to hardcode "cpython" and "linux"...
Digging in the other link provided, it looked like they used the following on windows:
"temp.*-{sys.version_info[0]}*{sys.version_info[1]}"

but I really don't know what the wildcards are doing and if sys.version is the same is version.

This probably comes from this StackOverflow question:
https://stackoverflow.com/questions/14320220/testing-python-c-libraries-get-build-path
There's a big warning there "This is an internal detail of setuptools (based on the bundled version of distutils), and so is not guaranteed to be stable." even before I edited it to include the current Setuptools. The internal build directory is a private detail and building the name yourself is going to be prone to breakage.
Look at the edited response to see how you can build this (and there's also an answer that avoids that using build_lib from build, if that's accessible).

AFAICK, old releases are not generally made - once 62 is out, 61 does not continue to get releases. That's the only reason to "cap" to a major version - you should only do that if the library continues to produce fixes for that major version. Very, very few Python packages do that, so you should not major-version cap. So capping <61.1 is no "worse" than your theoretical <62; both are equal; until you fix it (which should be done soon!), <61.1 is fine. Click 8.1 and pytest 7.1 are other examples of breaking releases on .1, that's pretty common in the Python ecosystem. See https://iscinumpy.dev/post/bound-version-constraints/ for warnings about trying to use SemVer blindly.

You said multibuild is broken? I'm not seeing an issue there.

Thanks to both of you for the help. The edited response on stack overflow is a lot of help, and will probably be found by people in the same situation as me!

Just to let you know: I just realized that the build directory uses the new format for the library, however, there is a second src directory that uses the old format. Everything works with 62.1.0, but I just wanted to let you know because it looks like this might be inconsistent (I never specified by src directory, so this was generaged automatically by setuptools):
$ ls build
src.macosx-10.9-x86_64-3.10/		temp.macosx-10.9-x86_64-cpython-310/

Hi @MarkWieczorek, I am not entirely sure if I understood this last comment. Can you specify a bit more (maybe showing the directory tree and pointing out what is wrong)?
Could it be the case of some cache? Is is the case if you remove build, dist, src/*.egg-info the second folder still exists?

The src directory is not a cache and is generated whenever I build my project. I am just pointing out that the directory naming convention is no longer consistent following 62.1.0. Here is the directory tree of the build directory.
$ tree build
build
├── src.macosx-10.9-x86_64-3.10
│   ├── build
│   │   └── src.macosx-10.9-x86_64-3.10
│   │       └── src
│   │           ├── fortranobject.c
│   │           └── fortranobject.h
│   ├── numpy
│   │   └── distutils
│   │       └── include
│   │           └── npy_cpu_dispatch_config.h
│   └── src
│       ├── _SHTOOLS-f2pywrappers.f
│       ├── _SHTOOLSmodule.c
│       ├── fortranobject.c
│       └── fortranobject.h
└── temp.macosx-10.9-x86_64-cpython-310
    ├── build
    │   └── src.macosx-10.9-x86_64-3.10
    │       └── src
    │           ├── fortranobject.o
    │           └── fortranobject.o.d
    ├── ccompiler_opt_cache_clib.py
    ├── ccompiler_opt_cache_ext.py
    ├── fftw3.mod
    ├── ftypes.mod
    ├── libSHTOOLS.a
    ├── shtools.mod
    └── src
        ├── BAtoHilm.o
        ├── BAtoHilmRhoH.o
        ├── CilmMinus.o
        ├── CilmMinusRhoH.o
        ├── CilmPlus.o
        ├── CilmPlusRhoH.o
        ├── ComputeDG82.o
        ├── ComputeDMap.o
        ├── ComputeDm.o
        ├── Curve2Mask.o
        ├── DHaj.o
        ├── DownContFilter.o
        ├── EigValSym.o
        ├── EigValVecSym.o
        ├── EigValVecSymTri.o
        ├── FFTW3.o
        ├── GLQGridCoord.o
        ├── MakeCircleCoord.o
        ├── MakeEllipseCoord.o
        ├── MakeGeoidGrid.o
        ├── MakeGradientDH.o
        ├── MakeGravGradGridDH.o
        ├── MakeGravGridDH.o
        ├── MakeGravGridPoint.o
        ├── MakeGrid2D.o
        ├── MakeGridDH.o
        ├── MakeGridDHC.o
        ├── MakeGridGLQ.o
        ├── MakeGridGLQC.o
        ├── MakeGridPoint.o
        ├── MakeGridPointC.o
        ├── MakeMagGradGridDH.o
        ├── MakeMagGridDH.o
        ├── MakeMagGridPoint.o
        ├── NormalGravity.o
        ├── PLegendre.o
        ├── PLegendreA.o
        ├── PLegendreA_d1.o
        ├── PLegendre_d1.o
        ├── PlBar.o
        ├── PlBar_d1.o
        ├── PlON.o
        ├── PlON_d1.o
        ├── PlSchmidt.o
        ├── PlSchmidt_d1.o
        ├── PlmBar.o
        ├── PlmBar_d1.o
        ├── PlmIndex.o
        ├── PlmON.o
        ├── PlmON_d1.o
        ├── PlmSchmidt.o
        ├── PlmSchmidt_d1.o
        ├── PreGLQ.o
        ├── PythonWrapper.o
        ├── Random.o
        ├── SHAdmitCorr.o
        ├── SHBias.o
        ├── SHBiasAdmitCorr.o
        ├── SHBiasK.o
        ├── SHBiasKMask.o
        ├── SHCilmToVector.o
        ├── SHConfidence.o
        ├── SHConvertCoef.o
        ├── SHExpandDH.o
        ├── SHExpandDHC.o
        ├── SHExpandGLQ.o
        ├── SHExpandGLQC.o
        ├── SHExpandLSQ.o
        ├── SHFindLWin.o
        ├── SHGLQ.o
        ├── SHLocalizedAdmitCorr.o
        ├── SHMTCouplingMatrix.o
        ├── SHMTDebias.o
        ├── SHMTVar.o
        ├── SHMTVarOpt.o
        ├── SHMagPowerSpectra.o
        ├── SHMultiTaperCSE.o
        ├── SHMultiTaperMaskCSE.o
        ├── SHMultiTaperMaskSE.o
        ├── SHMultiTaperSE.o
        ├── SHMultiply.o
        ├── SHPowerSpectra.o
        ├── SHPowerSpectraC.o
        ├── SHRead.o
        ├── SHRead2.o
        ├── SHReadJPL.o
        ├── SHReturnTapers.o
        ├── SHReturnTapersM.o
        ├── SHReturnTapersMap.o
        ├── SHRotateCoef.o
        ├── SHRotateRealCoef.o
        ├── SHRotateTapers.o
        ├── SHSCouplingMatrix.o
        ├── SHSCouplingMatrixCap.o
        ├── SHSjkPG.o
        ├── SHSlepianVar.o
        ├── SHTOOLS.o
        ├── SlepianCoeffs.o
        ├── SlepianCoeffsToSH.o
        ├── SphericalCapCoef.o
        ├── Wigner3j.o
        ├── YilmIndexVector.o
        ├── _SHTOOLS-f2pywrappers.o
        ├── _SHTOOLSmodule.o
        ├── _SHTOOLSmodule.o.d
        ├── djpi2.o
        └── ftypes.o

I am not sure what is happening here, it is indeed very confusing.
I am also finding some problems to run the build myself to investigate what is happening:
> docker run --rm -it python:3.10-bullseye /bin/bash

apt-get update
apt-get install -y g++ gfortran libfftw3-dev libblas-dev liblapack-dev
cd /tmp
git clone https://github.com/SHTOOLS/SHTOOLS.git
cd SHTOOLS
git checkout develop
python -m pip install -U setuptools pip numpy build
python -m build --no-isolation

...
numpy.distutils.system_info.FFTWNotFoundError:                                                                                FFTW (http://www.fftw.org/) libraries not found.                                                                                Directories to search for the libraries can be specified in the                                                                                numpy/distutils/site.cfg file (section [fftw]) or by setting                                                                                the FFTW environment variable.

Is there any step missing in the SHTOOLS missing?

distutils should find the fftw library, but we always had problems with this working properly. The easiest way would be to install fftw with conda.

Not sure if related, but I got this when building py-numpy now:
INFO: building library "npymath" sources
WARN: Could not locate executable gfortran
WARN: Could not locate executable f95

Version of py-setuptools is @62.3.2 though.


Doc for axonio read_protocol

Hello @samuelgarcia! Thanks for submitting the PR.

In the file neo/io/axonio.py, following                are the PEP8 issues :

Line 20:100: E501 line too long (161 > 99 characters)
Line 23:100: E501 line too long (158 > 99 characters)
Line 24:100: E501 line too long (112 > 99 characters)
Line 25:100: E501 line too long (163 > 99 characters)
Line 26:100: E501 line too long (232 > 99 characters)
Line 27:100: E501 line too long (202 > 99 characters)
Line 28:100: E501 line too long (105 > 99 characters)
Line 33:100: E501 line too long (121 > 99 characters)
Line 33:122: W291 trailing whitespace
Line 34:100: E501 line too long (121 > 99 characters)
Line 35:100: E501 line too long (116 > 99 characters)
Line 35:117: W291 trailing whitespace
Line 36:100: E501 line too long (122 > 99 characters)
Line 37:100: E501 line too long (120 > 99 characters)

This shoudl close #597

Coverage decreased (-0.004%) to 48.22% when pulling e159957437929518643a7339e668012b514307fc on samuelgarcia:axonio_doc into 937946b7bd03423a3f3a666c5c903897fc30c082 on NeuralEnsemble:master.


feat: improve messaging when both the title and every commit must be semantic

When every commit is required to be semantic, the PR hint should not only say "a commit".

@dargmuesli this has now been deployed (apologies for the delay!)


gh-754: Allow undefined count

Closes gh-754
Allow count be undefined... looks like it was supported before in js version as well as cli doesn't provide it but then count is typed as compulsory.
TODO:

[ ] add test
[ ] fix failures

@Shinigami92 does it make sense to you?

@Shinigami92 does it make sense to you?

Yeah, could make sense.
I'm just thinking about if we could use const { count = 1 } = options or something like that 🤔

hi @Shinigami92! Thanks for the tip to use default value :+1: i think it's ready to merge. Would appreciate if you take a look and approve


feat(linter/tree-shaking): pass CallExpression cases

#2839  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @mysteryven and the rest of your teammates on  Graphite


Is the debug instructions still accurate?

When I got a crash, I got this message:
  Most likely, this means you have an outdated version of homebrew-cask. Please run:

      brew update && brew upgrade brew-cask && brew cleanup && brew cask cleanup

  If this doesn’t fix the problem, please report this bug:

      https://github.com/caskroom/homebrew-cask/issues

However, when I ran brew update && brew upgrade brew-cask && brew cleanup && brew cask cleanup I got this error:
Already up-to-date.
Error: No available formula for brew-cask
==> Removing dead symlinks
Nothing to do
==> Removing cached downloads
Nothing to do

I thought brew-cask is now caskroom/cask/brew-cask instead?
BTW: The README has the same text at the bottom.
PS: I know why I got the error, I removed a directory it was scanning with rmdir -p and it got grumpy. So don't worry about that.

Yes, it is still accurate.

I thought brew-cask is now caskroom/cask/brew-cask instead?

Same thing. caskroom/cask/ refers to the repo.

Then why did I get an error about it not existing? Just curious.

I don’t have enough information to answer that. If it is causing an issue, try this first. Only if it persists, please open a new issue, and follow the procedure.


feature: logging++

Is your feature request related to a problem? Please describe
Time seems ripe for us to have a deep look at logging and make sure it is consistent across the different components, it is well documented, and testable.
Currently, we use hclog as well as zap. The combo works mostly fine, except for when one needs to deal with both at the same time, in which case it can create some potential and actual confusion (for example, see https://github.com/veraison/veraison/pull/55) .
If possible, describe a solution
One thing that I like to achieve is more homogeneity, i.e., picking one library, if possible.
Another thing is we bundle all log related things (e.g., the Logger interface to mock and use in testing, a bunch of COTS logger constructors with sensible Logger configuration choices) into a component that can be used across the codebase. This would in future accommodate other related things, for example function to anonymise PII, etc.

Inherited from original Feature request received from:
https://github.com/veraison/veraison/issues/56


Clarification and Justification Requested for OpenGPTs License Terms

We are seeking detailed clarification on the OpenGPTs License terms, particularly given its contrast with common open-source licenses like Apache 2.0 and MIT. OpenGPTs is promoted as a free alternative to OpenAI's GPT and Assistant APIs, yet its licensing terms present potential contradictions and practical challenges, especially considering LangChain's use of the MIT license for other projects.
Key Issues and Contradictions:

Sublicensing Restrictions:

The license prohibits sublicensing, limiting the ability to grant a license to third parties.
Impact: Restricts distribution and modification possibilities in projects that typically allow sublicensing (e.g., Apache 2.0, MIT).

Hosted Services Prohibition:

Forbids providing the software as a hosted or managed service.
Impact: Limits use in cloud-based or SaaS models, prevalent in modern software ecosystems.

License Key Mechanism and Distribution Challenges:

The license requires a license key but restricts its distribution and tampering.
Impact: Creates a contradiction where users of a derivative project need a license key from OpenGPTs but cannot obtain it through sublicensing.
Open Question: How can developers distribute projects using OpenGPTs if each end-user is required to obtain a separate license key directly from OpenGPTs?

Compatibility with Other OSS Licenses:

Concerns about compatibility with permissive OSS licenses like Apache 2.0 and MIT due to OpenGPTs restrictions.
Open Question: Can OpenGPTs be legally used in projects under Apache 2.0, MIT, GPLv2, GPLv3, etc., without violating its terms?

Scope of Hosting Restriction:

Unclear whether the hosting restriction applies to the entire derivative work or only the parts utilizing OpenGPTs code.
Open Question: Does the hosting limitation impact the derivative work as a whole, or is it limited to the OpenGPTs components?

Contradiction with OSS Principles:

The limitations contradict the openness and freedom associated with OSS.
Open Question: How do these restrictions align with OSS philosophy, particularly considering LangChain's use of MIT for other projects?

Request for Clarification:
We request thorough clarification on the above points to better understand the intended use, limitations, and practical implications of integrating OpenGPTs into various projects.
Justification for License Choice:
Additionally, we seek justification for the choice of this specific licensing model for OpenGPTs, especially given its contrast with the more permissive MIT license used in other LangChain projects.
The license seems to be designed to make the project something for building personal tools, never shared with anyone. It grants distribution rights, yet the ambiguities force conservative interpretation, and it doesn't seem to be possible to redistribute any project using OpenGPTs given the licensing restrictions. All this goes against the nature of the open-source software community as a whole. Please explain what is so special about OpenGPts that it cannot use an SPDX OCI license like Apache 2.0 on which much of this world runs on.

So I take it Harrison won't be answering this? If you can't spare a few minutes after over 3 weeks to explain your rationale and the gaping holes in the proprietary licence which prevents who knows how many from touching this project, then I take the point. Never for builders in the first place. Case closed.

Sorry for the delay @wfjt  - had completely missed this.
This license chosen was the same as the Airbyte License for largely the same reasons as when they first moved (https://airbyte.com/blog/a-new-license-to-future-proof-the-commoditization-of-data-integration). Namely we do not want huge companies to take this project and start offering it as a hosted service themselves (as has happened to many  other OSS projects)
I think the Airbyte License FAQ (https://airbyte.com/company/license-faq) can probably better respond to general questions - I think this part is the most salient
There are only a few high-level limitations. You cannot:

Provide the products to others as a managed service. For example, you cannot sell a cloud service that provides users with direct access to Airbyte. You can sell access to applications built and run using Airbyte.

The intent is for this to not affect anyone except for huge companies trying to offer it directly as a hosted offering.
More than happy to answer specific questions to your use case should you have any more!

The license you have chosen fails to accomplishes those goals.

It is a violation of the license of this library to incorporate it into existing open source libraries. It is a violation of this license to use this code as a reference to write a corresponding class in Hugging Face's transformers library, code to export models trained in GPT-NeoX or Megatron-LM to be compatible with this library, or code to evaluate models in this framework with LM Eval, HELM, and BigBench.
The entire clause about keys is logically incoherent. Taken literally, it seems to mean that the software functionally cannot be redistributed at all.
Projects that incorporate your code cannot be commercialized.

This seems like an extremely poor license to use for your stated goals. If you are really adamant about wanting to give companies the middle finger and don't believe in open source philosophical principles, why not do something like the LLaMA2 license or the Stability AI Community License which only limits the commercialization of the software (and in some cases only does so for large corporations?). I still think that those licenses are philosophically non-desirable, and depending on the exact wording may not address #1. But it would certainly make the code less radioactive.

I have no trouble with choosing to use a non-OSI license, but I do find it troublesome to simultaneously name the project as OpenGPT and describe it as "an open source effort to create a similar experience to OpenAI's GPTs and Assistants API".
It is source-available, not open source.
As someone who was curious about using the code, and am familiar with popular OSS licenses, I feel pretty discouraged by it having a non-standard license with unclear terms. I simply don't know if my use would fall under the restrictions, as they seem fairly broad and vague.
I think @StellaAthena's points are great, and further my concern.

Thank you for changing the license to MIT, now I can use this great project!


Add numbered viewer variables to be able to access multiple created viewers

Closes #40
Enable to access all the available viewers by adding a viewer{number} variables as needed. For example, if you create from a single script to viewers you will be able to access/interact with both: one via a viewer0 variable and the other one via viewer1. A preview:

Notes

One thing that this implementation is not covering is to remove the variable referencing the viewer once the viewer is closed/deleted. So, with this implementation, you will be able to access the deleted viewer from another viewer/napari mainwindow instance and then getting an error when trying to do any operation with the closed/deleted viewer variable. Tried setting some clean up logic at the console closeEvent using either self.push (setting the respective numbered viewer variable to be something like None) or calling the shell reset_selective and drop_by_id methods but none of those calls helped as far as I was able to check 🤔 Any ideas to handle this are appreciated!

Would it be possible to save the current viewer number and remove the reference in closeEvent?


PowerToys doesn't upgrade from 0.15.2 to 0.16.0, and after an upgrade, downgrades

Environment
Surface Laptop 3.
0.15.2 and 0.16.0
Windows build number: [run "ver" at a command prompt]
PowerToys version:
PowerToy module for which you are reporting the bug (if applicable):

Steps to reproduce
Run 0.15.2
Doesn't update automatically or prompts.
Manually upgrade to 0.16.0
After reboot, I got a message that PowerToys detected an older version and offered to uninstall.
I said yes.
After that, I discovered I was back with 0.15.2

Expected behavior

0.15.2 prompts and updates to 0.16.0
Actual behavior
0.15.2 didn't give me a notification.
After installing 0.16.0 on top of 0.15.2 and rebooting (for a different reason) a prompt offered to remove the old version, after which 0.15.2 version was installed.
Also note I had to reconfigure after going from 0.15.2 to 0.16.0.

Screenshots

Hi @amnonig
the MSIX was experimental and it's currently deprecated, sorry for the inconvenience, please manually uninstall 0.15.2 and install 0.16.0.

Thanks, I didn't know that.

Fixed by https://github.com/microsoft/PowerToys/pull/1920
Fix available in 0.16.1 https://github.com/microsoft/PowerToys/releases


add option to allow for an iteration count

Add option to allow an iteration count to be used with the enc command
All changes comming from pkcs8 command
Checklist

[x] documentation is added or updated
[x] CLA is signed

Description of change

Next step could be to integrate also the -v2 option ?

+1

FYI, I am ready to go to add PBKDF2 as soon as this PR will be closed.

I don't think this is the way to go. Under the hood this is using EVP_BytesToKey() which is no longer recommended. I know that by default enc is already using EVP_BytesToKey() with an iteration count of 1 - but it doesn't make any sense to me to add new ways of using a non-recommended approach. I'd suggest a separate -pbkdf2 option to force PBKDF2. You could also have a -iter option but if you use it then it would force PBKDF2.

For me this small PR was a preliminary stage before introducing PBKDF2. I mean -iter could be used for both legacy BytesToKey() and PBKDF2 options. But if I understand you correctly you would prefer another PR dedicated to PBKDF2 leaving BytesToKey() using only 1 iteration. Right ?

For me this small PR was a preliminary stage before introducing PBKDF2. I mean -iter could be used for both legacy BytesToKey() and PBKDF2 options. But if I understand you correctly you would prefer another PR dedicated to PBKDF2 leaving BytesToKey() using only 1 iteration. Right ?

Correct. I don't think we should provide additional capabilities on top of BytesToKey() - that will only encourage its use.

With due respect, I find it hard to believe that such a tool can be use for real-life encryption ;-)
For me, enc is more about learning, experimenting, debuging or testing.
And there is probably still a lot of software around calling EVP_BytesToKey() internally with an iteration count != 1 and for which it might be helpfull adding it to the command (ethical-hacking ?).
Don't you think it's a bit like the discussion some monthes ago about keeping MD5 or DES in OpenSSL 1.1.0 ?
However, you are the best judge, so if you just close this PR, I will propose another to support PBKDF2 in compliance with your opinion.

With due respect, I find it hard to believe that such a tool can be use for real-life encryption ;-)
For me, enc is more about learning, experimenting, debuging or testing.

People do use it.

And there is probably still a lot of software around calling EVP_BytesToKey() internally with an iteration count != 1 and for which it might be helpfull adding it to the command (ethical-hacking ?).

I don't see any demand for that.

Don't you think it's a bit like the discussion some monthes ago about keeping MD5 or DES in OpenSSL 1.1.0 ?

I'm not sure which specific discussion you are referring to.

However, you are the best judge, so if you just close this PR, I will propose another to support PBKDF2 in compliance with your opinion.

I do not consider this PR to be mergeable in its current state. If you want to implement PBKDF2 in a separate PR then I will happily look at that.


Change max_decoding_message_size and other config

I'm running into the following error when reading large rows:
Error, decoded message length too large: found 5134061 bytes, the limit is: 4194304 bytes

The BigtableClient generated by prost-build exposes functions like max_decoding_message_size to be able to configure the grpc client. On the bigtable connection I get to these settings via fn client() returning a BigTable instance, on which I can call get_client() to get to the right instance to call these config methods on and create a new internal client. However I cannot recreate the BigTable instance with this updated client.
So this works, but then I can't update the read_bigtable_client nor the bigtable_connection.
        let mut bigtable_client = bigtable_connection.client();

        let inner_client = read_bigtable_client
            .get_client()
            .clone()
            .max_decoding_message_size(usize::MAX);

It would be nice to have a set_client function on both of these types of structs.

Thanks for the contribution.


Enhance recombination algorithm to support 4x4 matrices (#4145)

This pull request enhances the recombination algorithm to support 4x4 matrices.
Changes:

Modified the recombination function to handle 4x4 matrices.
Added test cases to verify the correctness of the implementation for 4x4 matrices.

Related Issues:

Issue #4145

This is my first contribution to public project, so any feedback or suggestions for improvement would be greatly appreciated.

Thanks again for adding this feature, I made a couple of small changes and it's now on the main branch as commit https://github.com/lovell/sharp/commit/60c5c5083dd12fae04db2640bb5c40564bb0728d for inclusion in the v0.33.5 release.


Add Footer to FAQ's page

Find the footer component  and import it in faqs page and use it to display a footer on faq page.  Update the year in footer to 2023
You can take reference from how it is done on home page.

claim

Claim


Build Fails

I have a React Native project and trying to use this package to create a native midi module. I am having a problem that once I import this Package the build will fail. It fails even without importing and using it in any of my source code. Simple adding the package to the project causes my build to fail. I am not sure if there are configurations that are require or should be avoided.
I attached my Xcode logs
Build target test_react-macOS_2022-06-03T02-11-57.txt

I don't have experience with React Native but from Googling, it seems that it's possible to use Swift Packages in a RN application -- but there may be some boilerplate required. I won't have a chance to look deeper until next week.
Not sure if any of these help:

https://stackoverflow.com/questions/65130609/how-to-solve-undefined-symbols-for-architecture-x86-64-on-nsscanner-swift-pod
https://cuneyt.aliustaoglu.biz/en/using-native-ui-libraries-in-react-native-for-with-swift/

Found a working solution from the first link that you provided. Looks like there may be a conflict between React Native when MidiKit is added. Would be nice if there is a simple fix as I don't know what other issues may occur  in React Native after making the changes below. The working solution is:

The solution: Remove the swift-5.0 search path, or you will get an error about undefined symbols (try it if you don't believe me), do this -> https://github.com/react-native-community/upgrade-support/issues/62#issuecomment-622985723

Basically for React Native users trying to use MidiKit you have to remove all search path for debug and release.

Thanks for confirming that. I'll include this tip in troubleshooting documentation. It may be a workaround but if it works then it's worth noting for the time being.


fix(crux-ui): fixes the sidebar height to scrollable

It adds the sidebar to scrollable when the user screen height is small

Thanks for the PR!
I beleive it would be better solution to just simply let the menu's background grow vertically, so it fills the available space:

@polaroi8d what do you think?

Well, is this solution for only the sidebar settings section or complete sidebar?

If it is the complete sidebar to grow vertically, then paddings, margins respective to each element would dynamically change based on screen size

I think it should affect the complete sidebar.

No problem I will try that and share the screenshot for review

when the sidebar height grows vertically, each and every sections in the sidebar would take up same space. These above images are of different screen heights. Would like to know what you guys think about this ?

we can also do one thing, based on screen size , we can update the padding and text-size changes dynamically based on some event making sure that every section should take up equal space and fits to height of the sidebar.

It would be good if the first part, 'settings,' were fixed to the bottom of the screen, while the rest remains at the top as it is now, and if the screen is very large, then only a space between 'settings' and the top ('dashboard, project, component') grow. What do you think?

That means settings would become sticky to sidebar bottom and rest of components would appear on top even if the screen becomes larger ? @polaroi8d

Exactly!

would like you to see variations for different screen sizes? @polaroi8d

would like you to see variations for different screen sizes? @polaroi8d

Yes it's cool.

@polaroi8d pushed changes!

Well , it is because of the fixed font size , padding between elements not allowing settings section elements to be in the left side bar.

I feel elements font size should decrease with the screen height decreases making sure to have every element on the screen.

@polaroi8d hey I got you, I am working on it.

Due to inactivity, I'm closing this issue. Thanks for the contribution if you have time and energy we can. easily reopen this.


Mutable references: mut cells everywhere

Proposal for supporting mutable references. Closes #140
Original solution
The envisioned solution was to wrap mutably borrowed references in a case class MutRef[@mutable T](t: T) when the borrow happens. At the end of the loan's lifetime, we would reassign the changes made on the MutRef object's field back to the original object. A little example:
pub fn main() {
  let mut x = 1;
  let y = &mut x; // borrow happens
  *y = 2;
  assert!(x == 2) // original used again, hence loan over
}

Would have been translated to the following in Scala:
object Mutable {
  final case class MutRef[@mutable T](var t: T)
  def main() = {
    var x = 1
    val y = MutRef(x) // borrow as a wrap in MutRef
    y.t = 2
    x = y.t // propagate changes back to the original
    assert(x == 2)
  }
}

In the above example, the reassignment of changes to the original x is necessary because of Scala semantics. In other cases where the original is an object, Scala semantics would be correct for our use-case but Stainless is not able to track the aliasing that is created by the wrap into MutRef. The following examples shows a weird case of this:
pub struct Thing<T> { field: T }

pub fn change_thing<T>(thing: &mut Thing<T>, t: T) {
  *thing = Thing { field: t };
}

pub fn main() {
  let mut thing = Thing { field: 123 };
  change_thing(&mut thing, 456);
  assert!(thing.field == 456);

  let thing2 = &mut thing;
  change_thing(thing2, 789);
  assert!(thing.field == 789);

  let thing3 = &mut thing;
  *thing3 = Thing { field: 0 };
  assert!(thing.field == 0);
}

This is translated to the following in Scala (only the main is shown):
  def main() = {
    val thing = Thing(123)
    change_thing(MutRef(thing), 456) // MutRef is not bound, last binding is 'thing'
    assert(thing.field == 456)

    val thing2 = MutRef(thing)  // MutRef is bound to 'thing2'
    change_thing(thing2, 789)
    thing = thing2.t
    assert(thing.field == 789)

    val thing3 = MutRef(thing)  // MutRef is bound to 'thing3'
    thing3.t = Thing(0)
    thing = thing3.t
    assert(thing.field == 0)
  }

Stainless correctly tracks aliasing in the first block, where the borrow happens in the argument of the function. This is probably due to the fact that the last binding of that value is still thing. Then, the changes are correctly applied back to thing in the AntiAliasing phase.
For the two later blocks, this is not the case. The newly created MutRef(thing) is bound to a name and Stainless correctly tracks changes to the MutRef objects thing2 and thing3. But it does not track the fact that they are still aliasing thing and therefore it doesn't propagate changes back to thing.
The example demonstrates, how the manual reassignment of changes to the original object at the end of the loan's lifetime is necessary to guarantee correctness. However, to implement this revealed itself to be very hard/impossible by only looking at the HIR (high-level intermediated representation in rustc) because the HIR does not know when the lifetime of a reference is over. Rustc performs borrow checking and lifetime resolution only later in the compilation and uses the MIR (mid-level IR), a sort of CFG that is much further apart from Stainless' AST, to do so.
Work-around
Thanks to @romac, I found a solution to that problem. Instead of propagating the changes back to the original, we can lift a variable that is later borrowed into a MutRef from the start. In that manner, borrowing simply becomes aliasing and Stainless can track the changes to the field of the object all the way through.
object Mutable {
  final case class MutRef[@mutable T](var t: T)
  def main() = {
    var x = MutRef(1)
    val y = x // borrowing simply becomes aliasing
    y.t = 2
    assert(x.t == 2) // changes are already performed on the original object
  }
}

Thanks to the fact that y is a val, Stainless is able to track the aliasing that is introduced and hence correctly verifies the above example.

Closing in favour of #164. Most of the changes here are contained there.


I there a way to change the filter operator?

Custom filter component:
      filterComponent: ({ columnDef, onFilterChanged }) => (
        <TextField
          variant="standard"
          onChange={(e: React.ChangeEvent<HTMLInputElement>) => {
            onFilterChanged(columnDef.tableData.id, e.target.value); // send here the operator
          }}
        />
      ),

Filter object in query for async pagination:
{ column: 'some_column', operator: '=', value: 'something' } // <-- The operator is always '='

Question
Is there a way to change the operator for something else, like <, >, ~=, etc?

You can use the customFilterAndSearch prop of the columns:
customFilterAndSearch?: (
    filter: any,
    rowData: RowData,
    columnDef: Column<RowData>
  ) => boolean;

I did a PR solving this.


new blog: 献给新生女儿的礼物：基于wechaty和语义开放平台打造的育儿机器人

Please make sure to check the following boxes creating your pull request, thank you very much!

[x] Add Blog Header, including title, author, image, categories, tags, etc.

[x] Keep all filenames lowercase, and use - to connect words instead of space. e.g. 2017-10-06-wechat-pc-impactor.md instead of 2017-10-06-WeChat PC Impactor.md

[x] Embed video/pdf from YouTube/Bilibili/local (if there's any) by using {% include iframe.html src="URL_or_PATH" %}. (Learn more about how to embed file/url in the post by reading this blog post

[x] Save photo to local before publishing, to a folder named by ${MONTH}-${YOUR_BLOG_SLUG}/ under /assets/2021/ directory.

[x] Select a beautiful and meaningful teaser image for your blog

[x] Create your contributor profile (if you are a first time contributor)

To learn more about the contribute guideline, visit: https://wechaty.js.org/docs/contributor-program/

Congratulations to you for having your daughter!
Please make sure the following requirements have been met for your PR before we can continue reviewing it:

The PR has passed all the CI tests and turns green. (you can execute all tests locally by running npm test)


Issue completing scoped packages

Reproduce:

Have a scoped package installed in npm
Type enough to trigger completion on it, eg. import whatever from '@
This will bring up a completion list with @foo
Hit tab to insert the suggestion
You get  import whatever from '@@foo';

Expectation: that the @ not be doubled.

Yep, I know about this. I'm going to rewrite autocomplete mechanism, but don't have enough time for this.
PRs are welcome!

From a quick look at the code it isn't clear to me if that comes from this module or whatever calls it. If you have some pointers, I'm happy to give it a shot. (Also happy to try without pointers but it might take longer to get started.)


iPad Not working

Hi @tr3v3r
I am using Epson TM-T88VI printer with same network wifi
EscPosPrinter.discover()
.then((printers) => {
alert(printers[0]);
})
.catch((e) => console.log('Print error:', e.message));
alert not coming please assist me.
library v1.5.0

@devudr after investigation, I realized that the current SDK version doesn't support your printer.
I need to update it.

hi @tr3v3r
I need to add object of printing iteration please assist
.

@tr3v3r Just updated to v.1.8.0   with Android SDK updates. Could please confirm that printer is now could be detected on Android?

Hi @tr3v3r  Currently I am working only for  iOS Platform using v1.5.0.
yes its working

Hi @tr3v3r
printing.initialize().text("hello, is it me you're looking for").send();
how can i add text from loop its not working for me


TODO: code documentation

Deal breaking bugs
PHP-Files/Classes are missing - severely impacting the value of the documentation export
Incorrect output when typehinting a variable as string - makes output confusing
Would be nice at a later stage
Add support for @method tags
Documentation itself is mostly done.
docs/* will be deleted, once an export (either md-files or HTML (preferred)) can be exported without issues.
See https://fai.mr42.me/ for current status.

Stuck with the current docs for now, as https://github.com/yiisoft/yii2-apidoc/issues/162 have been encountered here as well.

Documentation itself is mostly done.
docs/* will be deleted, once an export (either md-files or HTML (preferred)) can be exported without issues.

Documention is on https://www.yii.rocks/yii2-fontawesome-inline/


Should support translations and semver versioning

I'm interested in refering your checklist(s) from a document, which might be used in the public sector. However in the public sector, there is at times quite strict obligations regarding, native language support. For now, I will most likely do an unoffical translation to Swedish of the document shown as https://best.openssf.org/Concise-Guide-for-Developing-More-Secure-Software and friends . But it would be even better if:

The guides were open for translation and took such contributions.
The versions were following semver so one would know when to update the translation
The online version had a choose lang-button so one could choose between available translations.

Would be great if It would be considered. Thanks.

Hello @janderssonse !  Thanks for the issue.  Historically we have accepted members' contributions for translations of our work.  We'd love to collaborate with your to help ensure we're able to support that.  I'll add my pal @david-a-wheeler into this, as he has had some experience working with translations of LF docs in the past, and can speak to the capabilities if we're able to add a button like you have suggested to select the language the viewer desires.

@SecurityCRob and @david-a-wheeler Here is an Swedish translation from me, initiated by @thomaswennersten which might act as motivator for OpenSSF to also work on the translation possibilities. Let me know if you need a few more "quality approvals proof-reads" before acceptance, I can think of few who would be interested in this. Of course you can always AI it back and see that it roughly matches your intent.

Concise-Guide-for-Evaluating-Open-Source-Software.sv.md


A cold-start "flutter test" takes multiple seconds

Running "flutter test" used to be pretty fast (a few hundred milliseconds until the test runs — not great, but workable). Now though it takes entire seconds to start up the test. This makes iterating on a test an unbearably slow process.
cc @mraleph @aam @a-siva @tvolkert

We can reduce script start-up time 0.3 seconds by switching to app jit (https://github.com/flutter/flutter/pull/27749).

Overall test performance is also influenced by the flutter_tester (see https://github.com/flutter/flutter/issues/22873). I noticed that running from a locally built profile build reduced overall test execution time to ~70% of the normal time.

We also spend about 1-2 seconds compiling to kernel for a test (more if there are more dependencies). We might be able to speed this up by incrementally compiling the dependencies into kernel once and reusing them for subsequent test runs. Currently we use the frontend server to keep the kernel in memory which helps for multiple tests, but everything gets thrown away on subsequent test runs. This will require https://github.com/flutter/flutter/issues/27141 to integrated into the tool.

We spend a couple hundred ms in the tool before even getting to the test execution:

jonahwilliams-macbookpro2:hello_world jonahwilliams$ flutter test -v
[  +25 ms] executing: [/Users/jonahwilliams/Documents/flutter/] git rev-parse --abbrev-ref --symbolic @{u}
[  +32 ms] Exit code 0 from: git rev-parse --abbrev-ref --symbolic @{u}
[        ] upstream/master
[        ] executing: [/Users/jonahwilliams/Documents/flutter/] git rev-parse --abbrev-ref HEAD
[   +8 ms] Exit code 0 from: git rev-parse --abbrev-ref HEAD
[        ] initialize_from_dill
[        ] executing: [/Users/jonahwilliams/Documents/flutter/] git ls-remote --get-url upstream
[   +9 ms] Exit code 0 from: git ls-remote --get-url upstream
[        ] git@github.com:flutter/flutter.git
[        ] executing: [/Users/jonahwilliams/Documents/flutter/] git log -n 1 --pretty=format:%H
[  +12 ms] Exit code 0 from: git log -n 1 --pretty=format:%H
[        ] 216a65b6697f2ca5cf3501e50c3f9592334fbe4b
[        ] executing: [/Users/jonahwilliams/Documents/flutter/] git log -n 1 --pretty=format:%ar
[  +12 ms] Exit code 0 from: git log -n 1 --pretty=format:%ar
[        ] 2 days ago
[   +1 ms] executing: [/Users/jonahwilliams/Documents/flutter/] git describe --match v*.*.* --first-parent --long --tags
[  +52 ms] Exit code 0 from: git describe --match v*.*.* --first-parent --long --tags
[        ] v1.2.0-140-g216a65b66
[ +314 ms] switching to directory LocalDirectory: 'test' to run tests
[   +1 ms] running test package with arguments: [-r, compact, --concurrency=6, --, /Users/jonahwilliams/Documents/flutter/examples/hello_world/test/hello_test.dart]

From investigating we spend approximately 0.4 seconds gathering analytics data. Times below on hello_world test
All usage and version gathering removed:
time flutter test
00:01 +1: All tests passed!
DONE: 1815 ms

real	0m2.428s
user	0m2.568s
sys	0m0.572s

master:
jonahwilliams-macbookpro2:hello_world jonahwilliams$ time flutter test
00:01 +1: All tests passed!
DONE: 1955 ms

real	0m2.803s
user	0m2.754s
sys	0m0.689s

We also spend a couple hundred milliseconds checking pub get. We could replace that by first checking if the pubspec.yaml was modified after the .packages.
without analytics, pub get, and with app jit snapshot we're almost fast:
jonahwilliams-macbookpro2:hello_world jonahwilliams$ time flutter test
00:01 +1: All tests passed!
DONE: 1916 ms
real	0m2.200s
user	0m2.334s
sys	0m0.540s

And to back up my claims about the host_profile, flutter test in the flutter gallery takes
default: 101463 ms
profile:  54277 ms
So actually about half the time

Currently we use the frontend server to keep the kernel in memory which helps for multiple tests, but everything gets thrown away on subsequent test runs.

This is not quite right - on subsequent runs incremental compiler gets bootstrapped from previously generated dill file: https://github.com/flutter/flutter/blob/master/packages/flutter_tools/lib/src/test/flutter_platform.dart#L274

Understood, we might be lumping in some additional start/warm-up time in the first compile then. I see If I can get a better breakdown of that time

We've moved the needle on the benchmarks a bit by switching to an app-jit snapshot: https://github.com/flutter/flutter/pull/27749

More improvements from the last change:
flutter_test_performance without_change_elapsed_time_ms:

From 2945 ms down to 2420 ms

flutter_test_performance6266 interface_change_elapsed_time_ms:

From 6800 ms down to 6200 ms

Even with an empty test directory it's slow

Any update on this please, 41 seconds to run 42 tests on a recent macbook pro.  TDD & githooks when checking in would be much more possible if this were faster.
It would help to know what to expect in terms of a solution for this so we can possibly look at workarounds

btw, I wrote this a couple of years ago when someone asked me how fast I think tests should run in flutter:

So, I've read a number of blogs and books over the years regarding TDD and they all say that when employing TDD, each individual unit test should complete quickly. This is because it means I can make a change to a single line of code and instantly see the success of that change. Personally I think waiting a second is kind of ok but anything more can damage 'flow' which is important in developing software (pschyologically flow is the thing that makes programming addictive and fun!) Also, if I have to wait too long people will be less likely to write tests, they won't get written which will then affect the robustness of the software being created.
This stack overflow post concludes that individual test should take <10ms up to 1s (over 1s is too slow), the highest voted answer suggests the entire suite of tests should run under 1s
https://stackoverflow.com/questions/10486/unit-test-execution-speed-how-many-tests-per-second
Roy Osherove states in his 'The Art of TDD' as one of his four TDD principles that tests should be fast.
Mark Seeman states that the entire suite should complete in 10 seconds; if you have thousands of tests each test should complete in <10ms. (He also mentions the fail-fast principle
http://blog.ploeh.dk/2012/05/24/TDDtestsuitesshouldrunin10secondsorless/
Kent Beck in Test Driven Development by Example
Quick to execute: If the tests don’t run fast, they won’t get run. If they don’t get run, they won’t be valuable. If they aren’t valuable, they won’t continue to be written.

For some reason, when I run it from the terminal flutter test it takes more time.
Anyone can help me to check why is that?
From command line
00:52 +2838: All tests passed!
From the IDE

FYI, I am running a unit test in Cloud Run.
Dart is much faster than Flutter for the same unit test.

@danielgomezrico should we put anything into the number of tests in the two different scenarios not matching (2838 vs 2839)? Also, how many tests vs how many test files do you have?
And is it an open source project by any chance? (so we can try to recreate the exact setup)
Maybe @scheglov or @devoncarew or @stevemessick can tell us how the intellij plugin runs tests. (I would naively assume that it'd just run flutter test but if the numbers are correct that can hardly be true).
@atreeon are the project you're working on (and the 42 tests) open source? (so we can try to recreate the exact setup)
Someone a year and a half ago (not at saying it because of the age of that comment) ran flutter pub run test --- I don't even know what that does... I'm guessing it's way different that flutter test.

The IDE does flutter test. There is an additional argument like --machine to cause it to emit json results.

Does anyone know how to fix this? I have around 900 tests and they take around 55 seconds. That is really long most of the tests should not take seconds. I am using flutter test --coverage.

@tjcampanella How long does flutter test take without --coverage?

Running without the --coverage flag it takes 17 seconds vs 40+ with the flag. It would be ideal to run it every time with that flag to see how the code coverage is changing.

/cc @a-siva @liamappelbe it seems that coverage processing is still incurring a large overhead (see above), is this expected? I see that reported issues (like https://github.com/flutter/flutter/issues/86722) are closed - anything planned in this area?

@tjcampanella The fix from that bug relies on having an up to date version of Dart, so make sure you're using a recent version. It roughly halves the time it takes to run with coverage. But there's always going to be overhead, due to the RPCs that have to be run.
If you're still having issues, you can try this mitigation where you wrap all your tests in a single script that runs all of them. It's annoying to have to do that, but it should give you a dramatic speed boost in your case, since you're running hundreds of small tests.

@liamappelbe I am using dart 2.16.1, so I don't think that is the issue. I will look into that mitigation ig. Thanks.

My 8 empty tests (1 file) are taking 5+ seconds to simply throw UnimplementedError for each test using both flutter test and the Dart test runner in VS Code. My whole test suite of ~150 tests takes over 30 SECONDS to complete, with each test file taking 3-7 seconds just to start running tests.

launch.json (for Dart test runner)
    {
      "name": "Dart: Run all tests",
      "type": "dart",
      "request": "launch",
      "program": "./test/"
    },

Timed test run:
00:05 +1 -7: Some tests failed.
real    0m9.532s
user    0m0.075s
sys     0m0.138s

I am using nested group blocks, but I don't believe this should cause such an enormous delay that makes TDD extremely slow

Example section of code
void main() {
  group('AuthCubit:', () {

    late AuthCubit          authCubit;
    late MockLogIn          logIn;
    late MockLogOut         logOut;
    late MockRestoreSession restoreSession;

    setUp(() {
      logIn         = MockLogIn();
      logOut        = MockLogOut();
      restoreSession= MockRestoreSession();
      authCubit     = AuthCubit(
        logIn         : logIn,
        logOut        : logOut,
        restoreSession: restoreSession,
      );
    });

    tearDown(() {
      authCubit.close();
    });

    test('initial state is unauthenticated', () async {
      expect(authCubit.state, equals(const AuthStateUnauthenticated()));
    });
    group('logIn():', () {
      //? Create late variables that all stages use, if needed
      setUp(() {});
      //? Create subgroups for stages this function will step through
      group('On failure:', () {
        setUp(() {});
        test('emits unauthenticated state', () async { throw UnimplementedError(); });
      });
      group('On success:', () {
        setUp(() {});
        test('emits authenticated state', () async { throw UnimplementedError(); });
      });
    });
  });
  // ... + similar empty tests for other functions ...
}

Is nested grouping like this the problem, or could it be an issue with my setup? Pretend I know nothing about Flutter or Dart; what questions would help find the cause and solutions for this?

@SabianF how do you run your example files?
When I run from the terminal I get this:
$ flutter clean
[...]
$ time flutter test test/test_test_*
[...]
00:09 +0 -20: Some tests failed.

real    0m10.940s
user    0m15.826s
sys     0m3.289s

$ time flutter test test/test_test_*
00:02 +0 -20: Some tests failed.

real    0m3.340s
user    0m4.926s
sys     0m1.925s

On the first run (after I've cleaned) it has to compile the flutter framework etc again, so it takes longer. In this case 9-10 seconds to compile and run the tests.
On the subsequent run it can reuse the existing compile (it still has to read all the files though, to make sure things haven't changed) so it skips the compilation of the flutter framework etc and the test runs are thus much faster. 2-3 seconds in this case.
The number-examples you gave did not show this. I wonder if either caching is - for whatever reason - not in effect, or if caching in effect (neither of the runs was "the first run") and it's just slow.
A number of questions that might help:

(again) How did you run your examples?
What are your numbers with/without a flutter clean first?
What CPU does your computer have?
How much CPU is used by other processes?
What version of flutter are you using?
What's your operating system?
What type of disk and/or how fast is the disk in your machine? (the one having flutter on it and the one with the project on it)
Whats the output from e.g. flutter test -v test/test_test_1.dart
If your on Linux (or possibly mac, although I'm not sure  --- but give it a try) what's the output from DART_CFE_ENABLE_INCREMENTAL_COMPILER_BENCHMARKING=true flutter test -v test/test_test_1.dart

Another interesting thing, though, is that your reported user time is so low compared to mine. So also

what does /usr/bin/time -v flutter test test/test_test_1.dart print?

an also @mraleph might know more about the output for user and sys that I do...

Quoted reply by @jensjoha ("@SabianF how do you run your example files...")

@SabianF how do you run your example files?
When I run from the terminal I get this:
$ flutter clean
[...]
$ time flutter test test/test_test_*
[...]
00:09 +0 -20: Some tests failed.

real    0m10.940s
user    0m15.826s
sys     0m3.289s

$ time flutter test test/test_test_*
00:02 +0 -20: Some tests failed.

real    0m3.340s
user    0m4.926s
sys     0m1.925s

On the first run (after I've cleaned) it has to compile the flutter framework etc again, so it takes longer. In this case 9-10 seconds to compile and run the tests.
On the subsequent run it can reuse the existing compile (it still has to read all the files though, to make sure things haven't changed) so it skips the compilation of the flutter framework etc and the test runs are thus much faster. 2-3 seconds in this case.
The number-examples you gave did not show this. I wonder if either caching is - for whatever reason - not in effect, or if caching is in effect (i.e. neither of the runs was "the first run") and it's just slow.
A number of questions that might help:
(again) How did you run your examples?
What are your numbers with/without a flutter clean first?
What CPU does your computer have?
How much CPU is used by other processes?
What version of flutter are you using?
What's your operating system?
What type of disk and/or how fast is the disk in your machine? (the one having flutter on it and the one with the project on it)
Whats the output from e.g. flutter test -v test/test_test_1.dart
If your on Linux (or possibly mac, although I'm not sure --- but give it a try) what's the output from DART_CFE_ENABLE_INCREMENTAL_COMPILER_BENCHMARKING=true flutter test -v test/test_test_1.dart

Another interesting thing, though, is that your reported user time is so low compared to mine. So also
what does /usr/bin/time -v flutter test test/test_test_1.dart print?

an also @mraleph might know more about the output for user and sys that I do...

To be as specific as possible, I did them with

The Run button directly above the test/group blocks inside the test files
VS Code's Run with debugging and Run without debugging according to the launch.json I showed
time flutter test <dir>

What are your numbers with/without a flutter clean first?

Everything in my previous comment was with NO flutter clean
I just retested all files using flutter clean

Commands
flutter clean && time flutter test test/test_tests/test_test_1_test.dart
flutter clean && time flutter test test/test_tests/test_test_2_test.dart
flutter clean && time flutter test test/test_tests/test_test_1_2_test.dart
flutter clean && time flutter test test/test_tests/*

Results
# test_test_1
00:15 +0 -5: Some tests failed.

real    0m22.504s
user    0m0.030s
sys     0m0.121s

# test_test_2
00:18 +0 -5: Some tests failed.

real    0m25.694s
user    0m0.091s
sys     0m0.090s

# test_test_1_2
00:19 +0 -10: Some tests failed.

real    0m31.579s
user    0m0.045s
sys     0m0.213s

# all 3 files in one run
00:19 +0 -20: Some tests failed.

real    0m26.931s
user    0m0.030s
sys     0m0.152s

What CPU does your computer have?

i5 4590 (4-cores) @ 3.3GHz

How much CPU is used by other processes?

20% at most, usually less

What version of flutter are you using?
Flutter 3.0.1 • channel stable • https://github.com/flutter/flutter
Framework • revision fb57da5f94 (13 days ago) • 2022-05-19 15:50:29 -0700
Engine • revision caaafc5604
Tools • Dart 2.17.1 • DevTools 2.12.2

What's your operating system?

Windows 10.0 build 19041 x64

What type of disk and/or how fast is the disk in your machine? (the one having flutter on it and the one with the project on it)

Samsung 860 EVO 250GB Sata3 SSD

Whats the output from e.g. flutter test -v test/test_test_1.dart

Verbose results
$ flutter test -v test/test_tests/test_test_1_test.dart
[  +93 ms] executing: [C:\src\flutter/] git -c log.showSignature=false log -n 1 --pretty=format:%H
[ +571 ms] Exit code 0 from: git -c log.showSignature=false log -n 1 --pretty=format:%H
[   +2 ms] fb57da5f945d02ef4f98dfd9409a72b7cce74268
[        ] executing: [C:\src\flutter/] git tag --points-at fb57da5f945d02ef4f98dfd9409a72b7cce74268
[  +74 ms] Exit code 0 from: git tag --points-at fb57da5f945d02ef4f98dfd9409a72b7cce74268
[   +1 ms] 3.0.1
[  +16 ms] executing: [C:\src\flutter/] git rev-parse --abbrev-ref --symbolic @{u}
[  +61 ms] Exit code 0 from: git rev-parse --abbrev-ref --symbolic @{u}
[   +1 ms] origin/stable
[   +1 ms] executing: [C:\src\flutter/] git ls-remote --get-url origin
[ +111 ms] Exit code 0 from: git ls-remote --get-url origin
[   +1 ms] https://github.com/flutter/flutter
[ +228 ms] executing: [C:\src\flutter/] git rev-parse --abbrev-ref HEAD
[  +51 ms] Exit code 0 from: git rev-parse --abbrev-ref HEAD
[   +1 ms] stable
[ +157 ms] Found 1 files which will be executed as Widget Tests.
[  +54 ms] Artifact Instance of 'AndroidGenSnapshotArtifacts' is not required, skipping update.
[   +2 ms] Artifact Instance of 'AndroidInternalBuildArtifacts' is not required, skipping update.
[  +12 ms] Artifact Instance of 'IOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterWebSdk' is not required, skipping update.
[  +28 ms] Artifact Instance of 'WindowsEngineArtifacts' is not required, skipping update.
[   +1 ms] Artifact Instance of 'WindowsUwpEngineArtifacts' is not required, skipping update.
[   +1 ms] Artifact Instance of 'MacOSEngineArtifacts' is not required, skipping update.
[   +7 ms] Artifact Instance of 'LinuxEngineArtifacts' is not required, skipping update.
[   +3 ms] Artifact Instance of 'LinuxFuchsiaSDKArtifacts' is not required, skipping update.
[   +1 ms] Artifact Instance of 'MacOSFuchsiaSDKArtifacts' is not required, skipping update.
[   +2 ms] Artifact Instance of 'FlutterRunnerSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerDebugSymbols' is not required, skipping update.
[ +194 ms] Artifact Instance of 'MaterialFonts' is not required, skipping update.
[        ] Artifact Instance of 'GradleWrapper' is not required, skipping update.
[        ] Artifact Instance of 'AndroidGenSnapshotArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'AndroidInternalBuildArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'IOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterWebSdk' is not required, skipping update.
[        ] Artifact Instance of 'FlutterSdk' is not required, skipping update.
[        ] Artifact Instance of 'WindowsEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'WindowsUwpEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxEngineArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'LinuxFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'MacOSFuchsiaSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerSDKArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FlutterRunnerDebugSymbols' is not required, skipping update.
[        ] Artifact Instance of 'IosUsbArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'IosUsbArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'IosUsbArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'IosUsbArtifacts' is not required, skipping update.
[   +1 ms] Artifact Instance of 'IosUsbArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'FontSubsetArtifacts' is not required, skipping update.
[        ] Artifact Instance of 'PubDependencies' is not required, skipping update.
[ +128 ms] Skipping pub get: version match.
[ +238 ms] Found plugin flutter_appauth at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_appauth-4.0.0\
[  +54 ms] Found plugin flutter_secure_storage at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage-5.0.2\
[  +13 ms] Found plugin flutter_secure_storage_linux at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_linux-1.1.0\
[   +8 ms] Found plugin flutter_secure_storage_macos at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_macos-1.1.0\
[  +29 ms] Found plugin flutter_secure_storage_web at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_web-1.0.2\
[  +11 ms] Found plugin flutter_secure_storage_windows at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_windows-1.1.2\
[ +224 ms] Found plugin path_provider_linux at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\path_provider_linux-2.1.6\
[  +13 ms] Found plugin path_provider_windows at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\path_provider_windows-2.0.6\
[  +70 ms] Found plugin shared_preferences at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences-2.0.15\
[   +9 ms] Found plugin shared_preferences_android at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_android-2.0.12\
[  +15 ms] Found plugin shared_preferences_ios at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_ios-2.1.1\
[   +7 ms] Found plugin shared_preferences_linux at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_linux-2.1.1\
[   +9 ms] Found plugin shared_preferences_macos at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_macos-2.0.4\
[  +23 ms] Found plugin shared_preferences_web at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_web-2.0.4\
[  +11 ms] Found plugin shared_preferences_windows at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_windows-2.1.1\
[ +295 ms] Found plugin flutter_appauth at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_appauth-4.0.0\
[   +5 ms] Found plugin flutter_secure_storage at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage-5.0.2\
[   +2 ms] Found plugin flutter_secure_storage_linux at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_linux-1.1.0\
[   +2 ms] Found plugin flutter_secure_storage_macos at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_macos-1.1.0\
[   +8 ms] Found plugin flutter_secure_storage_web at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_web-1.0.2\
[   +6 ms] Found plugin flutter_secure_storage_windows at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_windows-1.1.2\
[  +46 ms] Found plugin path_provider_linux at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\path_provider_linux-2.1.6\
[   +5 ms] Found plugin path_provider_windows at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\path_provider_windows-2.0.6\
[  +20 ms] Found plugin shared_preferences at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences-2.0.15\
[   +3 ms] Found plugin shared_preferences_android at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_android-2.0.12\
[   +8 ms] Found plugin shared_preferences_ios at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_ios-2.1.1\
[   +2 ms] Found plugin shared_preferences_linux at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_linux-2.1.1\
[   +2 ms] Found plugin shared_preferences_macos at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_macos-2.0.4\
[  +17 ms] Found plugin shared_preferences_web at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_web-2.0.4\
[   +7 ms] Found plugin shared_preferences_windows at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_windows-2.1.1\
[ +235 ms] Generating C:\src\Ugeddit\ugeddit_v2\android\app\src\main\java\io\flutter\plugins\GeneratedPluginRegistrant.java
[ +558 ms] running test package with arguments: [-r, compact, --timeout, 30s, --concurrency=2, --chain-stack-traces, --,
C:\src\Ugeddit\ugeddit_v2\test\test_tests\test_test_1_test.dart]
00:00 +0: loading C:\src\Ugeddit\ugeddit_v2\test\test_tests\test_test_1_test.dart                                                                                [ +262 ms] test 0: starting test C:\src\Ugeddit\ugeddit_v2\test\test_tests\test_test_1_test.dart
[  +11 ms] Stopping scan for flutter_test_config.dart; found project root at C:\src\Ugeddit\ugeddit_v2
[   +5 ms] Compiler will use the following file as its incremental dill file: C:\Temp\flutter_tools.41da03de\flutter_test_compiler.9b840528\output.dill
[        ] Listening to compiler controller...
[  +18 ms] Compiling file:///C:/Temp/flutter_tools.41da03de/flutter_test_listener.410f33b3/listener.dart
[  +66 ms] Found plugin flutter_appauth at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_appauth-4.0.0\
[  +13 ms] Found plugin flutter_secure_storage at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage-5.0.2\
[   +3 ms] Found plugin flutter_secure_storage_linux at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_linux-1.1.0\
[   +2 ms] Found plugin flutter_secure_storage_macos at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_macos-1.1.0\
[   +9 ms] Found plugin flutter_secure_storage_web at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_web-1.0.2\
[   +2 ms] Found plugin flutter_secure_storage_windows at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\flutter_secure_storage_windows-1.1.2\
[  +49 ms] Found plugin path_provider_linux at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\path_provider_linux-2.1.6\
[  +13 ms] Found plugin path_provider_windows at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\path_provider_windows-2.0.6\
[  +20 ms] Found plugin shared_preferences at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences-2.0.15\
[   +4 ms] Found plugin shared_preferences_android at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_android-2.0.12\
[   +5 ms] Found plugin shared_preferences_ios at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_ios-2.1.1\
[   +3 ms] Found plugin shared_preferences_linux at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_linux-2.1.1\
[   +2 ms] Found plugin shared_preferences_macos at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_macos-2.0.4\
[   +4 ms] Found plugin shared_preferences_web at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_web-2.0.4\
[   +7 ms] Found plugin shared_preferences_windows at C:\Users\Sabian\AppData\Local\Pub\Cache\hosted\pub.dartlang.org\shared_preferences_windows-2.1.1\
[  +56 ms] C:\src\flutter\bin\cache\dart-sdk\bin\dart.exe --disable-dart-dev C:\src\flutter\bin\cache\dart-sdk\bin\snapshots\frontend_server.dart.snapshot --sdk-root
C:\src\flutter\bin\cache\artifacts\engine\common\flutter_patched_sdk/ --incremental --no-print-incremental-dependencies --target=flutter --debugger-module-names
--experimental-emit-debug-metadata -DFLUTTER_WEB_AUTO_DETECT=true --output-dill C:\Temp\flutter_tools.41da03de\flutter_test_compiler.9b840528\output.dill --packages
C:\src\Ugeddit\ugeddit_v2\.dart_tool\package_config.json -Ddart.vm.profile=false -Ddart.vm.product=false --enable-asserts --track-widget-creation
--initialize-from-dill C:\src\Ugeddit\ugeddit_v2\build\test_cache\build\c075001b96339384a97db4862b8ab8db.cache.dill.track.dill --source
C:\src\Ugeddit\ugeddit_v2\.dart_tool\flutter_build\dart_plugin_registrant.dart --source package:flutter/src/dart_plugin_registrant.dart
-Dflutter.dart_plugin_registrant=file:///C:/src/Ugeddit/ugeddit_v2/.dart_tool/flutter_build/dart_plugin_registrant.dart
[  +98 ms] <- compile file:///C:/Temp/flutter_tools.41da03de/flutter_test_listener.410f33b3/listener.dart
00:09 +0: loading C:\src\Ugeddit\ugeddit_v2\test\test_tests\test_test_1_test.dart                                                                                [+8990 ms] <- accept
[   +1 ms] <- reset
[        ] Compiling file:///C:/Temp/flutter_tools.41da03de/flutter_test_listener.410f33b3/listener.dart took 9353ms
[   +1 ms] test 0: starting test device
[   +9 ms] test 0: awaiting connection to test device
[   +5 ms] test 0: Observatory uri is not available
[   +9 ms] test 0: test harness socket server is running at port:52014
[   +4 ms] Using this directory for fonts configuration: C:\Temp\flutter_tools.41da03de\flutter_test_fonts.bf76ecff
[   +3 ms] test 0: Starting flutter_tester process with command=[C:\src\flutter\bin\cache\artifacts\engine\windows-x64\flutter_tester.exe, --disable-observatory,
--enable-checked-mode, --verify-entry-points, --enable-software-rendering, --skia-deterministic-rendering, --enable-dart-profiling, --non-interactive,
--use-test-fonts, --packages=C:\src\Ugeddit\ugeddit_v2\.dart_tool\package_config.json,
C:\Temp\flutter_tools.41da03de\flutter_test_listener.410f33b3\listener.dart.dill], environment={FLUTTER_TEST: true, FONTCONFIG_FILE:
C:\Temp\flutter_tools.41da03de\flutter_test_fonts.bf76ecff/fonts.conf, SERVER_PORT: 52014, APP_NAME: ugeddit_v2, UNIT_TEST_ASSETS:
C:\src\Ugeddit\ugeddit_v2\build\unit_test_assets}
[  +85 ms] test 0: Started flutter_tester process at pid 4352
00:10 +0: loading C:\src\Ugeddit\ugeddit_v2\test\test_tests\test_test_1_test.dart                                                                                [ +878 ms] test 0: connected to test device, now awaiting test result
[   +2 ms] test 0: Waiting for test harness or tests to finish
00:10 +0 -1: empty test 1 [E]
  UnimplementedError
  test\test_tests\test_test_1_test.dart 7:35  main.<fn>
  test\test_tests\test_test_1_test.dart 7:24  main.<fn>

00:10 +0 -2: empty test 2 [E]
  UnimplementedError
  test\test_tests\test_test_1_test.dart 8:35  main.<fn>
  test\test_tests\test_test_1_test.dart 8:24  main.<fn>

00:10 +0 -3: empty test 3 [E]
  UnimplementedError
  test\test_tests\test_test_1_test.dart 9:35  main.<fn>
  test\test_tests\test_test_1_test.dart 9:24  main.<fn>

00:10 +0 -4: empty test 4 [E]
  UnimplementedError
  test\test_tests\test_test_1_test.dart 10:35  main.<fn>
  test\test_tests\test_test_1_test.dart 10:24  main.<fn>

00:11 +0 -5: empty test 5 [E]
  UnimplementedError
  test\test_tests\test_test_1_test.dart 11:35  main.<fn>
  test\test_tests\test_test_1_test.dart 11:24  main.<fn>

[ +608 ms] test 0: Test harness is no longer needed by test process
[  +10 ms] test 0: finished
[   +3 ms] test 0: cleaning up...
[  +20 ms] test 0: ensuring test device is terminated.
[   +8 ms] test 0: Terminating flutter_tester process
[   +3 ms] test 0: Shutting down test harness socket server
[  +29 ms] test 0: Test process is no longer needed by test harness
[  +23 ms] test 0: flutter_tester process at pid 4352 exited with code=-1
[   +1 ms] test 0: deleting temporary directory
[  +14 ms] test 0: finished
00:11 +0 -5: Some tests failed.
[  +34 ms] Deleting C:\Temp\flutter_tools.41da03de\flutter_test_compiler.9b840528...
[  +24 ms] killing pid 7352
[ +106 ms] Deleting C:\Temp\flutter_tools.41da03de\flutter_test_fonts.bf76ecff...
[  +10 ms] test package returned with exit code 1
[  +13 ms] "flutter test" took 14,051ms.
[  +20 ms]
           #0      throwToolExit (package:flutter_tools/src/base/common.dart:10:3)
           #1      TestCommand.runCommand (package:flutter_tools/src/commands/test.dart:471:7)
           <asynchronous suspension>
           #2      FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1183:27)
           <asynchronous suspension>
           #3      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
           <asynchronous suspension>
           #4      CommandRunner.runCommand (package:args/command_runner.dart:209:13)
           <asynchronous suspension>
           #5      FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:281:9)
           <asynchronous suspension>
           #6      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
           <asynchronous suspension>
           #7      FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:229:5)
           <asynchronous suspension>
           #8      run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:62:9)
           <asynchronous suspension>
           #9      AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
           <asynchronous suspension>
           #10     main (package:flutter_tools/executable.dart:94:3)
           <asynchronous suspension>

[ +117 ms] ensureAnalyticsSent: 102ms
[   +4 ms] Running shutdown hooks
[   +3 ms] Shutdown hooks complete
[   +4 ms] exiting with code 1

If your on Linux (or possibly mac, although I'm not sure --- but give it a try) what's the output from DART_CFE_ENABLE_INCREMENTAL_COMPILER_BENCHMARKING=true flutter test -v test/test_test_1.dart

Not using Linux or Mac

Thanks a ton for all the questions and detail, Joha.
What do you think of these answers?

Alright.
I'm not familiar with a Windows time command, but it probably explains why the user and sys is so different from mine.

I'm not sure exactly how tests are run when clicking the Run or Debug button on specific tests and/or groups in the IDE, so lets concentrate about the command line runs --- and those commands looks right to me, so far so good :)

That the times are much longer with flutter clean at least means that the caching is working. So no funny business there.
This naturally gives us a whole new set of numbers --- looking at the "real" numbers it takes from between 22 and 32 seconds on a clean build. Yikes.
Another interesting point here, though, is that test_test_1 and test_test_2 are the same test, i.e. really the time for compiling and running should be the same. Naturally we can't expect timings to be rock stable, but here we're talking 22.504s vs 25.694s (real, or 15s vs 18s as reported by flutter test). That's quite a bit of difference.
The same thing goes for test_test_1_2 which is included in the last run of all 3 tests, and yet the real time is 31.579s vs 26.931s (i.e. compiling and running more is more than 4.5 seconds faster).
This - I think - indicates that there's something interfering. Combined with Windows, antivirus software (and the like) comes to mind, but I surely can't say anything with certainty.

A i5 4590 (4-cores) @ 3.3GHz CPU looks to be pretty powerful (e.g. comparing to the one in my machine the "single threaded rating" is only 20% slower than mine (or mine is 25% faster) --- and with running only one tests having more cores probably doesn't matter much). I don't think this is the issue.

20% as reported by Windows, if I recall correctly, is - on a 4-core machine - almost an entire core being used for something else. In all likelihood it's not a single core being used though, but rather ~20% of all of them meaning that the new "single threaded rating" might again be ~20% slower. Said another way, something taking 3.340 seconds on my machine (running all the tests) could reasonably be expected to take 5.18 seconds on yours. Your timing said 12.696 seconds, so nowhere close.

Up-to-date, super!

Windows --- I've touched a bit on this above --- it probably indicates that you have some security software which could influence performance too.

The disk itself seems fast when I look it up.

This is interesting. Was this a clean build or not? (the self-reported 11 seconds before saying all tests failed doesn't seem to fit with any of the previous numbers --- if it's not a clean build this might indicate that printing the verbose output takes a very long time). Let me try to sum it up a bit:
Flutter launches git a few times before doing anything else. This takes a combined total of 1.2 seconds (!).
It then finds the test, does some Artifact Instance of (I have no idea what that is) and something with plugins (which it seems to do twice?). This takes a combined total of 1.75 seconds (!).
It then takes ~0.25 seconds generating some java file, and another ~0.5 seconds before it actually runs the test package. We're now at ~3.7 seconds
It does something with plugins again -- again seemingly the same ones (that's three times now) - this time it "only" takes ~0.2 seconds though. As a side note, for my runs (a new project with just the test files in it) I don't have any plugins.
We're at ~3.9 seconds --- now it launches the compilation process. This takes something like 100-150 ms. Let's say we're at 4 seconds before the compiler is asked to do anything.
It then takes almost 9.5 seconds compiling (well, sort of --- this step covers a lot of stuff --- for instance, if it starts from a previous compile it first has to read all the files to see if anything has changed etc.). This surely isn't great either. We're at ~13.5 seconds.
We're starting the test process (~100 ms) and running the tests (seemingly ~600 ms), does some cleanup (shuts down processes, deletes temporary files etc) (~300 ms). We're at ~15.5 seconds.
It says "flutter test" took 14,051ms..  This fits approximately with flutter test not spending the first ~1.2 seconds on git stuff (~300 ms is missing somewhere, I'm not sure where, but relative to the overall numbers it hardly matters).
I then sends some analytics, taking another ~100 ms.

On windows, try running set DART_CFE_ENABLE_INCREMENTAL_COMPILER_BENCHMARKING=true in the terminal first. Then run "as normal" (possibly with -v though).

On a higher level note I should probably add that for each file you have tests in, running tests are going to be slower because it has to do a compile, launch a new process etc. Similarly - as we saw - getting to the point where it actually starts doing something with the tests takes quite a while, so running flutter test specificfile.dart (at least for many specific files) isn't great.
As somewhat of a workaround - when you want to run all tests - you could try to create one test file like done in https://github.com/flutter/flutter/issues/86722#issuecomment-914688482 and only run that.

@jensjoha I'm going to reply to the rest of your points when I can, but just wanted to let you know quickly that the literal very last thing you mentioned (about combining tests via imports) made my entire (real) test suite run seamlessly
time flutter test test/app_combined_test.dart
...
00:07 +144 -4: Some tests failed.

real    0m11.258s
user    0m0.075s
sys     0m0.121s

flutter clean && time flutter test test/app_combined_test.dart
...
00:22 +144 -4: Some tests failed.

real    0m33.431s
user    0m0.045s
sys     0m0.228s

I'll review your other points when I have time, but thank you tons for sharing that. I'm using Clean Architecture, so it's simple to create "test repositories" and a "app all features" test, and config them using launch.json for more granular tests for specific features.
You rock.

I'm glad combining the tests seems to be a workaround.
With regard to 8) it's funny --- that seems to take ~exactly as long as without verbose. I wonder what happened with the previous verbose run which was slower than the non-verbose one.
As for 9), setting that won't make it faster. That it appears faster is coincidence. In fact, it appears not to have done the wanted thing at all. It was meant to enable printing extra timings from the compilation, but it's not in the output. I'm not quite sure why.
As for the OP's 100 ms startup that's surely not going to happen anytime soon (if at all --- not directly anyway). For instance we've seen how git runs when starting the flutter command on your machine takes ~1 second (1000 ms).
(that being said, having a watcher of sorts with a compiler already present and initialized etc we may conceivably get a lot closer).

Quoted reply from @jensjoha

I'm glad combining the tests seems to be a workaround.
With regard to 8) it's funny --- that seems to take ~exactly as long as without verbose. I wonder what happened with the previous verbose run which was slower than the non-verbose one.
As for 9), setting that won't make it faster. That it appears faster is coincidence. In fact, it appears not to have done the wanted thing at all. It was meant to enable printing extra timings from the compilation, but it's not in the output. I'm not quite sure why.
As for the OP's 100 ms startup that's surely not going to happen anytime soon (if at all --- not directly anyway). For instance we've seen how git runs when starting the flutter command on your machine takes ~1 second (1000 ms). (that being said, having a watcher of sorts with a compiler already present and initialized etc we may conceivably get a lot closer).

For (8), I'm thinking you mean it takes exactly as long without flutter clean, not verbose? But either way, no idea.
For (9) is there a distinct place or command I can check to verify the setting's current state?
We'll see 👍. Thanks again, Joha!

Hello folks!
I'm also trying to understand possible improvements on how long it takes for a flutter test to run.
Here, a single call to flutter test path-to-test-file.dart is taking ~10s.
One thing that caught my attention is that is seems like the flutter tester is being re-created and re-compiled every time I run the flutter test command.
For example, when I run with the --verbose flag, one part of the output is:
/Users/philippe.ehlert/sdk-flutter/bin/cache/artifacts/engine/common/flutter_patched_sdk/ --incremental --no-print-incremental-dependencies --target=flutter --debugger-module-names --experimental-emit-debug-metadata -DFLUTTER_WEB_AUTO_DETECT=true --output-dill
/var/folders/1n/vlfv1d455490yst2p7myk5w40000gn/T/flutter_tools.Yrc4td/flutter_test_compiler.D07Dco/output.dill --packages /Users/philippe.ehlert/dev/project/test/flutter/.dart_tool/package_config.json -Ddart.vm.profile=false -Ddart.vm.product=false --enable-asserts
--track-widget-creation --initialize-from-dill /Users/philippe.ehlert/dev/project/test/flutter/build/test_cache/build/c075001b96339384a97db4862b8ab8db.cache.dill.track.dill --source
/Users/philippe.ehlert/dev/project/test/flutter/.dart_tool/flutter_build/dart_plugin_registrant.dart --source package:flutter/src/dart_plugin_registrant.dart
-Dflutter.dart_plugin_registrant=file:///Users/philippe.ehlert/dev/project/test/flutter/.dart_tool/flutter_build/dart_plugin_registrant.dart
[  +30 ms] <- compile file:///var/folders/1n/vlfv1d455490yst2p7myk5w40000gn/T/flutter_tools.Yrc4td/flutter_test_listener.kqZoOi/listener.dart
[+3846 ms] ../../../../../.pub-cache/git/flare-flutter-7870c1a8d2acb25dca4b7dbbbef0b883e1535588/flare_flutter/lib/flare_render_box.dart:167:26: Warning: Operand of null-aware operation '?.' has type 'SchedulerBinding' which excludes null.

This last step (compile file:///var/folders....) consistently takes ~4s to run.
Every time I run flutter test, it creates a new folder inside /var/folders/1n/vlfv1d455490yst2p7myk5w40000gn/T/.
In this example, the folder name is flutter_tools.Yrc4td. If I ls this directory there are multiple folders called flutter_tools.a_hash.
Is there a way to reuse this element instead of creating a new one every time I run a test?


kvstore: Wait for kvstore watcher to exit

This fixes an overall watcher problem where kvstore watchers can continue running with events being emited and acted upon even after logic in code has decided to stop watching.

This change is

test-me-please

test-me-please

This fix has uncovered a bug in the etcd watcher handling where etcd continuously returns a closed channel

test-me-please

test-me-please

test-me-please

@tgraf does this need backporting to v1.0 and / or v1.1?

This also needs an update to the PR description to show which commit it fixes, c.f. http://docs.cilium.io/en/v1.1/contributing/#submitting-a-pull-request step 5.

@tgraf does this need backporting to v1.0 and / or v1.1?

Yes, labels added

This also needs an update to the PR description to show which commit it fixes, c.f.

It does not fix a particular commit, it's an overall bugfix that affects all existing watchers.

@ianvernon Are you OK to merge?

LGTM!

I attempted to backport this for 1.0 but it requires #4768 and dependencies of that PR as well, so the amount of backporting required seems to be more than is worthwhile.
Consulted with @tgraf on Slack and came to the conclusion that this PR is not critical for 1.0 backport.
Removing label.


Use tektoncd/plumbing instead of knative/test-infra

Changes
As Pipeline graduated from knative, we're trying, with plumbing to
make remove our dependency on knative test-infra scripts. Those
scripts have been migrated to tektoncd/plumbing, so let's start use those.
The end goal being to reduce even more the use of those script and
only use pipeline to build pipeline 👼.
Marking this as wip for now, as I expect some breaking
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:

[ :no_good_man: ] Includes tests (if functionality changed/added)
[ :no_good_man: ] Includes docs (if user facing)
[x] Commit messages follow commit message best practices

See the contribution guide
for more details.

Oh my god it's green :man_dancing:

/hold
cc @bobcatfish @abayer @dlorenc

/lgtm
yay! =)

/hold cancel

/hold
Need to do a small fix

/hold cancel

/lgtm

This is awesome! Good call @vdemeester , I love this incremental approach :heart_eyes: