Package all standalone resources into a single file option

Hello @kayhayen,
Let me first greet you for that project who opens up new horizons, really.
I would like to bring an idea to improve it (at least, for my usages): could you propose an option to pack all standalone resources into one single file like the --onefile option for pyinstaller ? IMHO it's even more convenient for users to manipulate a single file than rather use entire folders.
Pyinstaller packs all resources into a single zip file (which is executable) and unzip them at runtime (check one of my compiled tools, for instance https://github.com/maaaaz/tibcopasswordrevealer) to see what it looks like).
Cheers!

I don't think is as straight forward as packing resources into a zip. Using the standalone or portable flag gives you a dist folder with a compiled executable in it, alongside all DLLs required to run it. The idea being that you distribute the entire folder. I don't think creating some form of self-extracting archive instead of this folder is an elegant solution and it can be done outside of Nuitka using other tools (I wouldn't recommend it but IExpress).
I would however like to see an option to embed the DLLs that end up in the dist folder in the executable itself. I think this would be a great feature however sadly I'm not sure if developers are answering these requests. I really enjoy Nuitka and love the benefits, but shipping the dist folder to people can often lead to questions around trust, and there have been times where people have taken the exe out of the folder. Would be awesome to see this added.
If it helps, I've used this with great success in the past in Visual Studios and I believe C++ Builder have this feature built in. But I'm not sure how to include generated c++ source into such projects or if that's even possible:
https://github.com/Fody/Costura

I agree with you, my wish is to have only one standalone file: indeed the zip solution is not as a elegant as the DLL-embedded one.

On the mailing list, there was once a discussion. There is a library that py2exe uses, and that could also be integrated with Nuitka. I won't be doing that though.

I don't see a lot of benefits about packaging the whole nuitka generated content in one single file, you can do that easily with some custom tool or using batch files so everything will be tidy up for the end-user... But, what it'd be really useful would be nuitka packaging just the minimal content for the projects, right now is adding more content (dlls) than necessary in certain projects, for instance pyqt projects.
This has been already commented in #73 but it feels right to bring it up here as well, as this topic is quite popular (4 thumb ups in the opening section is quite a lot :))

I think the idea of embedding shared libraries onto an executable is really a nasty/bad/ugly way to proceed here, period... For those one who like to hack around you can read more about it in this post https://blog.kowalczyk.info/article/d7d08da8d2c643349463292a051293b5/embed-dll-in-an-exe.html.
Of course, a different matter would be allowing to add more static code to the application if possible, ie: Embedding visual studio runtime library statically or maybe embedding CPython statically as that would allow the compiler global optimizations, that'd be cool.
What we all want at the end of the day is being able to ship nicely our software with little size over the internet, right? But embedding a bunch of shared libraries into an executable is definitely not a great idea for few reasons.
Let's say you got A) app1.exe + lib1.pyd + lib2.pyd = N bytes. If you embedded the whole thing into megaapp1.exe = N bytes, what'd be the benefit of it? Shipping a compressed form of either A) or B) would remain practically the same.
I understand there is a real use-case to present the software properly to end-users, but... that should be responsability of providing a proper installer and adding proper shortcuts to the user, shouldn't? There is really nice existing software that already does that and I think these type of tasks should be out of scope of nuitka. Nuitka should be a python well-optimized compiler and shouldn't do extra stuff already done by package management software. Even more, if you ask me, I think the existing freezer shouldn't be nuitka's responsability although I see a point of giving that to the users to make nuitka more attractive. Even if that's not the real value of the product itself :)
Another example, consider:

A) app1.exe + lib1.pyd + lib2.pyd = Na bytes   <=> megaapp1.exe
B) app2.exe + lib1.pyd + lib2.pyd + lib3.pyd = Nb bytes <=> megaapp2.exe

Let's say you wanted to distribute both A and B, by distributing megaapp1/megaaapp2 you're shipping duplicated content... and the optimal way to ship both apps would be shipping app1.exe/app2.exe+{lib1.pyd,lib2.pyd, lib3.py} at once
Does it make sense?
And don't get me wrong, I'm a huge geeky fan of as tiny as possible well-optimized standalone executables... but putting and wasting dev time on these tasks... would bring any benefit in the long term? I don't think so...
If you ask me, I'd like to think more effort/time will be invested on much more interesting problems, for instance, optimizing further the generated c code and making it faster in all cases than vanilla python/pypy/... That'd be awesome and prove than nuitka is the best choice to help our lovely-but-slow dear python  :)

@maaaaz , @psutton00 . Hello guys, If you don't mind... let me ask you this, let's hypothetically speaking I eventually decide to implement that feature you seem to be interested on, dll embedding into one single file, which I don't think it'd be really hard to code. What are the benefits you think this would have?
First, let me fastly talk very fast about what that feature would imply:

In case you wanted to provide exe updates to your customers/community you'd have to force them to download few hundred of mbs over and over. And that fact would improve very little even when using exe compressors.
Uncompressing on memory/disk the whole stuff is really a crapy/nasty/ugly/intrusive thing to do, period. As a user I consider that not just intrusive but I think the software that does that has been poorly coded... even if dumping to temporary memory. Unless the reason is proper obfuscation/compression.
Would improve obfuscation of your software? Not the lightest bit, at some point the whole uncompressed code will be in memory ready to be used by reverse engineers. In fact, adding this feature and not warning properly to users would give this false sensation of being "safe".
Would it improve speed? On contrary, the unpacking adds a little overhead

And again, don't get me wrong, I'm a huge fan of standalone executables, even I love coding executables shipping my own startup runtime library to make the exes being as small as possible... but... in this particular case I don't see any real benefit other than having fun coding the packer/unpacker code.
I mean, if I wanted to ship 20 command line tools written in python and each one (~1mb per exe + 49mb of 3rdparty dlls)... would i have to ship 20mb + 980mb = 1Gb! plus decompressing... instead 20mb+49mb=69 (which is also quite a lot)? Nuts :)
But again, maybe I'm missing something here, so please, tell me so I can understand the rationale here, why do you think this would be a good feature worth considering? I don't get it
Thanks.

@brupelo There is no point is telling people to not want what they want, because they still are going to want it, until provided with a better alternative. So please do not discuss this issue in that way. Are you OK with me deleting the comments of yours, that I feel are distracting others from knowing that a solution is actually welcome?
For all I know, it's possible to package all in one and people do it. Clearly at least pyinstaller does it, and it makes life easy for deployment, surely.
On the mailing list back then, concerns were raised, if this is something in scope of Nuitka, or if it is not to be a third party tool instead. Actually, if somebody were to look at how pyinstaller or py2exe do it, we might well find they are doing the same thing, or using the same library. If integration with Nuitka has any benefits other than convenience, I don't know yet, I would have to see the solution.
But like I said, one of the people who wants to have it, needs to create the PR and implement it in Nuitka, not necessary by implementing the loading of DLLs, which I understand is duplicating Windows functionality, but by using existing solutions. Feel free to go ahead if you are one of them, that is what the "help_needed" tag is for.
My hope is that this is actually only a matter of research, and not so much going to be a burden for actual Nuitka maintenance, with this being generic work, and that there could be an upstream responsible for problems if there are any.

@brupelo There is no point is telling people to not want what they want, because they still are going to want it, until provided with a better alternative. So please do not discuss this issue in that way. Are you OK with me deleting the comments of yours, that I feel are distracting others from knowing that a solution is actually welcome?

@kayhayen Mmm, maybe I expressed myself in the wrong way (although I don't see anything wrong on my previous comments, I'll re-read them again). Just to clarify, I will never order/ask people what to do, what to code or not and definitely what to ask or not, period!
Plus, I code silly stuff all the time just for the https://en.wikipedia.org/wiki/Art_for_art%27s_sake, so my attitude about coding non-real-world useful stuff is rock solid :) . That said, I think in this case I was just giving my thoughts about the issue itself, and of course people can agree with me or not. It's not a bad thing explaining the disadvantages of some solution in particular cases, isnt? (even if it works for a very limited set of use-cases)
I like open discussions and in this case I was trying to understand the rationaly, that's why I asked them in a nice way (I think I did)... not only that, by understanding if the feature was useful or not I could consider to implement it myself and upload to upstream for people's benefit, dealing with PE/Packers/Unpackers is something I've got a bit of experience with in the first place.
What I definitely don't really like will be any kind of censorship attitude, If I haven't been unpolite or said anything out of scope... but of course, you're invited to point out if my comments won't favour nuitka to improve in certain way and if I agree with your view I will prevent those type of comments to happen again.
You need to think than writing comments like the ones I do isn't a 10s job and I've put quite a lot of effort writing them actually :) , so by just deleting them is like you're saying "I don't give a damm about what this guy has to say", which of course.... I wouldn't consider it a nice to do.
But again, let me repeat myself, telling me if certain attitude/comments are not good to improve Nuitka, please do so.
Does it make sense +/-?

I remember I had tried this twitter utility couple of years ago called PEX and it was also implementing PEP441. Here's a nice video from one of the authors that explains the underlying logic, it's actually quite simple stuff.
Not sure though whether that approach would allow you to load .pyd files though :/
Anyway, here's some links for those who want it to give it a shot to this feature (DISCLAIMER: windows stuff only):

https://github.com/sudachen/Molebox <-- abandoned, doesn't build properly on vs2010 so it needs a little bit of hacking, ie. build system is broken
enigma protector  <-- commercial
https://blog.kowalczyk.info/article/d7d08da8d2c643349463292a051293b5/embed-dll-in-an-exe.html

I've made a very fast test, which it's adding to a .zip file multiple .pyd files. Python3.6.x won't be able to load them, as expected... so I guess it can only load .pyc/.pyo/.py files from zip files, feels reasonable btw.

I think UPX could be a possible solution here. While PyInstaller is capable of packing files without it, most projects I've seen end up using UPX anyways because of it's performance.

Onefile is implemented for Nuitka and with Nuitka-commercial you can now even protect data files. https://nuitka.net/pages/commercial.html rendering this all obsolete.


Add "transgender" to the list of words that are counted as "spelled correctly"

I confirm (by marking "x" in the [ ] below: [x]):

[x] This is not a troubleshooting question. Troubleshooting questions go here: http://www.mattermost.org/troubleshoot/.
[x] This doesn't reproduce on web browsers (such as in Chrome). If it does, issue reports go to the Mattermost Server repository.
[x] I have read contributing guidelines.

Summary
There was a request to add the word "transgender" to the list of words that are counted as "spelled correctly". This issue was opened to track this project.
Another one is "Signage"

"online" is another


exception  "access violation" with minimal example

I'm getting an exception with access violation with the minimal example, change slightly to that below, see screenshot for place of exception (plot->first detail::get_array(X) -> PyList_New(v.size());
Any idea what is wrong?
Thank you in advance,
Thomas

#include
// according to: https://github.com/boostorg/system/issues/32
#define HAVE_SNPRINTF
#include <Python.h>
#define WITHOUT_NUMPY
#include "C:\Users\tsch\OneDrive\Dokumente\fiberdesk wo code\fiberdesk code dependencies\matplotlib-cpp-master\matplotlibcpp.h"
int main()
{
std::cout << "Hello world";
namespace plt = matplotlibcpp;
plt::plot({ 1,3,2,4 });

return 0;

}

One step further:
included
Py_SetPythonHome(L"C:\\Users\\tsch\\.conda\\envs\\TomsEnv");
Py_Initialize();

before plt::plot(...)
and then run into the error that matplotlib cannot be loaded:
throw std::runtime_error("Error loading module matplotlib!");
However, with the Anaconda promt, it works:
(base) C:\Users\tsch>conda activate TomsEnv
(TomsEnv) C:\Users\tsch>python
Python 3.7.7 (default, Mar 23 2020, 23:19:08) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.

import matplotlib

Hm, I've seen similar issues recently - I think there are a few places in the code that are missing calls to Py_Initialize() that were never detected because just happened to work in older python versions.
Could you test whether the following patch fixes the issue?
@@ -990,6 +990,8 @@ bool plot(const std::vector<NumericX>& x, const std::vector<NumericY>& y, const
 {
     assert(x.size() == y.size());

+    detail::_interpreter::get();
+
     PyObject* xarray = detail::get_array(x);
     PyObject* yarray = detail::get_array(y);

Ooh, that's interesting.  I've just been having a problem with named_plot not working. I'm using Python 3.7.
My code calling named plot is like this:
    for (pair<string, vector<double>> costByAlpha : costsByAlpha) {
        plt::named_plot(costByAlpha.first, iterations_v, costByAlpha.second);
    }

and I was finding the third line of named_plot have an access violation:
    PyDict_SetItemString(kwargs, "label", PyString_FromString(name.c_str()));

If I change it to this, my code works and a nice graph gets displayed:
    PyDict_SetItem(kwargs, PyString_FromString("label"), PyString_FromString(name.c_str()));

But, also, just adding the statement you suggest makes it work too:
    detail::_interpreter::get();

Ok, I just pushed a commit adding initialization of the interpreter to all user-facing functions, so hopefully this should be resolved now.

Sorry for my late reply. It is not working for me. Good to see that it helped others. I'm now getting always an error, because the basic module 'matplotlib' is not loaded, despite setting the right path, initializing python etc. I'm still investigating it ...
-- I just want to code and not waste my time in getting it running. --


feat: support username with password

Fixes: #204

The coverage went down... Is this a WIP?

The coverage went down... Is this a WIP?

Just not covered by integration testing, Now it is fixed, and covered by both it and st cases.

/cc @yhmo

/lgtm


cluster dropdown in Grafana Dashboard for multicluster

Describe the feature request
We have istio installed across multiple clusters in a multi-priomary setup and have configured prometheus according to monitoring-multicluster-prometheus. to aggregate in a central prometheus.
The default istio dashboards do not have the dropdown at cluster level and they seem to be built for a single cluster, which makes it very difficult to drill down to a single cluster for visualizing/debugging.
Istio already has cluster label attached to each metrics. It would be nice if we can add source and destination cluster labels in all relevant dashboards since we already have those labels for metrics, so it is easier to monitor/drill down to specific cluster.
Affected product area (please put an X in all that apply)
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[X] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Affected features (please put an X in all that apply)
[X] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane
Additional context

@zirain Are you planning to implement this ?

feel free to take over, I may be very busy in May.

https://github.com/istio/istio/pull/39081


🩹[BREAKING BUG] Fix missing dependecy definition of 'packaging'

Hi there, we just wanted to upgrade xarray to version 0.21.0 and our ASV workflow on the CI started failing.
File "/home/runner/work/pyglotaran/pyglotaran/benchmark/.asv/env/76391772e92136ec87b9940d70226329/lib/python3.8/site-packages/xarray/core/dask_array_compat.py", line 4, in <module>
       from packaging.version import Version
   ModuleNotFoundError: No module named 'packaging'

The reason is that packaging isn't a direct dependency of  xarray.
$ johnnydep xarray
2022-01-29 08:28:38 [info     ] init johnnydist                [johnnydep.lib] dist=xarray parent=None
2022-01-29 08:28:40 [info     ] init johnnydist                [johnnydep.lib] dist=numpy>=1.18 parent=xarray
2022-01-29 08:28:44 [info     ] init johnnydist                [johnnydep.lib] dist=pandas>=1.1 parent=xarray
2022-01-29 08:28:47 [info     ] init johnnydist                [johnnydep.lib] dist=numpy>=1.18.5 parent=pandas>=1.1
2022-01-29 08:28:51 [info     ] init johnnydist                [johnnydep.lib] dist=python-dateutil>=2.8.1 parent=pandas>=1.1
2022-01-29 08:28:53 [info     ] init johnnydist                [johnnydep.lib] dist=pytz>=2020.1 parent=pandas>=1.1
2022-01-29 08:28:56 [info     ] init johnnydist                [johnnydep.lib] dist=six>=1.5 parent=python-dateutil>=2.8.1
name                            summary
------------------------------  -----------------------------------------------------------------------
xarray                          N-D labeled arrays and datasets in Python
├── numpy>=1.18                 NumPy is the fundamental package for array computing with Python.
└── pandas>=1.1                 Powerful data structures for data analysis, time series, and statistics
    ├── numpy>=1.18.5           NumPy is the fundamental package for array computing with Python.
    ├── python-dateutil>=2.8.1  Extensions to the standard Python datetime module
    │   └── six>=1.5            Python 2 and 3 compatibility utilities
    └── pytz>=2020.1            World timezone definitions, modern and historical

The problem why your tests didn't catch this is that packaging is a dependency of pytest itself.
$ johnnydep pytest
2022-01-29 08:25:03 [info     ] init johnnydist                [johnnydep.lib] dist=pytest parent=None
2022-01-29 08:25:07 [info     ] init johnnydist                [johnnydep.lib] dist=atomicwrites>=1.0 parent=pytest
2022-01-29 08:25:09 [info     ] init johnnydist                [johnnydep.lib] dist=attrs>=19.2.0 parent=pytest
2022-01-29 08:25:11 [info     ] init johnnydist                [johnnydep.lib] dist=colorama parent=pytest
2022-01-29 08:25:14 [info     ] init johnnydist                [johnnydep.lib] dist=iniconfig parent=pytest
2022-01-29 08:25:16 [info     ] init johnnydist                [johnnydep.lib] dist=packaging parent=pytest
2022-01-29 08:25:18 [info     ] init johnnydist                [johnnydep.lib] dist=pluggy<2.0,>=0.12 parent=pytest
2022-01-29 08:25:20 [info     ] init johnnydist                [johnnydep.lib] dist=py>=1.8.2 parent=pytest
2022-01-29 08:25:23 [info     ] init johnnydist                [johnnydep.lib] dist=toml parent=pytest
2022-01-29 08:25:25 [info     ] init johnnydist                [johnnydep.lib] dist=pyparsing!=3.0.5,>=2.0.2 parent=packaging
name                              summary
--------------------------------  ---------------------------------------------------------------------
pytest                            pytest: simple powerful testing with Python
├── atomicwrites>=1.0             Atomic file writes.
├── attrs>=19.2.0                 Classes Without Boilerplate
├── colorama                      Cross-platform colored terminal text.
├── iniconfig                     iniconfig: brain-dead simple config-ini parsing
├── packaging                     Core utilities for Python packages
│   └── pyparsing!=3.0.5,>=2.0.2  Python parsing module
├── pluggy<2.0,>=0.12             plugin and hook calling mechanisms for python
├── py>=1.8.2                     library with cross-python path, ini-parsing, io, code, log facilities
└── toml                          Python Library for Tom's Obvious, Minimal Language

We also only saw our ASV workflow fail since ASV only installs the package and all direct runtime dependencies in the venv it runs the benchmark with.
I prepared a dummy PR on my fork to demonstrate that this fixes the issue.
Since the release is only 10h old maybe just do a quick post-release so most users won't even notice.

[ ] User visible changes (including notable bug fixes) are documented in whats-new.rst

The failing tests are unrelated to this change and originate from this commit.

Thanks. I think it makes sense to revert #6200 then it's not critical. I opened #6208 for this.
Thanks for the report and the PR.

cc @pydata/xarray

On second thought I think the problem is the release of dask 2022.01.1.

On second thought I think the problem is the release of dask 2022.01.1.

We don't have dask as a dependency

$ johnnydep pyglotaran
2022-01-29 10:55:08 [info     ] init johnnydist                [johnnydep.lib] dist=pyglotaran parent=None
2022-01-29 10:55:11 [info     ] init johnnydist                [johnnydep.lib] dist=asteval>=0.9.21 parent=pyglotaran
2022-01-29 10:55:24 [info     ] init johnnydist                [johnnydep.lib] dist=click>=7.0 parent=pyglotaran
2022-01-29 10:55:26 [info     ] init johnnydist                [johnnydep.lib] dist=netCDF4>=1.5.3 parent=pyglotaran
2022-01-29 10:55:29 [info     ] init johnnydist                [johnnydep.lib] dist=numba>=0.52 parent=pyglotaran
2022-01-29 10:55:32 [info     ] init johnnydist                [johnnydep.lib] dist=numpy<1.21,>=1.20.0 parent=pyglotaran
2022-01-29 10:55:36 [info     ] init johnnydist                [johnnydep.lib] dist=pandas>=0.25.2 parent=pyglotaran
2022-01-29 10:55:39 [info     ] init johnnydist                [johnnydep.lib] dist=rich>=10.9.0 parent=pyglotaran
2022-01-29 10:55:42 [info     ] init johnnydist                [johnnydep.lib] dist=ruamel.yaml>=0.17.17 parent=pyglotaran
2022-01-29 10:55:45 [info     ] init johnnydist                [johnnydep.lib] dist=scipy>=1.3.2 parent=pyglotaran
2022-01-29 10:55:49 [info     ] init johnnydist                [johnnydep.lib] dist=sdtfile>=2020.8.3 parent=pyglotaran
2022-01-29 10:55:51 [info     ] init johnnydist                [johnnydep.lib] dist=setuptools>=41.2 parent=pyglotaran
2022-01-29 10:55:54 [info     ] init johnnydist                [johnnydep.lib] dist=tabulate>=0.8.8 parent=pyglotaran
2022-01-29 10:55:56 [info     ] init johnnydist                [johnnydep.lib] dist=xarray!=0.20.0,!=0.20.1,>=0.16.2 parent=pyglotaran
2022-01-29 10:55:58 [info     ] init johnnydist                [johnnydep.lib] dist=colorama parent=click>=7.0
2022-01-29 10:56:00 [info     ] init johnnydist                [johnnydep.lib] dist=cftime parent=netCDF4>=1.5.3
2022-01-29 10:56:03 [info     ] init johnnydist                [johnnydep.lib] dist=numpy>=1.9 parent=netCDF4>=1.5.3
2022-01-29 10:56:07 [info     ] init johnnydist                [johnnydep.lib] dist=numpy>1.13.3 parent=cftime
2022-01-29 10:56:11 [info     ] init johnnydist                [johnnydep.lib] dist=llvmlite<0.39,>=0.38.0rc1 parent=numba>=0.52
2022-01-29 10:56:15 [info     ] init johnnydist                [johnnydep.lib] dist=numpy<1.22,>=1.18 parent=numba>=0.52
2022-01-29 10:56:18 [info     ] init johnnydist                [johnnydep.lib] dist=setuptools parent=numba>=0.52
2022-01-29 10:56:21 [info     ] init johnnydist                [johnnydep.lib] dist=numpy>=1.18.5 parent=pandas>=0.25.2
2022-01-29 10:56:25 [info     ] init johnnydist                [johnnydep.lib] dist=python-dateutil>=2.8.1 parent=pandas>=0.25.2
2022-01-29 10:56:27 [info     ] init johnnydist                [johnnydep.lib] dist=pytz>=2020.1 parent=pandas>=0.25.2
2022-01-29 10:56:29 [info     ] init johnnydist                [johnnydep.lib] dist=six>=1.5 parent=python-dateutil>=2.8.1
2022-01-29 10:56:31 [info     ] init johnnydist                [johnnydep.lib] dist=colorama<0.5.0,>=0.4.0 parent=rich>=10.9.0
2022-01-29 10:56:34 [info     ] init johnnydist                [johnnydep.lib] dist=commonmark<0.10.0,>=0.9.0 parent=rich>=10.9.0
2022-01-29 10:56:36 [info     ] init johnnydist                [johnnydep.lib] dist=pygments<3.0.0,>=2.6.0 parent=rich>=10.9.0
2022-01-29 10:56:38 [info     ] init johnnydist                [johnnydep.lib] dist=ruamel.yaml.clib>=0.2.6 parent=ruamel.yaml>=0.17.17
2022-01-29 10:56:40 [info     ] init johnnydist                [johnnydep.lib] dist=numpy<1.23.0,>=1.16.5 parent=scipy>=1.3.2
2022-01-29 10:56:44 [info     ] init johnnydist                [johnnydep.lib] dist=numpy>=1.15.1 parent=sdtfile>=2020.8.3
2022-01-29 10:56:48 [info     ] init johnnydist                [johnnydep.lib] dist=numpy>=1.18 parent=xarray!=0.20.0,!=0.20.1,>=0.16.2
2022-01-29 10:56:51 [info     ] init johnnydist                [johnnydep.lib] dist=pandas>=1.1 parent=xarray!=0.20.0,!=0.20.1,>=0.16.2
2022-01-29 10:56:55 [info     ] init johnnydist                [johnnydep.lib] dist=numpy>=1.18.5 parent=pandas>=1.1
2022-01-29 10:56:55 [info     ] init johnnydist                [johnnydep.lib] dist=python-dateutil>=2.8.1 parent=pandas>=1.1
2022-01-29 10:56:55 [info     ] init johnnydist                [johnnydep.lib] dist=pytz>=2020.1 parent=pandas>=1.1
2022-01-29 10:56:55 [info     ] init johnnydist                [johnnydep.lib] dist=six>=1.5 parent=python-dateutil>=2.8.1
name                                  summary
------------------------------------  ----------------------------------------------------------------------------------------------------------------

------------
pyglotaran                            The Glotaran fitting engine.
├── asteval>=0.9.21                   Safe, minimalistic evaluator of python expression using ast module
├── click>=7.0                        Composable command line interface toolkit
│   └── colorama                      Cross-platform colored terminal text.
├── netCDF4>=1.5.3                    Provides an object-oriented python interface to the netCDF version 4 library.
│   ├── cftime                        Time-handling functionality from netcdf4-python
│   │   └── numpy>1.13.3              NumPy is the fundamental package for array computing with Python.
│   └── numpy>=1.9                    NumPy is the fundamental package for array computing with Python.
├── numba>=0.52                       compiling Python code using LLVM
│   ├── llvmlite<0.39,>=0.38.0rc1     lightweight wrapper around basic LLVM functionality
│   ├── numpy<1.22,>=1.18             NumPy is the fundamental package for array computing with Python.
│   └── setuptools                    Easily download, build, install, upgrade, and uninstall Python packages
├── numpy<1.21,>=1.20.0               NumPy is the fundamental package for array computing with Python.
├── pandas>=0.25.2                    Powerful data structures for data analysis, time series, and statistics
│   ├── numpy>=1.18.5                 NumPy is the fundamental package for array computing with Python.
│   ├── python-dateutil>=2.8.1        Extensions to the standard Python datetime module
│   │   └── six>=1.5                  Python 2 and 3 compatibility utilities
│   └── pytz>=2020.1                  World timezone definitions, modern and historical
├── rich>=10.9.0                      Render rich text, tables, progress bars, syntax highlighting, markdown and more to the terminal
│   ├── colorama<0.5.0,>=0.4.0        Cross-platform colored terminal text.
│   ├── commonmark<0.10.0,>=0.9.0     Python parser for the CommonMark Markdown spec
│   └── pygments<3.0.0,>=2.6.0        Pygments is a syntax highlighting package written in Python.
├── ruamel.yaml>=0.17.17              ruamel.yaml is a YAML parser/emitter that supports roundtrip preservation of comments, seq/map flow style, and m

ap key order
│   └── ruamel.yaml.clib>=0.2.6       C version of reader, parser and emitter for ruamel.yaml derived from libyaml
├── scipy>=1.3.2                      SciPy: Scientific Library for Python
│   └── numpy<1.23.0,>=1.16.5         NumPy is the fundamental package for array computing with Python.
├── sdtfile>=2020.8.3                 Read Becker & Hickl SDT files
│   └── numpy>=1.15.1                 NumPy is the fundamental package for array computing with Python.
├── setuptools>=41.2                  Easily download, build, install, upgrade, and uninstall Python packages
├── tabulate>=0.8.8                   Pretty-print tabular data
└── xarray!=0.20.0,!=0.20.1,>=0.16.2  N-D labeled arrays and datasets in Python
    ├── numpy>=1.18                   NumPy is the fundamental package for array computing with Python.
    └── pandas>=1.1                   Powerful data structures for data analysis, time series, and statistics
        ├── numpy>=1.18.5             NumPy is the fundamental package for array computing with Python.
        ├── python-dateutil>=2.8.1    Extensions to the standard Python datetime module
        │   └── six>=1.5              Python 2 and 3 compatibility utilities
        └── pytz>=2020.1              World timezone definitions, modern and historical

As far as I can see xarray/__init__.py triggers the following import chain :
xarray.testing->xarray.core.duck_array_ops->xarray.core.dask_array_compat  which triggers the import of packaging

Traceback starting from our package

```python
File "/home/runner/work/pyglotaran/pyglotaran/benchmark/.asv/env/76391772e92136ec87b9940d70226329/lib/python3.8/site-packages/glotaran/io/prepare_dataset.py", line 6, in
       import xarray as xr
     File "/home/runner/work/pyglotaran/pyglotaran/benchmark/.asv/env/76391772e92136ec87b9940d70226329/lib/python3.8/site-packages/xarray/__init__.py", line 1, in
       from . import testing, tutorial, ufuncs
     File "/home/runner/work/pyglotaran/pyglotaran/benchmark/.asv/env/76391772e92136ec87b9940d70226329/lib/python3.8/site-packages/xarray/testing.py", line 8, in
       from xarray.core import duck_array_ops, formatting, utils
     File "/home/runner/work/pyglotaran/pyglotaran/benchmark/.asv/env/76391772e92136ec87b9940d70226329/lib/python3.8/site-packages/xarray/core/duck_array_ops.py", line 24, in
       from . import dask_array_compat, dask_array_ops, dtypes, npcompat, nputils
     File "/home/runner/work/pyglotaran/pyglotaran/benchmark/.asv/env/76391772e92136ec87b9940d70226329/lib/python3.8/site-packages/xarray/core/dask_array_compat.py", line 4, in
       from packaging.version import Version
   ModuleNotFoundError: No module named 'packaging'
```

packaging is imported in 16 other python files e.g. indexing.py, I think there is no way around adding it as a dependency.
Hiding all the imports in closures will be quite error prawn especially since tests won't catch it since pytest bring packaging along with it.

https://docs.python.org/3/whatsnew/3.10.html#distutils-deprecated recommends importing packaging instead, so if we want to do version checking maybe we should just add it as a dependency?

Yes, I think I just expressed it wrong. What I wanted to say: zhe failure you see here is not caused by #6200 but by a new release of dask. And this is independent of your issue with packaging.

Yes, I think I just expressed it wrong. What I wanted to say: zhe failure you see here is not caused by #6200 but by a new release of dask. And this is independent of your issue with packaging.

Ahhh now I get it 😅

Thanks everyone! Are we ok to close this then?

no, I think we should still add packaging to the dependencies, even if it doesn't cause any failures: we unconditionally import it in at least xarray.core.pycompat. We need to add a reasonable minimum version, though.

Dask has packaging>=20 which was released over 2y ago guess this is kinda reasonable

Following dask seems like a good idea! We probably should add packaging in our ci installs as well:
https://github.com/pydata/xarray/tree/main/ci/requirements
and the asv config:
https://github.com/pydata/xarray/blob/main/asv_bench/asv.conf.json

just for the record, we decided that packaging related tools (i.e. setuptools, not sure if that also applies to packaging) should be released just before the earliest supported python version, so we would have to decide between 19.2 and 20. However, since the release notes of packaging state that 20 is the first version to officially support py38 I'd say using packaging>=20 should be uncontroversial.

other than the environment files (in ci/requirements) we also need to update doc/getting-started/installing.rst: the Required dependencies section, and maybe also the Minimum dependency versions. For the latter, we need to figure out a rule, though.

@pydata/xarray - we should try to get this merged and a new release up ASAP as bug reports are starting to pile up. I've pushed changes to the CI configs, documentation, and elsewhere. I've also pulled in the recent changes on main which should hopefully fix the failing CI in the first commit.

I can do the release later today if no one gets there first

This is ready to merge. @max-sixty - whenever you are ready, go ahead and merge and make the release.

I'll merge now though do the release this eve


Tests requires secrets.json but no mention in README

Testing this repo (via tests/Plais.MSTest) requires a secrets.json file but there's no mention in the README of that file, it's format or what to put in it.
Could one be added to the repo (with fake values that need substitution) and the README updated to indicate it's format/use?

Ok I will.  The secrets.json is at the root of the project and is contain the following
"plaid":
{
"secret": null,
"client_id": null,
"public_key": null,
"access_token": null
}


Add isDefined function to Enum type

The changes are pointing to add a straightforward way to find out whether an EnumItem or either its constant or value have been defined as part of Enum type instance.

v2.4.0

It might be a good idea to document this in the README.


Reconciliation results are not sorted by score

In building my own reconciliation service, I noticed that OpenRefine displays the results in the order returned by the remote API, rather than sorting them by the score object. For example, a query for "McCarthy, Joseph M." returns 3 results, which are displayed in the following order, even though the 3rd match has the highest score:
{"q0": {"result": [
    {
        "name": "McCarthy, Joseph R.",
        "score": 6.4928436,
        "type": ["person"],
        "id": "6603"
    },
    {
        "name": "McCarthy, Senator Joseph R.",
        "score": 6.4928436,
        "type": ["person"],
        "id": "3558"
    },
    {
        "name": "McCarthy, Joseph M.",
        "score": 11.757351,
        "type": ["person"],
        "id": "6536"
    }
]}}

OpenRefine should use the score value rather than the order of objects sent by the API (in particular since JSON lacks a notion of document ordering that either server or client can depend on).
(I'm using 2.6RC1 with current versions of Mac OS X and Google Chrome.)

Your assertion that JSON arrays aren't ordered is incorrect.
http://stackoverflow.com/questions/7214293/is-the-order-of-elements-in-a-json-list-maintained
The reconciliation service is already filtering and ranking the results.   It makes far more sense, at least to me, to order them correctly at that point, particularly since the reconciliation service presumably has the most information available as to what the correct order is.

Thanks for correcting my misunderstanding about JSON arrays being ordered.  That insight also helped me track down the cause of the misordering in my reconciliation service.

Cool.  Always happy to hear about new reconciliation services!


[Feature] Replace X/Wayland reliance for a Rust Wayland

Rust Wayland: https://github.com/Smithay/wayland-rs

Invalid. Related to: #8


Modified Workers to stop or start based on success rate

With the change to no worker pool I don't think we need this pr.


znc: 1.6.5 -> 1.6.6

Motivation for this change
This version of znc fixes --makepem segfault. See the changelog here: znc 1.6.6 changelog
Additionally, this change keeps service's file configuration's Version up to date with znc's version.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

The bots are taking our jobs! #37146
But the module update would still be needed. In addition I found out that parallel building can be enabled for znc ("if you are on a dedicated server and your CPU has more than one core, you can use make -jX where X is the number of CPU cores to speed up compilation"). Those things could be done in a single PR.

Sounds good. Were you thinking of making that a configurable option, or filling it with the result of nproc or something like that?

@mdsib Just add enableParallelBuilding = true; to the derivation :)

This PR is conlicting

Yeah, closing. I'll put up another with updates to the module soon.


BaseActiveRecord after refresh incorrect oldAttributes

In the BaseActiveRecord->refresh() field _oldAttributes is the same as _attributes. I think it is wrong.
See patch https://github.com/yiisoft/yii2/compare/master...only-victor:patch-1
The $record->_oldAttributes may be different than $record->_attributes.
Therefore $this->_oldAttributes must be the same as $record->_oldAttributes.
For example I can change $record->_attributes in afterFind handler and now $this->_oldAttributes will not be equal $record->_oldAttributes.

@only-victor, can you open a pull request for this?

I don't know to do it.

https://github.com/yiisoft/yii2/blob/master/docs/internals/git-workflow.md

@samdark Sorry, but I don't understand what exactly I must to do.

Ah, you have branch already. Then it's simpler and can be done via GitHub UI: https://help.github.com/articles/creating-a-pull-request/

Resolved by commit 4f7b1b081714018d3d086b2379701da0255d9855

@klimov-paul Thanks.


Test failure after upgrading to Pester 5.3.0-beta

Following test failure found:
[-] Describe Rename-Many (Internal) failed
 CommandNotFoundException: Could not find Command rename-FsItem

Tests completed in 3.34s
Tests Passed: 0, Failed: 1, Skipped: 0 NotRun: 222
BeforeAll \ AfterAll failed: 1
  - Rename-Many (Internal)

But this looks like a test that has been skipped. Perhaps, the Skip functionality has changed since 5.1.1

Looks like there are changes to mocking since 5.1.1. Also, the BeforeAll code block is being executed now regardless of whether there are any active tests in the correspnding block. Previously, a BeforeAll block would not be executed if the blocks tests were individually skipped, however it looks like this has now changed.


1.11 updates (part 1)

Signed-off-by: chipzoller<EMAIL_ADDRESS>Related issue
Supersedes #940
Closes #971
Closes #968
Closes #966
Fixes #958
Closes #946
Closes #934
Closes #824
Proposed Changes
Main changes are this adds/updates cleanup docs based on 1.11 changes.
Checklist

[x] I have read the contributing guidelines.
[x] I have inspected the website preview for accuracy.
[x] I have signed off my issue.

cc @VedRatan


Pass workingdir-relative paths to gc to improve error messages

Do not change directory before invoking the gc tool. Instead,
relativize each filepath passed to the gc tool to the current
working directory. This greatly improves error messages produced
by gc since the broken file is now identified by an unambiguous
path.
The testing infrastructure required for this involves introducing
a new runOut variable to gcToolchain so it can be mocked. A similar
run variable is also introduced for parity of implementation and
possible future tests.
Fixes https://github.com/constabulary/gb/issues/341

Looks like I broke something with cgo usage, not entirely sure what or why though, will take a closer look.

I like this approach, we used to do it a while back, see #57. The problem is not everything is relative to the project root. Temporary files, from cgo, and from tests are relative to context.Workdir().
One idea I had for fixing this bug was to delay printing the name of the package til after it was compiled, rather than before it is compiled, which would make the name of the failed package more obvious.

So, two questions:

Is this fundamentally the wrong approach then and we should close?

Is there something special I need to have set up for the tests to pass? On a fresh checkout of gb's master branch (without this change), I get failures in those exact tests, but with different output than what Wercker shows:

--- FAIL: TestTest (6.31s)
	test_test.go:81: Test(cgoonlynotest): want <nil>, got # /home/vagrant/go/pkg/tool/linux_amd64/6c -F -V -w -trimpath /tmp/gb300528713 -I /tmp/gb300528713/cgoonlynotest/_test -I /home/vagrant/go/pkg/linux_amd64 -o /tmp/gb300528713/cgoonlynotest/_test/_cgo_defun.o -D GOOS_linux -D GOARCH_amd64 /tmp/gb300528713/cgoonlynotest/_test/_cgo_defun.c: exit status 1
		<unknown line number> 6c: No such file or directory: /tmp/gb300528713/cgoonlynotest/_test/_cgo_defun.c
	test_test.go:81: Test(cgotest): want <nil>, got # /home/vagrant/go/pkg/tool/linux_amd64/6c -F -V -w -trimpath /tmp/gb681991048 -I /tmp/gb681991048/cgotest/_test -I /home/vagrant/go/pkg/linux_amd64 -o /tmp/gb681991048/cgotest/_test/_cgo_defun.o -D GOOS_linux -D GOARCH_amd64 /tmp/gb681991048/cgotest/_test/_cgo_defun.c: exit status 1
		<unknown line number> 6c: No such file or directory: /tmp/gb681991048/cgotest/_test/_cgo_defun.c

I've updated the commit with a fix that should be general enough to handle all cases (and makes all of the tests pass). It changes the path-relativization process to skip all absolute paths.

@inconshreveable thanks for this. I think this need to be tested on windows 'cos often their tempdir cannot be made relative to their workdir. @ChrisHines are you able to test on windows ?

It's hard to compare on windows as master is broken there as well (go 1.5.1 on Win7):
C:\Users\alan\gopath\src\github.com\constabulary\gb>go test -short github.com/co
nstabulary/gb/...
a
a
b
a
d.v1
c
d.v1
cc1.exe: sorry, unimplemented: 64-bit mode not compiled in
cc1.exe: sorry, unimplemented: 64-bit mode not compiled in
notestfiles
cc1.exe: sorry, unimplemented: 64-bit mode not compiled in
cc1.exe: sorry, unimplemented: 64-bit mode not compiled in
--- FAIL: TestBuild (0.62s)
        build_test.go:74: ctx.Build(cgomain): want <nil>, got # C:\Go\pkg\tool\w
indows_amd64\cgo -objdir C:\Users\alan\AppData\Local\Temp\gb281314287\cgomain\_c
go -importpath cgomain -- -I C:\Users\alan\AppData\Local\Temp\gb281314287\cgomai
n\_cgo -I C:\Users\alan\gopath\src\github.com\constabulary\gb\testdata\src\cgoma
in main.go: exit status 2
        build_test.go:74: ctx.Build(cgotest): want <nil>, got # C:\Go\pkg\tool\w
indows_amd64\cgo -objdir C:\Users\alan\AppData\Local\Temp\gb997494402\cgotest\_c
go -importpath cgotest -- -I C:\Users\alan\AppData\Local\Temp\gb997494402\cgotes
t\_cgo -I C:\Users\alan\gopath\src\github.com\constabulary\gb\testdata\src\cgote
st basic.go: exit status 2
        build_test.go:74: ctx.Build(cgoonlynotest): want <nil>, got # C:\Go\pkg\
tool\windows_amd64\cgo -objdir C:\Users\alan\AppData\Local\Temp\gb557978884\cgoo
nly\_cgo -importpath cgoonlynotest -- -I C:\Users\alan\AppData\Local\Temp\gb5579
78884\cgoonly\_cgo -I C:\Users\alan\gopath\src\github.com\constabulary\gb\testda
ta\src\cgoonlynotest cgoonly.go: exit status 2
        build_test.go:74: ctx.Build(cppmain): want <nil>, got # C:\Go\pkg\tool\w
indows_amd64\cgo -objdir C:\Users\alan\AppData\Local\Temp\gb119974520\cppmain\_c
go -importpath cppmain -- -I C:\Users\alan\AppData\Local\Temp\gb119974520\cppmai
n\_cgo -I C:\Users\alan\gopath\src\github.com\constabulary\gb\testdata\src\cppma
in main.go: exit status 2
a
a
b
a
d.v1
c
d.v1
a
b
--- FAIL: TestPackageBinfile (0.00s)
        package_test.go:68: (b).Binfile(): want C:\Users\alan\gopath\src\github.
com\constabulary\gb\testdata\bin\b, got C:\Users\alan\gopath\src\github.com\cons
tabulary\gb\testdata\bin\b.exe
FAIL
FAIL    github.com/constabulary/gb      1.409s

@inconshreveable could you please log a fresh bug for that one. Ta

I've rebased on top of the latest Windows fixes (hooray!) and all tests pass there as well (only verified Win7)
C:\Users\alan\gopath\src\github.com\constabulary\gb>go test -short github.com\co
nstabulary\gb\...
ok      github.com/constabulary/gb      5.903s
ok      github.com/constabulary/gb/cmd  17.999s
ok      github.com/constabulary/gb/cmd/gb       0.133s
?       github.com/constabulary/gb/cmd/gb-vendor        [no test files]
?       github.com/constabulary/gb/log  [no test files]
ok      github.com/constabulary/gb/vendor       0.123s

@inconshreveable thank you for this. There are a few problems
lucky(~/src/github.com/juju/juju) % gb build
github.com/juju/juju/version
version.go:8: non-declaration statement outside function body
version.go:10: syntax error: unexpected import
version.go:12: non-declaration statement outside function body
version.go:13: non-declaration statement outside function body
version.go:14: non-declaration statement outside function body
version.go:15: non-declaration statement outside function body
version.go:16: non-declaration statement outside function body
version.go:17: non-declaration statement outside function body
version.go:18: non-declaration statement outside function body
version.go:19: non-declaration statement outside function body
version.go:19: too many errors
FATAL command "build" failed: exit status 2

I think this should say version/version.go:NN
lucky(~/src/github.com/juju) % gb build juju/...
github.com/juju/juju/version
version.go:8: non-declaration statement outside function body
version.go:10: syntax error: unexpected import
version.go:12: non-declaration statement outside function body
version.go:13: non-declaration statement outside function body
version.go:14: non-declaration statement outside function body
version.go:15: non-declaration statement outside function body
version.go:16: non-declaration statement outside function body
version.go:17: non-declaration statement outside function body
version.go:18: non-declaration statement outside function body
version.go:19: non-declaration statement outside function body
version.go:19: too many errors
FATAL command "build" failed: exit status 2

and this should say juju/version/version.go:NN
Using $pwd as the base for this may be tricky, gb supports a -R flag which lets you change projects without changing the working directory (this essentially overrides the base of the project detection code).
I wonder if it would be more straight forward to always make all errors relative to the project root.
/cc @fatih because I know we've had this conversation before.

@davecheney Can you double check that you used a build of gb you tested with includes the proposed change? I can't duplicate the output you're seeing, I get properly-relativized error messages:
inconshreveable [~/gopath/src/github.com/juju/juju] ♡ : gb build
github.com/juju/juju/version
gopkg.in/juju/charmstore.v4/internal/v4
version/version.go:32: non-declaration statement outside function body
version/version.go:32: syntax error: unexpected name, expecting semicolon or newline
github.com/juju/juju/provider/gce/google
provider/gce/google/instance.go:212: cannot use value (type string) as type *string in field value
provider/gce/google/instance.go:228: cannot use item.Value (type *string) as type string in assignment
../../../gopkg.in/juju/charmstore.v4/internal/v4/content.go:49: not enough arguments in call to jujusvg.NewFromBundle

The -R flag forces an interesting UX decision.
Without -R, I personally would prefer paths to be printed to be relative to $pwd, although it would be nice to know if others disagree.
With -R, I'm unsure because I don't have that use case. It seems like in that case it would be most appropriate to relativize either to the project root or to the directory passed to -R.
I would suggest, though, that changes to make the -R case better could be a separate issue as the proposed change is not set in stone and would not make the output any less user-friendly for -R than it already is.

IMO, paths always being relative to project root is a really simple and
unambiguous solution (reasonably easy for tooling wrapping gb to
parse/understand too).

Opps. Let me check again, I was pretty sure I was on the right branch and
had rebuilt, but please let me check again.
On Mon, Sep 21, 2015 at 10:49 AM, Alan Shreve<EMAIL_ADDRESS>wrote:

@davecheney https://github.com/davecheney Can you double check that you
used a build of gb you tested with includes the proposed change? I can't
duplicate the output you're seeing, I get properly-relativized error
messages:
inconshreveable [~/gopath/src/github.com/juju/juju] ♡ : gb buildgithub.com/juju/juju/versiongopkg.in/juju/charmstore.v4/internal/v4
version/version.go:32 http://gopkg.in/juju/charmstore.v4/internal/v4version/version.go:32: non-declaration statement outside function body
version/version.go:32: syntax error: unexpected name, expecting semicolon or newlinegithub.com/juju/juju/provider/gce/google
provider/gce/google/instance.go:212 http://github.com/juju/juju/provider/gce/googleprovider/gce/google/instance.go:212: cannot use value (type string) as type *string in field value
provider/gce/google/instance.go:228: cannot use item.Value (type *string) as type string in assignment
../../../gopkg.in/juju/charmstore.v4/internal/v4/content.go:49: not enough arguments in call to jujusvg.NewFromBundle
The -R flag forces an interesting UX decision.
Without -R, I personally would prefer paths to be printed to be relative
to $pwd, although it would be nice to know if others disagree.
With -R, I'm unsure because I don't have that use case. It seems like in
that case it would be most appropriate to relativize either to the project
root or to the directory passed to -R.
I would suggest, though, that changes to make the -R case better could be
a separate issue as the proposed change is not set in stone and would not
make the output any less user-friendly for -R than it already is.
—
Reply to this email directly or view it on GitHub
https://github.com/constabulary/gb/pull/377#issuecomment-141851311.

@inconshreveable thanks for your patience. I found the error, I was using a stale version of gb. I need to fix that bug first (coming up)

Thanks for working on this. I'll merge this now as it is an improvement over the current situation. I'll raise an issue to discuss the remaining points about making the output relative to a fixed point.

This change broke gb test on Windows when the project is not on the same drive as temp dirs.
I apologize for not having the time to test this before it was merged, but now my CI server has done the work.
Here is some output from the CI sever. (Portions in [brackets] show where I anonymized the log.)
D:\jenkins-slave\workspace\[project]>D:\jenkins-slave\workspace\[project]\tools\bin\gb test
FATAL command "test" failed: Rel: can't make \Windows\TEMP\gb528445211\cmd\bqtest\_test\cmd\_testmain.go relative to \jenkins-slave\workspace\[project]

It is strange that the drive letters are missing from the printed paths. \Windows\... should be C:\Windows\... and \jenkin-slave\... should be D:\jenkins-slave\...

That's weird. The code should leave all of the absolute paths alone but I'm clearly missing something: https://github.com/constabulary/gb/commit/7ef316330a51e59320880424cf98e6fbb8597648#diff-71395ae51f99d575c9342c11090a29c0R177
Since we've decided on a different behavior (https://github.com/constabulary/gb/issues/406) let's revert this. Any change along these lines is going to need a much more robust set of tests, too.


Added some callbacks to dynamic channel

Allow listener removal of channel during runtime
Allow retrieval of channel name

@hardening @bmiklautz @mfleisz anyone want to merge this?


Use Fedora 31 as a base for the workers

This is to make the tests work again. On the long run we should figure
out why the worker pods failed to start when the image was based on
Fedora 32 and why they had this issue only in the test environment but
not on the stage deployment.
Signed-off-by: Hunor Csomortáni<EMAIL_ADDRESS>

:+1:


Create a Contributor Guidelines Document

What happened (or feature request):
We will be implementing a Contributor License Agreement across the organization that will apply to all repositories.  This brings up the need for a general contributors document that outlines how we would like to engage with contributors.
What you expected to happen:
A document outlining our philosophy and procedures around contributing should be drafted.
Anything else do we need to know:
from @rfay
We're starting to collect some of these("team norms"), unstated but firmly held:

Approving a PR almost always means having actually tested it live
The PR creator generally pulls own PR after it's been approved and passed test, when the creator is sure it's the right time.
Nontrivial PRs normally get two full reviews before pull. Smaller ones may be fine with just one.
Significant PRs may need to be rebased near the end of their life so that the tests run with all current code (to minimize the potential of breaking master on pull)
PRs are generally expected to include tests for the new features they're adding, or to prevent relapses on bugs they're fixing.
The PR template includes a section for a manual testing procedure. Nearly every PR should have explicit instructions on how to do the manual testing.

It may be worth listing a few examples from other organizations to help people understand the practice.
Here are some example CLA agreements from other software projects you can use to compare:

Kubernetes https://github.com/kubernetes/community/blob/master/CLA.md
React https://code.facebook.com/cla
Apache https://www.apache.org/licenses/#clas
GitHub https://cla.github.com/
Golang https://golang.org/doc/contribute.html#cla%

@cyberswat Can you confirm (with the PR merged) that this is done?


Refactor executor main

Which issue does this PR close?

Part of https://github.com/apache/arrow-ballista/issues/613
Rationale for this change

Make it easier to build custom versions of Ballista with extensions and to wrap the executor in Python.
What changes are included in this PR?

Refactor existing code:

Add new ExecutorProcessConfig struct
Move logic from main.rs to executor_process.rs
Move main.rs out of lib and into src/bin/main.rs

Are there any user-facing changes?

No

@avantgardnerio You have experience in building custom executors that extend Ballista. Is this approach reasonable as a starting point to simplify doing this?
@jdye64 Would be good to get your thoughts too

Is this approach reasonable as a starting point to simplify doing this?

Yes, I definitely think we should make this change.
That being said, it can be painful extending the dependency graph to this many layers deep:
                         +-> delta-rs -+
                         |             |
                         |             |
arrow-rs ----> datafusion -------------+--> ballista --> custom-app

So it's my hope at S&T that we can get as much functionality merged upstream as possible to minimize TTL on arrow-rs changes.


[redis-plus-plus] update to 1.2.3

Fix #18331
Update redis-plus-plus to the latest version 1.2.3
All features are tested successfully in the following triplets:

x86-windows
x64-windows-
x64-windows-static

The port wouldn't link with c++17 app due to different header interfaces depending on the standart, so please add this option in vcpkg_configure_cmake as well:
        -DCMAKE_CXX_STANDARD=17

Cool, thanks Liu :)


Add example from R2 Advanced::Thin pod to the test suite

Ensure example from Marpa::R2::Advanced::Thin, or an equivalent, is in the test suite.

Issue moved from #123.


Error for pytorch-lightning

Thanks for sharing your excellent work! But I confronted an error and thus asking for help. When I run the trainer.fit(system) in train_ctc.py, there is an error said: The LightningModule.on_epoch_end hook was removed in v1.8. Please use LightningModule.on_<train/validation/test>_epoch_end instead.
But I can't find any possible code in train_ctc.py to fix it. I think this happens to the wrong version of pytorch-lightning, but the version is exactly what you wrote in the readme. So can you help me fix this problem?

#8

Thanks for your reply. I see that issue but get confused where to put the new code def on_validation_epoch_end(self). Could you explain it more explicitly?


Improve public key pinning

Currently the fingerprint of the certificate is pinned.
There are things to improve this pinning:

When cert expires and/or is renewed,. the fingerprint will change, so this breaks clients regularly!
To prevent this, pin the public key of the key pair instead of the fingerprint.
When the key pair has to be changed (e.g. in case of a server breach) there should be a backup key pinned, which can be used in this case. As Threema uses HPKP, they specify one. More information here.

As this may be an upstream issue with aiohttp, I also opened an issue there: https://github.com/KeepSafe/aiohttp/issues/1187

Also related: https://github.com/KeepSafe/aiohttp/issues/1185

Yeah, but HPKP is about dynamic pinning. This issue here is about a static (hardcoded) pin.

Yeah, but HPKP is about dynamic pinning. This issue here is about a static (hardcoded) pin.

Another good point made by @thorsteneckel: You still use SHA-1…

It's not like I have a choice. You should raise that issue towards the aiohttp project. :)

Maybe @thorsteneckel could look into it? I think you've already looked into the code…

I'll take the next free slot of time and create a PR.

@rugk It was actually MD5 but don't tell anyone. :fearful:
Changed it to SHA256 a6840a28041a1c43d90c21122ea324272f5c90c82dd64f52701f3a8f1a2b395b. However, this of course does not resolve the issue of public key pinning.

@rugk is right about that one, pinning the fingerprint is sometimes not an option in production, since it might break the gateway without notice.
Verifying the public key and backup key (via HPKP) would be the better approach.
@thorsteneckel any news on a free timeslot? :)

That's why verifying the fingerprint is turned off by default at the moment... which of course isn't ideal.

Actually, the aiohttp devs do not really care for that feature here…
Also remember https://github.com/aio-libs/aiohttp/issues/1187 is about static key pinning (what we need here) and https://github.com/aio-libs/aiohttp/issues/1185 about dynamic one (i.e. HPKP). And I would bet I know the argument they'll come up with if you urge them to implement HPKP or implement it yourself in a PR…


Powertoys url protocoll doesn't work

Microsoft PowerToys version
0.33.1
Running as admin
Yes
Area(s) with issue?
Settings window
Other Software
No response
Steps to reproduce

Open a "powertoys://" url

✔ Expected Behavior
PowerToys open
❌ Actual Behavior
It doesn't open (Not when powertoys is active nor if it's not active)

Can't also find PowerToys URL in Windows 10 settings > Default apps
Not sure if wanted or not

Cpuld be interesting for some of our features. Example: powertoys://ColorPicker

was this ever a thing?  What build did this work?

As I know this wasn't implemented actively at any time. So it's not a bug. It's more like a new feature/enhancement.

I know we had discussed deep linking settings but that can be done a few different ways.

was this ever a thing? What build did this work?

IDK. But it's a registred protocoll on my computer.
The funny thing is it doesn't appear in the settings app, but with 3rd party software like FileTypesMan it appears.

It seems like that gots implemented with 0.33.1

I'd say this is https://github.com/microsoft/PowerToys/issues/7408 but yeah, unsure how the protocol there works.  Looking at the PR listed in #7408 and the fact we haven't closed it, in the pure sense, i don't think this fully works as you'd expect it.
@enricogior close this out and point at #7408?

Closing this since the protocol is currently used for a complete different scope.


requirements.txt error?

(PokemonGo-Bot) Rorys-MBP:PokemonGo-Bot roryhellwig$ pip install -r requirements.txt
Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
ive done everything before this and now im stuck

check that file is there.. it cannot find it at the moment

the file isnt there

excuse me what do i do
(PokemonGo-Bot) Rorys-MBP:PokemonGo-Bot roryhellwig$ brew update && brew install --devel protobuf
Updated 1 tap (homebrew/core).
==> Updated Formulae
etcd                          gource                        libphonenumber                offlineimap                   purescript                    utf8proc
Warning: protobuf-3.0.0-beta-4 already installed
(PokemonGo-Bot) Rorys-MBP:PokemonGo-Bot roryhellwig$ git clone --recursive -b master https://github.com/PokemonGoF/PokemonGo-Bot
fatal: destination path 'PokemonGo-Bot' already exists and is not an empty directory.
(PokemonGo-Bot) Rorys-MBP:PokemonGo-Bot roryhellwig$ cd PokemonGo-Bot
(PokemonGo-Bot) Rorys-MBP:PokemonGo-Bot roryhellwig$ virtualenv .
New python executable in /Users/roryhellwig/PokemonGo-Bot/PokemonGo-Bot/bin/python2.7
Also creating executable in /Users/roryhellwig/PokemonGo-Bot/PokemonGo-Bot/bin/python
Installing setuptools, pip, wheel...done.
(PokemonGo-Bot) Rorys-MBP:PokemonGo-Bot roryhellwig$ source bin/activate
(PokemonGo-Bot) Rorys-MBP:PokemonGo-Bot roryhellwig$ pip install -r requirements.txt
Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'
(PokemonGo-Bot) Rorys-MBP:PokemonGo-Bot roryhellwig$


Move T's in the ASCII art up a bit.

The T's being too low was bugging me a bit more than it should have :)

Haha, thanks :)


Support react-native-fast-image

I would love to see support for react-native-web

Are you looking for the FastImage functionality on web? Or are you just trying to get your react-native-fast-image code working cross-platform?
With respect to the later, I'm exploring the possibility of using a babel plugin to rewrite all imports of the native Image component to use FastImage instead. You could conditionally use such a plugin only on native, then use the standard Image component throughout your code.

The latter. Cross platform working. I'll trust the browser to deal with the performance

I added it here: https://github.com/DylanVann/react-native-fast-image/pull/481


branch 0.13 add paritition fail

--- create table
CREATE table xng_ma.dwm_show_test_2 (
dt                          int COMMENT 'yyyyMMdd',
mid                         bigint COMMENT 'user id',
aid                         bigint COMMENT 'content id'
)
ENGINE = olap
DUPLICATE KEY(dt,mid, aid)
PARTITION BY range(dt) (
PARTITION dt20200817 VALUES [("20200817"),("20200818"))
)
DISTRIBUTED BY HASH(dt) BUCKETS 16
PROPERTIES (
"replication_num" = "3"
);
-- add material view
CREATE MATERIALIZED VIEW count_dwm_show_test_2 as
select
dt
count(mid)
from xng_ma.dwm_show_test_2
group by dt;
CREATE MATERIALIZED VIEW bitmap_dwm_show_test_2 as
select
dt,
bitmap_union(to_bitmap(mid)),
bitmap_union(to_bitmap(aid))
from xng_ma.dwm_show_test_2
group by dt;
-- add partition:
 alter table xng_ma.dwm_show_test_2 ADD PARTITION IF NOT EXISTS dt20200818 values [("20200818"),("20200819")) DISTRIBUTED BY HASH(dt) BUCKETS 32 PROPERTIES (     "replication_num" = "3" );
doris come out with error:
ERROR 1064 (HY000): errCode = 2, detailMessage = Failed to create partition[dt20200818]. Timeout. Unfinished mark: 10002=11121, 10002=11129, 10002=11137
i have see the log on be 10002, no 11121, 11129,11137 found.
I want to use the MATERIALIZED VIEW,
master branch do not have this error, but master branch can not desc dwm_show_test_2 all,  end in "ERROR 2027 (HY000): Malformed packet"
which version can i use?

#4342 Same as this issue. I already fix this issue in master branch by pr #4343 and #4408. You can use master branch directly.
According to "ERROR 2027 (HY000): Malformed packet" this error,  this may be caused by the incompatibility between the client you are using and the result of desc. You can try this statement directly through the mysql-client .

#4342 Same as this issue. I already fix this issue in master branch by pr #4343 and #4408. You can use master branch directly.
According to "ERROR 2027 (HY000): Malformed packet" this error, this may be caused by the incompatibility between the client you are using and the result of desc. You can try this statement directly through the mysql-client .

$ mysql --version
mysql  Ver 14.14 Distrib 5.7.31, for Linux (x86_64) using  EditLine wrapper
which version of mysql-client can be used for master branch?


Bls exports

replace custom bls types with eth2.0 types
export all related bls classes and methods from index
remove some extra dependency

resolves #385

This also fixes missing bls dependency (deprecated) that's breaking master build


Electron v10

Upgraded to v9 originally, fixed the issues.
Then I saw with the end of support message here: https://www.electronjs.org/blog/electron-12-0
So I decided to try to go to v10 as well, only breaking change from 9 -> 10 was the explicit enableRemoteModule: true
This is a PR to the exports PR branch that is also open. Made a decent amount of changes so I didn't want to miss any issue in those in the version upgrades

Seems to be working well 👍


Swipe to Unsubscribe

We currently have Swipe to Read on notifications, but as per SwipeCellKit we can have multiple options
One I'd find quite helpful is Swipe to Unsubscribe which stops you from getting further notifications on the given issue/pr
https://developer.github.com/v3/activity/notifications/#set-a-thread-subscription

Marking low since IMO unsub from the issue itself would be more useful (use it in multiple places).

We have this now


Thumbnail images are not properly referenced when writing AASX files

Thumbnail images for AASX files are currently simply exported along with other files, but are not properly referenced according to the specification.
This results in the problem, that e.g. the AASX explorer won't display the thumbnail image.
The AASX file as such is not broken or does cause actual errors, but the thumbnail image is kind of dangling inside the AASX.
To be correctly embedded, the file would need to be referenced from the root package, using the http://schemas.openxmlformats.org/package/2006/relationships/metadata/thumbnail relationship type.
I locally tested a fix and would be happy to provide a PR for this.

Thank you for bringing this to our attention. We would be very grateful if you would consider contributing a PR!


Upgrade Azure Firewall to Premium

As part of our network security requirements, we will be exploring the use of Azure Firewall Premium features.
This requires updating the existing CAF Terraform module settings (specifically the settings.connectivity.tf file), and changing the Azure Firewall SKU to "Premium".
            azure_firewall = {
              enabled = true
              config = {
                enable_dns_proxy = true
                dns_servers = [
                  "<IP_ADDRESS>" # Prvate IP address of Private DNS Resolver inbound endpoint
                ]
                sku_tier                      = "Premium"
                base_policy_id                = ""
                private_ip_ranges             = []
                threat_intelligence_mode      = ""
                threat_intelligence_allowlist = {}
                availability_zones = {
                  zone_1 = true
                  zone_2 = true
                  zone_3 = true
                }
              }
            }

Upgrade Observations
Terraform Plan
After changing the sku_tier property to Premium, terraform plan it shows that the Azure Firewall resource can be changed "in-place" (aka non-destructive), but, the Firewall Policy is a destructive upgrade. Notice the most important point, that the Rule Collection Groups will be removed (because we can't control those through the CAF deployment).
Terraform will perform the following actions:

  # module.connectivity.module.alz.azurerm_firewall.virtual_wan["/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/azureFirewalls/bcgov-managed-lz-forge-fw-hub-canadacentral"] will be updated in-place
  ~ resource "azurerm_firewall" "virtual_wan" {
        id                  = "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/azureFirewalls/bcgov-managed-lz-forge-fw-hub-canadacentral"
        name                = "bcgov-managed-lz-forge-fw-hub-canadacentral"
      ~ sku_tier            = "Standard" -> "Premium"
        tags                = {
            "deployedBy" = "azure-lz-core-forge"
        }
        # (8 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # module.connectivity.module.alz.azurerm_firewall_policy.virtual_wan["/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy"] must be replaced
-/+ resource "azurerm_firewall_policy" "virtual_wan" {
      - auto_learn_private_ranges_enabled = false -> null
      ~ child_policies                    = [] -> (known after apply)
      ~ firewalls                         = [
          - "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/azureFirewalls/bcgov-managed-lz-forge-fw-hub-canadacentral",
        ] -> (known after apply)
      ~ id                                = "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy" -> (known after apply)
        name                              = "bcgov-managed-lz-forge-fw-hub-canadacentral-policy"
      - private_ip_ranges                 = [] -> null
      ~ rule_collection_groups            = [
          - "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy/ruleCollectionGroups/WindowsAdminCenter_RuleCollectionGroup",
          - "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy/ruleCollectionGroups/e833c2_RuleCollectionGroup",
          - "/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy/ruleCollectionGroups/Core_RuleCollectionGroup",
        ] -> (known after apply)
      ~ sku                               = "Standard" -> "Premium" # forces replacement
        tags                              = {
            "deployedBy" = "azure-lz-core-forge"
        }
        # (3 unchanged attributes hidden)

        # (2 unchanged blocks hidden)
    }

Terraform Apply
When applying the above listed changes, the following error was thrown:
│ Error: deleting Firewall Policy (Subscription: "09bd024b-fbda-417d-b8db-694680c2b44e"
│ Resource Group Name: "bcgov-managed-lz-forge-connectivity"
│ Firewall Policy Name: "bcgov-managed-lz-forge-fw-hub-canadacentral-policy"): performing Delete: unexpected status 400 (400 Bad Request) with error: FirewallPolicyHasAzureFirewallReferences: Firewall Policy '/subscriptions/09bd024b-fbda-417d-b8db-694680c2b44e/resourceGroups/bcgov-managed-lz-forge-connectivity/providers/Microsoft.Network/firewallPolicies/bcgov-managed-lz-forge-fw-hub-canadacentral-policy' can not be deleted since there are Azure Firewalls using this policy.

According to the following GitHub Issue (Unable to change the firewall SKU from Standard to Premium), it states

It seems that we you might need to manually unlink the policy from the firewall to get around this. We will look into fixing this in a future release.

[!NOTE]
It is unknown at this time if this issue has been fixed in the CAF Terraform module, though it appears it has not been.
Additionally, the original GitHub Issue was closed due to lack of activity. We may want to open a new GitHub Issue to re-surface this issue.

How-To Unlink Azure Firewall Policy
To "unlink" (aka disassociate) an Azure Firewall Policy from an existing Azure Firewall, in the portal navigate to Firewall Manager > Firewall Policy > Manage Associations > Remove Hub Association.

[!WARN]
The wording on the UI can be misleading and cause concern. For example, "Remove Hub Association" may sound like you're removing the Firewall (not the policy) from the vHub. Also, the UI says "Remove" whereas the warning info says "Deleting policy associated..." Further, the bottom of the dialog also shows "Total VNets that will be dissociated", which causes concern about Hubs and VNets becoming disconnected.
The other concern is about having a disassociated Policy object in an orphaned state, and then the CAF Terraform module attempting to create a new Firewall Policy (with the same name), and causing a conflict. This did not occur though.

Durations
Manually removing the Policy association completed in approximately 5 minutes.
Applying the CAF Terraform module (with the updated SKU), completed in approximately 20 minutes.

The firewall has been upgrade to Premium only in the FORGE environment, to allow for testing and investigating (before it will potentially be enabled in LIVE).


Transform SequenceAt to split for special cases

Certain PyTorch.onnx.export will break the LSTM op into lower level operations and generate SplitToSequence and SequenceAt operation pattern. For example:
  %15 = onnx.Constant dense<0> : tensor<i64>
  %38 = onnx.Constant dense<1> : tensor<i64>
  %65 = onnx.Constant dense<100> : tensor<i64>
  %66 = "onnx.SplitToSequence"(%arg0, %65) {axis = 2 : si64, keepdims = 1 : si64} : (tensor<1x1x400xf32>, tensor<i64>) -> !onnx.Seq<tensor<1x1x100xf32>>
  %67 = "onnx.SequenceAt"(%66, %15) : (!onnx.Seq<tensor<1x1x100xf32>>, tensor<i64>) -> tensor<1x1x100xf32>
  %68 = "onnx.SequenceAt"(%66, %38) : (!onnx.Seq<tensor<1x1x100xf32>>, tensor<i64>) -> tensor<1x1x100xf32>
  %40 = "onnx.Add"(%67, %68) : (tensor<1x1x100xf32>, tensor<1x1x100xf32>) -> tensor<1x1x100xf32>

ONNX-MLIR currently does not have the lowering for SplitToSequence. In general, sequence related ops are not well optimized in ONNX-MLIR. I found that such code pattern can be converted into tensor operations to avoid the sequence ops.
    %0 = onnx.Constant dense<100> : tensor<4xi64>
    %1:4 = "onnx.Split"(%arg0, %0) {axis = 2 : si64} : (tensor<1x1x400xf32>, tensor<4xi64>) -> (tensor<1x1x100xf32>, tensor<1x1x100xf32>, tensor<1x1x100xf32>, tensor<1x1x100xf32>)
    %2:4 = "onnx.Split"(%arg0, %0) {axis = 2 : si64} : (tensor<1x1x400xf32>, tensor<4xi64>) -> (tensor<1x1x100xf32>, tensor<1x1x100xf32>, tensor<1x1x100xf32>, tensor<1x1x100xf32>)
    %3 = "onnx.Add"(%1#0, %2#1) : (tensor<1x1x100xf32>, tensor<1x1x100xf32>) -> tensor<1x1x100xf32>

The two onnx.Split is supposed to be able merged into one. But current onnx-mlir didn't. Need further investigation.
But the exported model has another type of SequenceAt, in which the output type of SequenceAt is different from the element type of the input sequence type. I think it is an error in the exporter. However, I tried to fix this issue in the transformation of this PR.
    %26 = onnx.Constant dense<0> : tensor<i64>
    %27 = onnx.Constant dense<1> : tensor<i64>
    %32 = "onnx.SplitToSequence"(%arg0, %27) {axis = 0 : si64, keepdims = 0 : si64} : (tensor<1x1x100xf32>, tensor<i64>) -> !onnx.Seq<tensor<1x1x100xf32>>
    %33 = "onnx.SequenceAt"(%32, %26) : (!onnx.Seq<tensor<1x1x100xf32>>, tensor<i64>) -> tensor<1x100xf32>

Output from this PR:
    %0 = onnx.Constant dense<0> : tensor<1xi64>
    %1 = onnx.Constant dense<1> : tensor<1xi64>
    %2 = "onnx.Split"(%arg0, %1) {axis = 0 : si64} : (tensor<1x1x100xf32>, tensor<1xi64>) -> tensor<1x1x100xf32>
    %3 = "onnx.Squeeze"(%2, %0) : (tensor<1x1x100xf32>, tensor<1xi64>) -> tensor<1x100xf32>

Other small fixes in this PR:

Handle the case that onnx.ConstantOp has not been normalized yet. Otherwise, --EmitONNXBasic will fail for this model.
It is not needed to check the tensor range size to be larger than one. In the model, the number of output from Split happened to be one.

@chentong319 Dominic is eager to test this. Can you implement the minor suggestions and merge it into main? Thanks

thanks @chentong319 .

Jenkins Linux s390x Build #16054 [push] Transform SequenceAt to ... started at 13:34

Jenkins Linux ppc64le Build #15081 [push] Transform SequenceAt to ... started at 13:50

Jenkins Linux amd64 Build #16051 [push] Transform SequenceAt to ... failed after 1 min 36 sec

Jenkins Linux s390x Build #16054 [push] Transform SequenceAt to ... passed after 1 hr 41 min

Jenkins Linux ppc64le Build #15081 [push] Transform SequenceAt to ... passed after 2 hr 18 min


Adding private IP address

It might be very useful to also add the private IP address of droplets to the created inventory file

Hi @markverg, I've just released an update that introduced a new --private-ips flag that uses private IPs instead of public IPs 😄


Agent PER account

Is the scope of the new vsts-agent on account level on Mac? So basically for each client I can create a different use account on the mac thus seperating the builds?

You can setup an agent per account or multiple agents on the same account in different directories.  It's self contained and xcopy bits.  The account is a security boundary - we store everything within that agent and account.  Even the credentials will be stored encrypted within the account once this PR comes in: https://github.com/Microsoft/vsts-agent/pull/97
Of course a machine boundary is even a better security boundary :)
Considerations are running as a service.  We run as LaunchAgents so even that is per account.  But you need to auto logon (and lock) - see service instructions.  We configure the service plist file (we create as a convenience) to run as the logged on user.  You could also create multiple agents in different directories under the same accounts logon but the plist says to run as a different user.  To do that you would have to create the plist, specify the user and ACL/chown the files and dirs.
What are you after here?  What are your goals.


useless for now

Domain
https://ckk.ai/
Link
https://ckk.ai/OF5suDdqVO
Version
0.0.0
What browsers are you seeing the problem on?
Firefox
What OS are you seeing the problem on?
Windows
Anything else?
doesn't work

Duplicate of #668.


Sorting of channels / conversations in left sidebar

Currently, newly added channels are just added to the bottom. This is quite strange, especially with private conversations inbetween.
Instead, in the first pass: All channels should be sorted first, and then private conversations. Sorted alphabetically.
In a second pass, we could sort the channels by the amount you use / message in them or something similar. That would make sorting more relevant.
For sure we can sort the private conversations to have the ones with the most recent message first. That’s how any mobile messenger like WhatsApp, Signal etc does it.

Don't forget users can re-order the server list by dragging the channels.

Yeah, I was just thinking that, too. I'm not sure that ordering it based on usage would be a good thing to do. Happy to start a new channel by putting it above the PMs though, that sounds like a sensible change.

I've found it very useful the way Textual does it: Channels are listed on top of PMs, and when a new one of either is added it goes to the bottom of that section. From there it can be dragged anywhere within that section (channels cannot be put in the middle of the PMs and vice versa).

Happy to start a new channel by putting it above the PMs though, that sounds like a sensible change.

@YaManicKill yup, definitely. The current mix is confusing. ;)
@xPaw @dgw For channels I am not sure what the best sorting method is. Alphabetical is fine for few channels, but then it quickly gets annoying. And manually dragging is just putting the work off to every single user. We can do better here.
For private messages, sorting by most recent up top is definitely a good way, as described above like others do it.

For reference: Slack (which is admittedly solving a slightly different problem, but is useful as a point of reference anyway) addresses this issue by having 3 sections:

"Starred" channels go way up at the top, sorted alphabetically.
All the rest of your channels go next, sorted alphabetically.
All your PMs go last, also sorted alphabetically.

There's an option to hide channels (& PMs) that don't have any activity in them, which is useful if you're in a large number of channels.
There's also a fantastic quick-switcher that lets you jump to a channel or PM by hitting a shortcut (Cmd+T) and typing part of the name.

The fancy UI propositions are nice, but as a first step, even only having a "Sort channels" button at the bottom of the channels would do it for me. When you have hundreds of channels, sorting by hand is a hassle, so I often find myself just using "CTRL+F"...

:+1: for a sort button !

Anybody working on this? Might give it a go some time next week.
Was thinking of adding an option in settings to sort alphabetically, and also leave it open for other sorting designs to be added, such as the most used channels design mentioned. Starred channels also sounds like a cool idea, so I might see if I can implement this if I have extra time.
Long time user, first time contributor, so some input on this would be greatly appreciated.

Was thinking of adding an option in settings to sort alphabetically, and also leave it open for other sorting designs to be added

In general, I like this, but we need to have a think where this will happen. I imagine it'll be a server-side option, because that's where the sorting happens (I think). But that makes it awkward to edit.
Also, a "manual" option would be good that would just pop it on at the bottom and you can manually reorder them and it'll never re-order.
We really need to get settings syncing :-\ that would make this a lot nicer.

Oh are settings stored locally? Never actually realized this, since I've never changed anything. I just assumed they were being pulled from the config file

We have server-side user settings, server-side global settings, and client-side settings. Client-side settings are entirely local, and don't sync at all.
The ideal situation is where the server-side user settings and client-side settings are 1 (with options to override on local machines) so for things that happen on the server, the user can edit locally, and the clients can sync their settings.
There was a PR for this ( #275 ) but it never got merged. It got lost in time.

Alright, I'm going to give a try at that.


Allow specifying auxilaryAttributes

which get dumped with the rest of data, very heplful for ploting performance

Got it. Kind of messy, but probably okay. @deepakn94: thoughts?
@viveksjain is going to add another way to specify a column as time so I'm curious if there is a more generic way we want to handle this.

From #64: timeColumn, attributes, lowMetrics, highMetrics are used together in a bunch of places - maybe it's worth refactoring them out into a single wrapper class.
It's not really a more generic way to handle this, more like duct tape to hide the excrescence. Maybe convert the rest to something like a DataTransformation so you have inversion of control? And then you could have have a wrapper class containing a List.
BTW, this is probably going to cause merge conflicts with #64, but since my branch already has merge conflicts I'm going to wait for this one to go in first.

And I'd prefer any refactoring to be done after #64 is in.

Cool. I like the idea of putting all those variables in a wrapper.
I'll wait for a word from @deepakn94 and merge this after that.

Sure, we can put these in a wrapper class.

Sure, we can do a refactoring push after #64.
@viveksjain: more incentive to get it in!


python3Packages.sphinx: 7.2.6 -> 7.3.7

Description of changes
Update to latest release:
https://github.com/sphinx-doc/sphinx/releases/tag/v7.3.7
This release includes a patch which improves reproducibility: https://github.com/sphinx-doc/sphinx/commit/8e768e6c231c67caadecd5b43c20eb1f3a594079
Requires slight modification to the postFetch due to test files moving around.
Also a new dependency in the test fixtures. These are used by downstream dependencies, so it has to be propagated.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[x] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Oh you skipped the graphviz tests...

@ofbuild build python3Packages.sphinx python3packages.sphinx-rtd-theme

The sphinx update to 7.3.7 is part of #305152.


FlatList in Collapsible fails on Second Open

Possibly related to issue #97 ?
In the following code, the first time I press on the Collapsible, it works AOK, press again and it collapses back down...
The Second Time, however, weirdness happens and the resulting element is neither scrollable nor the output of the lotsOfData...
import React, { Component } from 'react';
import { FlatList, StyleSheet, Text, View, TouchableHighlight, } from 'react-native';
import * as Animatable from 'react-native-animatable';
import Collapsible from 'react-native-collapsible';
import Accordion from 'react-native-collapsible/Accordion';

const BACON_IPSUM = 'Bacon ipsum dolor amet chuck turducken landjaeger tongue spare ribs. Picanha beef prosciutto meatball turkey shoulder shank salami cupim doner jowl pork belly cow.  I made an example project here you can see https://snack.expo.io/rJg_GM2pb where I replicated the simplest part of your code. The problem I'm assuming is because of the asynchronous way in which Flatlist handles elements.
On first opening, the Flatlist has not yet loaded all the items in the list, since with Flatlist the elements are not all rendered instantly. The second time more items have been loaded, in which you can see in the example that list is shown and even pushes the toggle button out of view.

Closed as inactive

@robertoclavijob Are you referring to a FlatList being inside one of the Accordion items?

Correct @iRoachie , the issue is reproduced using a FlatList inside of Accordion

@robertoclavijob can you make a snack for me to test this with? https://snack.expo.io

Hi,
I'm having the same issue : I have a button which manages the "collapsibility" of the Collapsible component.
In this Collapsible component, I have a FlatList with many items.
When I open the app, the FlatList is scrollable.
I push the button to mask the list, and push it again to display the list.
Then, the FlatList is no more scrollable.
Here is a snack which reproduces the problem.


Pal-132

Relevant Issue
[PAL-132] https://pallinet.atlassian.net/browse/PAL-132
Description
added detailed view page for treatments, now box is clickable and opens new page.

added a new entry to the database called detailedInstructions and pulled from that when displaying in detailed page.
when clicked

fixed the whitespace, just thought it looked clean like that. We will need to implement a way for doctors to be able to add into the database, same for prescriptions


Visual Studio 2015 - Update 3 - .ts files not compiling with tsconfig.json present

TypeScript Version:  1.8.35
For some reason, the .ts files aren't being compiled to .js on save using Visual Studio Professional 2015 Update 3
If I remove the tsconfig.json file (or rename it) the compilation runs properly.
Here's my tsconfig.json:
{
  "compilerOptions": {
    "target": "es5",
    "module": "system",
    "moduleResolution": "node",
    "sourceMap": true,
    "emitDecoratorMetadata": true,
    "experimentalDecorators": true,
    "removeComments": false,
    "noImplicitAny": true
  },
  "compileOnSave": true,
  "exclude": [
    "node_modules",
    "typings/main",
    "typings/main.d.ts",
    "wwwroot"
  ]
}

If the file is present in the root folder, when I save a .ts file, nothing happens. No error. It just doesn't compile.
The automatically compile TypeScript files that are not part of a project is unchecked.

Described here as well: https://github.com/Microsoft/TypeScript/issues/5995

I have tried your tsconfig.json then added a ts file into the same folder as tsconfig.json and the file seems to be compiled just file. I am currently used TypeScript 2.0.0 Beta on Update3, could you give 2.0.0 Beta a try?

Thanks for the response. I tried installing TypeScript 2.0.0 Beta. It shows up in the VS about windows as expected, but the behavior seems to be the same. VS Code, works but slowly and Atom with the Typescript plugin works without any problem.

can you share a project that exhibits the issue?

I'm using .NET Core 1.0, but I tried following this tutorial, using .NET 4.x: https://www.typescriptlang.org/docs/handbook/asp-net-4.html
It only compiles .ts if I build. Before I algo did try follow the tutorial for Typescript tutorial for .NET Core, adapting from RC1 to 1.0. Neither work as expected, since it shouldn't really matter which version of .NET Core is being used as long as there are typescript files and a tsconfig.json. Anyway, those are two projects where it didn't work.

Can you share the project?

It's the most basic project possible. Everything works fine with VS Code and Atom with the typescript plugin. It's specifically VS2015. I tried repairing it with no luck. Is there anyway to delete some Typescript cache definitions or something that may reset the integration between VS2015 and typescript?

By the way, it also seems to be ignoring the tsconfig.json options, since it complains gives the "experimental support for decorators is a feature..." message. If I switch it to false, the message shows up in VS Code, otherwise it doesn't.
I'm also getting 90000 errors from lib.d.ts and all the modules from @angular and rxjs are said to be "not found". Again, only in VS2015.
After a suggestion here, I installed Typescript 2.0beta, which didn't help. In the about page that's the only version that shows up and I'm unable to install it, although I supposedly uninstalled it by running it's installer.
After uninstalling VS2015 and installing it again, everything stayed the same, including the "2.0" Typescript version in the about window.

Anyway after multiple reinstalls and repairs and deleting chache and temp folders and registry entries and more repairs and some more random stuff, I once again installed Typescript 2.0 beta (which did nothing) and then repaired it aaaaaand now it works.
So for anyone with the same problem, just follow those steps :P

I had the same problem.
Solved it by checking version:
>tsc -v
make sure the value is > 1.7.
If not, update it using this tutorial:
http://stackoverflow.com/questions/31750964/typescript-compile-on-save-feature-not-working-in-visual-studio-2015/35258446#35258446

Hi,
I may be very late to answer but you can follow the steps mentioned in the following, it worked for me.
http://sharepointpals.com/post/Creating-a-SharePoint-App-with-TypeScript-and-Angular-Js-Step-by-Step


[Suggestion] Set default settings for custom args and filename

It'd be nice if custom args and filename doesn't get reset for every download. Would be very useful if for downloading bunch of videos with the same quality and same format and doesn't seem to be difficult to implement. I can make a PR if you're interested in adding this feature

I'll consider removing https://github.com/dsymbol/yt-dlp-gui/blob/main/app/app.py#L107 - 08 which would stop clearing of custom args and filename after download is added.

The custom args not being reset every time I open the program, would be great.


Polish Rainbow Brush to be Boss tier (thanks to Discord's kiooeht#3584 for the tip about Retain + Ethereal)

Ideas:

Status & Curses are added to the draw pile instead of the hand; - OK
Remove Void from the Curse pile if the change above does not happen; - OK
Change the card given after each time it gives one; - OK
Relic won't create Common cards. - OK
Relic increases the player's hand size by one. - No.
Tweak card generation to never give Ethereal to cards that have Retain (thanks kiooeht) - OK
Relic gives cards faster, but they have Retain AND Exhaust. - No need.
Relic gives cards faster, but they have Ethereal and Exhaust. - No need.
Tweak the chance variable from 6 to 5 - OK
See if Keywords can make the description smaller.

This goes to a wish. No energy, no motivation, the essential idea from the relic is flawed.

The hell this is a wish -_-

Removed "Rainbow cards & curses (so the relic is less dependable of mods to be interesting);" - Too much work that's better spent making the relic's mechanics better.

"Status & Curses are added to the draw pile instead of the hand;"
The main problem here is that this makes the relic description even bigger. -_- But it has to be done. So. Yeah.

Ok, we have enough already by now. Let's see if it holds up as a Boss relic.

After all the changes, I buffed the relic a bit. It now should be more reliable in its... unreliability : p

Ok, closing this issue, if the relic has any other problem, I will create a new issue.


Fall 2018 Project

This epic houses tasks for the Fall 2018: Tutoring Notifications project

This project has been completed on the fall-2018 branch. Closing