Starting the terminal in a non-WSL shell boots a WSL distribution whose shell is set as default in settings

Environment
Windows build number: 10.0.19042.0
Windows Terminal version (if applicable): 1.4.3141.0

Any other software? Ubuntu 18.04 and Ubuntu WSL distros installed, the latter of which is a WSL 2 distro.

Steps to reproduce

With a WSL distribution installed and its shell configured in the Windows Terminal, set that distribution's shell as the default profile in the Windows terminal settings (the distribution is usually added automatically, you should only need to modify the defaultProfile key in the settings JSON).
Ensure that the WSL distribution whose shell is currently set as the default in Windows terminal is not currently running (consider using wsl --shutdown to stop all distros).
Open the start menu, find the Windows Terminal application shortcut, and right click on it.
Under Tasks, open any non-WSL shell such as PowerShell or cmd.
Run wsl --list --verbose and observe that the WSL distribution whose shell is set to the default has been started and is now running.

Expected behavior
When starting the Windows Terminal directly to a specified profile via right clicking on the application icon and selecting one of the Tasks, the default shell is not invoked. If the default shell is a WSL distribution shell, this means not booting the WSL distribution.
Actual behavior
After starting Windows terminal directly to the powershell profile, I observed that the Ubuntu WSL distribution, whose shell is set as my default in Windows Terminal, was getting booted. The terminal would still load the Powershell profile correctly, it just starts the WSL distro even though it won't be getting used.
Settings.json
Note: I've removed the colour schemes, and I had no keybindings configured.
// To view the default settings, hold "alt" while clicking on the "Settings" button.
// For documentation on these settings, see: https://aka.ms/terminal-documentation
{
    "$schema": "https://aka.ms/terminal-profiles-schema",
    "defaultProfile": "{2c4de342-38b7-51cf-b940-2309a097f518}",
    "profiles": {
        "defaults": {
            // Put settings here that you want to apply to all profiles
        },
        "list": [
            {
                // Make changes here to the powershell.exe profile
                "guid": "{61c54bbd-c2c6-5271-96e7-009a87ff44bf}",
                "name": "Windows PowerShell",
                "commandline": "powershell.exe",
                "hidden": false
            },
            {
                // Make changes here to the cmd.exe profile
                "guid": "{0caa0dad-35be-5f56-a8ff-afceeeaa6101}",
                "name": "cmd",
                "commandline": "cmd.exe",
                "hidden": false
            },
            {
                "guid": "{c6eaf9f4-32a7-5fdc-b5cf-066e8a4b1e40}",
                "hidden": false,
                "name": "Ubuntu-18.04",
                "source": "Windows.Terminal.Wsl",
                "startingDirectory": "//wsl$/Ubuntu-18.04/home"
            },
            {
                "guid": "{b453ae62-4e3d-5e58-b989-0a998ec441b8}",
                "hidden": false,
                "name": "Azure Cloud Shell",
                "source": "Windows.Terminal.Azure"
            },
            {
                "guid": "{2c4de342-38b7-51cf-b940-2309a097f518}",
                "hidden": false,
                "name": "Ubuntu",
                "source": "Windows.Terminal.Wsl",
                "startingDirectory": "//wsl$/Ubuntu/home"
            },
            {
                "guid": "{00000000-0000-0000-ba54-000000000002}",
                "commandline": "%PROGRAMFILES%/git/usr/bin/bash.exe -i -l",
                "icon": "%PROGRAMFILES%/Git/mingw64/share/git/git-for-windows.ico",
                "name": "GBash",
                "startingDirectory": null,
                "colorScheme": "Tinacious Design (Dark)",
                "acrylicOpacity": 0.5,
                "useAcrylic": true
            }
        ]
    },

Please share a copy of your settings.json.

I've updated the description @DHowett , let me know if there is anything else you'd like.

Thanks.
If you run wt -p {61c54bbd-c2c6-5271-96e7-009a87ff44bf} directly (from the Run dialog) after terminating your WSL instances, what happens?
When you launch Terminal via the Tasks list, and it goes to the wrong profile, what icon does it display? Does it use the settings from one profile (color scheme, font, etc.) but start an application from a different profile?

Looks like I get the same behaviour when invoking it with wt -p {61c54bbd-c2c6-5271-96e7-009a87ff44bf}.
Also, to clarify @DHowett, it is going to the correct profile, using the correct icons and settings, etc. The only problem that I think exists is that it seems to be starting the WSL distro as well.

And if you do not set this WSL distribution as your default, it's definitely not booted?
How about if you launch Windows PowerShell outside of terminal after shutting down WSL?
WT doesn't do anything that should result in processes that are not part of a current active profile being run...

When the WSL shell isn't set as my default profile, it doesn't start the distro. Additionally, if I set the default profile to the shell for my Ubuntu 18.04 distro, it starts that distribution instead of the Ubuntu one, so it's definitely only starting the shell that's configured as the default. This also means it's not specific to WSL 1 or WSL 2, it occurs on both.
Starting powershell or CMD from outside of Windows Terminal does not cause this issue.

Small note here, there seems to be a mismatch between the build number listed by [Environment]::OSVersion in PowerShell and ver in CMD.

I reported the one listed in PowerShell, but the version in CMD is reported when I run winver or go to Settings -> System -> About.

Yeah, that version number thing is annoying. PowerShell doesn't know where to find the fourth component!
Would you mind grabbing a process monitor trace? It might be enlightening, and help us see what Terminal is doing behind the scenes.

Shut down WSL -- get ready to reproduce the issue
Install process monitor
Launch it.
Configure it with the following filters in place:

Process Name is WindowsTerminal.exe: Include
Process Name is wsl.exe: Include
Process Name is powershell.exe: Include

Hit OK
Reproduce the issue! Launch terminal with a profile that is incorrectly launching WSL.
Press the capture icon to stop capturing

File > Save

E-mail me the PML file -- or post it here (but it will include things like your username and some details about the file layout on your machine) -- at the address on my GitHub profile.
Thanks! :smile:

I've gathered the logs @DHowett and have sent the email. Let me know if there's anything else that you may need.

I think I've got it. Thanks so much!
We create a whole TerminalSettings for the default profile to measure how big the window should be (to translate rows/columns to pixels through the renderer).
Creating one of those evaluates the starting directory for the default profile, which causes it to be checked for viability.
Since yours is //wsl$/Ubuntu/xxx, it starts up the distribution to see if that path exists. Ew.
https://github.com/microsoft/terminal/blob/6b503ba8878b084280fa7de7fc36378f73807000/src/cascadia/TerminalApp/TerminalSettings.cpp#L97
@carlos-zamora remember how we talked about TerminalSettings proxying through to a profile? This is one issue where eagerly evaluating the profile values causes a problem.

Awesome, glad you were able to narrow down a cause 🙂 From: Dustin L. HowettSent: November 16, 2020 22:46To: microsoft/terminalCc: Kyle Anderson; AuthorSubject: Re: [microsoft/terminal] Starting the terminal in a non-WSL shell boots a WSL distribution whose shell is set as default in settings (#8285) I think I've got it. Thanks so much!We create a whole TerminalSettings for the default profile to measure how big the window should be (to translate rows/columns to pixels through the renderer).Creating one of those evaluates the starting directory for the default profile, which causes it to be checked for viability.Since yours is //wsl$/Ubuntu/xxx, it starts up the distribution to see if that path exists. Ew.https://github.com/microsoft/terminal/blob/6b503ba8878b084280fa7de7fc36378f73807000/src/cascadia/TerminalApp/TerminalSettings.cpp#L97@carlos-zamora remember how we talked about TerminalSettings proxying through to a profile? This is one issue where eagerly evaluating the profile values causes a problem.—You are receiving this because you authored the thread.Reply to this email directly, view it on GitHub, or unsubscribe.

The fix will probably be removing EvaluatedStartingDirectory from the profile interface & evaluating it in ConPtyConnection.

Hey! I think we actually did fix this in 1.9. We shopped evaluating the starting directory of any profile, likely much to folks' dismay.
See #10045, which closed #9541 and #9114.

/dup #9541 (sorry to dupe you to a higher number!)


pip: 21.0.1 -> 21.1.3; setuptools: 54.2.0 -> 56.2.0; wheel: update hompage

Motivation for this change
https://github.com/pypa/pip/blob/HEAD/NEWS.rst#2113-2021-06-26
https://github.com/pypa/setuptools/blob/HEAD/CHANGES.rst#v5620
setuptools >= 57.0.0 fails to build in bootstrap.py, and I'm not sure how to rework that step. The bootstrap.py now says to use a PEP-517 compatible installer method. But how do we bootstrap that one without setuptools?
❯ nom-build -A python3Packages.pip
these derivations will be built:
  /nix/store/jljikdxvyz3f9c6xbh3khyd67ci2b3a9-setuptools-57.0.0-sdist.tar.gz.drv
  /nix/store/zrdrkw3j25nbji5zf89p2937q6s2am4x-python3.9-bootstrapped-pip-21.1.3.drv
  /nix/store/0wlis1lri41ifg3xqqzx2mmdajnx0w04-python3.9-setuptools-57.0.0.drv
  /nix/store/gffmfx5aq98p1gafa8q6ry9kr1wh3zwh-python-catch-conflicts-hook.drv
  /nix/store/hi079vkq9gw4lk9jzhai0a7vqcgisc14-python3.9-pip-21.1.3.drv
building '/nix/store/jljikdxvyz3f9c6xbh3khyd67ci2b3a9-setuptools-57.0.0-sdist.tar.gz.drv' on<EMAIL_ADDRESS>unpacking sources
unpacking source archive /nix/store/xpggxixcp7dwsrg6046v4s952v9q8qk7-setuptools-57.0.0-source
source root is setuptools-57.0.0-source
patching sources
applying patch /nix/store/0qzs4kq2lvl09k80nf54bl00hd2ylfam-tag-date.patch
patching file setup.cfg
Hunk #1 succeeded at 144 with fuzz 2 (offset 143 lines).
configuring
no configure script, doing nothing
building
adding minimal entry_points
Regenerating egg_info
Traceback (most recent call last):
  File "/build/setuptools-57.0.0-source/pkg_resources/__init__.py", line 2679, in version
    return self._version
  File "/build/setuptools-57.0.0-source/pkg_resources/__init__.py", line 2813, in __getattr__
    raise AttributeError(attr)
AttributeError: _version

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/build/setuptools-57.0.0-source/setup.py", line 100, in <module>
    dist = setuptools.setup(**setup_params)
  File "/build/setuptools-57.0.0-source/setuptools/__init__.py", line 152, in setup
    _install_setup_requires(attrs)
  File "/build/setuptools-57.0.0-source/setuptools/__init__.py", line 145, in _install_setup_requires
    dist.parse_config_files(ignore_option_errors=True)
  File "/build/setuptools-57.0.0-source/setuptools/dist.py", line 770, in parse_config_files
    self._parse_config_files(filenames=filenames)
  File "/build/setuptools-57.0.0-source/setuptools/dist.py", line 652, in _parse_config_files
    opt = self.warn_dash_deprecation(opt, section)
  File "/build/setuptools-57.0.0-source/setuptools/dist.py", line 685, in warn_dash_deprecation
    commands = distutils.command.__all__ + self._setuptools_commands()
  File "/build/setuptools-57.0.0-source/setuptools/dist.py", line 699, in _setuptools_commands
    dist = pkg_resources.get_distribution('setuptools')
  File "/build/setuptools-57.0.0-source/pkg_resources/__init__.py", line 466, in get_distribution
    dist = get_provider(dist)
  File "/build/setuptools-57.0.0-source/pkg_resources/__init__.py", line 342, in get_provider
    return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
  File "/build/setuptools-57.0.0-source/pkg_resources/__init__.py", line 626, in find
    if dist is not None and dist not in req:
  File "/build/setuptools-57.0.0-source/pkg_resources/__init__.py", line 3124, in __contains__
    item = item.version
  File "/build/setuptools-57.0.0-source/pkg_resources/__init__.py", line 2687, in version
    raise ValueError(msg, self) from e
ValueError: ("Missing 'Version:' header and/or PKG-INFO file at path: /build/setuptools-57.0.0-source/setuptools.egg-info/PKG-INFO", setuptools [unknown version] (/build/setuptools-57.0.0-source))
Traceback (most recent call last):
  File "/build/setuptools-57.0.0-source/bootstrap.py", line 57, in <module>
    __name__ == '__main__' and ensure_egg_info()
  File "/build/setuptools-57.0.0-source/bootstrap.py", line 38, in ensure_egg_info
    run_egg_info()
  File "/build/setuptools-57.0.0-source/bootstrap.py", line 54, in run_egg_info
    subprocess.check_call(cmd)
  File "/nix/store/j70c02srryah1fjvf36yj54bqx04rqgp-python3-3.9.5/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/nix/store/j70c02srryah1fjvf36yj54bqx04rqgp-python3-3.9.5/bin/python3.9', 'setup.py', 'egg_info']' returned non-zero exit status 1.

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Relase notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Replaced by https://github.com/NixOS/nixpkgs/pull/130250

a3e5f70  can still be pushed

Just pick it into yours.


Im getting this error when trying to load a model in DreamBooth

Im using a 1.5 model but this happen

make sure you choose the right version in the cell

it crossed my mind, but with any version i get the same error

make sure you choose the right version in the cell

it crossed my mind, but with any version i get the same error


mpi run process() function hangs at some cases

Hi Lion,
I am trying to run a data processing using the process() function in mpi environment.
In the following case, I got 10 asdf file to process and I am using 4 processors. So I go through them one by one and delete the dataset when I am done with it. I got such errors after the program runs for a certain time(for example, after processing 4 files and now it is processing the 5th):
====================
Dir info:
input asdf: /lustre/atlas/proj-shared/geo111/Wenjie/DATA_SI/ASDF_new/raw/synt/C200502151442A.Mpp.synthetic.h5
output asdf: /lustre/atlas/proj-shared/geo111/Wenjie/DATA_SI/ASDF_new/proc/synt/C200502151442A.Mpp.proc_synt_50_100.h5
Processing info:
filter band: [40.0, 50.0, 100.0, 150.0]
tag map: {"synthetic" ==> "proc_synt_50_100"}
interp relative start and end(to eventcentroid time): (0.00    ,  6000.00)
interp deltat: 0.5
output_asdf exists and removed:/lustre/atlas/proj-shared/geo111/Wenjie/DATA_SI/ASDF_new/proc/synt/C200502151442A.Mpp.proc_synt_50_100.h5
Launching processing using MPI on 4 processors.
Jobs (running 2.17 seconds): queued: 142 | finished: 0 | total: 161
	Worker 1: 9 active, 0 completed jobs
	Worker 2: 9 active, 0 completed jobs
	Worker 3: 1 active, 0 completed jobs

Jobs (running 4.36 seconds): queued: 123 | finished: 0 | total: 161
	Worker 1: 19 active, 0 completed jobs
	Worker 2: 18 active, 0 completed jobs
	Worker 3: 1 active, 0 completed jobs

Jobs (running 6.51 seconds): queued: 105 | finished: 0 | total: 161
	Worker 1: 27 active, 0 completed jobs
	Worker 2: 28 active, 0 completed jobs
	Worker 3: 1 active, 0 completed jobs

Jobs (running 8.67 seconds): queued: 90 | finished: 0 | total: 161
	Worker 1: 35 active, 0 completed jobs
	Worker 2: 35 active, 0 completed jobs
	Worker 3: 1 active, 0 completed jobs

Jobs (running 10.81 seconds): queued: 76 | finished: 0 | total: 161
	Worker 1: 42 active, 0 completed jobs
	Worker 2: 42 active, 0 completed jobs
	Worker 3: 1 active, 0 completed jobs

Jobs (running 12.87 seconds): queued: 62 | finished: 0 | total: 161
	Worker 1: 49 active, 0 completed jobs
	Worker 2: 49 active, 0 completed jobs
	Worker 3: 1 active, 0 completed jobs

Jobs (running 15.02 seconds): queued: 45 | finished: 0 | total: 161
	Worker 1: 57 active, 0 completed jobs
	Worker 2: 58 active, 0 completed jobs
	Worker 3: 1 active, 0 completed jobs

Jobs (running 17.09 seconds): queued: 29 | finished: 0 | total: 161
	Worker 1: 66 active, 0 completed jobs
	Worker 2: 65 active, 0 completed jobs
	Worker 3: 1 active, 0 completed jobs

Jobs (running 19.13 seconds): queued: 13 | finished: 0 | total: 161
	Worker 1: 74 active, 0 completed jobs
	Worker 2: 73 active, 0 completed jobs
	Worker 3: 1 active, 0 completed jobs

Jobs (running 21.90 seconds): queued: 0 | finished: 0 | total: 161
	Worker 1: 80 active, 0 completed jobs
	Worker 2: 80 active, 0 completed jobs
	Worker 3: 1 active, 0 completed jobs

It seems the "Worker 3" is not reponding. So the program just hangs up. I am not quite sure what trigers that.
It might be some memory issues here since if I am using 16 cores, the problem never shows up. I am not quite sure. I will dive into the code and see if I can find the bug.
Or maybe it is not good to run several dataset processing in one mpirun(maybe the python is not good at this kind of memory management). So in the future, it will be safer to process one file at one mpiexec call.

Hey Wenjie,
does it always happen after the 4th file or it is random? I thought I found all the MPI communication edge cases :-( It should definitely work and I'd be quite happy to help debug the issue.
Can you run it with
ASDFDataSet(..., debug=True)

It will produce an enormous amount of output as it prints out the whole MPI communication but it should help find whats wrong. Can you please upload the output to pastebin or somewhere else?
Thanks!
Lion

I did another 5 test.
First three stops at the 3rd file on Worker 2.
Last two stops at the 6th file on Worker 1.
I will turn on the debug and keep you updated.

the "debug=True" flag is not working on my machine... I will try to figure it out now.
mpiexec -n 2 process_synthetic.py -f proc_synt.dirs.json -p proc_synt.params.json -v
====================
Dir info:
input asdf: /lustre/atlas/proj-shared/geo111/Wenjie/DATA_SI/ASDF_new/raw/synt/C200502151442A.synthetic.h5
output asdf: /lustre/atlas/proj-shared/geo111/Wenjie/DATA_SI/ASDF_new/proc/synt/C200502151442A.proc_synt_50_100.h5
Processing info:
filter band: [40.0, 50.0, 100.0, 150.0]
tag map: {"synthetic" ==> "proc_synt_50_100"}
interp relative start and end(to eventcentroid time): (0.00    ,  6000.00)
interp deltat: 0.5
output_asdf exists and removed:/lustre/atlas/proj-shared/geo111/Wenjie/DATA_SI/ASDF_new/proc/synt/C200502151442A.proc_synt_50_100.h5
/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/asdf_data_set.py:129: ASDFWarning: Compression will be disabled as parallel HDF5 does not support compression
  warnings.warn(msg, ASDFWarning)
/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/asdf_data_set.py:129: ASDFWarning: Compression will be disabled as parallel HDF5 does not support compression
  warnings.warn(msg, ASDFWarning)
Launching processing using MPI on 2 processors.
Traceback (most recent call last):
  File "/ccs/home/lei/source_inversion_wf/src/preproc_asdf/test_wf/process_synthetic.py", line 126, in <module>
Traceback (most recent call last):
  File "/ccs/home/lei/source_inversion_wf/src/preproc_asdf/test_wf/process_synthetic.py", line 126, in <module>
    proc.smart_run()
  File "/ccs/home/lei/source_inversion_wf/src/preproc_asdf/test_wf/process_synthetic.py", line 111, in smart_run
    proc.smart_run()
  File "/ccs/home/lei/source_inversion_wf/src/preproc_asdf/test_wf/process_synthetic.py", line 111, in smart_run
    self.proc_synt_asdf(_par, _file)
  File "/ccs/home/lei/source_inversion_wf/src/preproc_asdf/test_wf/process_synthetic.py", line 95, in proc_synt_asdf
    self.proc_synt_asdf(_par, _file)
  File "/ccs/home/lei/source_inversion_wf/src/preproc_asdf/test_wf/process_synthetic.py", line 95, in proc_synt_asdf
    interp_deltat=par_obj.interp_deltat)
  File "/autofs/nccs-svm1_home1/lei/source_inversion_wf/src/preproc_asdf/proc_util.py", line 204, in process_synt
    interp_deltat=par_obj.interp_deltat)
  File "/autofs/nccs-svm1_home1/lei/source_inversion_wf/src/preproc_asdf/proc_util.py", line 204, in process_synt
    ds.process(process_function, outputfn, tag_map=tag_map)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/asdf_data_set.py", line 1417, in process
    ds.process(process_function, outputfn, tag_map=tag_map)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/asdf_data_set.py", line 1417, in process
    station_tags, tag_map)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/asdf_data_set.py", line 1433, in _dispatch_processing_mpi
    station_tags, tag_map)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/asdf_data_set.py", line 1430, in _dispatch_processing_mpi
    output_data_set, tag_map)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/asdf_data_set.py", line 1569, in _dispatch_processing_mpi_worker_node
    station_tags, tag_map)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/asdf_data_set.py", line 1496, in _dispatch_processing_mpi_master_node
    self._send_mpi(None, 0, "WORKER_REQUESTS_ITEM", blocking=False)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/asdf_data_set.py", line 1775, in _send_mpi
    pretty_receiver_log(source, self.mpi.rank, status.tag, msg)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/utils.py", line 981, in pretty_receiver_log
    _pretty_log(prefix, rank, source, tag, payload)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/utils.py", line 1008, in _pretty_log
    pretty_sender_log(dest, self.mpi.rank, tag, obj)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/utils.py", line 975, in pretty_sender_log
    tag = tag_colors[tags.index(tag) % len(tag_colors)] + tag + \
ValueError: u'WORKER_REQUESTS_ITEM' is not in list
    _pretty_log(prefix, destination, rank, tag, payload)
  File "/autofs/nccs-svm1_home1/lei/software/pyasdf/pyasdf/utils.py", line 1008, in _pretty_log
    tag = tag_colors[tags.index(tag) % len(tag_colors)] + tag + \
ValueError: u'WORKER_REQUESTS_ITEM' is not in list
-------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code.. Per user-direction, the job has been aborted.
-------------------------------------------------------
--------------------------------------------------------------------------
mpiexec detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[1621,1],1]
  Exit code:    1
--------------------------------------------------------------------------

Hi Lion,
It seems I have located the bug for the "debug" flag.
    # Deterministic colors also on Python 3.
    msg_tag_keys = sorted(MSG_TAGS.keys(), key=lambda x: str(x))
    tags = [i for i in msg_tag_keys if isinstance(i, (str, bytes))]
    print("see 1:", msg_tag_keys)

    tag = MSG_TAGS[tag]
    print("see here:", tag, tags)

    tag = tag_colors[tags.index(tag) % len(tag_colors)] + tag + \
        colorama.Style.RESET_ALL

The debug=True bug should be fixed in a081348dc6738621ff25eab1308ca7d50afe7a62.

run1.log.txt
Hi Lion,
Please see line 7423 where it starts to process the last file.
Check Line
/ccs/home/lei/source_inversion_wf/src/preproc_asdf/test_wf

aHi Lion,
Could you clarify here a little bit? Line 1505 - 1507 in asdf_data_set.py.
            if (len(workers_requesting_write) >= 0.5 * self.mpi.comm.size) or \
                    (len(workers_requesting_write) and
                     jobs.all_poison_pills_received):

Could you use parenthese to make the logic more clear? I am little confused about the or and and

There are already parentheses that make it clear: if (more than half the MPI size are requesting write) or (any workers are requesting a write and all jobs have received a poison pill).

There are already parentheses that make it clear: if (more than half the MPI size are requesting write) or (any workers are requesting a write and all jobs have received a poison pill).

Exactly. The 0.5 is arbitrary but seems to work quite well. The problem is that before any processor can write to the file the metadata must be synchronized across all ranks. This would be terribly inefficient to do after every processed stream thus each rank buffers data up to a certain memory limit and then requests a write from the master rank. Once more then 50 % of the worker request a write the master will force all ranks to start the synchronization.
@wjlei1990 Can you upload the files + scripts you use so I can test them locally?
Are you on Python 2 or Python 3? I sometimes have issue to get the parallel I/O to work on Python 3 for some reason.

@wjlei1990 Does 484f41f211fa5d6c753e793c5d215d1cb30124c7 also fix this issue here? If not - can you upload files and scripts somewhere and I'll have a look.

Hi Lion,
The commit you mentioned above still doesn't fix the problem.
I will upload files somewhere so you can have a look.
You have access to clover, right?
My data file is about 100MB for each. And I will give you 10, so the total size would be 1GB.

Yes I have access to clover so putting the files there would be great. I'll have a look tomorrow.

Yes. I will try to upload the files tonight.
The disk on clover is full now and we are trying to delete some files to makeup some space :imp:

Did you upload the files somewhere? If its only a GB you can also just put it into dropbox or google or any other service. I would also need your scripts with the processing.

Hi Lion,
I have upload files on<EMAIL_ADDRESS>It is located at: /home/sdf/preproc_lion_test.
There are two example bash files you can run:

run_one_file_test.bash: process just one file. So this test should pass with no error.

run_multi_file_test.bash: process 10 files at one time. So this test should fail at a certain point. One thing I noticed that if you use more cores, you will fail more quickly. For example, if you use 4 cores, it can fail after 7 or 8 files. If you use 16 cores, it could fail after 2-3 files.

Let me you if you have problem with getting those files.

Thanks. I'll try this tomorrow!

This was nasty but I think I fixed it in 05f21b4d006feeced0904f8a3d87c951f981704a. Can you test on your machine?
The problem was that some MPI messages that were sent while processing one file did not get received so they got received while processing the next file which of course screws everything up. It only happened in specific situations. It now collects any left-over messages before shutting down the nodes so it is ready to process the next file without any previous state.
Man...our brains are really not made for dealing the parallel programs...

AHA! Great job, Lion.
I have tested it on my machine, using 8, 16 and 32 processors. It works like a charm :)
I initially thought it was some MPI library issue.

Man...our brains are really not made for dealing the parallel programs...

Yea, that's why there's so many RabbitMQ, ZeroMQ, etc....


Remove an allocation from addHandlers

Remove an allocation from addHandlers
Motivation:
Fewer allocations should improve performance.
Modifications:
Split out a sub function from addHandlers.
I originally thought I'd have to change the part of this
function which reads var handlers = handlers as there was
a surprising allocation at the beginning of this function.
It seems that breaking out some of the logic is sufficient
to remove an allocation.
Result:
1 fewer allocation.

Sorry @Davidde94 I had to add an extra commit.


Website Scraping Limit is 15 pages?

Hello,
Is there a way to increase the amount of scraped pages, or give it certain pages we want scraped?
Thank you!

You can update the value here:
backend-server/app/Http/Listeners/StartRecursiveCrawler.php:48

for something like:
        $this->crawl($url, $crawledUrls, 100, $rootUrl, $chatbotId, $dataSource->getId());


error in php artisan db:seed

Seeding: AdminGroup
In Carbon.php line 9:
Declaration of Illuminate\Support\Carbon::__set_state($array) should be com
patible with Carbon\Carbon::__set_state(array $state)

What your operating system

open this path
vendor/laravel/framework/composer.json
go to line 26
change this line
 "nesbot/carbon": "~1.20",
with
 "nesbot/carbon": "~1.24.1",
then composer update
and enjoy

mac

done


vendored the latest version of openshift-ansible for 3.4

Approved in PR #1458

Tests are failing due to harmless pylint errors from upstream


Burn .DS_Store with fire

Steps to reproduce:

Start uploading build
Open folder in Finder
Finder writes to .DS_Store
If you're really really unlucky, butler was reading from .DS_Store at this very moment & creates an inconsistent patch

The issue was actually that walking the fs respected ignore rules but walking zip did not, so they didn't agree on the layout.
Walking a zip now respects ignore patterns.


Find a better way to initialise component.

Currently, we use BuildConfig to inject SupervisorD to the image.
This is really slow, creates unnecessary BC that can confuse some users,  and it doesn't work with some images.
We should find something that is more universal and quicker than this.
One way could be to use initContianers to inject supervisor. Instead of using python SupervisorD we could use https://github.com/ochinchina/supervisord written in Go that doesn't require any dependencies.

closed in favor of #556


The error text created by a TestField adds an extra space character behind the expected value.

Describe the bug
I don't know if it is a (recent) bug, of if it is by design or has been in the system for a long time.
When creating a negative test I found the following.
To Reproduce
Steps and to reproduce the behavior:
Create a test function with the code below:
procedure TestFieldSpace()
var
    Vendor: Record Vendor;
    LibraryPurchase: Codeunit "Library - Purchase";
begin
    LibraryPurchase.CreateVendor(Vendor);
    asserterror Vendor.TestField("Prices Including VAT", true);
    LibraryAssertG.ExpectedError(StrSubstNo('Prices Including VAT must be equal to ''Yes''  in Vendor: No.=%1. Current value is ''No''.', Vendor."No."));
end;

Expected behavior
As seen above, the expected error text is:
StrSubstNo('Prices Including VAT must be equal to ''Yes''  in Vendor: No.=%1. Current value is ''No''.', Vendor."No.")
I'd expected:
StrSubstNo('Prices Including VAT must be equal to ''Yes'' in Vendor: No.=%1. Current value is ''No''.', Vendor."No.")
The difference is just on space less after ''Yes''.
Note, It seems that GitHub hides the second space. Look at the AL code and you will see 2 spaces.
Screenshots
If applicable, add screenshots to help explain your problem.
5. Versions:

AL Language: v7.0.341259

Business Central: Version: W1 17.0 (Platform 17.0.17020.18681 + Application 17.0.17126.18712)

Hi! Please take a minute to read the guidelines for this repository https://github.com/microsoft/AL and our collection of frequently asked questions https://github.com/Microsoft/AL/wiki/frequently-Asked-Questions .
As this repository is only for issues related to the AL compiler in latest developer preview environment for Dynamics 365 Business Central. We suggest that you open a support case or file a bug in Collaborate, to ensure that all Business Central users benefit from your catch as soon as possible.
To open a support case, you can:

Open a support request to CSS through the PartnerSource portal
Contact your Service Account Manager (SAM) in your local subsidiary to understand what is included in your contract as of support incident and PAH (Partner Advisory Hours). Your SAM might also direct you "step by step" how to open a support request or how to get credentials, if this is the first time for you or your company

If you file the bug in Collaborate, remember to include steps to reproduce the issue, and the Business Central build number and country version you're using.


feat: add support for redis 7.2 version

Description
Icd service, used inside tests, return 7.2 redis version. We added support for it.

Release required?

[ ] No release
[ ] Patch release (x.x.X)
[ ] Minor release (x.X.x)
[ ] Major release (X.x.x)

Release notes content

Run the pipeline
If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.
Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:
/run pipeline

Checklist for reviewers

[ ] If relevant, a test for the change is included or updated with this PR.
[ ] If relevant, documentation for the change is included or updated with this PR.

For mergers

Use a conventional commit message to set the release level. Follow the guidelines.
Include information that users need to know about the PR in the commit message. The commit message becomes part of the GitHub release notes.
Use the Squash and merge option.

/run pipeline

good catch, updated

/run pipeline

:tada: This PR is included in version 1.8.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Implemented essential parts of HttpListenerTimeoutManager

Adds support for a subset of HttpListenerTimeoutManager's timeouts, improving the network code's worst case failure modes.
This is the same as the previous pull request, but with the nonessential reusable-timer code removed.

Here you go! IMO this does the bare minimum to make this HttpListener code useful in production.

And now it has conflicts. Should I just close the PR and maintain a fork?

You're welcome to keep the pull request open because I think the feature is interesting and could always incorporate it later. But for immediate resolution you should put them on a separate fork. I have a requirement of being able to consume this from both .NET and .NET Core, so I've chosen to go the route of PCL since our application already has the necessary abstractions such as a socket factory.
I'm not putting those changes here in this repository because I do not want to disrupt people who are already using the nuget package. For now on a temporary basis I've embedded the PCL into our application, although I may extract it later:
https://github.com/MediaBrowser/Emby/tree/dev/SocketHttpListener.Portable
I did add a nice optimization that can be added to this repository as time allows. I was able to remove the need for the ResponseStream wrapper class except in the case of chunked responses. I also switched the whole stack be async Task-based since that's all that's allowed in PCL.

Thanks, that all makes perfect sense now. The PCL version is much cleaner. I hacked in some workarounds and am getting by with the Nuget version for the moment, but I'm pretty sure I'm still losing threads occasionally. Why does my code always seem to be deployed in the worst case scenario? :) I may temporarly clone the code, there's not a lot of code there really.
FYI, taking a quick look at the Portable branch/fork/version, with an infinite socket timeout you can still leak resources (thread pool threads) if the client fails to use timeouts. Async by itself does not protect you, but async with timeouts does, and the Listener code will close the socket from the parent thread when the handler returns. However it is important that 100% of remote I/O use a timeout around the async calls, both in the client code and internally. Or hack it and hard code a "large" timeout on the socket, chosen to be 20% longer than the largest application timeout. This is what I'd recommend, if it were my app.


Tone down emphasis on pure, functional programming

I find two issues with this lesson:

I think while this lesson is well-intended, it delivers the message that pure, functional programming is the only right way to do programming. There is no mention of other paradigms. For instance, object-oriented programming, when done right, makes code reuse easier through inheritance.

I agree that pure functions and functional programming paradigm is usually desirable for a lot of use cases which are fast to compute. However for computational sciences (for example, computational fluid dynamics) there are two recurring themes:

array operations: Array creation is expensive.
states: Arrays have states. These are mutable and evolve over time

Take an example of an imaginary function foo which updates an array every time step.
# pure function
def update(old_array):
    new_array = foo(old_array)
    return new_array

# impure function
def update(old_array):
    old_array[:] = foo(old_array)

# impure method, but states are tracked as attributes of a class
class State:
    def __init__(self, N):
        self.array = np.zeros(N)

    def update(self):
        self.array[:] = foo(self.array)

Which is better?
In other languages such as Fortran, modules are very often used as a container for arrays. This IMHO mimics classes in Python and structs or classes in C++. To quote the Zen of Python:

Special cases aren't special enough to break the rules.
Although practicality beats purity.

Thanks for the good suggestion! I agree that the presentation should be more balanced. There are indeed situations where mutable data structures are preferred. I will teach this lesson in a week and will try to adjust it until then.
My goal with this lesson was not to say that functional style is always good but rather to say that object oriented programming is not always the best solution. But sometimes it is. I agree that the text needs to be more balanced for this.

Okay, so we are on the same page :)
I agree it is good practice to resort to functional style when it leads to clean codes. It is hard to define a rule for when to use it or not.


Disable issue/PR comment button given empty input

Given an empty issue/PR comment, the comment history would not be updated if the user were to submit it. Therefore, it would make since to just disable the comment button when the text editor is empty.
This is inline with what GitHub does when given empty text editor input.

Before:

After:

Usually disabling buttons is not good UX practice as users cannot know why it's disabled. It's better to disallow submitting form with error notification about what is missing/invalid to submit form

Usually disabling buttons is not good UX practice as users cannot know why it's disabled. It's better to disallow submitting form with error notification about what is missing/invalid to submit form

Thanks for the review! As I explained in the desc I followed how GitHub handles this case (though I'm not sure if they inform the user about why it's disabled and I'm unable to check ATM).
With that said, I do agree with you and will look into your approach when I get the time.

Usually disabling buttons is not good UX practice as users cannot know why it's disabled. It's better to disallow submitting form with error notification about what is missing/invalid to submit form

I do not think it is a must for this case. Everyone should know that empty content are not allowed to be submitted if the button is disable.
And even more, it is quite hacky to add tooltip for a disabled element. So I prefer to go with this one, no more tooltip.

Usually disabling buttons is not good UX practice as users cannot know why it's disabled. It's better to disallow submitting form with error notification about what is missing/invalid to submit form

I do not think it is a must for this case. Everyone should know that empty content is not allowed to be submitted if the button is disabled.
And even more, it is quite hacky to add tooltip for a disabled element. So I prefer to go with this one, no more tooltip.

I'm not suggesting to add tooltip to disabled button but to not disable button and when clicked inform user that comment content is required

I'm not suggesting to add tooltip to disabled button but to not disable button and when clicked inform user that comment content is required

I still prefer the current "disabled button" solution. The reason is:

It has a intuitive UI feedback. But by using "informing after clicking", it actually misleads the users that "oh the form can be submitted" and they could only get the feedback at the second step (after the click).
Injecting translation strings for a prompt message is still hacky in this case.
At least, the "disabled button" is consistent with GitHub's form.


Routine dependency updates: Netty 4.1.91.Final, ASM 9.5, ByteBuddy 1.14.3

Description
Routine dependency updates: Netty 4.1.91.Final, ASM 9.5, ByteBuddy 1.14.3
Issues Resolved
N/A
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[X] Commits are signed per the DCO using --signoff
[X] Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #6981 (52acc77) into main (95c6ed9) will decrease coverage by 0.03%.
The diff coverage is n/a.

:exclamation: Current head 52acc77 differs from pull request most recent head 4af2609. Consider uploading reports for the commit 4af2609 to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@             Coverage Diff              @@
##               main    #6981      +/-   ##
============================================
- Coverage     70.78%   70.75%   -0.03%
- Complexity    59305    59339      +34
============================================
  Files          4813     4813
  Lines        283781   283781
  Branches      40924    40924
============================================
- Hits         200864   200795      -69
- Misses        66420    66564     +144
+ Partials      16497    16422      -75

see 490 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Gradle Check (Jenkins) Run Completed with:
* **RESULT:** FAILURE x

* **URL:** https://build.ci.opensearch.org/job/gradle-check/13406/

* **CommitID:** [4af2609](https://github.com/opensearch-project/OpenSearch/commit/4af2609326918d05f0bd6dc11a3f2a7cce4a7b5b)
  Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
  Is the failure [a flaky test](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#flaky-tests) unrelated to your change?

-----------
* What went wrong:
Execution failed for task ':distribution:docker:buildDockerImage'.
> A failure occurred while executing org.opensearch.gradle.docker.DockerBuildTask$DockerBuildAction
   > Failed to pull Docker **** image [centos:8], all attempts failed


Package maintained

So I am going to ask again, is this package maintained? I've bene told yes before but with PR's not merged for months and no activities on the repo, I am wondering if it is.

I haven't been able to work on it for a while due to work commitments. I just released a new version.


Removed useless alpha channels, optimized PNG file sizes

Fixes #330.
Several PNG files were containing a useless alpha channel.

Note that this PNG file needs to have a alpha channel given that it is a completely transparent texture:

tests/api/worlds/textures/noise_mask_transparent.png

Note that this PNG file needs to have a alpha channel given that it is a completely transparent texture:

tests/api/worlds/textures/noise_mask_transparent.png

It has, but the new version is smaller because I removed useless data from the PNG file.

There are many other PNG images that seem wrong to me, for example: asphalt_roughness.png weights 3,479,027 bytes whereas it's JPEG equivalent (with 95% quality) is only 1,790,422 bytes. I believe we should be very careful when adding PNG textures and always try to add the JPEG equivalent if smaller and if no transparency is needed.

Yes, when switching to PBR a lot of the new textures generated were in PNG :-/

We should take care of them in a separate PR, as it not related to the alpha channel.

test me

test me

Build Passed :ok_hand:
You can safely merge 'fix-unused-png-alpha-channel' into 'master'.

👍

Build Passed :ok_hand:
You can safely merge 'fix-unused-png-alpha-channel' into 'master'.

test me

Build Passed :ok_hand:
You can safely merge 'fix-unused-png-alpha-channel' into 'master'.

👍


The code implementation for more actions

Hi! In your paper, you showed some examples of generalizing to 'jump up', 'turn to', 'open' and 'place' motions. I wonder if you can provide the code of generating these types of motions? Because in data/align_motion.py (line 1798-1808), I only find support of the 4 default actions:
## sample valid position and rotation according to sit action
        if action == 'sit':
            action_align = SitAlign(self.annotations, self.instance_to_semantic, self.label_mapping, scene_path, static_scene, static_scene_label, translate_mat, body_vertices, joints_traj)
        elif action == 'stand up':
            action_align = StandUpAlign(self.annotations, self.instance_to_semantic, self.label_mapping, scene_path, static_scene, static_scene_label, translate_mat, body_vertices, joints_traj)
        elif action == 'walk':
            action_align = WalkAlign(self.annotations, self.instance_to_semantic, self.label_mapping, scene_path, static_scene, static_scene_label, translate_mat, body_vertices, joints_traj)
        elif action =='lie':
            action_align = LieAlign(self.annotations, self.instance_to_semantic, self.label_mapping, scene_path, static_scene, static_scene_label, translate_mat, body_vertices, joints_traj)
        else:
            raise Exception('Unsupport action: {}'.format(action))

Thanks!

Hi! In your paper, you showed some examples of generalizing to 'jump up', 'turn to', 'open' and 'place' motions. I wonder if you can provide the code of generating these types of motions? Because in data/align_motion.py (line 1798-1808), I only find support of the 4 default actions:
## sample valid position and rotation according to sit action
        if action == 'sit':
            action_align = SitAlign(self.annotations, self.instance_to_semantic, self.label_mapping, scene_path, static_scene, static_scene_label, translate_mat, body_vertices, joints_traj)
        elif action == 'stand up':
            action_align = StandUpAlign(self.annotations, self.instance_to_semantic, self.label_mapping, scene_path, static_scene, static_scene_label, translate_mat, body_vertices, joints_traj)
        elif action == 'walk':
            action_align = WalkAlign(self.annotations, self.instance_to_semantic, self.label_mapping, scene_path, static_scene, static_scene_label, translate_mat, body_vertices, joints_traj)
        elif action =='lie':
            action_align = LieAlign(self.annotations, self.instance_to_semantic, self.label_mapping, scene_path, static_scene, static_scene_label, translate_mat, body_vertices, joints_traj)
        else:
            raise Exception('Unsupport action: {}'.format(action))

Thanks!

Sorry, these codes are not merged into my repository. I can't find them now.


test PR with Github API

Please pull this in!

Docs Build status updates of commit c7b01c1:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details
Dependent Files

:x:Error

Details

[Error] Cannot sync git repo to specified state. Please make sure the pull request is mergeable. Details: Git merge results in conflict, repository remote url: https://github.com/Peachying/testinlineedit, working directory: W:\r4rn-s, target branch: HEAD, target branch commit: 96f8c02284f7ffefe8fb929195b36f182b032c9f, source commit: c7b01c1b4dedcec570d6755c32854e611746d9e2.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Remove zlib binaries, add build logs to ignore file.

Everything is already set up to build zlib from source, no need to include the binary artifacts.

Cool.  thank you, nice clean up :+1:


Adjust README dev instructions

This PR adjust the repository's documentation to include development NFD installation instructions.
Signed-off-by: Michail Resvanis<EMAIL_ADDRESS>

Codecov Report
Base: 81.43% // Head: 81.43% // No change to project coverage :thumbsup:

Coverage data is based on head (446c073) compared to base (2138684).
Patch has no changes to coverable lines.

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #25   +/-   ##
=======================================
  Coverage   81.43%   81.43%
=======================================
  Files           9        9
  Lines         797      797
=======================================
  Hits          649      649
  Misses        137      137
  Partials       11       11

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Fix/Semantic-release

Codecov Report

Merging #76 (639a186) into main (4f46d36) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 639a186 differs from pull request most recent head 44fb0e3. Consider uploading reports for the commit 44fb0e3 to get more accurate results

@@           Coverage Diff           @@
##             main      #76   +/-   ##
=======================================
  Coverage   87.62%   87.62%
=======================================
  Files          13       13
  Lines         493      493
=======================================
  Hits          432      432
  Misses         61       61

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4f46d36...44fb0e3. Read the comment docs.


Browserless Auth

If we can't open a browser we can provide a URL for students to go to. Also an issue here: https://github.com/Cal-CS-61A-Staff/ok/issues/878
Useful for:

iPython Notebooks
Submitting while on SSH

Duplicate of #212


bug minning the blocks

the blocks need silk touch to be picked up (the only block affected is the chiseled bookshelf) i'm playing on 1.20.1 with the mod version being 1.3.6

Fabric or Forge?

nm looks like its both versions, i'll sort it today, apologies

This is vanilla behaviour for the Chiseled Bookshelf.
https://minecraft.wiki/w/Chiseled_Bookshelf#Breaking


Support custom columns and grouping

Closes #80.
This PR allows users to customize the columns in the generated BOM and group them by the columns of their choice. This adds two arguments to the BOM generation function: a list of columns to group by, and which column corresponds to the designator. The list of columns is optional, and if not present, the BOM will be "flat", i.e. will have one row for each component instance.

Outdated
The reason we need a designator column has to do with how we generate the designator for components that are repeated due to repeated sheets. For example, imagine `Thermocouples.SchDoc` is repeated five times. A component with designator `C1` will have to become `C1A`, `C1B`, `C1C`, `C1D` and `C1E` in the BOM. To do this, we need to know which column corresponds to the designator, as otherwise we cannot add the letter at the end of the designator when collecting components.
One alternative that could remove this need is to assume that the designator will always be the Designator attribute in the component, but that could fail if the name of the attribute is changed.

Stacked on #83.
What's left:

[x] Assume 'Designator' is the designator attribute and remove the designator configuration.

Marking ready for review as we don't need the designator_column configuration any more. Another thing I added is the extraction of the part_id and description properties from the component top level, as they are useful fields for the BOM too. We'll have to educate users about this.

I have been looking at the code for a while now but think I am missing the context around the change.
What is the use case of grouping by a different column for BOM generation? Do we have a customer example?

What is the use case of grouping by a different column for BOM generation?

Fundamentally, we have to support custom grouping because we support column selection. Since you can pick your own columns and rename them to whatever you want, we need some way for you to pick which columns you group by. In the previous version, since we had hard-coded the columns and their names, grouping by part number was simple.
Secondly, Altium allows you to do this, and it makes sense. It's entirely possible you don't have a part number in your BOM and choose to group by something else (like a description). I've seen projects that didn't have a part number in the attributes at all, and the BOM can be grouped by other fields.

@shrik450 can you rebase this on main when you have a moment?

We need to update generate_bom.py so that it's consistent here for several reasons:

I've updated the example script (and I think the update will help clarify some of your issues too.)

Primarily, the return type for generate_bom_for_altium is a list of dictionaries with ALL component attributes [...]

I'm not sure why you're getting that. The returned dictionaries should only have the columns you've selected in the attributes mapping. You can see this in the test files of this PR, which only have the columns defined in the mapping. I double checked with the updated example script and only got the columns I selected.
I guess some of this confusion might be due to the old --attributes_to_extract option in the script. However, that argument was never used, so I'm not sure. Another possibility is that your attributes_mapping.json has all the columns. I'd be happy to help figure out why this is happening, but as far as I can tell this issue isn't in the generate_bom_for_altium code.
Given that,

get_filtered_bom(attributes: list[str], group_by, repo_data...) -> returns the list of BOM lines in the form list[list[str]] only with the attributes I specified, basically ready to be dumped to a csv (or whatever other tabular file I need).

Shouldn't be necessary - this is the expected output of generate_bom_for_altium. I'd also add that the dict response is strictly better than a list of strings for a csv. This is because Python ships with csv.DictWriter and csv.DictReader, so it is just as simple to write a list of dicts to a csv as a list of strings, and a list of dicts allows the user to do more things (as you point out.) This is reflected in the updated example script as well, and is used in the tests and the packaged action script.

get_components(repo_data...) -> returns flat list of components (non--grouped) in the form list[dict] (or dict[dict] might be better if we want to look up components by part id).

This is already the default, as not passing a group_by will give a flat BOM, so I'm not sure it's worth adding another alias to it.

We should have a test with and without group_by variable set.

We have these in https://github.com/AllSpiceIO/py-allspice/pull/84/files#diff-33c13e0b177bacd2f02e29bcb8aea5b49e7ce34901fd8f41fefb65defba1bd33R53 and https://github.com/AllSpiceIO/py-allspice/pull/84/files#diff-33c13e0b177bacd2f02e29bcb8aea5b49e7ce34901fd8f41fefb65defba1bd33R143

Hmm, when I run a the following I get:
› ALLSPICE_AUTH_TOKEN="db72c0eeaaabde1e128cb498e202263bcf42fc82" ./examples/generate_bom/generate_bom.py "kyle/Dual-Lane-Feeder-Controller" "Dual Feeder Controller.PrjPCB" --output_file "out.csv" --allspice_hub_url="https://staging.allspice.dev" --source_ref="master"
Generating BOM...
ATTRIBUTES MAP:
{'description': 'Comment', 'designator': 'Designator', 'manufacturer': ['Manufacturer', 'MANUFACTURER'], 'part_number': ['Part Number', 'MANUFACTURER #']}
bom_rows[0]:
{'Part Number': 'BKP1608HS101-T', 'PartId': '6594', 'Value': '100 Ohm @ 100MHz', 'Manufacturer': 'Taiyo Yuden', 'Supplier 1': 'Digi-Key', 'Supplier Part Number 1': '587-1922-2-ND', 'ComponentLink1Description': 'Datasheet', 'ComponentLink1URL': 'http://ds.yuden.co.jp/TYCOMPAS/ut/detail.do?productNo=BKP1608HS101-T&fileName=BKP1608HS101-T_SS&mode=specSheetDownload', 'ComponentLink2Description': 'Digi-Key listing', 'ComponentLink2URL': 'http://digikey.com/product-detail/en/BKP1608HS101-T/587-1922-2-ND/957775', 'Price': '0.0247', 'Type': 'Differential Mode - Single', 'Device Package': '0603 (1608 Metric)', 'DC Resistance': '50 mOhm Max', 'Current Rating': '1.7A', 'Designator': 'FB2', 'Comment': '=Value', '__part_id': 'BKP1608HS101-T', '__description': 'FERRITE CHIP 100 OHM 0603', 'Quantity': '1'}
Traceback (most recent call last):
  File "/home/kyle/1-develop/py-allspice/./examples/generate_bom/generate_bom.py", line 97, in <module>
    writer.writerows(bom_rows)
  File "/home/kyle/.pyenv/versions/3.11.8/lib/python3.11/csv.py", line 157, in writerows
    return self.writer.writerows(map(self._dict_to_list, rowdicts))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kyle/.pyenv/versions/3.11.8/lib/python3.11/csv.py", line 149, in _dict_to_list
    raise ValueError("dict contains fields not in fieldnames: "
ValueError: dict contains fields not in fieldnames: 'Power - Peak Pulse', 'Voltage (Clamping)', 'Current - Peak Pulse', 'Operating Temperature', 'Voltage (Reverse Standoff)', 'Voltage (Breakdown)'

(I added a couple debug lines)

Hmm, when I run a the following I get:
› ALLSPICE_AUTH_TOKEN="<TOKEN>" ./examples/generate_bom/generate_bom.py "kyle/Dual-Lane-Feeder-Controller" "Dual Feeder Controller.PrjPCB" --output_file "out.csv" --allspice_hub_url="https://staging.allspice.dev" --source_ref="master"
Generating BOM...
ATTRIBUTES MAP:
{'description': 'Comment', 'designator': 'Designator', 'manufacturer': ['Manufacturer', 'MANUFACTURER'], 'part_number': ['Part Number', 'MANUFACTURER #']}
bom_rows[0]:
{'Part Number': 'BKP1608HS101-T', 'PartId': '6594', 'Value': '100 Ohm @ 100MHz', 'Manufacturer': 'Taiyo Yuden', 'Supplier 1': 'Digi-Key', 'Supplier Part Number 1': '587-1922-2-ND', 'ComponentLink1Description': 'Datasheet', 'ComponentLink1URL': 'http://ds.yuden.co.jp/TYCOMPAS/ut/detail.do?productNo=BKP1608HS101-T&fileName=BKP1608HS101-T_SS&mode=specSheetDownload', 'ComponentLink2Description': 'Digi-Key listing', 'ComponentLink2URL': 'http://digikey.com/product-detail/en/BKP1608HS101-T/587-1922-2-ND/957775', 'Price': '0.0247', 'Type': 'Differential Mode - Single', 'Device Package': '0603 (1608 Metric)', 'DC Resistance': '50 mOhm Max', 'Current Rating': '1.7A', 'Designator': 'FB2', 'Comment': '=Value', '__part_id': 'BKP1608HS101-T', '__description': 'FERRITE CHIP 100 OHM 0603', 'Quantity': '1'}
Traceback (most recent call last):
  File "/home/kyle/1-develop/py-allspice/./examples/generate_bom/generate_bom.py", line 97, in <module>
    writer.writerows(bom_rows)
  File "/home/kyle/.pyenv/versions/3.11.8/lib/python3.11/csv.py", line 157, in writerows
    return self.writer.writerows(map(self._dict_to_list, rowdicts))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kyle/.pyenv/versions/3.11.8/lib/python3.11/csv.py", line 149, in _dict_to_list
    raise ValueError("dict contains fields not in fieldnames: "
ValueError: dict contains fields not in fieldnames: 'Power - Peak Pulse', 'Voltage (Clamping)', 'Current - Peak Pulse', 'Operating Temperature', 'Voltage (Reverse Standoff)', 'Voltage (Breakdown)'

(I added a couple debug lines)

We need to update generate_bom.py so that it's consistent here for several reasons:

I've updated the example script (and I think the update will help clarify some of your issues too.)

Primarily, the return type for generate_bom_for_altium is a list of dictionaries with ALL component attributes [...]

I'm not sure why you're getting that. The returned dictionaries should only have the columns you've selected in the attributes mapping. You can see this in the test files of this PR, which only have the columns defined in the mapping. I double checked with the updated example script and only got the columns I selected.
I guess some of this confusion might be due to the old --attributes_to_extract option in the script. However, that argument was never used, so I'm not sure. Another possibility is that your attributes_mapping.json has all the columns. I'd be happy to help figure out why this is happening, but as far as I can tell this issue isn't in the generate_bom_for_altium code.

Nice, glad you figured this out!

get_filtered_bom(attributes: list[str], group_by, repo_data...) -> returns the list of BOM lines in the form list[list[str]] only with the attributes I specified, basically ready to be dumped to a csv (or whatever other tabular file I need).

Shouldn't be necessary - this is the expected output of generate_bom_for_altium. I'd also add that the dict response is strictly better than a list of strings for a csv. This is because Python ships with csv.DictWriter and csv.DictReader, so it is just as simple to write a list of dicts to a csv as a list of strings, and a list of dicts allows the user to do more things (as you point out.) This is reflected in the updated example script as well, and is used in the tests.

get_components(repo_data...) -> returns flat list of components (non--grouped) in the form list[dict] (or dict[dict] might be better if we want to look up components by part id).

This is already the default, as not passing a group_by will give a flat BOM, so I'm not sure it's worth adding another alias to it.

The reason I'm harping on the architecture here is because we're also going to want to support other cases where we iterate components to generate data (eg. to check RHOS data). We're going to want to make it ECAD-platform agnostic, so that Altium/Orcad behave the same, so having an intermediate function with all component data in a standard format (then filtering/squashing that) is going to be useful. However, I'll certainly accept that this is premature optimization, and we can wait until we implement those functions to re-address this.

We should have a test with and without group_by variable set.

We have these in https://github.com/AllSpiceIO/py-allspice/pull/84/files#diff-33c13e0b177bacd2f02e29bcb8aea5b49e7ce34901fd8f41fefb65defba1bd33R53 and https://github.com/AllSpiceIO/py-allspice/pull/84/files#diff-33c13e0b177bacd2f02e29bcb8aea5b49e7ce34901fd8f41fefb65defba1bd33R143

:100:

One thing I noticed is that by default only strings with , are quoted in the csv output. I could see this causing unexpected issues - is there an option to quote all of the output strings?

Yes, if we set quoting=csv.QUOTE_ALL python will quote every field. Although, I think any csv-compliant parser should be able to python's defaults here pretty well. We can make this an argument for the packaged action.

@McRaeAlex If you are happy with the comments/flow here lmk and I'll go ahead and merge 👍

@McRaeAlex If you are happy with the comments/flow here lmk and I'll go ahead and merge 👍

Yup


Http Status 503 when deploying function app

When I try to deploy a function app I does not work properly. I encounter a 503 Status which will be retried for a very log time:
2019-12-11T08:23:38.1476676Z /d/a/jobreporter/jobreporter
2019-12-11T08:23:38.1736984Z ##[group]Run Azure/functions-action@v1
2019-12-11T08:23:38.1737167Z with:
2019-12-11T08:23:38.1737283Z   app-name: jobreporter-dev
2019-12-11T08:23:38.1737547Z   slot-name: staging
2019-12-11T08:23:38.1737652Z   package: ./dist
2019-12-11T08:23:38.1739072Z   publish-profile: ***
2019-12-11T08:23:38.1739194Z ##[endgroup]
2019-12-11T08:23:38.6519038Z ##[Initialize]
2019-12-11T08:23:38.7084531Z ##[ValidateParameter]
2019-12-11T08:23:38.7090779Z Finding files matching input: ./dist
2019-12-11T08:23:38.7091111Z ##[ValidateAzureResource]
2019-12-11T08:23:38.7095291Z Using SCM credential for authentication, GitHub Action will not perform resource validation.
2019-12-11T08:23:38.7096935Z [GET]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-11T08:23:39.8100532Z Sucessfully acquired app settings from function app (SCM)!
2019-12-11T08:23:39.8138061Z ##[PreparePublishContent]
2019-12-11T08:23:39.8248726Z This is folder package
2019-12-11T08:23:39.8249813Z Will archive ./dist into d:\a\_temp\temp_web_package_7695150422348975.zip as function app content
2019-12-11T08:23:39.8250727Z Archiving ./dist to d:\a\_temp\temp_web_package_7695150422348975.zip
2019-12-11T08:23:47.4235862Z Successfully created archive d:\a\_temp\temp_web_package_7695150422348975.zip
2019-12-11T08:23:47.4251341Z Will use api/zipdeploy to deploy (scm credential)
2019-12-11T08:23:47.4268170Z [GET]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-11T08:23:48.0940558Z ##[PublishContent]
2019-12-11T08:23:48.0951576Z Setting SCM_DO_BUILD_DURING_DEPLOYMENT in Kudu container to false
2019-12-11T08:23:48.0952566Z Update using Client.updateAppSettingViaKudu
2019-12-11T08:23:48.0959278Z [POST]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-11T08:23:49.0023080Z Response with status code 204
2019-12-11T08:23:54.0056713Z [GET]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-11T08:23:55.2901895Z App setting SCM_DO_BUILD_DURING_DEPLOYMENT propagated to Kudu container
2019-12-11T08:23:55.2902316Z Setting ENABLE_ORYX_BUILD in Kudu container to false
2019-12-11T08:23:55.2902721Z Update using Client.updateAppSettingViaKudu
2019-12-11T08:23:55.2904318Z [POST]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-11T08:23:56.0990328Z Response with status code 204
2019-12-11T08:24:01.1078446Z [GET]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-11T08:24:02.7097354Z App setting ENABLE_ORYX_BUILD propagated to Kudu container
2019-12-11T08:24:02.7098709Z Waiting for function app to spin up after app settings change.
2019-12-11T08:24:07.7115489Z [GET]http://jobreporter-dev.azurewebsites.net
2019-12-11T08:24:50.2085079Z Could not parse response: {}
2019-12-11T08:24:50.2087040Z Response: undefined
2019-12-11T08:24:50.2093970Z Encountered a retriable status code: 503. Message: 'Service Unavailable'.
2019-12-11T08:24:55.2201075Z [GET]http://jobreporter-dev.azurewebsites.net
2019-12-11T08:24:55.7204766Z Could not parse response: {}
2019-12-11T08:24:55.7207056Z Response: undefined
2019-12-11T08:24:55.7207503Z Encountered a retriable status code: 503. Message: 'Service Unavailable'.
2019-12-11T08:25:25.7264769Z [GET]http://jobreporter-dev.azurewebsites.net
2019-12-11T08:25:25.9490397Z Could not parse response: {}
2019-12-11T08:25:25.9492096Z Response: undefined
2019-12-11T08:25:25.9492738Z Encountered a retriable status code: 503. Message: 'Service Unavailable'.
2019-12-11T08:28:00.9522596Z [GET]http://jobreporter-dev.azurewebsites.net
2019-12-11T08:28:01.2803169Z Could not parse response: {}
2019-12-11T08:28:01.2803813Z Response: undefined
2019-12-11T08:28:01.2805049Z Encountered a retriable status code: 503. Message: 'Service Unavailable'.
2019-12-11T08:33:11.8229678Z ##[error]The operation was canceled.
2019-12-11T08:33:11.8272504Z Cleaning up orphan processes

when I stop the function app and the retrigger the github workflow the app gets deployed perfectly:

2019-12-12T07:21:19.9536094Z ##[group]Run<EMAIL_ADDRESS>2019-12-12T07:21:19.9536318Z with:
2019-12-12T07:21:19.9536446Z   app-name: jobreporter-dev
2019-12-12T07:21:19.9536569Z   slot-name: staging
2019-12-12T07:21:19.9536689Z   package: ./dist
2019-12-12T07:21:19.9537805Z   publish-profile: ***
2019-12-12T07:21:19.9537936Z ##[endgroup]
2019-12-12T07:21:20.5021745Z ##[Initialize]
2019-12-12T07:21:20.5119116Z ##[ValidateParameter]
2019-12-12T07:21:20.5131941Z Finding files matching input: ./dist
2019-12-12T07:21:20.5134233Z ##[ValidateAzureResource]
2019-12-12T07:21:20.5141610Z Using SCM credential for authentication, GitHub Action will not perform resource validation.
2019-12-12T07:21:20.5158336Z [GET]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-12T07:21:21.4781663Z Sucessfully acquired app settings from function app (SCM)!
2019-12-12T07:21:21.4782596Z ##[PreparePublishContent]
2019-12-12T07:21:21.4944138Z This is folder package
2019-12-12T07:21:21.4945458Z Will archive ./dist into d:\a\_temp\temp_web_package_3297521192034041.zip as function app content
2019-12-12T07:21:21.4947176Z Archiving ./dist to d:\a\_temp\temp_web_package_3297521192034041.zip
2019-12-12T07:21:29.7378151Z Successfully created archive d:\a\_temp\temp_web_package_3297521192034041.zip
2019-12-12T07:21:29.7388353Z Will use api/zipdeploy to deploy (scm credential)
2019-12-12T07:21:29.7401129Z [GET]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-12T07:21:30.3968706Z ##[PublishContent]
2019-12-12T07:21:30.3980246Z Setting SCM_DO_BUILD_DURING_DEPLOYMENT in Kudu container to false
2019-12-12T07:21:30.3981977Z Update using Client.updateAppSettingViaKudu
2019-12-12T07:21:30.3988574Z [POST]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-12T07:21:31.2416493Z Response with status code 204
2019-12-12T07:21:36.2433449Z [GET]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-12T07:21:37.0166920Z App setting SCM_DO_BUILD_DURING_DEPLOYMENT propagated to Kudu container
2019-12-12T07:21:37.0167454Z Setting ENABLE_ORYX_BUILD in Kudu container to false
2019-12-12T07:21:37.0167886Z Update using Client.updateAppSettingViaKudu
2019-12-12T07:21:37.0169562Z [POST]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-12T07:21:37.6964260Z Response with status code 204
2019-12-12T07:21:42.9642073Z [GET]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-12T07:21:43.2069025Z App setting ENABLE_ORYX_BUILD propagated to Kudu container
2019-12-12T07:21:43.2069702Z Waiting for function app to spin up after app settings change.
2019-12-12T07:21:48.2209700Z [GET]http://jobreporter-dev.azurewebsites.net
2019-12-12T07:21:48.4332097Z Could not parse response: {}
2019-12-12T07:21:48.4332992Z Response: undefined
2019-12-12T07:21:48.4334183Z Package deployment using ZIP Deploy initiated.
2019-12-12T07:21:48.4344847Z [POST]***jobreporter-dev.scm.azurewebsites.net/api/zipdeploy?isAsync=true&deployer=GITHUB_ZIP_DEPLOY
2019-12-12T07:21:52.7600389Z [GET]https://jobreporter-dev.scm.azurewebsites.net/api/deployments/latest?deployer=GITHUB_ZIP_DEPLOY&time=2019-12-12_07-21-52Z
2019-12-12T07:22:13.3857118Z [GET]https://jobreporter-dev.scm.azurewebsites.net/api/deployments/latest?deployer=GITHUB_ZIP_DEPLOY&time=2019-12-12_07-21-52Z
2019-12-12T07:22:26.4042185Z [GET]https://jobreporter-dev.scm.azurewebsites.net/api/deployments/latest?deployer=GITHUB_ZIP_DEPLOY&time=2019-12-12_07-21-52Z
2019-12-12T07:22:33.9184615Z [GET]https://jobreporter-dev.scm.azurewebsites.net/api/deployments/latest?deployer=GITHUB_ZIP_DEPLOY&time=2019-12-12_07-21-52Z
2019-12-12T07:22:40.5830000Z [GET]https://jobreporter-dev.scm.azurewebsites.net/api/deployments/latest?deployer=GITHUB_ZIP_DEPLOY&time=2019-12-12_07-21-52Z
2019-12-12T07:22:47.1492445Z [GET]https://jobreporter-dev.scm.azurewebsites.net/api/deployments/latest?deployer=GITHUB_ZIP_DEPLOY&time=2019-12-12_07-21-52Z
2019-12-12T07:22:53.6213123Z [GET]https://jobreporter-dev.scm.azurewebsites.net/api/deployments/latest?deployer=GITHUB_ZIP_DEPLOY&time=2019-12-12_07-21-52Z
2019-12-12T07:22:55.1628692Z [GET]***jobreporter-dev.scm.azurewebsites.net/api/deployments/87d2c4ce62994c0f8c2de7d16b7fa17f
2019-12-12T07:22:55.9502758Z Deploy logs can be viewed at https://jobreporter-dev.scm.azurewebsites.net/api/deployments/87d2c4ce62994c0f8c2de7d16b7fa17f/log
2019-12-12T07:22:55.9503420Z Successfully deployed web package to App Service.
2019-12-12T07:22:55.9512757Z [GET]***jobreporter-dev.scm.azurewebsites.net/api/vfs/site/deployments/87d2c4ce62994c0f8c2de7d16b7fa17f/manifest
2019-12-12T07:22:56.8565047Z Could not parse response: {}
2019-12-12T07:22:56.8566163Z Response: undefined
2019-12-12T07:22:56.8602105Z [PUT]***jobreporter-dev.scm.azurewebsites.net/api/vfs/site/deployments/87d2c4ce62994c0f8c2de7d16b7fa17f/manifest
2019-12-12T07:22:57.8399271Z [PUT]***jobreporter-dev.scm.azurewebsites.net/api/deployments/87266d56249c9ef341009c0e3f622457ae9af7db1576135377830
2019-12-12T07:22:58.9179120Z Successfully updated deployment History at https://jobreporter-dev.scm.azurewebsites.net/api/deployments/87266d56249c9ef341009c0e3f622457ae9af7db1576135377830
2019-12-12T07:22:58.9182502Z [POST]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-12T07:22:59.4786322Z Response with status code 204
2019-12-12T07:22:59.4788438Z [POST]***jobreporter-dev.scm.azurewebsites.net/api/settings
2019-12-12T07:23:00.0953664Z Response with status code 204
2019-12-12T07:23:00.0960620Z [DELETE]***jobreporter-dev.scm.azurewebsites.net/api/settings/WEBSITE_RUN_FROM_PACKAGE
2019-12-12T07:23:00.6883853Z Response with status code 204
2019-12-12T07:23:00.6885279Z ##[ValidatePublishedContent]

Therefore it is not really clear to me what is happening and why I only can deploy when the function app is stopped.
Github Workflow is:
name: Azure_Deployment_Development

on:
  push:
    branches:
    - develop

jobs:
  build-and-deploy:
    runs-on: windows-latest
    steps:
    - name: 'Checkout GitHub Action'
      uses: actions/checkout@master

    - name: Setup Node 10.x
      uses: actions/setup-node@v1
      with:
        node-version: '10.x'

    - name: 'Run npm build and prepare deploy'
      shell: bash
      run: |
        pushd .
        npm ci
        npm run build --if-present
        npm run deploy:prepare
        popd

    - name: 'Deploy to Azure Function App'
      uses<EMAIL_ADDRESS>      id: fa
      with:
        app-name: jobreporter-dev
        slot-name: staging
        # If your function app project is not located in your repository's root
        # Please consider prefixing the project path in this package parameter
        package: './dist'
        # If you want to use publish profile credentials instead of Azure Service Principal
        # Please uncomment the following line
        publish-profile: ${{ secrets.SCM_CREDENTIALS_DEV }}

Thanks for reporting this and sorry for the late reply.
There was an issue on the Linux Python function app service during the time of deployment.
If you're still seeing this issue, please let me know.

i still see this behavior with Node.js functions as well.
I need to stop the slot before deploying to it, otherwise the function action hangs forever
here's an example run https://github.com/andycmaj/typescript-azure-functions/runs/575127010?check_suite_focus=true

I also faced the same issue on .net functions. I stopped the slot and deployed it and it got deployed again. Any Idea how to make it work without stopping the azure function?

I'm experiencing this issue right now with C# functions, intermittent failures.
I have a mono-repo with 8 function app projects, each project is deployed to it's own function app in azure using a matrix and reusable workflow, no single project is failing, it's random, sometimes all projects deploy successfully.
I'm also considering having a separate workflow per project, this way I can use the path to detect changes to a single project and decide whether or not to the deploy that project, I hope to reduce the billable time and hopefully the time to deploy, right now it takes over 90 minutes of billable time, about 11 minutes to deploy all functions.

I have the same issue right now, deploying dotnet net6.0 to linux functionapp.

Also having this issue deploying Python Linux function app.

I have the same issue with Flex Consumption plan. Other than me trying to provide custom retry logic, I couldnt find any other way. Please reopen the issue.


Feat(web, web-react): FileUploader: zobrazení změněné fotky #DS-954

Description

Additional context

Issue reference

Before submitting the PR, please make sure you do the following

[ ] Read the Contributing Guidelines.
[ ] Follow the PR Title/Commit Message Convention.
[ ] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[ ] Ideally, include relevant tests that fail without this PR but pass with it.

It is looking good. I think you are on the right track. 👍

coverage: 96.46% (+25.5%) from 70.949% when pulling 24847205dca10180eae682cb8e54470c682e3a1d on feat/DS-954-FU-support-crop-image into 2094a04e0a1d7d254f2df3260da7f0667f0e2a87 on main.

Great 👏 Thanks for this work. From my point of view, all demos are working. We should maybe add some text to the demo, that the edit button changes the crop. Also write a better copy of the documentation so that is even possible to change the image preview. I can help you with both if you want.
I have also added some suggestions to the logic. But they do not change the behavior so they are non-blocking.
Once again, thanks.


onElementsRemove: When composing multiple flows, removing element(s) from one flow will deselect element(s) from other flows

Test case: https://github.com/sarasfive/react-flow-remove-elements
STR

Select node 1 (first flow)
Select node 2 (second flow)
Press backspace

Expected
Selected element(s) will be passed to onElementsRemove for the specific flow where the element(s) are to be removed. Only those elements will be deselected.
Actual
Selected element(s) in each flow are deselected.
For this test case, selecting node-1 (first flow) and node-2 (second flow) and pressing backspace deselects both node-1 and node-2. Console prints Removing elements from 1st flow and Removing elements from 2nd flow.
My initial solution was to track the 'active' flow and somehow block the deselection in the non-'active' flow. The only way I could see to do this is to useStoreActions hook and select the elements again, but this doesn't seem to be the right approach. I couldn't get the call stack (can't block deselection) to work properly without introducing a setTimeout. It works, but this solution causes a deselect->select flicker (not shown in test case).
Is this a bug or would another solution be recommended?
Cheers!
Code
import React from 'react'
import ReactFlow, { ReactFlowProvider } from 'react-flow-renderer'

function App() {
  return (
    <>

        <ReactFlowProvider>
          <ReactFlow
            elements={[
              {
                id: 'node-1',
                data: { label: 'node 1' },
                position: { x: 0, y: 0 }
              }
            ]}
            deleteKeyCode={8}
            onElementsRemove={() =>
              console.log('Removing elements from 1st flow')
            }
          />
        </ReactFlowProvider>

        <ReactFlowProvider>
          <ReactFlow
            elements={[
              {
                id: 'node-2',
                data: { label: 'node 2' },
                position: { x: 0, y: 0 }
              }
            ]}
            deleteKeyCode={8}
            onElementsRemove={() =>
              console.log('Removing elements from 2nd flow')
            }
          />
        </ReactFlowProvider>

    </>
  )
}

export default App

This is not really an issue. The flows don't know of each other. You would need to implement a deselection by yourself. You could use setSelectedElements([]) for flow A when you select a node from flow B for example.

Awesome. I should have slept on it! :)
I've updated the test case with a solution.
onElementsRemove={
  activeFlow === 'flow-1'
    ? (elements) => {
        console.log('Removing elements from 1st flow')
        console.log(elements)
      }
    : null
}


weather information query

query weather information data near RC

commit eb5ec41fa2c2223d1aa17e5c6032a415e015ba0b queries:

the current weather situation in Clementi
the 24 hour and 3 hour PSI levels in West Singapore


Block wise transfer tests

Improve old tests and add some new ones.

Upstream PR: https://github.com/eclipse/wakaama/pull/682


chore(cd): fix binaries matcher

Proposed changes
macos & linux doesn't follow exactly the same pattern as windows, so this fix that.
THis has been used in the current release with the branch release-temp-locked

macos & linux doesn't follow exactly the same pattern as windows, so this fix that.

You mean windows does not follow the same pattern as every one else :P

Well, if I was accurate, I'd say they all do their own stuff, 'cause linux and macos doesn't behave the same too.


Feedback command

To help devs provide feedback or log an issue a 'feedback' command has been suggested.
nitric feedback

This command should open a pre-filled new GitHub issue page. GitHub providers query params to support issue prefill (see: https://github.com/nitrictech/cli/issues/146).
Should this command prompt for the type of feedback? e.g. issue, feature request?

We could also consider a feedback survey in the CLI to ask for the type of feedback and offer the survey to anyone who's interested. The main goal here is to capture any issues, suggestions or other feedback people want to provide so we can make the CLI and framework better.

Implement nitric feedback command to open a pre-filled new GitHub issue page. GitHub providers query params to support issue prefill (see: https://github.com/nitrictech/cli/issues/146).
In this version, allow the users to select the type of feedback once they reach the issues page.

Add some information that lets people run a diagnostic to provide the information we need.
They would manually paste it into the issue in the appropriate spot.


Nifi-1808-0.x-add-license

MQTT bundle was committed with a NOTICE in the nar, adding that and fixing a MQTT provenance issue

Commits got messed up, will force push a new one


fix: Duplicated example dataset

SUMMARY
https://github.com/apache/superset/pull/28789 duplicated the wb_health_population dataset which made our command that load the example data to fail. This PR simply removes the only chart that referenced the duplicated dataset and also removes the YAML file that was the root cause for the duplication.
TESTING INSTRUCTIONS
Check the you're able to load the examples correctly.
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

/testenv up

@sadpandajoe @mistercrunch @eschutho @rusackas I can confirmed that this PR also fixed the ephemeral environment problem.


Can't make player

I am getting this error
makefile:33: C:\CEdev/include/.makefile: No such file or directory make: *** No rule to make target 'C:\CEdev/include/.makefile'.  Stop.
I have the windows CEdev installed and its in path. I checked the specified folder and there is no .makefile there. Thanks in advance

This is because the project is so out of date that it doesn't work with the latest toolchain. I'd definitely like to see this project updated to use the new tools, or at least compiled and uploaded as a pre-release!

Alright, I installed an old version of the tool chain and fixed the folder structure enough to get a compiled program. You probably have to convert and send some videos to the calculator first before it runs though. I included the necessary c libraries as well.
CEVidium.zip
for the program to run.


Cookies URL takes to nothing related to our Cookie policy

That URL takes to: https://cookiesandyou.com/
Our Cookie policy URL is: https://info.whichost.com/eng/legal#cookies-section
Please change this today, asap!

I'll deal with this one and should be done by Friday.

This have been fixed and pushed to master and staging.


java.string.lang

Error:(302, 5) error: method does not override or implement a method from a supertype

From version 3.1.3 we support only<EMAIL_ADDRESS>If you want to use<EMAIL_ADDRESS>please, consider using<EMAIL_ADDRESS>


updated admin links to community repo

Hi,
I updated the links in the admin UI to the new community repository as this makes it easier for new people using this adapter.
Also updated the version in admin.js to the latest release.

Thank you!


factorial using recursion added

To which issue is it linked?

https://github.com/Algo-Phantoms/Algo-Tree/issues/276

I was  not aware of the same. I'll work on that.

Changes have been done!


Reading single line from file

Is it possible to do this with figment?
If so, where should I be looking?

Can you be more specific about what you want to do? An example would be great.

Something like this.
fn build_cli() -> Result<Cli> {
	let mut cli = Figment::new().merge(Serialized::defaults(Cli::parse()));
	if let Some(path) = CONFIG_FILE_PATH.as_ref() {
		cli = cli.merge(Data::<SingleLine>::file(path.as_path()));
	}
	Ok(cli.extract()?)
}

So what if I want a custom figment::providers::Data like SingleLine, for instance.?

It's not clear from your example which line should be read, what should happen if the file is empty, and how the line should be parsed. But in any case, yes it would be straightforward to implement such a provider. Alternatively, you can simply read the line from the file using the lines() I/O reader and pass the string to any existing provider.

I see you've added to your previous comment. It sounds like you want a custom provider, since your file doesn't seem to have any obvious relation to existing formats. It's not clear what the file should be parsed as, or which line you want to keep and what that line means.
The figment docs are complete. See Format for what you probably need.

Yeah, I haven't figured out any specifics in terms of how things will be parsed. I just wanted to find out how to extend it. Thank you.


Uncertain about content/text vs iter_content(decode_unicode=True/False)

When requesting an application/json document, I'm seeing next(r.iter_content(16*1024, decode_unicode=True)) returning bytes, whereas r.text returns unicode. My understanding was that both should return a unicode object. In essence, I thought "iter_content" was equivalent to "iter_text" when decode_unicode was True. Have I misunderstood something? I can provide an example if needed.
For reference, I'm using python 3.5.1 and requests 2.10.0.
Thanks!

what does (your response object).encoding return?

There's at least one key difference: decode_unicode=True doesn't fall back to apparent_encoding, which means it'll never autodetect the encoding. This means if response.encoding is None it is a no-op: in fact, it's a no-op that yields bytes.
That behaviour seems genuinely bad to me, so I think we should consider it a bug. I'd rather we had the same logic as in text for this.

r.encoding returns None.
On a related note, I originally thought iter_text might be clearer/more consistent than iter_content(decode_unicode=True) if there's room for change in the APIs future (and iter_content_lines and iter_text_lines I guess), assuming you don't see that as bloat.

@mikepelley The API is presently frozen so I don't think we'll be adding those three methods. Besides, iter_text likely wouldn't provide much extra value outside of calling iter_content(decode_unicode=True).

I added a pull request #3362 that adds the same check used int text() to set the encoding type if there is none. It seems to solve the raw bytes issue. Let me know if there's any needed tweaking.

Glad we fixed this, this was definitely an oversight.

I was reading through this and the related issues and PRs and I'd love to help out. However, I'm not sure what direction to head with it (or if it's something more suitable for the next version of requests).
Just so I understand correctly, we want to get it so iter_content and text both return Unicode strings. The original PR solved this by grabbing the apparent_encoding, which caused a problem when using streaming responses (chardet.detect consumes the entire stream).
I'm assuming detecting the encoding incrementally will still lead to problems, as part of the stream will be consumed. Is this something that may be best solved via documentation, suggesting encoding is set before using iter_content? It seems like we'd need some sort of knowledge about the response, which may be hard to do without consuming it (or part of it).
Could we find some middle ground? For streaming responses, documentation calls out that encoding should be set. If it isn't, return bytes and let the users decode to their heart's desire (seemed like it worked well as a workaround). While nonstreaming responses can use the apparent_encoding, as originally suggested.
I know its a bit cold, but thoughts @Lukasa, @sigmavirus24?

I'm inclined to suggest that we just use encoding and, if that is not set, that we throw an exception that requires that users set it, rather than subtly returning an incompatible type.

I've been playing around with this since we reverted and I don't think that there's a nice way to avoid the exception. I'm thinking that an encoding param for iter_content that @Lukasa suggested in #3481 would be useful. It allows the user to avoid having to set Response.encoding on every PreparedRequest that might be read.

I know @sigmavirus24 brought up a concern about raising an exception if encoding isn't specified for the current major version. Would the suggested fix go against the 3.0 branch? @Lukasa, when you say encoding, I'm assuming you're referring to the response's encoding rather than the additional parameter on iter_content?

Any change to the response's encoding logic (e.g. throwing exceptions if it's not present but decode_unicode is) is definitely breaking, and so would have to go against the 3.0.0 branch. I was not planning to add an extra parameter to iter_content.

I think @shellhead's work in #3574 should have this addressed in 3.0.0.

I'm pulling this comment out of #3675 and putting it here to avoid derailing the other issue.
#3675 suggests to me again that there's a use case for a user supplied encoding fallback. Right now, you get unexpected bytes with decode_unicode=True. Even with the 3.0.0 fix, anytime decode_unicode is used, it must be wrapped in a try/except or conditional block to guarantee the request will be useful. This isn't the end of the world, but the fact that iter_* will fail on (semi?) common APIs like Amazon S3, Pinterest, and Stripe seems like a gap worth addressing. Why not provide the interface instead of forcing the users to program around it every time?

Users are encouraged to set response.encoding: the docs point out that you can and should do it. I think that means we already provide the interface.

Users are encouraged to set response.encoding: the docs point out that you can and should do it.

I don't know if I would go so far as to assert the docs say you "should do it", in fact I think they give a false sense of assurance about decoding.
The encoding section only discusses how Response.text will either use the header encoding/chardet to decode the body, or you can get raw bytes from Response.content. If you want to change the default encoding you can supply a new one. They don't really touch on that default being a no-op, or in the future halting entirely.
If if r.encoding is None: r.encoding = 'my-chosen-fallback-1' is a requirement to reliably use the streaming interface because decode_unicode=True isn't sufficient, then that should definitely be detailed. I guess I'm beating a dead horse on integrating this into iter_* though.


Build fails when library uses submodules in bundle

Some libraries, such as Capybara and Hallon use a vendored dependency, which is pulled in through a submodule, and then use bundler to install this dependency. It would be convenient if Travis automatically pulled in a project's submodules before bundle install is run.
Note that this is the reason we can't simply use a before_script, since before_script is run after bundling.
#5 attempted to solve this problem, but was reverted. I am adding this issue, since #5 is closed but the issue still remains, and we don't know if #5 is the correct way of solving this problem.

/cc @Burgestrand, @joliss and @ahx

Hey Jonas,
For the time being, until the issue is really fixed, you can use before_install which will run before bundle install.
Hope this helps.
J

Sent from my Sega Master System
On 6/11/2011, at 12:34 PM, Jonas<EMAIL_ADDRESS>wrote:

Some libraries, such as Capybara and Hallon use a vendored dependency, which is pulled in through a submodule, and then use bundler to install this dependency. It would be convenient if Travis automatically pulled in a project's submodules before bundle install is run.
Note that this is the reason we can't simply use a before_script, since before_script is run after bundling.
#5 attempted to solve this problem, but was reverted. I am adding this issue, since #5 is closed but the issue still remains, and we don't know if #5 is the correct way of solving this problem.

Reply to this email directly or view it on GitHub:
https://github.com/travis-ci/travis-worker/issues/22

Oh yeah, that helps a lot, actually!

Sweet!

I don't think there is any need for special support for submodules, before_install solves this and many other similar problems.

Not to bikeshed, but ...

I don't think there is any need for special support for submodules, before_install solves this and many other similar problems.

Is there ever any reason not to pull submodules though? I'm thinking that if git is up-to-date on the workers, we might as well pass --recurse-submodules to clone and fetch.

My 2c is that before_install allows people to handle all these issues. If we add everything to the core then it will bloat and means that we would have to maintain it all.

I am not adverse to adding --recurse-submodules, in fact maybe you might be
interested in adding this to the worker yourself and sending a pull
request? :)
On Sun, Nov 6, 2011 at 11:44 PM, Philip Arndt <
<EMAIL_ADDRESS>
My 2c is that before_install allows people to handle all these issues.
If we add everything to the core then it will bloat and means that we would
have to maintain it all.

Reply to this email directly or view it on GitHub:
https://github.com/travis-ci/travis-worker/issues/22#issuecomment-2648304

That was #5; it got merged in 44d0d79, but reverted in 59a702e83. Michael says in #5, "Now we have git 1.7.5 which should support --recurse-submodules but we did not try it yet."
That's all I know. So ... Feel free to re-merge. Or not. :-)

Why have a specialized solution like this one while we have a generic one everyone can use when they need to? before_install lets you handle the trickiest of cases with submodules, build dependencies you download tarballs and so on. And by default, travis won't do anything to make sure it does not get in your way.
Last time we tried --recurse-submodules something caused builds to fail. Since vast majority of projects on travis do not use submodules, I see no need to add it. Once you add something, you then have to maintain it forever. See email notification features & logic to find out how crazy it can get.


md-icon SVG's registered with different names but second SVG overrides things in the first SVG

Bug, feature request, or proposal:
This is a bug with md-icon here is the use case I have, I have two SVG icons that are similar but the icon of them in the middle and the fill change. What happens is the second SVG overrides the fill of the first SVG for some reason.
What is the expected behavior?
I would expect since these two icons are named differently and are different svgs they shouldn't have conflicts with each other and change things about each other.
What is the current behavior?
The second SVG icon will override the first SVG icon's fill to its fill etc...

How the first icon is supposed to look:

What are the steps to reproduce?
Have two SVG icons with different fills and the first will override the second's fill.
Providing a Plunker (or similar) is the best way to get the team to see your issue.
Plunker template: https://goo.gl/DlHd6U
Another bug is the Plunker template is currently broken... so I am going to create the example via a commit to my fork's master:
https://github.com/alexw10/material2/commit/19d234ff2e22c565fc6a60ee0f10135b65ea5602
In the above example if I take away the second SVG icon and only put test1 in the dom it renders as the proper icon and is red etc...
Above is to show what I am seeing.
What is the use-case or motivation for changing an existing behavior?
Use case is having separate svgs should always render svgs as they are.
Which versions of Angular, Material, OS, TypeScript, browsers are affected?
Whatever is in master currently for each of these, Mac, Chrome.
Is there anything else we should know?

Ah, okay I think this can be closed, I think the solution is to just make sure the SVG icons classes are not the same... this solved it.
Sorry I didn't know much about SVG and such before creating this. I am going to go ahead and close.


Write example in Elixir

Hi @guy, has there been any progress here?
I'm interested in using quic_dist in a large project. More details about the project can be found here https://github.com/eigr/spawn/issues/90
We have a discord channel if you're interested in joining the discussions. We would love to try out using quic_dist for our purposes.

Hi @sleipnir
glad to hear that you are interested in using quic_dist.
Unfortunately, I have so much other work at the moment but PRs are very welcomed. I can also assist you in a free time slightly_smiling_face

Cool. Do you think the project is usable in production?

I remember there was at least one bug i.e. sometimes you might not get connected?  https://github.com/mickel8/quic_dist/issues/20

Ok. Thanks!

This issue is still open?

Yes, unfortunately I am focused on other things at the moment but any PRs are welcomed