asusctl: init at 4.0.6

Motivation for this change
Add new package asusctl to improve support for ASUS Laptops.
Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[X] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[X] Fits CONTRIBUTING.md.

Hm, for some reason the systemd system unit service isn't copied to share/systemd while the user services are

should probably have more patching since asusctl tries to edit config files (example)
would be nice to have a module as well

I could be wrong, but I think the convention is to have separate asusctl: init at 4.0.6 and nixos/asusctl: init commits.

Fixed :+1:

For the most part, it seems to be working fine.
Just a few problems, at least on my configuration.
supergfxd:
sudo systemctl status supergfxd
● supergfxd.service - SUPERGFX
     Loaded: loaded (/etc/systemd/system/supergfxd.service; linked; vendor preset: enabled)
     Active: active (running) since Thu 2021-12-16 10:29:19 CST; 18min ago
   Main PID: 4725 (supergfxd)
         IP: 0B in, 0B out
         IO: 0B read, 20.0K written
      Tasks: 1 (limit: 18444)
     Memory: 284.0K
        CPU: 0
     CGroup: /system.slice/supergfxd.service
             └─4725 /nix/store/ysk35xrh97hb2lrw67h2ndhkk5wzv0mg-supergfxctl-2.0.4/bin/supergfxd

Dec 16 10:29:21 tsuki supergfxd[4725]: INFO: GFX: 0000:07:00.4: Function for 0000:07:00.0
Dec 16 10:29:21 tsuki supergfxd[4725]: INFO: GFX: 0000:07:00.2: Function for 0000:07:00.0
Dec 16 10:29:21 tsuki supergfxd[4725]: INFO: GFX: 0000:07:00.0: Function for 0000:07:00.0
Dec 16 10:29:21 tsuki supergfxd[4725]: INFO: GFX: 0000:07:00.5: Function for 0000:07:00.0
Dec 16 10:29:21 tsuki supergfxd[4725]: INFO: GFX: 0000:07:00.3: Function for 0000:07:00.0
Dec 16 10:29:21 tsuki supergfxd[4725]: INFO: GFX: 0000:07:00.1: Function for 0000:07:00.0
Dec 16 10:29:21 tsuki supergfxd[4725]: INFO: GFX: 0000:01:00.0: NVIDIA graphics, setting PM to auto
Dec 16 10:29:21 tsuki supergfxd[4725]: INFO: GFX: Writing /etc/X11/xorg.conf.d/90-nvidia-primary.conf
Dec 16 10:29:21 tsuki supergfxd[4725]: INFO: GFX: Writing /etc/modprobe.d/supergfxd.conf
Dec 16 10:29:21 tsuki supergfxd[4725]: ERROR: Gfx controller: Command exec error: "modprobe" "nvidia_drm": No such file or directory (os error 2)

asusd:
● asusd.service - ASUS Notebook Control
     Loaded: loaded (/etc/systemd/system/asusd.service; linked; vendor preset: enabled)
     Active: active (running) since Thu 2021-12-16 10:43:10 CST; 4s ago
    Process: 11348 ExecStartPre=/nix/store/fvprxgcxf4px865gdjd81fbwnxcjrg41-coreutils-9.0/bin/sleep 2 (code=exited, status=0/SUCCESS)
   Main PID: 11350 (asusd)
         IP: 0B in, 0B out
         IO: 3.3M read, 12.0K written
      Tasks: 4 (limit: 18444)
     Memory: 2.4M
        CPU: 0
     CGroup: /system.slice/asusd.service
             └─11350 /nix/store/rfdn03vqb8s2kjc5llqi9hmbcrbmnrrn-asusctl-4.0.6/bin/asusd

Dec 16 10:43:10 tsuki asusd[11350]: ERROR: rog_bios_control: Path /sys/firmware/efi/efivars/AsusPostLogoSound-607005d5-3f75-4b2e-98f0-85ba66797a3e: No such fil>
Dec 16 10:43:10 tsuki asusd[11350]: INFO: Battery charge limit: 80
Dec 16 10:43:10 tsuki asusd[11350]: INFO: Device has profile control available
Dec 16 10:43:10 tsuki asusd[11350]: ERROR: AniMe control: Missing functionality: ASUS AniMe device node not found
Dec 16 10:43:10 tsuki asusd[11350]: INFO: Matched to ROG Zephyrus G15 GA503Q
Dec 16 10:43:10 tsuki asusd[11350]: WARN: led_node: Missing functionality: ASUS LED device node not found
Dec 16 10:43:10 tsuki asusd[11350]: WARN: led_node: Missing functionality: ASUS LED device node not found
Dec 16 10:43:10 tsuki asusd[11350]: WARN: led_node: Missing functionality: ASUS LED device node not found
Dec 16 10:43:10 tsuki asusd[11350]: INFO: Using device at: "/dev/hidraw5" for LED control
Dec 16 10:43:10 tsuki systemd[1]: Started ASUS Notebook Control.

supergfx can't seem to modprobe, for some reason.
asusd can't seem to read an efivar, which I have confirmed is there.
asusd won't start without a rebuild, and it goes without mentioning, doesn't start on boot, so I have to nixos-rebuild and then start it manually.

Also, supergfxd creates /etc/supergfxd.conf.
This should be controlled by the nix configuration in nixos/supergfxctl.nix preferably.

From a cursory look, you should be able to more or less do what you did with asusctl and it's systemd service to supergfxctl.

Something interesting I noticed, if I modprobe -i vfio-pci, and then restart supergfxd, and then restart it again, it keeps failing until I nixos-rebuild switch.

Thanks for the feedback @Madouura ! I've added the options to supergfxctl and fixed the paths to modprobe and rmmod. That should fix the last thing you've mentioned re: vfio-pci. I'll have to look into why asusd fails to start.

asusd can't seem to read an efivar, which I have confirmed is there.

I think it actually can read the file, but it can't write to it since efivars is mounted as ro. I don't think we have an option in NixOS to remount it as r-w, so I'm not sure if we can do something about that.

I'll test this a bit later.
You may want to follow the contributing guidelines and add release notes.

I can't find much logs to go off of either, the two top things I can think of for the reason of asusd failing would be either a hard-coded path or something with starting supergfxd.
It's possible asusd may just be starting too early, or not early enough as well.

Yeah, maybe it’s starting too early. I don’t think it’s due to hard coded paths, then it wouldn’t work sometimes. I’ll try fixing it tomorrow

thread 'main' panicked at 'The directory /etc/asusd/ is missing', daemon/src/config.rs:28:33

@Madouura Are you sure that's from asusd from the latest version of this PR? The logs look like this for me:
Jan 05 15:20:04 nixos systemd[1]: Stopped ASUS Notebook Control.
Jan 05 15:20:04 nixos systemd[1]: asusd.service: Deactivated successfully.
Jan 05 15:20:04 nixos systemd[1]: Stopping ASUS Notebook Control...

Which would suggest that systemd is stopping it (for some reason)

It should be, I'll retry it this weekend.

Try replacing the contents of service.patch with
diff --git a/data/asusd.service b/data/asusd.service
index 9f0ee84..8b5d466 100644
--- a/data/asusd.service
+++ b/data/asusd.service
@@ -1,16 +1,18 @@
 [Unit]
 Description=ASUS Notebook Control
-StartLimitInterval=200
+StartLimitInterval=0
 StartLimitBurst=2
 Before=multi-user.target
+After=supergfxd.service
+Requires=supergfxd.service

 [Service]
 Environment=IS_SERVICE=1
-ExecStartPre=/bin/sleep 2
 ExecStart=/usr/bin/asusd
+StateDirectory=asusd
 Restart=on-failure
 Restart=always
-RestartSec=1
+RestartSec=4
 Type=dbus
 BusName=org.asuslinux.Daemon
 SELinuxContext=system_u:system_r:unconfined_t:s0

I have it working with this.

Huh weird,  I'm pretty sure I've tried something like that too - let me apply that patch, thanks a lot for the suggestion :)

I couldn't find a good way to separate supergfxd and asusd unfortunately, but I fixed a couple issues in this patch.
You should just be able to git apply this right onto your nixpkgs fork.
lol.patch.txt
diff --git a/pkgs/tools/misc/asusctl/default.nix b/pkgs/tools/misc/asusctl/default.nix
index b44af812..ac8dc453 100644
--- a/pkgs/tools/misc/asusctl/default.nix
+++ b/pkgs/tools/misc/asusctl/default.nix
@@ -29,8 +29,8 @@ rustPlatform.buildRustPackage rec {
     for file in data/*.service; do
       substituteInPlace $file \
         --replace /usr/bin/sleep ${coreutils}/bin/sleep \
-        --replace /bin/sleep ${coreutils}/bin/sleep \
-        --replace /bin/mkdir ${coreutils}/bin/mkdir \
+        --replace =/bin/sleep =${coreutils}/bin/sleep \
+        --replace =/bin/mkdir =${coreutils}/bin/mkdir \
         --replace /usr/bin $out/bin
     done

diff --git a/pkgs/tools/misc/asusctl/service.patch b/pkgs/tools/misc/asusctl/service.patch
index 7fd232d5..cc113539 100644
--- a/pkgs/tools/misc/asusctl/service.patch
+++ b/pkgs/tools/misc/asusctl/service.patch
@@ -1,9 +1,12 @@
 diff --git a/data/asusd.service b/data/asusd.service
-index 9f0ee84..66add89 100644
+index 9f0ee84..8b5d466 100644
 --- a/data/asusd.service
 +++ b/data/asusd.service
-@@ -3,10 +3,13 @@ Description=ASUS Notebook Control
- StartLimitInterval=200
+@@ -1,16 +1,18 @@
+ [Unit]
+ Description=ASUS Notebook Control
+-StartLimitInterval=200
++StartLimitInterval=0
  StartLimitBurst=2
  Before=multi-user.target
 +After=supergfxd.service
@@ -11,9 +14,14 @@ index 9f0ee84..66add89 100644

  [Service]
  Environment=IS_SERVICE=1
- ExecStartPre=/bin/sleep 2
-+StateDir=asusd
+-ExecStartPre=/bin/sleep 2
  ExecStart=/usr/bin/asusd
++StateDirectory=asusd
  Restart=on-failure
  Restart=always
-
+-RestartSec=1
++RestartSec=4
+ Type=dbus
+ BusName=org.asuslinux.Daemon
+ SELinuxContext=system_u:system_r:unconfined_t:s0
+
\ No newline at end of file

With the patch you've provided it seems to work just fine! Weird, I'm pretty sure I've tried something similar. Thanks for the help :)

@Cogitri Looks like there's some merge conflicts, you'll need to rebase.
Also, I would suggest you add some release notes: https://github.com/NixOS/nixpkgs/blob/af21d41260846fb9c9840a75e310e56dfe97d6a3/CONTRIBUTING.md#generating-2205-release-notes

Done, thanks :)

Rebased against master. This has worked on my laptop for a few weeks now :)

Same for me, don't see any major changes so still LGTM.

@itsfarseen Clone the PR branch, and do sudo nixos-rebuild switch -I nixpkgs=/path/to/nixpkgs

@leo60228, that will also update every other package right? Is there a way to somehow pick just asusctl?

This is what I ended up doing:
let
  nixpkgs-tars = "https://github.com/NixOS/nixpkgs/archive/";
  pr_asusctl_tar = fetchTarball "${nixpkgs-tars}a4a81b6f6c27e5a964faea25b7b5cbe611f98691.tar.gz";
in
  { config, lib, pkgs, ... }:

  {
    imports =
      [ # include the results of the hardware scan.
        ./hardware-configuration.nix
        "${pr_asusctl_tar}/nixos/modules/services/misc/asusctl.nix"
        "${pr_asusctl_tar}/nixos/modules/services/misc/supergfxctl.nix"
      ];

    nixpkgs.overlays = [
      (self: super: {
        asusctl = pkgs.callpackage "${pr_asusctl_tar}/pkgs/tools/misc/asusctl/default.nix" {};
        supergfxctl = pkgs.callpackage "${pr_asusctl_tar}/pkgs/tools/misc/supergfxctl/default.nix" {};
      })
    ];

    environment.systemPackages = with pkgs; [
      # .. snip ..
      asusctl
    ];

    services.asusctl.enable = true;

    # .. snip ..
  }

I've been testing this, and it works almost perfectly, but I've been having trouble getting the Quiet profile to persist between reboots. From what I can tell, asusd loads it from /etc/asusd/profile.conf when it starts, but the postPatch in asusctl/default.nix switches things so it uses /var/lib/asusd/profile.conf. When I apply a profile interactively with asusctl profile -P Quiet, the contents of profile.conf change to reflect the active profile, but after a reboot it's back to Balanced every time. Any ideas on what might be causing this?

Recent kernels manage those profiles now, maybe there's some kind of conflict?

That may be the issue. asusd is definitely what's overwriting the file though, I tried setting the profile to Quiet, disabling the asusd service, then rebooting, and profile.conf doesn't change until I the asusd service is started again. I think what's happening is the profile isn't surviving the reboot inside the bios/whatever low level system is responsible for that, then when asusd starts it realizes the file is out of sync with what's actually happening so it updates it. I'm going to do some testing with the behaviour on my arch installation to see if the profile survives reboots without asusd running.

Ok, sorry for cluttering up the thread, it seems I misunderstood the way asusctl does profile control. It looks like most of the work is done by power-profiles-daemon, and asusctl just provides an subcommand that does a similar thing to powerprofilectl, which is the usual interface for power-profiles-daemon. So I'm not sure whether it makes sense to include a dependency on power-profiles-daemon here, because while the profile you set doesn't persist between reboots without it, some users might not want that functionality.
For reference, the necessary configuration to get power-profiles-daemon working is:
  services.power-profiles-daemon.enable = true;
  systemd.services.power-profiles-daemon = {
    enable = true;
    wantedBy = [ "multi-user.target" ];
  };

Should we add a setting for the asusctl service that does this, or should we leave it up to the user to configure manually?

@Cogitri

Would it be better naming the service asusd and supergfxd instead? Since the *ctl names are for the CLIs interacting with the daemon.

Sounds like a good idea, thanks

I believe all concerns should be addressed now. For those of you using flakes, you can use my repository if you want to: https://github.com/Cogitri/cogitri-pkgs

Gave this a quick try, something I noticed: supergfxctrl creates it's own modprobe config at /etc/modprobe.d/supergfxd.conf, pretty unexpected to turn on the service, run supergfxctrl once, then turn off the service and still have the nvidia kernel module blacklisted on next reboot.
Not sure the best fix, since it seems to rely on doing this for any of its features that require a reboot. Ideally the file would be cleaned up if the service is disabled but I don't believe there is a clean way to do that, maybe a patch to prevent it from writing to modprobe.d, and a config option to enable/disable the patch, so the user is aware of non-declarative changes to the system?

Gave this a quick try, something I noticed: supergfxctrl creates it's own modprobe config at /etc/modprobe.d/supergfxd.conf, pretty unexpected to turn on the service, run supergfxctrl once, then turn off the service and still have the nvidia kernel module blacklisted on next reboot.
Not a showstopper but might throw some people for a loop, not sure the best fix either, since it seems to rely on doing this for any of its features that require a reboot. Ideally the file would be cleaned up if the service is disabled but I don't believe there is a clean way to do that, maybe a patch to prevent it from writing to modprobe.d, and a config option to enable/disable the patch, so the user is aware of non-declarative changes to the system?

I'd rather get this merged first before pouring more work into this PR

I'd rather get this merged first before pouring more work into this PR

For sure, its a fairly niche situation in an already niche pkg that's only a portion of the branch, with no obviously correct solution, I agree on getting it merged as it is, just wanted to note it down

cc @SuperSandro2000
Could we get this merged please?

Done :heavy_check_mark:

Why do we want /etc/supergfxd.conf managed by nix? Would you, in some cases, want ot be able to switch graphics modes on the fly without having to rebuild your system?

It would be great to have some variant of asusctl in NixOS, even if it had some quirks.

Why do we want /etc/supergfxd.conf managed by nix?

Because NixOS tries to be mostly stateless and most system wide settings are managed by it.

Would you, in some cases, want ot be able to switch graphics modes on the fly without having to rebuild your system?

Do we? I don't know but a specialization could fix that.

Superseded by https://github.com/NixOS/nixpkgs/pull/201081


Add CI and docs

Questions:

should we include the license file in this repo? (pending ongoing discussion)
anything else I missed from the checklist: https://github.com/dotnet-foundation/projects/issues/353#issue-2096112950 ?

Pending:

how do we add the CLA?
~add the relevant PAT token to the secrets~ done

Thanks! This is ready for review.

don't hesitate to ask on the other issue we have under DNF

I'll do it as soon as we get the two PRs merged and we do a successful test release.


Introduce a pluralisable string type

PRing this in a working state to discuss about integrating this in the localisation pipeline.
This implements a PluralisableString type deriving from TranslatableString, that can display a pluralised variant of a translation according to the plural context.
I initially had the 'separator character' delimiting two variants in localised text as a ctor parameter but I guess it should be fine to have it be hardcoded as a '|' for now, which corresponds to what's used on web-side to separate plural variants altough I'm opened to make it a parameter again.
The integration should be transparent on osu!-side with required changes being on osu-localisation-analyzer end.
Plans to do so sum up to adding heuristics to detect strings which support pluralisation (contains the | delimiter) and use PluralisableString in place of TranslatableString in that case. I'll mention that I haven't delved into it yet but according to my memory from a previous look at the source, that should be pretty straightforward.
The diffstat makes this PR looks big where in fact more than half of it comes from the locale -> plural count mapping.
Implementation references
https://github.com/laravel/framework/blob/f60d3da8f9ad236d339562cd89eb5dfde4eb7dce/src/Illuminate/Translation/MessageSelector.php#L99 for code-based plural rules
Spec at : https://unicode-org.github.io/cldr-staging/charts/latest/supplemental/language_plural_rules.html

Not 100% on the PluralisableString : TranslatableString inheritance. Feels it complicates code and maybe it may be easier to duplicate the pieces that need to be duplicated for it to work, dunno.

On a second thought, reverting the inheritance and simply copying over the format logic sounds cleaner, as that would remove the need to override the equal checks which are becoming messy. Will do that instead.

would this allow for omitting variables (including cases with multiple ones)?
o ora ({1}) | {0} ore ({1}) | {0} de ore ({1}) would be required for languages such as romanian

As far as I can tell this PR already supports that, yes. (see getPluralIndex()).


update FloodAgent for stack

Modifications requested:

Update icon name in PostGIS table according to water level and trends
Add blazegraph endpoint in PostGIS table for GeoServer
Specify Blazegraph's namespace via environment variable instead of hardcoding
Update to use the latest base-lib for updated time series namespace

Updating icon name is not done yet


Add Batch (.bat) Syntax Highlighting

Please could batch (.bat)/cmd syntax highlighting be added to codeblocks?
Reproduce Issue

Insert code block and after the first ``` enter either bat, batch or cmd
No syntax highlighting provided

Thanks in advance!

I couldn't find any good codemirror mode for batch files, so this is probably something that has to wait until there is a proper plugin api for language modes.

Indeed, I would have hoped it's available from here: https://github.com/codemirror/legacy-modes/tree/main/mode but I don't see it. So indeed somebody would have to create one first. Unless it's compatible with powershell, but I don't think so?


fix: use Base36 for all code indexers

Use Base36 for all code indexers. Should fix #12829.

Codecov Report

Merging #12830 into master will increase coverage by 0.01%.
The diff coverage is 66.66%.

@@            Coverage Diff             @@
##           master   #12830      +/-   ##
==========================================
+ Coverage   43.15%   43.16%   +0.01%
==========================================
  Files         654      654
  Lines       72205    72205
==========================================
+ Hits        31159    31167       +8
+ Misses      35998    35989       -9
- Partials     5048     5049       +1

Impacted Files
Coverage Δ

modules/indexer/code/bleve.go
71.16% <ø> (-0.27%)
:arrow_down:

modules/indexer/code/indexer.go
35.95% <66.66%> (+0.72%)
:arrow_up:

modules/queue/workerpool.go
57.95% <0.00%> (-0.82%)
:arrow_down:

modules/git/repo.go
49.74% <0.00%> (-0.51%)
:arrow_down:

services/pull/pull.go
41.57% <0.00%> (-0.47%)
:arrow_down:

models/gpg_key.go
55.40% <0.00%> (+0.58%)
:arrow_up:

routers/repo/view.go
38.11% <0.00%> (+0.64%)
:arrow_up:

models/unit.go
49.31% <0.00%> (+2.73%)
:arrow_up:

modules/git/utils.go
77.04% <0.00%> (+3.27%)
:arrow_up:

modules/util/timer.go
85.71% <0.00%> (+42.85%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 53b6565...5f98b8d. Read the comment docs.

Oh. So even bleve indexer has been effected by this bug? It's strange we haven't recieved report before.

~Oh. So even bleve indexer has been effected by this bug? It's strange we haven't recieved report before.~

This bug will only occurs if the count of repositories >= 10


log file problem, modification extension

warning in 3aac4505bb3ec2e8: in annotation extension for SPAC22A12.15c.1, can't parse identifier: 10
looks fine to me:

or did anyone fix it already?

did @mah11  or @Antonialock  fix this annotation?

I have never done anything with this paper or session.
It may have something to do with the fact that we added the "occupancy" option for bulk modification data (and then the submitter decided not to use it after all!), and never properly thought about how it would play with Canto.

smidge of background in https://github.com/pombase/pombase-chado/issues/392

Are OK, this is probably the only use case. It's likely that Kim needs to  so something to allow it?
First off is the config file correct? It says:
MOD:00000       is_a    occupancy       %       proportion of proteins with modification                0,1     user

seems to be....
FYPO:0000002    is_a|has_part   has_penetrance  FYPO_EXT:1000002|%      penetrance      proportion of the population that shows the phenotype (can be quantitative or qualitative)      0,1     user

FYPO_EXT:1000002|%

Thanks Val.  I think that's the clue I needed.  I had a look at the code I couldn't see a problem.
Now you've pointed out the config, I think the code is being dumb and tries to parse the "10" as a FYPO extension - it's not handling the "|".

Its working for FYPO, but not for modification. Maybe the  config file isn't quite right?

Its working for FYPO, but not for modification.

Ah, thanks.  It was working for has_penetrance but not for occupancy.  Fixed now.


How many options are possible with OptionsConsoleCommand?

Hello,
i have a question:
How many options in one command are possible with OptionsConsoleCommand?
I asked because i search the problem in this project: https://github.com/wezhunter/ESP32_LinuxCNC_MotionController_RealTime
It uses your ESP32Console for the setup, some commands have more than 3 option for one command and that makes problems ( described here: https://forum.linuxcnc.org/18-computer/51816-esp32-s2-s3-linuxcnc-controller-6-axis-hardware-step-gen-usb-plug-and-play?start=134 )
I searched but i don't find the limitation or an error in the implementation.
Thanks!
Best regards
Michael

Hello,
i found the limitation and solved my problem:

A wish for a future version: Pleas ad an error message if the number of arguments are higher than the limit.
Best regards
Michael


Error in MqttLib/Core/QosManage.cs

Hi,
Please check out Core/QosManage.cs  Line 91. I believe that should be a MqttPubrecMessage rather than MqttPubrelMessage that is added to response. I tried to fixed it, but still didn't get it work properly. Is there anyone here who can help fix this?

It really seems to be wrong, since the response of a Publish with QoS=2 should be Pubrec.

It sounds like this issue is related to #3 ?
I'll happily accept a pull request if someone has a fix and there is consensus on the correct behaviour.


proposal: Go 2: add collect statements to the language

#21161 has seen a lot of proposals regarding error handling. Based on the discussions there and my experience writing go, I'd like to propose what I call a collect statement. The root of this idea is that the complaints leveled against Go's error handling being repetitive/verbose are really complaints that checking for nil in go code is repetitive/verbose.
I could see it being beneficial to have a way to "collect" to a variable - that is, try a series of statements until a non-nil value is encountered. I've proposed syntax and control flow here, but I'm not particularly married to either. My proposed style isn't without reason, though. I feel that it fits with go's philisophy of being able to read code linearly, and having to explicitly announce intent in code.
A collect statement would be of the form collect [IDENT] [BLOCK STMT], where ident must me an in-scope variable of a nil-able type. Within a collect statement, a special variable _! is available as an alias for the variable being collected to. _! cannot be used anywhere but as an assignment, same as _. Whenever _! is assigned to, an implicit nil check is performed, and if _! is not nil, the block ceases execution.
Theoretically, this would look something like this:
func TryComplexOperation() (*Result, error) {
    var result *Result
    var err error

    collect err {
        intermediate1, _! := Step1()
        intermediate2, _! := Step2(intermediate1, "something")
        // assign to result from the outer scope
        result, _! = Step3(intermediate2, 12)
    }
    return result, err
}

which is equivalent to this code, which will compile (provided all necessary functions etc. are defined, of course)
func TryComplexOperation() (*Result, error) {
    var result *Result
    var err error

    {
        var intermediate1 SomeType
        intermediate1, err = Step1()
        if err != nil { goto collectEnd }

        var intermediate2 SomeOtherType
        intermediate2, err = Step2(intermediate1, "something")
        if err != nil { goto collectEnd }

        result, err = Step3(intermediate2, 12)
        // if err != nil { goto collectEnd }, but since we're at the end already we can omit this
    }

collectEnd:
    return result, err
}

Aside from error handling, I think a collect statement has some other value additions.
For example, imagine you'd like to try to retrieve a value in a series of ways

retrieve from a local shared cache
retrieve from a global shared cache
instantiate a new object

With collect you could write that as
// try several approaches for acquiring a value
func GetSomething() (s *Something) {
    collect s {
        _! = getLocalCache()
        _! = getGlobalCache()
        _! = new(Something)
    }
    return s
}

Also consider the scenario where you have several subsets of a larger operation, which can be annotated separately to give better error contxt
func TryComplexOperation() (*Result, error) {
    var (
        ioErr error
        bytes1, bytes2 []byte
    )

    collect ioErr {
        bytes1, _! = GetBytes()
        bytes2, _! = GetOtherBytes()
    }

    if ioErr != nil {
        return nil, fmt.Errorf("read error: %s", ioErr)
    }

    var (
        parseErr error
        target1 SomeStruct
        target2 SomeStruct2
    )

    collect parseErr {
        _! = json.Unmarshal(bytes1, &target1)
        _! = json.Unmarshal(bytes2, &target2)
    }

    if parseErr != nil {
        return nil, fmt.Errorf("parse error: %s", err)
    }

    return &Result{target1, target2}, nil
}

New syntax features required:

keyword collect (I'd need to see how often this ident occurs in wild go code, I haven't run a scan on my GOROOT or GOPATH yet)
special ident _! (I've implemented this in the parser to see if it would be possible, and it's not terribly difficult to make this match as an ident without allowing users to declare it as a name for a variable, etc)

I think something like this would provide developers with a clean and concise way to express code that may encounter nil values, while keeping the control flow nice and linear.
I'd really like to see this discussed, even if it ends up not being adopted, because as I've developed the concept I've increasingly fallen in love with the idea. Because conceptually equivalent code can be created today (as demonstrated above), I think the impact on the language would be minimal except for at the syntactic level.

How would this work with nesting?

I imagine nesting wouldn't be an issue. If you imagine the code as written in current go (using block scopes and goto), it would look like this:
with collect
func TryComplexOperation() (*Result, error) {
	var result *Result
	var err error

	collect err {
		var (
			subError error
			intermediate1 SomeType
			intermediate2 SomeType2
		)

		collect subError {
			intermediate1, _! = trySomething()
			intermediate2, _! = trySomething2()
		}
		_! = subError
		result, _! = trySomething3(intermediate1, intermediate2)
	}

	return result, err
}

with goto using current go syntax
func TryComplexOperation() (*Result, error) {
	var result *Result
	var err error

	{
		var (
			subError      error
			intermediate1 SomeType
			intermediate2 SomeType2
		)

		{
			intermediate1, subErr = trySomething()
			if subErr != nil {
				goto endInnerCollect
			}

			intermediate2, subErr = trySomething2()
			if subErr != nil {
				goto endInnerCollect
			}
		}
	endInnerCollect:

		err = subError
		if err != nil {
			goto endOuterCollect
		}
		result, err = trySomething3(intermediate1, intermediate2)
		// if err != nil { goto endOuterCollect }, omitted because we're at the end anyway
	}
endOuterCollect:
	return result, err
}

I think it wouldn't make much sense to allow using _! as a collection target. I think the proposal forbids that, anyway, as it wouldn't technically count as an "assignment".

@mccolljr I really like the idea of a collect statement. It's almost what I was trying to convey in my gist (https://gist.github.com/KernelDeimos/384aabd36e1789efe8cbce3c17ffa390), with the only difference (non-syntactically, that I can see) being there's no option to continue control flow from the same place after checking the nil-able type (and control flow returns to the parent block, whereas my idea requires the not-nil-handling-code to be in a sub-scope is it may be called more than once like in a loop structure). I assume that's a worthy sacrifice for how much simpler what you propose would be to implement, and how useful it would be in a wide variety of cases.
I think what you propose could also work with more than one variable at a time, provided the compiler checks that any assignment with a _! is consistent with types specified after collect
// try several approaches for acquiring a value
func GetSomething() (s *Something) {
var err error
collect (s, err) {
_!, _! = fetchOrNil1()
_!, _! = fetchOrNil2()
// The below line would cause a compiler error
_! = errorOrNil()
// The behaviour for the below line should
// probably also be a compiler error
_!, _ = new(Something)
}
return s
}

@KernelDeimos Interesting suggestion for collecting more than one variable. If that were the approach, though, I'd want to not use the same _! identifier for each collected variable. I'd prefer something like _1, _2 etc.
However, I'm not really sure what the example above would be doing. When you have _!, _! = fetchOrNil1(), you'd exit the block if the function returned an error OR returned a valid *Something. I'm not sure that's what you'd want in that scenario. I realize that's just some pseudocode that I put together and that you modified, though. Would you be able to provide an example from some real world code where collecting on 2 variables would be helpful? I'm not saying it can't be, I'm just having trouble thinking of what that situation would be.

Is there any purpose in the _!? Why not continue using err (or whatever variable you are collecting?) It may be more clear as to what's going on.
Also, for personal curiosity, is there any expected behavior for non-pointer types? Like what would happen with var n int; collect n { ... }?

I feel like this would just bring about a collect statement for every function.  I'm not necessarily opposed to the idea, but it feels unclean to think that a vast majority of function bodies will be indented twice because they all start with collect statement.

@luke-park Perhaps it could be convention to have collect be on the same indent-level, similar to switch or select? Although switch and select only share the indent with case statements, so I'm not sure as to how it would look for collect...

I'm happy people are engaging with the idea! I chose _! for a couple of reasons - 1 being that ! is used in other languages to denote something that warrants extra attention / consideration. Ruby has an idiom where methods that modify their receivers end with !. Kotlin uses ! to assert a nillable type is non-nil. Rust uses ! to call a macro to make clear it's not just a function call. It could also be _? for similar reasons. Why it should be any of these at all is reason 2 - it makes it clear that something more than just an assignment is happening. Along with these, there's the added benefits of being easy to visually scan and see where in a collect statement an assignment to the collection target is happening, is familiar to go developers since _ is something we're used to seeing when we want to ignore an assignment. It also seemed reasonable to keep the _ and this new symbol similar, since they both have similar restrictions on when they could be used. You wouldn't be able to use _! as a value, same as _.
As for collecting to values of non-pointer types, I'm on the fence. On the one hand, I can see the usefulness if a developer treats zero values as invalid, but on the other hand, value types often have useful/meaningful zero values.
I'm not sure if it makes sense to include non-pointer types, but I'd be interested to hear others thoughts as to why or why not.

@luke-park I can understand that concern, but I also think you're overestimating the useful scope of a collect statement. For small functions I could see that being the case, certainly, but I also think it can make those functions much cleaner, ala the GetSomething() example.
I feel like it wouldn't be any more or less bad than switch statements, which also get used to keep a bunch of if statements from muddying the flow of the code. You trade an extra level of indentation for clarity elsewhere.
I personally like the indentation because it helps me visually parse the code. I think the vast majority of the code inside collect statements would be a series of function calls, and I think being able to read through that series of calls linearly without having to skip over interspersed nil checks etc. makes it much easier to follow the intended path of the code, despite the extra indentation.

I get the overall idea but this proposal as is introduces 2 new elements with a special handling flow not obvious at first sight, it seems a lot to reduce the burden of error handling to me.

@antham

this proposal as is introduces 2 new elements with a special handling flow not obvious at first sight

I don't think I agree with this assessment. Go has unique keywords that aren't "obvious at first sight" to someone who has yet to learn the language, but that become de rigueur very quickly. go, for example, suggests that it may "go" do something, but what that actually is isn't obvious until you read about it. It does introduce one new construct with a handling flow that differs from the rest of the code... like switch statements, or select statements. I actually think if you'd never read a collect statement before, or this proposal, you might still have a decent idea what was going on.
The choice of keyword collect is certainly up for discussion. There may be a more descriptive keyword that could be chosen. I can see the term collect being unclear. Perhaps collapse or until would be more clear.

it seems a lot to reduce the burden of error handling to me

It's meant to address far more than error handling. In the original proposal I put up the idea that the underlying problem that people want addressed when they complain about error handling is really the handling of nil values in go. It's a new control flow statement that is meant to allow go developers to write more concise and linearly readable code when dealing with potentially nil values. I believe the savings in lines of code more than justifies the addition of a single keyword. _! is not a breaking change, because it's not currently a valid identifier, and it would even be possible to introduce a keyword including an illegal character like ! so that the addition of the keyword isn't a breaking change.
I would be interested in your thoughts after going through some of your code and re-writing to use collect statements. What are the savings in lines of code? Is it more / less than I expect? Does a side-by-side comparison of the old version demonstrate that the collect version is easier or harder to reason about (assuming the reader knows what collect does)?
That would help me understand how others would think of and use this construct. I'm admittedly close to it, as I've spent some time whittling down versions of this idea until I came to this version. Outside perspective is always helpful.

switch statement exists in many languages so I don't think it's a good counter example, most of the people know this statement and the way it works.
Concerning select and go they handle asynchronous code, so how they are working is related to the purpose they are intended for, the flow is special but to handle a non orthogonal thing which is concurrency.

It's meant to address far more than error handling. In the original proposal I put up the idea that the underlying problem that people want addressed when they complain about error handling is really the handling of nil values in go.

I missed this point from your original proposition, thanks to enlighten me.

I would be interested in your thoughts after going through some of your code and re-writing to use collect statements. What are the savings in lines of code? Is it more / less than I expect? Does a side-by-side comparison of the old version demonstrate that the collect version is easier or harder to reason about (assuming the reader knows what collect does)?

Sure, it saves lines and the code is more concise, I have nothing to say against that. What I wanted to say with my comment is, the changes this proposal wants to introduce seems big to me regarding what they solve. In my opinion, it doesn't seem to fit what I understand from the go philosophy.

Regarding the !_ of which @deanveloper suggested could be identifiers, what if identifiers were prefixed with ! to indicate a != nil check? For instance, data, !err := somefunction() or !err1, !err2 := somefunction()
If you did this, you could also (and I'm deviating a bit here from the original proposal) use labels instead of defining which variables are being collected
  var err error
  var data []byte
  collect MyCoolLabel {
    data, !err = somefunction()
  }
MyCoolLabel:
  return err

Perhaps there could be different prefixes for different purposes:

! goto label if not nil (must be nil-able type)
? goto label if false (must be bool type)

@mccolljr in response to your question earlier about the use for multiple not-nil checks (_!, _! := fetchOrNil1()), I hadn't actually thought of one before I wrote the example. However, if would enable checking multiple error objects if that was useful for any reason.

@mccolljr

The choice of keyword collect is certainly up for discussion. There may be a more descriptive keyword that could be chosen. I can see the term collect being unclear. Perhaps collapse or until would be more clear.

At first I thought until was really good, but then I had realized that until is used by several other languages as a type of loop. May seem confusing. Also, collapse just feels unintuitive.
Also, in terms of !err... definitely not. It should be err!, as "!" Expression is already used for inverting bools and would definitely be confusing.
Also, labels are a bad idea. They make messy code imo, I really like the original syntax with collect Variable and _! much better.
I would like to point out that this collect pattern is very similar to try/catch, which I didn't like at first...
try {
    // several io operations
} catch (Exception err) {
    // handle error
}

versus
var err error
collect err {
    // several io operations
}
if err != nil {
    // handle error
}

BUT the redeeming thing in Go is that both options are available.
The problem with try/catch is for pesky one-liners which have a lot of boilerplate if you use try/catch but in Go we would still have if err != nil to handle one-liners, which is much more useful.

I think the similarity with try/catch is purely visual. panic/defer, functionally speaking, is Go's closest equivalent to try/catch. collect lacks any and all ability to affect the flow of the program outside of the function in which it is defined, which I think is a very key difference.

It's hard for me to evaluate your proposal because your examples that use the current Go language are written in a style that I find unusual and unnecessarily complicated.
The second code listing in the initial post could be rewritten to be something like this:
func TryComplexOperation() (*Result, error) {
	result1, err := Step1()
	if err != nil {
		return nil, err
	}
	result2, err := Step2(result1, "something")
	if err != nil {
		return nil, err
	}
	return Step3(result2, 12)
}

And this is the same for the second example in the third post.
I don't see a benefit to adding collect based on this example. It doesn't show off the power of collect. Rather, it shows how awkward Go code can be made better. (The obvious response to that is "Don't write awkward Go code in the first place.")
To make a convincing case fo collect (to me, at least) you'll need to show a problematic function (or ideally many such functions) written in today's idiomatic Go, and how that function would be improved with the use of collect.

@mccolljr

I think the similarity with try/catch is purely visual.

The point I was making is that I really don't like preambles, I should have been more clear about that. I want to think about handling an error after (or when) it happens, not before. I hate typing out a function, then when I realize I need to error check, need to go above (for the try), below (for the catch), and in front of (to indent it) just to check for errors. Again, because we still have if err != nil, which is purely after the error-prone function, this isn't a problem.

@deanveloper If I understand what you're saying, I'd argue that still counts as being "visual". If you're using vim, for example, you can type Otry to put a "try" above the line, a plugin indents the rest for you, 2ja}catch from normal mode and you're golden.
Of course, a language shouldn't require you to have a fancy editor, but my point is that I don't think a language should really care about stuff like that.
Another example that I think will make my argument more concrete is that it's relatively easy to preempt when you need some kind of error handler. The same thought process is necessary when writing loops - rarely do you write the code to operate on an array element before realizing that you need a loop structure.

I have a very rudimentary implementation of collect statements: https://github.com/mccolljr/go/tree/collect
The error messages are not great, although better than I expected before doing anything to correct.
If you want to test it out, clone the repo, switch to the collect branch, and build as you would normally install build go from source. Make sure you set the GOBINDIR environment variable to the directory you want these new go binaries to end up in.
Notes

The build will succeed, but there are some test that fail towards the very end of all.bash. I'll get to those ASAP, but for now the go binaries will still have been built, and you can still use them to build & run code using collect statements. The failed tests don't appear significant at first glance.
Keyword collect is implemented as __collect__ in this implementation, for testing & to avoid collisions.
You can't currently use _! on the left-hand side of a := statement, only =. I made this decision because it made some problem easier to solve, I'm just drawing a blank as to what it was. It's not a permanent solution, it just helped in getting a reasonably working implementation faster.
Some tools will not work with this fork (as expected). guru still works in most cases, autocomplete via gocode still seems mostly there, but gofmt and goimports fail miserably.

Here's a sample program, I've been using to test:
package main

import "io/ioutil"
import "flag"
import "fmt"
import "encoding/json"
import "os"

import _ "github.com/mattn/go-sqlite3"
import "database/sql"

type Conf struct {
	Server   struct {
		Host string
		Port string
	}

	Database struct {
		Name string
	}
}

var (
	CONF Conf
	DB   *sql.DB
)

func LoadConf() error {
	var (
		err  error
		data []byte
	)

	__collect__ err {
		data, _! = ioutil.ReadFile("conf.json")
		      _! = json.Unmarshal(data, &CONF)
	}

	return err
}

func ParseFlags() error {
	flag.StringVar(&CONF.Server.Host, "sh", CONF.Server.Host, "-sh=<IP_ADDRESS>")
	flag.StringVar(&CONF.Server.Port, "sp", CONF.Server.Port, "-sp=8080")
	flag.StringVar(&CONF.Database.Name, "db", CONF.Database.Name, "-db=\":memory:\"")
	flag.Parse()
	return nil
}

func LaunchDB() (err error) {
	__collect__ err {
		DB, _! = sql.Open("sqlite3", CONF.Database.Name)
		    _! = DB.Ping()
	}
	return
}

func main() {
	var err error

	// perform initialization
	__collect__ err {
		_! = LoadConf()
		_! = ParseFlags()
		_! = LaunchDB()
	}

	// exit on error
	if err != nil {
		fmt.Println("fatal:", err)
		os.Exit(1)
	}

	fmt.Printf("%+v\n", CONF)
}

@adg You're definitely right about the current examples being poor.
I've got a fork with this implemented enough to test, and I'm going to convert some of my existing code to get better example material.
If you want, you can install my fork and play with it as well.

@mccolljr
Even if switch can have capabilities that you need to know that's right, the overall behaviour is mostly what you could expect from a switch.
The same for the if and for statement, there are things particular you need to learn yes, but mostly they behave like in other languages, no suprise about it.
When you say

they provide shorthand syntax for very specific cases, or introduce a lot of new visual complexity to the language

I feel the same when I look at the example you provided, the flow is not as straightforward as a list of if check on error. You need to switch your mind to understand the way things behave in a collect block, "only" (I'm not saracastic as well) to handle nil value where plain if check are really expressive and without any ambiguity.
I can't say more regarding to my original comment.

@smasher164
I like that in concept but you'd have to define more explicitly what the collect function does. I don't like the idea of adding the overhead of closures and function calls just for some visual clarity.
A general note - I've been reworking my fork into a much cleaner and more general state. I will update here when I have it building and passing all tests, stay tuned.

Did you consider try as the keyword instead of collect? If so, why the latter?
This feature tries things until a failure. Also I imagine that try is clearer to those with C++, Java, or Javascript chops.

It may be helpful to extend the proposed construct to support two kinds of errors:

expected/manageable: as proposed
unexpected/fatal: execute log.Fatal(), panic(), or user-defined func

try err, log.Fatal { // optional 2nd arg is a func taking argument of type error
   _! = f1() // any error lands in err
   _# = f2() // any error is passed to log.Fatal
}

Many APIs (e.g. os.File) can report catastrophic errors, which should terminate the program. It would be nice to avoid this:
try err {
   _! = f1()
   ebad := f2()
   if ebad != nil {
      log.Fatal(ebad)
   }
}

@mccolljr, @mikeschinkel  for clarity I formalized my syntax suggestion and split it out as a new proposal. I suppose it's different enough that it should not be discussed in detail here. Sorry for the spam, and thank you for the inspiration!

@smyrman

" A problem with your example @mikeschinkel is the :=, as it looks like err inside the break scope is shadowed, and would therefore be ignored by the break check."

Good point. But it was not shadowed, it was just a typo.  Fixed.
BTW, I can get on board with @networkimprov's keyword try instead of break as it would be just as good a word to you IMO.

it's not really clear when the break/collect/try check should be re-calculated. For every line? For every semicolon? Every time either of the variables in the check are updated? What if you send a pointer to the variable to a sub-scope or function? What is a line in the eyes of the compiler?"

Yes, I can see that. I made an assumption that it would be obvious when it would be recalculated, but obvious I was wrong in thinking it would be obvious.  You are correct that lines are not appropriate, which is why I was envisioning it would break from the scope when err was assigned a not-nil value.
This could be done by the compiler implementing assignment to err as a closure with a guard condition and any assignment would thus trigger a break out of the scope. Easy peasy.
As for the explicit _! syntax that is what I object to most. _ is basically /dev/null and so what does /dev/null! mean?  How does an '!' modify a special variable implemented to allow values assigned to it disappear?  You say its not clear what break err!=nil means? I'd say it would be a lot easier to learn and remember that it breaks on change in the expression rather than to grok something that fundamentally changes the meaning of the special _ variable name.
Further use of _! adds yet another "magic symbol" to the language and squats on a syntax that could not reasonably be in any other scope in the future in the case that a better, more obvious use for that syntax were to emerge whereas what I proposed would be explicitly limited to the scope it was used with.
BTW, I have been using the following construct for a while to address the use-case that @mccolljr's proposal appears to be trying to address, and it works extremely well if not rather verbose. So my inline proposal was simply using syntax sugar (and I am a big fan of syntax sugar) to do what I have already proven works extremely well in practice:
var err error
for {
	err = foo()
	if err!=nil {
		break
	}
	num,err := bar()
	if err!=nil {
		break
	}
	num,str,err := baz()
	if err!=nil {
		break
	}
	if something_else_is_wrong() {
		err= errors.New("Something else is wrong!")
		break
	}
	fmt.Printf("Num: %v and Str: %v", num, str )
             break
}
if err != nil {
	fmt.Printf("Oh, the HORROR!" )
}

@mikeschinkel, that's some nice points. This reply first showed up for me after I creating #27075. Feel free to continue the discussion there.

@adg, the most benefit is in situations where you handle the error in the same function:
func F(stream) {
   var a, b, c T
   var err error

   a, err = f1()
   if err == nil {
      b, err = f2()
      if err == nil {
         c, err = f3()
         // this pattern is hard to read and may continue to great depth
         // using if err != nil { goto HandleError } may be preferable to indenting
      }
   }
   if err != nil {
      fmt.Fprintf(stream, "error %s", err.Error())
   } else {
      fmt.Fprintf(stream, "result %d", a+b+c)
   }
}

versus:
try err {
   a, _! = f1()
   b, _! = f2()
   c, _! = f3()
}
if err != nil {
   ...
}

And it's better not to read three lines of ceremonial boilerplate for each function call when returning errors:
a, err = f1()
if err != nil {
   return err
}
b, err = f2()
if err != nil {
   ...

@networkimprov I really like your syntax, but not the adoption of the _! construct. Better to keep it simple like so:
try err {
   a, err := f1()
   b, err := f2()
   c, err := f3()
}
if err != nil {
   ...
}

@networkimprov The problem with the suggestion of using a triggering expression to implicitly define possible checking points is that it's hard in general to detect all possible such points statically.  For example:
try err != nil {
    err = foo()  // this is one
    p = &err
    q := &otherErr
    if someComplicatedCondition() {
        q = p
    }
    *q = foo() // is this one?
}

Once you take the address of err, all of the assignments to it become hard to track. They can happen in other functions, even.
You either have to give up and say assignments by pointer don't count, or enforce that you can't take the address of any variable in a condition (and also foo can't be a global variable).  Either seems unsatisfactory.

@randall77 the triggering expression was suggested by @smyrman, and is currently under discussion in https://github.com/golang/go/issues/27075
My suggestions were keyword try, and flagging fatal errors with _#.

I haven't been here in a while, but I'd like to give an update with my experience. Work has gotten busy again so I had to put work on this on hold for a while. That being said, I found as I was writing test cases that I use this feature far less than I expected I might. As mentioned by someone else (I apologize, not sure who at the moment), this does tend to add an extra level of indentation to many functions... something I'm not too jazzed about in practice. I see a lot of suggestions in this thread and I will read through them when I have a little more time, but as it stands I actually feel my proposal, having tried it in the wild, is not as helpful I once thought it to be.

A better solution, perhaps, would be to allow a block of go code to break with a value. Maybe (100% spitballing):
result, err := {
    e, temp1 := trySomething()
    break (nil, e) if e // implicitly means "if e is non-zero"
    e, temp2 := tryOther(temp1)
    break (nil, e) if e
    break (alwaysOk(temp2), nil) // no if here, always break
    // if no break were to be called, the values assigned from the block would contain their zero values
}

Still, though, I don't think it's as helpful of a construct as I had originally thought

FYI: There are some pretty similar semantics to collect in @rsc's draft proposal here with a handler keyword (similar to defer for registering error handler) and a corresponding explicit check keyword:

https://go.googlesource.com/proposal/+/master/design/go2draft.md - See Error Handling

One interested detail seams to be a suggestion that this could work:
func printSum(a, b string) error {
	handle err { return err }
	fmt.Println("result:", check strconv.Atoi(x) + check strconv.Atoi(y))
	return nil
}

A check applies to an expression of type error or a function call returning a list of values ending in a value of type error.

The collect block is essentially try, and the new check/handle proposal is essentially catch, so they're similar ideas at root. Meanwhile I'm working on a revision of that, the alternate handler concept


Add Particle Culling

Found here:
https://www.curseforge.com/minecraft/mc-mods/particle-culling

added!


Netflix Clone searchbar is not working

Describe the bug
A clear and concise description of what the bug is.
The search bar in Netflix Clone doesn't give any results when filled.
As a participant of GSSOC'21 , I would like to work on this issue.

@sanjaysunil34 Its working for me. Can you please show me the demo?


Add an event router

It would help with servers that receives webhook events.

While dispatching on an event itself is straightforward, dispatching another level down is harder as there is no consistency in the payload of an event. Probably will want to accept **kwargs, limit it to 1 item, and then dispatch on that.
As for when one dispatches, an option is to take *args and/or **kwargs past the event and then pass them down to the handler(s).

One thing to potentially remember is the route should be flexible enough to not be global in case different URL endpoints in the same server need to dispatch to different functions.

This will probably live in gidgethub.routing.
Looking at the routers for Django, Flask, and Bottle, it seems Flask and Bottle use decorators while all three support explicit specifying of the routes in a sequence. My current thinking is leaning towards a Router class whose instances can either have a route added through a method call or through a decorator. A Router instance can also accept other routes to copy things over. That way a submodule can set up routes in a module-level router attribute which an application-level router can then take in to copy over its routes to build the larger router. So in the submodule you could do:
router = Router()

@router.register("comment", action="added")
async def foo(event):
    pass

And in __main__:
router = Router(sub.router)
...
router.dispatch(event)

That allows the router  to be built up in parts where the dividing line is a module instead of at the actual bot level. This allows for less resource usage and better caching (if #20 is implemented).


Stepper Component

Hi you can check my custom stepper component we built on GWT-Material.
http://gwtmaterialdesign.github.io/gwt-material-demo/snapshot/#steppers
Resource Files
You can fork and add to Materializecss about the component.
http://codepen.io/kevzlou7979/pen/QyWbyr
Demo

It looks nice! Currently we are focused on bug fixing and stability so anyone can make a PR of this but it may take a while for us to get to it ourselves.

Yes you can check our updates on gwt-material about the horizontal stepper
and error feature
On Sat, Dec 5, 2015, 10:12 AM Alan Chang<EMAIL_ADDRESS>wrote:

It looks nice! Currently we are focused on bug fixing and stability so
anyone can make a PR of this but it may take a while for us to get to it
ourselves.
—
Reply to this email directly or view it on GitHub
https://github.com/Dogfalo/materialize/issues/2414#issuecomment-162131956
.

+1 for steppers!

+1

+1 @kevzlou7979

+1

+1

+1

+1

+1

Well, it's not perfect yet, but I made a small plugin =)
https://github.com/Kinark/Materialize-stepper
I hope you like it!

+1

+1

Have a look at this


Fix typo in README.md: is_confort_noise --> is_comfort_noise

@googlebot #trivial
(I am not a lawyer, but I recall reading somewhere that trivial changes do NOT require a CLA.  Have a nice day!)

Thanks for the PR.   I believe we do need the CLA in these cases as well.


Use Redis as ephemeral storage for JTI

What is the context of this PR?
This PR introduces Redis as a storage backend for JTI claims.
This also disables the unique checking within the datastore implementation as it has its limitations.
How to review
Test that you are still able to launch a survey.
Check that the expiry is working and that records are being removed from the Redis instance
Related to https://github.com/ONSdigital/census-eq-terraform/pull/17

So, I've been able to run this successfully with the expiry reduced and it correctly expires as expected. Not sure how I'd check that redis is cleaning up expired tokens though.

@iwootten in GCP it shows your the key count in the DB. you can see it reducing over time

@MebinAbraham Yeh i was unsure about where to put that and ended up leaving it in terraform as then it has access to the values

https://github.com/ONSdigital/census-eq-terraform/pull/17


feat(changelogs): apply prettier formatting on changelogs text

Background
This PR adds a new feature, apply prettier formatting on the generated changelogs text
Checklist

[ ] The Pull Request has been fully tested
[x] There's an entry in the CHANGELOGS
[ ] There is a user-facing docs PR against https://github.com/daeuniverse/dae

Full changelogs

feat(changelogs): apply prettier formatting on changelogs text

Issue reference
NA
Test Result
NA

Please do npm run bundle before we merge it.

Please do npm run bundle before we merge it.

No worries, already done that


[Fix-6228][UI] fix yarn queue dialog bug

Fix #6228
Purpose of the pull request

Brief change log

Verify this pull request

This pull request is code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:

Codecov Report

Merging #6264 (bd4e8f4) into dev (fab4eae) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##                dev    #6264   +/-   ##
=========================================
  Coverage     38.71%   38.71%
  Complexity     3212     3212
=========================================
  Files           637      637
  Lines         25321    25321
  Branches       2752     2752
=========================================
  Hits           9804     9804
  Misses        14638    14638
  Partials        879      879

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fab4eae...bd4e8f4. Read the comment docs.


Add new user guide URL to Help menu

Resolve #43

Codecov Report

Merging #44 (7fbdd48) into master (72a1451) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #44   +/-   ##
=========================================
  Coverage     72.15%   72.15%
  Complexity      399      399
=========================================
  Files            70       70
  Lines          1232     1232
  Branches        125      125
=========================================
  Hits            889      889
  Misses          311      311
  Partials         32       32

Impacted Files
Coverage Δ

src/main/java/seedu/address/ui/HelpWindow.java
0.00% <ø> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Reopened PR due to new integration with package name (foodwhere)

Reopened PR due to new integration with package name (foodwhere)


SVG Draw produces very low-res PNG

Description
I am using the following code to render an SVG on top of an image:
        private async Task<Stream> GetStampedImageStreamAsync(Stream sourceStream, GetWatermarkResponseSettings watermarkSettings, bool withSpecial)
        {
            var destinationStream = new MemoryStream();

            using (var assetImg = Image.FromStream(sourceStream))
            {
                var svg = string.Empty;

                if (watermarkSettings.IsText)
                {
                    svg = GetTextWatermarkSvg(assetImg, watermarkSettings, withSpecial);
                }
                else
                {
                    svg = await GetImageWatermarkSvgAsync(assetImg, watermarkSettings, withSpecial);
                }

                _logger.Log($"SVG: {svg}");

                var svgDocument = Svg.SvgDocument.FromSvg<Svg.SvgDocument>(svg);

                using (var g = Graphics.FromImage(assetImg))
                //using (var wm = svgDocument.Draw(assetImg.Width, assetImg.Height))
                {
                    //g.DrawImage(wm, 0, 0, assetImg.Width, assetImg.Height);
                    svgDocument.Draw(g);
                    assetImg.Save(destinationStream, assetImg.RawFormat);
                }

                return destinationStream;
            }
        }

The SVG contains some text as a watermark.  On simple SVG rendering websites (such as here), the text all renders correctly.  An example PNG is attached of the expected outcome:

However, attached is the PNG that ends up being rendered and saved:

Notice that the text is cut off after a certain width.  Also, the text is slightly larger than expected, and in certain views, is more pixelated/has more anti-aliasing/is more bold (this is reminiscent of #253).  Both images are 1920 x 1080.  The PNG is also much smaller then I would expect (26KB as opposed to 118KB).  If there is some sort of compression going on that is causing a problem, I would like to remove that - but I don't see where or how that would be happening.  The stream to which the SVG is writing is just a clear BMP of the same dimensions.
Most importantly, I need the text to not be cut off.  Other factors, such as the anti-aliasing and font sizing, are fudgeable if need be.
Example data
Here is the SVG in question:
<svg style="align-items: baseline; display: flex; height: auto;" height="1080px" width="1920px" xmlns="http://www.w3.org/2000/svg">
<text style="fill: #fff; font-family: Arial, sans-serif; text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);" font-size="87.35545px" x="386.5" y="488.5">Karl Collection Watermark</text>
<text style="fill: #fff; font-family: Arial, sans-serif; text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);" font-size="16px" x="0" y="0">
<tspan style="fill: #fff; font-family: Arial, sans-serif; text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);" font-size="16px" dy="1.4em" x="0">Karl Speer</tspan>
<tspan style="fill: #fff; font-family: Arial, sans-serif; text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);" font-size="16px" dy="1.4em"<EMAIL_ADDRESS><tspan style="fill: #fff; font-family: Arial, sans-serif; text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);" font-size="16px" dy="1.4em" x="0"><IP_ADDRESS></tspan>
<tspan style="fill: #fff; font-family: Arial, sans-serif; text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);" font-size="16px" dy="1.4em" x="0">Sample Organization</tspan>
<tspan style="fill: #fff; font-family: Arial, sans-serif; text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);" font-size="16px" dy="1.4em" x="0">Karl's Account</tspan>
<tspan style="fill: #fff; font-family: Arial, sans-serif; text-shadow: 0 1px 2px rgba(0, 0, 0, 0.6);" font-size="16px" dy="1.4em" x="0">This collection belongs to Karl</tspan>
</text>
</svg>

Nothing complicated - the overall height and width is matched to the original image size.
Used Versions
I am using the latest SVG package (3.3.0) in a dotnet core 3.1 lambda container (public.ecr.aws/lambda/dotnet:core3.1) with libgdiplus installed.  The container is using CentOS.7 (64-bit).

An interesting development: when run on a Windows machine via a console app, the SVG renders correctly, and the text is not cut off.  This may indicate that the issue is with running in CentOS container - which would be very unfortunate for us, as we need that particular OS to run this container as a Lambda with AWS.

This may be an issue with libgdiplus. It could help if someone could test this under other OSes (Ubuntu, MacOS) to see if it is a general *NIX problem, or specific to CentOS.

Same Problem (text cutoff) on Arch using SVG 3.4.4


fix the confirmation dialog

Description
Fixing the confirmation deletion dialog to make it conform to the design system of the application.

Related Issue

Motivation and Context

How Has This Been Tested?

Checklist:

[ ] My code follows the style guidelines of this project
[ ] I have performed a self-review of my code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works

Screenshots (if appropriate):

This is a little bit lengthy

It's not fixed yet @uKiJo

Replace "Yes I am sure " with Delete

Fix linting and we are good to go

Also check #62

Fix linting and we are good to go

i don't have those error when running the linter script...


how to get mAP of different scale ?

hello, thank you for very great work ,and i have a question is that ,how to get the mAP of  different scale ?

hello, what do you mean by different scale? You mean a different overlap (IoU)?

no,@Cartucho,just like mAPs,mAPm,mAPL

you mean small, medium and large objects? Like in the YOLOv3 paper?

Yes. Do we have such feature support? Thanks!

following :)


 Sourasish Mitra
What happened?
Period Calculator returning wrong values.
Also not working on pressing Enter on keyboard.

@nisha331
I accidentally submit this 15 mins before.
can u assign me now?

Yes smitra you can work on it .

I want to work on this issue under JWOC


Reuse _TORCHVISION_AVAILABLE in examples

What does this PR do?
We are defining _TORCHVISION_AVAILABLE twice
Before submitting

[ ] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[n/a] Did you make sure to update the documentation with your changes? (if necessary)
[n/a] Did you write any new necessary tests? (not for typos and docs)
[n/a] Did you verify new and existing tests pass locally with your changes?
[n/a] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

if so, should we move torchvision to be an extra requirement to simplify the core lightning dependencies?

torchvision is not in the core dependencies
https://github.com/PyTorchLightning/pytorch-lightning/blob/master/requirements.txt

I would probably in the init just import from Pl and later in all examples hold import to this imported var

In that case we would just be avoiding one single function call. If you prefer it that way to having it defined in a single place, we can close this PR


"Răchiți", "Cișmea", "Costești" and "Roșiori" are listed in the drop-down menu

Description:
"Răchiți", "Cișmea", "Costești" and "Roșiori" are listed in the drop-down menu alongside its corresponding county and commune, according to the law
Precondition:
The website is up an running.

Step 1
Type "Răchiți" in the search bar.

Expected results
The location is listed in the drop-down menu as "Răchiți, Botoșani (Răchiți)".

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Type "Cișmea" in the search bar.

Expected results
The location is listed in the drop-down menu as "Cișmea, Botoșani (Răchiți)".

Step 4
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 5
Type "Costești" in the search bar.

Expected results
The location is listed in the drop-down menu as "Costești, Botoșani (Răchiți)".

Step 6
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 7
Type "Botoșani" in the search bar.

Expected results
The location is listed in the drop-down menu as "Botoșani, Botoșani (Botoșani)".


Can ELMO return a list of similar words?

See attachment. So I tried word2vec. I enter 'swim' and it returns me a list of related/similar words like paddle, row, swimming, swam, etc.
Now I want to see what Bert/Elmo return me. But will they return me a list? Or do they just compare 2 inputs face to face? I DON'T want the latter. I want a list returned, of inter-exchangeable alternative words.
Also, if it can do this, I'd be thankful if someone here could snapshot me the list of similar words to 'swim' for Bert/Elmo. As I can't install these guys, too hard.

This is not possible with BERT / ELMo.
BERT / ELMo are contextualized word embeddings, i.e., the embedding for a words depends on the surrounding words in a sentence.
Example:
Can I swim here?
Don't swim in the pool
In these two sentences, the vector for swim will be computed on the fly by the other words in the sentence. Hence, swim will have in the two sentences two different vectors.
The goal is that words in a similar context have similar vectors, i.e. the two swims in the above sentence should be close in vector space. But if swim is used like this:
Swim is a new app for car sharing.
Then this Swim should get a different vector, a vector that is maybe close to Uber or Lyft.


Bulk ingest docs - 1st PR

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have signed the Adobe Open Source CLA.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

+1 This is exciting. Thank you for adding these docs!


Popup: expose reference for react-popper instance

Feature Request
Problem description
I want to recalculate Popup position when the size of content is changing.
However, it seems like there's no easy way to achieve this without having popper instance.
https://github.com/FezVrasta/react-popper/issues/274
Proposed solution

simple way: Just add reference prop for <Popper /> element to the PopupProps
complex way: Add feature to recalculate popup position to the Popup element.

I don't want to expose it with renderProps because it will be not aligned with existing API. Rather it can be positionDependencies that is an array:
const myDataFromServer = [...]

return <Popup positionDependencies=[myDataFromServer.length] />
// or
return <Popup positionDependencies=[myDataFromServer.queryId, isFetched] />

It's aligned with dependencies idea in React hooks. What do you think? 🤔

Yay, hook-like dependency prop! What a great idea!

@rokoroku can you take a look on #3657 and provide some feedback? 🐱


[Spy mode]  The id of State is changing

In Spy mode the Id's of the state is changing all the time between two identifications.
It's blinking.
This is for all states.

It seems that the text "LESSSYSTEM" appears and disappears in the Title of the SUT.
This causes the existence of two widgets, one with that text and the other without the text in the Title property.
Therefore, the State Identificator constantly changes, including or not "LESSSYSTEM" text like a property.

Changing the ID due to the changed Title is not a bug but a confusing aspect of Spy mode for users.
The development from #220 will allow users to customize displayed properties.

The ID of the state was changing due to the dynamic Title property.
This is the expected behavior of the State ID calculation but a dynamic aspect of the SUT.


set x-ms-client-name

MSFT employees can try out our new experience at OpenAPI Hub  - one location for using our validation tools and finding your workflow.

Contribution checklist:

[ ] I commit to follow the Breaking Change Policy of “no breaking changes
[ ] I have reviewed the documentation for the workflow.
[ ] Validation tools were run on swagger spec(s) and errors have all been fixed in this PR. How to fix?

If any further question about AME onboarding or validation tools, please view the FAQ.
ARM API Review Checklist

[ ] Ensure to check this box if one of the following scenarios meet updates in the PR, so that label “WaitForARMFeedback” will be added automatically to involve ARM API Review. Failure to comply may result in delays for manifest application. Note this does not apply to data plane APIs, all “removals” and “adding a new property” no more require ARM API review.

Adding new API(s)
Adding a new API version
Adding a new service

[ ] Please ensure you've reviewed following guidelines including ARM resource provider contract and REST guidelines. Estimated time (4 hours). This is required before you can request review from ARM API Review board.

[ ] If you are blocked on ARM review and want to get the PR merged with urgency, please get the ARM oncall for reviews (RP Manifest Approvers team under Azure Resource Manager service) from IcM and reach out to them.

Breaking Change Review Checklist
If there are following updates in the PR, ensure to request an approval from API Review Board as defined in the Breaking Change Policy.

[ ] Removing API(s) in stable version
[ ] Removing properties in stable version
[ ] Removing API version(s) in stable version
[ ] Updating API in stable version with Breaking Change Validation errors
[ ] Updating API(s) in preview over 1 year

Please follow the link to find more details on PR review process.

/azp run automation - sdk

for fix issue: https://github.com/Azure/azure-sdk-for-python/issues/14150

Hi @hyonholee @grizzlytheodore  would you help to review this PR?
This is a fix for issue https://github.com/Azure/azure-sdk-for-python/issues/14150.
The reported Python SDK breaking change comes from previous PRs, don't for this PR.

hi @dingmeng-xue will Powershell generator read x-ms-client-name from Swagger?

hi @dingmeng-xue will Powershell generator read x-ms-client-name from Swagger?

Yes.


remark@next (13)

Hi!
remark is switching to a new parser internally (micromark, see remarkjs/remark#536 for more info).
From a quick glance at the code, it seems this plugin should be fine. However, it would be good to check that in the future (there is 13.0.0-alpha.0 published now, and one or two more will come before the release)

@wooorm Thanks for the heads up. I will check if everything still works correctly.


1.19 Update

Hello,
Sorry to bother you. I was trying to update your lib on 1.19 but unfortunately I run into some issue with the pose. I don't really get how to make it work ^^' Looks like it needs an OptionalInt instead of the actual pose. I don't really understand what's the issue, and the little website for the remap is down so I can't really look into it ^^'. So,  I was wondering if you could update the lib to 1.19. Thanks

Yes, I have been working on the 1.19 NPC util with additional functions.
However, since 1.19 is a bit different, I have to test every functionality before publishing the newest version.
Therefore, it will take some time for me to finish it, but I am aiming for a release before Friday.

Thanks a lot for your hard work!

Hey ZeTioZ
Good news. I will release the NPC-1.19 utility tonight before 12 am CEST.
Thank you for your patience.
Kind regards,
Daniël.

Thanks a lot !


feat: replicate boundary vectors to multiple partitions

just one concern: can we define compute_partitions in terms of compute_multiple_partitions instead of duplicating the code?
same for compute_multiple_memberships

sure we can, just duplicated the code for now because compute_partitions is extremely optimized and I paid as less as possible effort to get this feature worked first.
will remove the dup later


GoogleDrive is Link failure

Hello, the link to the model seems to be broken, Can you update it?

@meteorshowers Hi Li, would you please update the GoogleDrive link? Its broken


Close SQLite connection if session is deleted and thread is still running

Here is one possible solution for #187. This works based on the logic of aiosqlite.Connection.run() (implementation of Thread.run()). It can be forced to exit without blocking or scheduling any tasks by setting an internal flag and emptying its task queue. The underlying sqlite3.Connection object will not be closed (which is a blocking operation), but in my experience this doesn't cause any problems.
This will only be used in the case where the SQLiteCache object is being garbage-collected and the connection has not yet been closed implicitly (via contextmanager exit) or explicitly (via close()).
This does have the potential for data loss, if somehow the session or cache objects are deleted while async tasks have been created but not awaited. This seems unlikely to happen in practice, and I think it's a reasonable tradeoff, but it does leave some room for possible future improvement.

Hey @JWCook! I think this solves most of #187. The thread can still hang if the process receives a SigInt (ctrl + c), but I am not sure how much can be done to avoid that.
I pulled the branch and ran the sqlite integration tests locally and pytest showing a RuntimeWarning being thrown in test_concurrent_bulk_commit. This looks like it has to do with the interaction between the async context manager, self.init_cache(), and bulk_commit_items coroutine which captures the context manager. See below:
pytest test/integration/test_sqlite.py

test/integration/test_sqlite.py::TestSQLiteCache::test_concurrent_bulk_commit
  /home/test/aiohttp-client-cache/sqlite-nocontext-autoclose/aiohttp_client_cache/backends/sqlite.py:116: RuntimeWarning: coroutine 'AsyncMockMixin._execute_mock_call' was never awaited
    self._connection._tx.queue.clear()
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

At least on my machine, wrapping self.init_cache() and bulk_commit_items in a async context manager resolved the issue.
@@ -53,17 +53,24 @@ class TestSQLiteCache(BaseStorageTest):
         mock_connection = AsyncMock()
         mock_sqlite.connect = AsyncMock(return_value=mock_connection)

-        async with self.init_cache() as cache:
+        from contextlib import asynccontextmanager
+
+        @asynccontextmanager
+        async def bulk_commit_ctx():
+            async with self.init_cache() as cache:
+
+                async def bulk_commit_items(n_items):
+                    async with cache.bulk_commit():
+                        for i in range(n_items):
+                            await cache.write(f'key_{n_items}_{i}', f'value_{i}')
+                yield bulk_commit_items

-            async def bulk_commit_items(n_items):
-                async with cache.bulk_commit():
-                    for i in range(n_items):
-                        await cache.write(f'key_{n_items}_{i}', f'value_{i}')
+        async with bulk_commit_ctx() as bulk_commit_items:

-        assert mock_connection.commit.call_count == 1
-        tasks = [asyncio.create_task(bulk_commit_items(n)) for n in [10, 100, 1000, 10000]]
-        await asyncio.gather(*tasks)
-        assert mock_connection.commit.call_count == 5
+            assert mock_connection.commit.call_count == 1
+            tasks = [asyncio.create_task(bulk_commit_items(n)) for n in [10, 100, 1000, 10000]]
+            await asyncio.gather(*tasks)
+            assert mock_connection.commit.call_count == 5

While resolving the above warning, I ended up running pytest with the -s flag to print stdout and noticed in the log output two instances of ERROR:asyncio:Unclosed client session. I figured running pytest with tracemalloc enabled will show where this is occurring. I will look into this later this afternoon and report back.

While resolving the above warning, I ended up running pytest with the -s flag to print stdout and noticed in the log output two instances of ERROR:asyncio:Unclosed client session. I figured running pytest with tracemalloc enabled will show where this is occurring. I will look into this later this afternoon and report back.

I should have read more closely when reviewing the tests, I now see the comment in the  test_without_contextmanager test:
    async def test_without_contextmanager(self):
        """Test that the cache backend can be safely used without the CachedSession contextmanager.
        An "unclosed ClientSession" warning is expected here, however.

The thread can still hang if the process receives a SigInt (ctrl + c), but I am not sure how much can be done to avoid that.

I was thinking about that as well. Options would include handling KeyboardInterrupt in CachedSession.__aenter__(), or adding a custom handler for SIGINT. I would prefer to avoid adding that until that specific case becomes an issue for you (or someone else).

pytest showing a RuntimeWarning being thrown in test_concurrent_bulk_commit

I wasn't able to reproduce this locally, but this makes sense, so I applied your patch. Thanks for the suggestion!


added my profile README

Hey, @Prathamkrishna please add some more content in the readme and maybe add some dynamic widgets which will make it look good.

Yes please add some more content in the readme than either me or one of the maintainers will merge it!


Warnings During Global Install [fsevents] - Unsupported Platform

Bug Report or Feature Request (mark with an x)
- [ X ] bug report -> please search issues before submitting
- [ ] feature request

Versions.
@angular/cli: 1.4.9
node: 6.11.4
os: win32 x64
Repro steps.

Open command prompt
Run: npm install -g @angular/cli@latest

The log given by the failure.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules@angular\cli\node_modules\chokidar\node_modules\fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"win32","arch":"x64"})
Desired functionality.
No warnings or errors would be displayed after the install, just the dependency tree.
Mention any other details that might be useful.

This is just a warning that displays on windows. Install should be fine

Closing as answered by @deebloo. This is a fairly common question so I'm marking it as a FAQ.


Sorry to nit-pick, but if the build is explicitly private API, can it be its own top-level class, rather than being an inner class of the public API? I think it's confusing to downstream developers when we inter-mingle public and private API resources in the same class.
otherwise, +1

@ndimiduk I hear you, but honestly I feel like there are pros and cons on both sides of that. If I break it out, I'd need to make BalanceResponse's constructor package-private at least, and the org.apache.hadoop.hbase.client package is quite large. I'd probably also get rid of the convenience BalanceResponse.newBuilder() since putting that in BalanceResponseBuilder would be sort of redundant. So we'd change the access pattern to new BalanceResponseBuilder().build(). Not a huge deal, but it's less intuitive and now there's an asymmetry with BalanceRequest. I could do the same with BalanceRequest, but that's just expanding non-intuitiveness. I could leave the asymmetry, but I do think there's some value there and I feel like inner Builder classes are sort of conventional.
None of that is a non-starter, but I'm not sure if it's explicitly better than having the inner class be IA.Private, or even IA.Public. I can do it if you feel strongly about it, but just wanted to put that out there for consideration.

Sorry for the belated reply. @bbeaudreault my ask was to move BalanceResponse$Builder to be private, not BalanceRequest$Builder to be private. Users are expected to be able to use BalancerRequest, as this is on Admin#balance(BalanceRequest).
https://github.com/apache/hbase/blob/a15e94a47f06ae665a0c213d092761431e907eac/hbase-client/src/main/java/org/apache/hadoop/hbase/client/Admin.java#L847
My point was that the Master is the only thing which should be creating BalanceResponse objects, thus end users don't need to know about the BalanceResponse$Builder class. Because the constructors on BalanceRequest are private, users must use the Builder class.

Object
Client Visibility

BalanceRequest
Public

BalanceRequest$Builder
Public

BalanceResponse
Public

BalanceResponse$Builder
Private

Please correct me if I'm wrong. I think we want to revert the change to BalancerRequest$Builder and apply it to BalancerRespons$Builder instead.

Oh, thanks @joshelser for pointing this out.
The IA.Private class should be BalanceResponse, not BalanceRequest...

Ugh! You are right, I totally messed that up. I'll push a fix tomorrow. Sorry all


Update to phoenix_live_view 0.18

As stated in issue #2 the phoenix_live_view package moved to live_component/1 function to Phoenix.Component.
This PR creates a helper macro because we have to check for the version at compile time to conditionally import the Phoenixe.LiveView.Helper module.
I also updated the LiveView version and removed some warnings the new LiveView version generates about variable assigns (both not fundamentally necessary to fix the issue). See the following example:
warning: you are accessing the variable "toggle_terminal" inside a LiveView template.

Using variables in HEEx templates are discouraged as they disable change tracking. You are only allowed to access variables defined by Elixir control-flow structures, such as if/case/for, or those defined by the special attributes :let/:if/:for. If you are shadowing a variable defined outside of the template using a control-flow structure, you must choose a unique variable name within the template.

Instead of:

    def add(assigns) do
      result = assigns.a + assigns.b
      ~H"the result is: <%= result %>"
    end

You must do:

    def add(assigns) do
      assigns = assign(assigns, :result, assigns.a + assigns.b)
      ~H"the result is: <%= @result %>"
    end

  lib/terminal_button_component.ex:23: Underthehood.TerminalButtonComponent.render/1

fixes #2

I think you actually want "use Phoenix.LiveComponent", and so maybe the code should be roughly:
if (phoenix live view version checking ~> 0.18) do
  use Phoenix.LiveComponent
else
  use Phoenix.Component
  import Phoenix.LiveView.Helpers, only: [live_component: 1]
end

@kevinschweikert Thanks a lot for the PR - I'm sorry for getting back to you only now, some real-world issues kept me busy. 😔
Supporting LiveView 0.18 would be awesome, but I wonder if we can support both LiveView 0.17 and LiveView 0.18 at the same time. 🤔

Closing this, it was superseded by #16  -- underthehood 0.2.x will support LiveView 0.18.


NextStepManager.GetNextStep counts all previous actions taken against an resource even if they dont apply to a certain Policy

Rule Engine

OS Type: Linux
Java version:
1.8

Summary
Rule engine during fixing of a existing resource violation a policy will check the amount of actions already taken against the resource to be able to determine if the property:
pacman.auto.fix.max.email.notifications
has been reached. Once the max email notifications have been reached the fix method is invoked and applied.
If you have a resource that is a routine offender the following steps could occur:

Resource in violation created
Resource found by policy in violation
After max emails sent, Resource is fixed (in this case say deleted)
Resource with same identifier recreated
Resource found by policy in violation
Fix will pull the last actions based on compliance/v1/get-last-action/?resourceId=$resourceId
which will pull the actions taken on the first instance of the resource created.
Autofix will invoke the fix method without sending notifications

Reproduce steps

Create a resource that violates a policy where the fix is something like destroying the resource.
Once the resource has been destroyed by the policy recreate it with the same violation and resourceId
Resource will be destroyed without any notification emails sent about the second instance of the resource

Expected Results

Violated resource is sent notification emails up to the max notification property
executeFix executed
Resource recreated
Violation resource is sent notification emails up to the max notification property
executeFix executed

Actual Results

Violated resource is sent notification emails up to the max notification property
executeFix executed
Resource recreated
executeFix executed

This can also occur if you have two different policies enact on the same resource type and an certain resourceId is in violation of both policies. Example where the max notification is set to 2

Resource in violation of two different policies
Policy 1 runs, create issue, sends 1 notification
Policy 2 runs, creates issue, sends 1 notification
Policy 1 runs again, pulls get-last-actions, sees 2 emails sent for resourceId, executes executeFix early

Where you would expect each policy to send individual notifications of up to 2 each.
It looks like this is mostly caused by the pac_rule_engine_autofix_actions table not having any reference to what annotation/issue/autofixplan the actions are in reference to. Because the table is just resourceId, lastActionTime, action there is no way to identify which policies did which actions and therefore all past actions are counted against the resource during getNextStep
I could see an additional column being added to pac_rule_engine_autofix_actions to correlate a certain action to a certain annotation/issue/autofixplan that during the AutoFixManager calls NextStepManager.GetNextStep the lastActions list would only be filled out with actions that applied to the current issue/annotation/autofixplan execution.
Although that is only a rough thought and maybe you guys know of/have a better way for this to be solved.
Let me know if i can explain anything in more detail.

@jacob-kinzer acknowledged this is a bug, and we will fix it in next release.

@kaykumar Sounds good. Is there any tentative date for the next release? I understand its hard to pin down actual releases so i obviously wouldn't expect it to be accurate. But if you have a idea like its planned for end of Q3 that might help use decide on how we want to proceed.
Thanks for the response!

@kaykumar Any thoughts on when the next release might be that might include this fix?
Thanks!


Create route or ingress of MicroFrontendClass if defined in spec

Is your feature request related to a problem? Please describe.
When I create MicroFrontendClass I want to expose it via Gateway or Ingress API. I don't want to duplicate the information and I want to have the possibility to specify that I want the operator to create the Gateway or Ingress API resources.
Describe the solution you'd like
I want to have the option to add configuration to MicroFrontendClass that will specify whether I want the operator to create Gateway resources (HTTPRoute) or Ingres resources.
Describe alternatives you've considered
The other option is creating the resources manually this should remain possible.
Additional context
Both HTTPRoute or Ingress need to specify all the required fields that the operator cannot get from the existing fields and are required for the concrete resource to work.
Consider whether we want to support Ingress here as HTTPRoute is more generic.

Implemented in #29


bring back webgpu

CI tests succeed locally, and so does test_search (which can't run in CI). I
don't have access to a machine with METAL, so I don't know if any errors are
specific to CI, but I'll try to get access to one.
Implemented bool tensors as i32 using rewrite rules. Allocation is not
great, and it makes some tests slow, but we don't use the Allocator for
exported models. Suggestions here would be appreciated.
Tweaked compile_efficient to make it work again.

We can add f16 support when wgpu supports it. It should come soon https://github.com/gfx-rs/wgpu/pull/5701.

Bounty locked.

Still working on this? I closed that standalone PR which changed types because it's definitely possible to do better

Yes I'm going to keep working on it this week. I appreciate your feedback on that PR, and I agree it can be done better.

Still working?

I see webgpu test failing

I haven't been working on this much, but I pushed changes to merge this branch with the new cstyle (which is really nice).
The merge also made threefry the default RNG.  This broke randomness functions since threefry uses uint64 in _threefry_random_bits, so a lot of things are broken right now.
Something also broke fancy indexing tests, but it works with NOOPT=1. I'm going to look at recent commits to see what's causing this, but it might just be a rendering issue.
I haven't been able to fix the errors on Metal; they are all related to inf/nan. Webgpu allows handling of runtime infs and nans to be implementation specific (https://www.w3.org/TR/WGSL/#differences-from-ieee754), and they can yield indeterminate values. Testing locally on Vulkan these tests pass fine, but I'm not sure if it's possible to work around this on Metal. It's probably sensible to skip these cases since we can't guarantee behavior anyways. We skipped a couple inf/nan tests with webgpu before; there are just more tests with inf/nan cases now.
I don't know what to do about threefry yet; maybe we could create our own int64 type in wgsl. If someone else wants to work on these issues and polish things, they're welcome to, as I'm not sure how much I'm going to keep working on this.


cmd/contour, internal/contour: --envoy-{http,https}-access-log flags and implementation untested

Spotted during #454
  --envoy-http-access-log="/dev/stdout"
                               Envoy HTTP access log
  --envoy-https-access-log="/dev/stdout"
                               Envoy HTTPS access log

Options are not tested.
These are fields on Translator.ListenerCache which can be unit tested and e2e tested.

+1 for paying down this tech-debt for 0.6.0

bumping this to 0.7 as its a p2 item