LUKS single-password unlock

One of the ways to achieve a single password unlock for multiple LUKs encrypted drives is to use a randomly generated keyfile, which is set as a key for each encrypted drive, then encrypt the key itself into an image file, that has to unlocked and mounted prior to unlocking all subsequent drives at stage-1 boot.
I achieved this before using boot.initrd.luks.devices with the first submodule set to keyfile image, and subsequent submodules keyFile parameter set to the mounted unlocked keyfile.
Since I have upgraded to release-17.03, it still works, but I'm seeing some weird messages on the stage 1 console.
# this happens twice
Waiting 10 seconds for key file /dev/mapper/luks-key-encrypted to appear .....
...
Enter passphrase for /luks-key.img
...
# this happens 3 times
killall: cryptsetup: no process killed

Here is the relevant configuration:
boot.initrd.luks.devices = [
    { device = "/luks-key.img"; name = "luks-key-encrypted"; keyFile = null; }
    { device = "/dev/disk/by-id/ata..."; name = "main-1"; keyFile = "/dev/mapper/luks-key-encrypted"; }
];

Has something changed in the stage 1 luks booting script so that the device list gets unlocked out of order?
Previously stage 1 will first try to unlock the first submodule, and only then try the next devices, but if there's the Waiting 10 seconds... messages, then that seems like it's trying the other devices before finding my keyfile device.
I am also perplexed by the killall: cryptsetup: no process killed message.

Relevant files: https://github.com/NixOS/nixpkgs/blob/release-17.03/nixos/modules/system/boot/luksroot.nix

The way I do this, with an encrypted /boot on the same partition as /, is as follows:
  boot.initrd.luks.devices = [{
    name = "cryptlvm"; device = "/dev/disk/by-uuid/<redacted>";
    allowDiscards = true;
    keyFile = "/crypto_keyfile.bin";
  }];
  boot.initrd.prepend = ["${/crypto_keyfile.cpio.gz}"];

Works fine on nixos unstable without the errors you're mentioning. You could set up secondary encrypted filesystems with crypttab if you wanted to.
iirc crypto_keyfile.cpio.gz is created with
cpio -o -H newc <<< "crypto_keyfile.bin" > crypto_keyfile.cpio
gzip crypto_keyfile.cpio

I think the difference between your method and mine is that I'm also de
rypting the keyfile image. That is a single password to unlock the image
and then all subsequent drives pointing to the unlocked device. It used to
work without those errors, and now it has those errors but still works in
the end. That makes me think something changed in the unlocking order.
On 29/03/2017 10:25 AM, "Nick Hu"<EMAIL_ADDRESS>wrote:

The way I do this, with an encrypted /boot on the same partition as /, is
as follows:
boot.initrd.luks.devices = [{
name = "cryptlvm"; device = "/dev/disk/by-uuid/";
allowDiscards = true;
keyFile = "/crypto_keyfile.bin";
}];
boot.initrd.prepend = ["${/crypto_keyfile.cpio.gz}"];
Works fine on nixos unstable without the errors you're mentioning. You
could set up secondary encrypted filesystems with crypttab if you wanted
to.
iirc crypto_keyfile.cpio.gz is created with
cpio -o -H newc <<< "crypto_keyfile.bin" > crypto_keyfile.cpio
gzip crypto_keyfile.cpio
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/issues/24386#issuecomment-289935501,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAnHHdSgRQNWF2SGlZWKdzfY-SMjEvaXks5rqZb9gaJpZM4MqSCc
.

Yes I have grub prompt me for a password and then unlock my filesystems afterwards - just thought you might like an insight on a different approach to largely achieve the same thing. I don't think it's documented anywhere that nixos is supposed to respect the order of your entries, and it's not evaluated sequentially like a bash script. The standard approach across most Linux distros is that crypttab is used to unlock dependant/second-stage encrypted drives.

@NickHu Sorry but I don't understand, I'm using boot.initrd.prepend in the same way you are proposing. Your one is called /crypto_keyfile.bin and mine is called /luks-key.img. I noticed that you don't have your keyfile image being part of the Luks devices list, maybe that's the thing I'm missing?

@NickHu How was your /crypto_keyfile.bin created? My method was through this: https://gist.github.com/CMCDragonkai/6456d974982e09d4fe71f339f9029bd3

There's no need to make the keyfile a block device, a simple file will suffice:
dd bs=512 count=4 if=/dev/urandom of=/crypto_keyfile.bin
cryptsetup luksAddKey /dev/sda1 /crypto_keyfile.bin
chmod 000 /crypto_keyfile.bin

Also, the kernel parameter defaults to look for /crypto_keyfile.bin in the root if it exists, otherwise it has to be specified with the cryptkey parameter on the kernel command line.

I didn't know about that feature. That's very useful. What kernel parameter defaults to looking at /crypto_keyfile.bin?

It's a Linux default as far as I'm aware.

How is your crypto_keyfile.bin password protected? I didn't see any gpg encryption or luks encryption in your examples.

It's in the root partition which is decrypted by grub at boot time. I have a unified boot/root partition. When grub hands off to the Linux kernel, the partition gets locked again, but as the keyfile is in the initramfs (also on the encrypted partition), it can be unlocked by the kernel without prompting for a password the second time.

My zfs root is composed of several partitions that needs to be decrypted prior to zpool import. I don't know what you mean by "unified boot/root". Are you saying you use grub to unlock something and then it relocks it when in stage 1 boot phase? I'm currently using systemd boot and there's nothing like that.

Also my root and boot can't be unified because of uefi firmware which requires a fat32 boot partition. So that's where my keyfile image and initramfs currently reside.

Yes, as far as I'm aware, this only works with grub. If you don't have an encrypted /boot anyway, there's no need to do this and you can just use crypttab. The crypto keyfile is only useful for unlocking the root partition from the bootloader/initramfs.

I have just discovered why my system still works even when in the presence of the 2 errors.
First I found that my l2arc devices were not working:
 ♜ sudo zpool status                                                                                pts/2 19:53:02
[sudo] password for cmcdragonkai:
  pool: rpool
 state: ONLINE
status: One or more devices could not be used because the label is missing or
        invalid.  Sufficient replicas exist for the pool to continue
        functioning in a degraded state.
action: Replace the device using 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-4J
  scan: none requested
config:

        NAME                                                                        STATE     READ WRITE CKSUM
        rpool                                                                       ONLINE       0     0     0
          mirror-0                                                                  ONLINE       0     0     0
            dm-name-main-encrypted-ata-ST2000LM003_HN-M201RAD_S321J9BFB03122-part1  ONLINE       0     0     0
            dm-name-main-encrypted-ata-ST2000LM003_HN-M201RAD_S321J9BFB03123-part1  ONLINE       0     0     0
        logs
          mirror-1                                                                  ONLINE       0     0     0
            dm-name-zil-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180196-part2         ONLINE       0     0     0
            dm-name-zil-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180209-part2         ONLINE       0     0     0
        cache
          dm-name-l2arc-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180196-part3         UNAVAIL      0     0     0
          dm-name-l2arc-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180209-part3         UNAVAIL      0     0     0

errors: No known data errors

Then I found that the within /dev/mapper, the l2arc block devices no longer existed:
 ♖ ls /dev/mapper                                                                                pts/2 19:59:35
control                                                          zil-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180196-part2@
main-encrypted-ata-ST2000LM003_HN-M201RAD_S321J9BFB03122-part1@  zil-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180209-part2@
main-encrypted-ata-ST2000LM003_HN-M201RAD_S321J9BFB03123-part1@

My investigation into dmesg showed no errors, and upon restart, I remembered the 2 messages appearing:
# this happens twice
Waiting 10 seconds for key file /dev/mapper/luks-key-encrypted to appear .....

I tested the actual partitions using:
sudo cryptsetup status l2arc-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180196-part3
sudo cryptsetup --test-passphrase luksOpen /dev/disk/by-id/ata-PLEXTOR_PX-G128M6e_P02445180196-part3
sudo cryptsetup status l2arc-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180209-part3
sudo cryptsetup --test-passphrase luksOpen /dev/disk/by-id/ata-PLEXTOR_PX-G128M6e_P02445180209-part3

And those partitions are working.
Then it occurred to me, that I have 7 luks devices specified under nixos configuration.nix.
luks-key-encrypted
main-encrypted-ata-ST2000LM003_HN-M201RAD_S321J9BFB03123-part1
main-encrypted-ata-ST2000LM003_HN-M201RAD_S321J9BFB03122-part1
zil-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180196-part2
l2arc-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180196-part3
zil-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180209-part2
l2arc-encrypted-ata-PLEXTOR_PX-G128M6e_P02445180209-part3

And it turns out that in alphabetical order, l2arc-... is ahead of luks....
This meant NixOS at stage 1 tried to unlock the 2 l2arc-... devices, but couldn't because both relied on the existence of the unlocked /dev/mapper/luks-key-encrypted device. So it showed those 2 failure messages, then it tried to unlock luks-... device, which unlocked /dev/mapper/luks-key-encrypted, and then all subsequent drives were automatically unlocked and the system proceeded as normal.
Which all the more points to the comments above mentioning the change in the luks order of operations: https://github.com/NixOS/nixpkgs/issues/24386#issuecomment-289939950
@NickHu it does appear that since UEFI doesn't allow a unified boot and root, then crypttab should be used. I'm still not sure how NixOS will ask for unlocking a password protected /crypto_keyfile.bin and then feed that result into the subsequent luks devices.

So I looked at crypttab, and it appears that using crypttab is no different from just using NixOS's boot.initrd.luks.devices specification. Both achieve the same thing. What neither achieves is the usage of a password-protected /crypto_keyfile.bin. It is trivial to use one that is not password protected, as that is just the a property of keyFile.
I think that we need a new NixOS luks option, that allows a designation of a primary keyfile device, whether that is /crypto_keyfile.bin, which then gets mapped to a name, and where that name can be used as the keyFile setting for all other subsequent luks devices. This would achieve a password protected keyfile, that doesn't involve any extra tooling like gnupg, but only using existing luks tools.
Previously I was relying an insertion ordered luks opening operations, but with the new alphabetical ordering, a designated keyfile option should be used instead.

I can try tackling a PR for this, but I would like some feed back on this idea.

My configuration works perfectly fine on my UEFI laptops. What you can do is put just the EFI booting stubs in the FAT32 EFI system partition mounted to /boot/efi or something, and leave the kernel, initramfs, crypto_keyfile etc. on the /boot partition unified to / and use GRUB to boot it. As far as I am aware, GRUB is the only bootloader which supports loading a kernel/initramfs from an encrypted partition.

If I leave crypto_keyfile etc on /boot unified to /, this means the key (keys?) required to unlock the devices (listed above in my previous comment) needed for zpool import will be inside the zpool. Importing the zpool requires all LUKS devices to be unlocked prior. I don't see how your solution solves the problem?
It's not a good idea to partially import a zpool just to get access to the crypto_key, to unlock all subsequent devices and finally reimporting the entire zpool. My zpool consists of 2 drives and 2 ssds, of which the ssds are split in /boot, zil and l2arc.

Yeah, the way it works for me is that GRUB is able to mount encrypted filesystems - although I don't know if this extends to ZFS. In principle this has nothing to do with /boot, so what you could do is put a crypto_keyfile.bin which will unlock your ZFS drives on a partition that is in a LUKS device (could be separate) and have GRUB unlock that and then pass it to the kernel to decrypt your ZFS drives. That LUKS device could contain a tiny ext4 filesystem, and GRUB will prompt for the password when it tries to read from it. In essence, this is a way to emulate a password protected crypto_keyfile.bin.
Alternatively, there may already be facilities in GRUB to handle password protected crypto_keyfile.bin files, so switching bootloader might solve your problems.

So I see 3 ways around this:

Reprogram GRUB somehow so it supports password protected files. This would most likely use gnupg somehow.
Use the LUKS file device trick I suggested above, and relied on before the change to alphabetical luks unlocking.
Use the your suggestion which isn't really that different from my suggestion.

The main issue, is that there must always be some part of the disk that is unencrypted so the UEFI can load the bootloader. I prefer symmetrical partition layouts, so I had previously researched into soft raid capabilities of UEFI, but it appears it's not recommended, as they only officially support using a FAT32 partition. In my system, I have 2 (refer to my preference for symmetricity) unencrypted boot partitions that are mirrored (but not in soft raid style) manually.
The next issue is that the bootloader can be programmed to do anything, so that's where you need to put the keyfile. If that keyfile is not password protected, it's super easy. Password protecting it requires some crypto system available inside there. So that's why I'm suggesting solution 2, since it makes use of existing luks infrastructure, it works and doesn't change the partitioning layouts, it's just a file that exists in the unencrypted boot partition that can be first unlocked and fed into the rest of the partitions that constitute a zpool or lvm setup.

Hi @CMCDragonkai,
I run into the same problem as you but with another use case: LVM on LUKS on multiple disks. It means that all my LUKS partitions have to be unlock before mounting LVM volume group.
Your solution seems very nice and simpler then the GRUB with crypted /boot option option.
So, I am really interested in a PR around the solution 2.

Also, I'm trying to find a workaround  by prefixing the key name with "aa" but it do not work. Even if the keyfile device is loaded first I have the same error as you: waiting 10 second...
I used the gist you provide (https://gist.github.com/CMCDragonkai/6456d974982e09d4fe71f339f9029bd3)  , but I think I am missing the extra steps describe at the end of it:
 # prepend the CPIO archive to the initial ramdisk
 # it will be available at stage-1 boot as /luks-key.img

How do you prepend the CPIO archive to the initial ramdisk?
Here is my config just in case:
  boot.initrd.luks.devices =
  let
    luks_key_name = "aa-luks-key-encrypted";
    luks_key = "/dev/mapper/${luks_key_name}";
  in
  [
    {
      name = luks_key_name;
      device = "/luks-key.img";
    }
    {
      name = "root";
      device = "/dev/disk/by-uuid/c3a6ae03-368b-4877-b8a3-9d02c0a64d47";
      keyFile = luks_key;
      preLVM = true;
      allowDiscards = true;
    }
    {
      name = "home";
      device = "/dev/disk/by-uuid/edea857a-8e93-4eaf-b7cd-e94b753cd573";
      keyFile = luks_key;
      preLVM = true;
      allowDiscards = true;
    }
  ];

@CMCDragonkai @mickours you might be able to do something like this:
boot.initrd.luks.devices = [
  {
    name = "root";
    device = "/dev/disk/by-uuid/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
  }
  {
    name = "crypto_zfs_00";
    device = "/dev/disk/by-uuid/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
    keyFile = "/mnt-root/root/tank.keyfile";
  }
  {
    name = "crypto_zfs_01";
    device = "/dev/disk/by-uuid/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx";
    keyFile = "/mnt-root/root/tank.keyfile";
  }
];

systemd.generator-packages = [
  pkgs.systemd-cryptsetup-generator
];

environment.etc = {
  "crypttab" = {
    enable = true;
    text = ''
      crypto_zfs_00 UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /mnt-root/root/tank.keyfile luks
      crypto_zfs_01 UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx /mnt-root/root/tank.keyfile luks
    '';
  };
};

I only realised that /mnt-root was a thing after reading this: https://github.com/NixOS/nixpkgs/pull/29343

What does that systemd generator package do?

@CMCDragonkai it automatically creates unit files from the entries it sees in /etc/crypttab: https://www.freedesktop.org/software/systemd/man/systemd-cryptsetup-generator.html

May I ask why you are using a key file anyways? In my current Gentoo setup I just use the same passphrase for all LUKS devices (the key is derived from header information anyways) and try to unlock all luks devices with the phrase entered once.
In NixOS that might need changes to the initramfs though, but apart from that I see no real downside to your approach.

Since #29441 we have the password reuse feature. I think this should be enough to solve this issue. Using an extra keyfile, depending on its location, doesn't really change any security properties because both the passphrase and key eventually unlock a master key that's stored encrypted in the LUKS header.
Feel free to reopen if you disagree though.

For those looking for a solution in 2021 (this is one of the first results searching for similar terms), a key file can be included in the initrd using boot.initrd.secrets. This might look something like
{
  boot.initrd.luks.devices.root = {
    device = "...";
    preLVM = true;
    allowDiscards = true;
    keyFile = "/crypto_keyfile.bin";
  };

  boot.initrd.secrets = {
    "/crypto_keyfile.bin" = "/boot/initrd/crypto_keyfile.bin";
  };
}

And, while writing this comment, I found this gist which basically does the same thing.

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/how-to-unlock-some-luks-devices-with-a-keyfile-on-a-first-luks-device/18949/11


OSIDB-677: Add OSV collector

This PR adds a collector of OSV records from osv.dev. These records are saved as snippets and can be used to create a draft flaw (once that logic exists). OSV is an aggregator of various sources of vulnerabilities so this PR selects only a few of them. We can adjust the list of OSV ecosystems later on if needed.

@jobselko @osoukup This is sort of ready for initial review. Please have a look.
Each OSV record can have one or more CVE IDs, so I opted to create separate snippets for each CVE ID so we could keep the relationship of having one snippet for one CVE ID. We could also store CVE IDs as a list on the snippet, and then use a many-to-many relationship between Flaws and Snippets. This should also work with the current CVE ID-based snippet to flaw conversion method.

@mprpic The Snippet to Flaw conversion is already in the NVD collector. Do you plan to create a new PR for it later? And I am sorry for removing this from the merge queue without letting you know :)

@mprpic The Snippet to Flaw conversion is already in the NVD collector. Do you plan to create a new PR for it later? And I am sorry for removing this from the merge queue without letting you know :)

No worries, at least I was able to fix that incorrect test name before it got merged :)
I think the conversion will have to be a bit more complicated than the NVD one since it would be a shame not to use the affects data that is a part of each OSV record and converting those to our Affects might require some changes to the Affects model itself. I'll file an issue for this (same one as for the should_create_snippet() usage you mentioned above) to track this work, but I guess someone else will have to pick this up instead of me :-/

Filed OSIDB-1741 to improve the snippet collection filtering.

LGTM, just two minor things (which can be ignored if you want):
* collector and its tests should be in one commit

* the migration name does not match its content, so maybe rename it to sth like `0110_snippet_miscellaneous.py`

Both addressed.


Bad example

The first example on this page returns a List<Product>. It should not directly return this type according to framework guidelines , various FxCop/analyzers and numerous online references: see https://docs.microsoft.com/en-us/visualstudio/code-quality/ca1002-do-not-expose-generic-lists?view=vs-2019

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fa03501b-5e71-4c18-0269-39babbf9d93c
Version Independent ID: b51bc104-a6e2-b21f-8362-45cee22e8c22
Content: Controller action return types in ASP.NET Core web API
Content Source: aspnetcore/web-api/action-return-types.md
Product: aspnet-core
Technology: aspnetcore-webapi
GitHub Login: @scottaddie
Microsoft Alias: scaddie

This rule isn't applicable to controllers. For an explanation, see https://github.com/aspnet/AspNetCore.Docs/pull/16982#issuecomment-585335171.

Someone should tell the analyzer team that. Either the example controller method here should be updated or the documentation for the analyzer should be updated.
I can explain it to developers but I cannot convince my auditors that I am justifiably ignoring this rule without some documentation to point at.

@bbarry I think the problem is https://docs.microsoft.com/visualstudio/code-quality/design-warnings?view=vs-2019. The link text (at the top of the doc) should be changed from ".NET Design Guidelines" to ".NET Framework Design Guidelines". I'll submit a PR to address that low-hanging fruit.


chore(vitest): add vitest browser beta 2 test

This is just a test using Vitest beta v2.0 packages to run the new added test in the browser: src/components/TresCanvas.test.ts cloned from my previous branch (https://github.com/Tresjs/tres/blob/userquin/feat-vitest-browser/src/components/TresCanvas.test.ts).
To run the browser test:

pnpm install && pnpm build from root folder
pnpm test:browser:none from root

We can also use webdriverio or playwright, you need to add the corresponding dependencies as dev dependencies and run:

pnpm test:browser:webdriverio from root or
pnpm test:browser:playwright from root

This PR using vitest packages from local tgz from this PR: https://github.com/vitest-dev/vitest/pull/5815
Once we release a new beta version we can use vitest from npm registry.
For CI, we need to include webdriverio or playwright and configure headless to true.

Once the browser run finish running the tests, run the TresCanvas test to see the UI:

e2e tests would be a big help. Looking forward to this.

@andretchen0 we need to cover e2e tests, we have some internal team discussion about it, stay tuned


[idea] Device name (netbox) as visible hostname (zabbix)

Hi! Great project!
I needed support for long device host name, so and I made a small patch, it might be interesting.
I changed the variable mapping, self.name is now "visible name" field in zabbix:

createInZabbix:
                host = self.zabbix.host.create(host=self.ip,
                                                     name=self.name,

zabbixHostnameExists:
        host = self.zabbix.host.get(filter={'name': self.ip}, output=[])

ConsistencyCheck:
        if(host["name"] == self.name):
            logger.debug(f"Device {self.name}: visible name in-sync.")
        else:
            logger.warning(f"Device {self.name}: visible name OUT of sync. "
                           f"Received value: {host['name']}")
            self.updateZabbixHost(name=self.name)
        if(host["host"] == self.ip):
            logger.debug(f"Device {self.name}: hostname in-sync.")
        else:
            logger.warning(f"Device {self.name}: hostname OUT of sync. "
                           f"Received value: {host['host']}")
            self.updateZabbixHost(host=self.ip,name=self.name)

Glad to hear that you like the project! I used the "host name" field as a starting point given that the "host name" field is mandatory and the "visible name" field is optional. I might implement a simple function that would allow users to set the "visible name" field as preferred instead of the "host name" field.
However since the host name field is mandatory, what would your preference be? A shorter version of the visible name or replace it with the IP address (like you did.)

It's clear.. ;) But, look at zabbix docs about hostname field:
Alphanumerics, spaces, dots, dashes and underscores are allowed. However, leading and trailing spaces are disallowed.
Device name in netbox can contain any characters.
Due to this, an error may occur during synchronization
There are two ways: filter device name variable, or change hostname variable mapping to visible name, primary_ip -> hostname
I can also note that "device name" field in netbox is not unique, but "hostname" in zabbix - does.

I agree it would be useful to utilize 'visible name' and 'host name' fields separately, as 'host name' is often a technical name and 'visible name' a more user friendly name for the user using the dashboard.
However, I'm not fond of using the IP address as the 'host name' in Zabbix as that could easily result in naming clashes in multi tenant or multi-vrf environments.
Food for thought?

Maybe you're right.
What about another suggestions? 'host name' can be filled as 'asset tag' value, if it is missing, set 'host name' as "device[netbox_device_id]"

The asset tag could be useful, however this is not an option in clustered setup where the name is overwritten by the cluster name. Assuming that you give similar names to clustered devices (eg. long names, umlauts, non-allowed characters.)
Personally i don't see a lot of Netbox deployment utilizing the asset field. However this doesn't mean its not viable. It could provide for an easy "backdoor" trick for long names and special characters.

On second thought this might also clash with the requirement of unique hostnames. Given this scenario with 2 switches:

........Very long name in Netbox declared which is longer than 128 characters switch01
........Very long name in Netbox declared which is longer than 128 characters switch01

Cutting off the last couple of characters would end the string with something like "in Netbox declared which is longer than 128 chara". Therefore making them both identical and failing the hostname unique requirement.
On the other hand starting at the front isn't ideal either since this comes with the same risks.
@k109x Can you elaborate your use case on why you use such long names? If it's just one or 2 hosts within an inventory of a few 100s then i'm more tempted to make a check for the length of a hostname and skip a host if its name is longer than 128 characters.

So after some code digging i found the following:
Testing on Netbox 4: it seems that i hit a hard limit of 64 characters for the device name. My fix does seem to work for #42. Looking forward to your reply and use case since i don't see a problem with an enforced 64 character limit in Netbox syncing to a 128 character limit field in Zabbix.

@TheNetworkGuy I have this cases:

The name field in the netbox does not reflect a network name, so device names can be assigned descriptions, either automatically generated hostnames, or the field can be empty.
the hostname can contains special characters or non-English characters

I use this construct in production:
 host = self.zabbix.host.create(name=self.name,
                                               host=f"netbox_device_{self.id}",
                                               status=self.zabbix_state,
                                               interfaces=interfaces,
                                               groups=groups,
                                               templates=self.zbx_templates,
                                               proxy_hostid=self.zbxproxy,
                                               description=f"https://mynetboxhost.domain/dcim/devices/{self.id}/")

Okay that makes sense. I've updated the code to use the same method as you did but only for "special names" to fix the point that you bring up: "the hostname can contains special characters or non-English characters".
However i am unable to replicate your initial comment about the character limit. Can i conclude that this fix for now is enough for you to close this issue? :)

Yes, sure
Thanks a lot


Add Action workflow

Add Action workflow

:tada: This PR is included in version 1.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Gulp V4 need advanced sourcemap settings

As of new Gulp v4 I can get rid of gulp-sourcemaps plugin.
As far as I know Gulp v4 uses its own sourcemap.
I refactored my project to use Gulp v4 sourcemap, however I am missing advanced settings in sourcemap.
I have following project structure:
├── dist
│   ├── app.bot.js
│   ├── app.bot.js.map
│   └── authorize
│       ├── authorize.js
│       └── authorize.js.map
├── gulpfile.js
├── package.json
├── package-lock.json
├── src
│   ├── app.bot.ts
│   └── authorize
│       └── authorize.ts
└── tsconfig.json

As you can see after typescript compilation all files that were in src/*.ts goes into dist/*.js.
Path from generated *.map file to original *.ts file is incorect. It is correct only for one nested level.
I need correct path for debug in VSCode.
E.g. path from app.bot.js.map to app.bot.ts is correct:
"sources":["../src/app.bot.ts"]

E.g. path from authorize.js.map to authorize.ts is invalid:
"sources":["../src/authorize/authorize.ts"]

With gulp-sourcemaps plugin I managed to do correct path as following:
const gulp = require('gulp');
const ts = require('gulp-typescript');
const sourcemaps = require('gulp-sourcemaps');
const tsProject = ts.createProject('tsconfig.json');
const outputDir = './dist';

gulp.task('build', () => tsProject.src()
  .pipe(sourcemaps.init({ loadMaps: true }))
  .pipe(tsProject())
  .js.pipe(sourcemaps.write('./', {
    includeContent: false,
    sourceRoot: '.'
  })).pipe(gulp.dest(outputDir))
);

With Gulp v4 sourcemap I can not do above example, right now I have this (but it does not help):
const gulp = require('gulp');
const ts = require('gulp-typescript');
const tsProject = ts.createProject('tsconfig.json');
const outputDir = './dist';

function build() {
  return gulp.src('./src/**/*.ts', { sourcemaps: true })
  .pipe(tsProject()).js
  .pipe(gulp.dest(outputDir, { sourcemaps: '.' }))
}

Is there any possibility to setup advanced settings for Gulp v4 sourcemap (so it will generate correct sourcemap path)?

Please reopen this issue. It's not about TS. You can reproduce the issue with babel too.
In fact, you can reproduce the issue without any transpiling step. It's a gulp issue.
https://github.com/3cp/gulp-sm-demo
gulp.src('src/**/*.js', {sourcemaps: true})
  .pipe(gulp.dest('dist', {sourcemaps: '.'}));

In this tiny demo, there are two files in src/folder: src/index.js and src/foo/index.js.
Because of the extra folder under src/, the compiled dist/foo/index.js.map is incorrect in the context without sourceRoot.
{
  "version":3,
  "names":[],
  "mappings":""
  ,"sources":["foo/index.js"],
  "sourcesContent":["module.exports = \"Hello Foo\";\n"],
  "file":"foo/index.js"
}

For an app that uses this npm module, it thinks:

the source for dist/index.js is at path dist/index.js. (not src/index.js, but not a big issue)
the source for dist/foo/index.js is at path dist/foo/foo/index.js. (incorrect duplicated folder structure)

This can be fixed by using gulp-sourcemaps:
https://github.com/3cp/gulp-sm-demo/tree/gulp-sourcemaps
gulp.src('src/**/*.js', {sourcemaps: true})
    .pipe(gulpSourcemaps.write('.', {
      sourceRoot: '../src/'
    }))
    .pipe(gulp.dest('dist'));

gulp-sourcemaps smartly does

add "sourceRoot": "../src/" to dist/index.js.map, so the source path will becomes src/index.js.
add "sourceRoot": "../../src/" (note it smartly added extra ".." to match deeper folder) to dist/foo/index.js.map, so the source path will becomes src/foo/index.js.

So the default gulp v4 sourceMap offering is incorrect in term of source path. But this can be fixed in gulp v4 gracefully because the sourceMap writing is handled by gulp.dest, it can compare the dest location with original source location, then smartly mutate sourceMap "sources" field or add "sourceRoot" field.
I can try to make a PR if this makes sense.

@3cp Thanks for your good reproduction codes. I'm trying to solve this issue.

@3cp it was never intended that gulp support the full set of gulp-sourcemaps options. Gulp just loads the initial sourcemaps and writes the final ones to disk—this is to avoid any filesystem calls within the middle of a pipeline.  If you want extra features, you can reach for an @gulp-sourcemaps/... plugin, like.
For your sourceRoot example, it looks like the gulp-sourcemaps team hasn't had the chance to implement a separate plugin for that yet. If you want to help, or just ask for it to be created, I'd suggest visiting https://github.com/gulp-sourcemaps/gulp-sourcemaps/issues/285
If you think there is a bug in gulp on how source paths are calculated, please open an issue (with a test case) at https://github.com/gulpjs/vinyl-sourcemap

Sure, I understand. I will do some study and try to open an issue on vinyl-sourcemap.
One thing I noticed, is that gulp-sourcemaps without explicit sourceRoot option behaves exactly like gulp v4 built-in sourceMap support. Both did not care about the duplicated source path segments.
I thought it probably makes more sense to get a correct default behaviour (for both gulp-sourcemaps and gulp v4 built-in). gulp v4 doesn't need to have any new features or new options to get this right, gulp.src and gulp.dest got enough information to make it right out of the box.

@3cp vinyl-sourcemap was created using gulp-sourcemap code as a basis. If there was a bug in gulp-sourcemaps, it would be present in vinyl-sourcemap as well.


Order of search plugins

[x] I am on the latest Cerebro.app version
[x] I have searched the issues of this repo and believe that this is not a duplicate

OS version and name: Linux Debian Stretch
Cerebro.app version:
Relevant information from devtools (See above how to open it):

Issue

Hello,
I would like to find a way to manage order of my search plugins. When I installed github search plugin it automatically became first search plugin, but I would like to Google search plugin to be the first one. I even tried to reinstall G search plugins hoping it will make it latest/preferred.  In Details about G search plugin there is gif and there IS G search before github search. I tried searching in issues but failed to find anything. Thanks for your work, I love this app.

Hey, @tenorz007 what do you think about this one? Can we change order key for github plugin so it is moved below google result?

@KELiON I was planning to do that but just haven't gotten around to it. I should have that updated by today.

Cool, this solves my problem I guess, thank you.

@KELiON @Taskkill I have updated the plugin with some fixes too. It shouldn't show up ahead of the google plugin

@tenorz007 It's working like a charm! Thank you a lot for your help.

@Taskkill Glad I could help, I hope the plugin turns out to be useful to you!

@tenorz007 It indeed is.

Is there a way to do the same for other plugins? I'd like to have Google Appear above the Everything plugin
https://github.com/masterperas/es-everything-plugin


Fix cert.pem being copied into each destination .pem file

I can't believe I made that bug! Thank you so much for filing a fix PR


Amazon Error setting/closing connection: Not Connected

I get the following error when running a test job from my PC. I have connectivity to the redshift cluster because at the same time I am running Aginity workbench session.
Any idea what settings can be tweaked to avoid it? it may be a timeout...
16/01/04 21:17:48 INFO RedshiftWriter: CREATE TABLE IF NOT EXISTS "staging"."messages_20160104_205059" ("message_id" ...
Exception in thread "main" java.sql.SQLException: [Amazon](500150) Error setting/closing connection: Not Connected.
    at com.amazon.redshift.client.PGClient.isConnected(Unknown Source)
    at com.amazon.redshift.client.PGClient.reprepareStatement(Unknown Source)
    at com.amazon.redshift.dataengine.PGAbstractQueryExecutor.pushMappedParamTypes(Unknown Source)
    at com.amazon.jdbc.common.SPreparedStatement.updateParameterMetadata(Unknown Source)
    at com.amazon.jdbc.common.SPreparedStatement.execute(Unknown Source)
    at com.databricks.spark.redshift.RedshiftWriter.com$databricks$spark$redshift$RedshiftWriter$$doRedshiftLoad(RedshiftWriter.scala:163)
    at com.databricks.spark.redshift.RedshiftWriter.saveToRedshift(RedshiftWriter.scala:381)
    at com.databricks.spark.redshift.DefaultSource.createRelation(DefaultSource.scala:106)
    at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:170)
    at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:146)
    at com.amazonaws.proserve.jelez.spark.kinesis2redshift.FlattenJsonToRedshiftAndS3Batch$.main(FlattenJsonToRedshiftAndS3Batch.scala:183)
Caused by: com.amazon.support.exceptions.GeneralException: [Amazon](500150) Error setting/closing connection: Not Connected.
    ... 11 more

Did you try Googling the exception? When I searched for "Amazon Error setting/closing connection: Not Connected.", in quotes, I found two relevant links:

http://wiki.pentaho.com/display/analysis/PA_CR_PA-<IP_ADDRESS>-RC1_redshift_08.00.0002_RedshiftJDBC-<IP_ADDRESS>3
https://sagebionetworks.jira.com/browse/BRIDGE-559

The second link suggests that this could be caused by MTU issues.
Is the "Aginity workbench session" initiating the JDBC connection from your PC or is it using some sort of client-server architecture where the actual connection is being established from a different machine?
In either case, I'm not sure that this is an issue with spark-redshift itself, so I'm going to resolve this as "not-a-spark-redshift" issue and suggest that you contact Amazon support to troubleshoot this connection  issue.
Other readers: if you also see this exception, don't hesitate to bump this thread with a "me, too!". I'm only closing this issue to avoid cluttering the open issues queue with things that aren't actionable from my end.

me, too!
I am using 1.1.6 Redshift driver in JasperServer 6.01 to connect Redshift.
Occasionally, JasperServer will error with 'Amazon Error setting/closing connection: Not Connected.', we tried to reproduce this issue but failed, do not know how it can happen. It can be recovered by JapserServer restartup.

This worked for me....

Sign-in into AWS Console
Go into EC2 Management Console
On the left navigation pane, look for Network & Security header and click on Security Groups.
Notice the row with "Default" in the Group Name's column. Click that. Then select the Inbound tab, and click Edit.
Add Rule: Type=Redshift, Source=MyIP. (auto-fill)
Save it.
Try to connect again from within SQLWorkbench/J. Feel good!

I tried the steps above  but it still doesnt work. What am i missing?

I've also been having this issue. Assigned an IAM role that had full permissions and using that hadn't worked. Looking at the flow log on AWS it seems that the outgoing communication from Databricks to Redshift is going to the correct location and is accepted, but no response is given - causing it to timeout as it waits.

I have a similar issue as @adzsroka. Have assigned the proper IAM role that works fine locally to extract data from Redshift via Spark. I spun up a cluster in Databricks a few hours ago, loaded "RedshiftJDBC41-<IP_ADDRESS>7", getting the timeout.

I'm having a similar issue in cluster mode, using spark-redshift 2.0.1. Works fine on simple queries, but on more complicated queries, after 2 hours (and we see the temp directories being written to in s3) we get this timeout.

@pigate is right, I have a the same issue, its not working on complicated queries.

I was experiencing the same issue, when queries take more than 2 hours. Found an interesting post: https://kb.informatica.com/solution/23/Pages/57/485601.aspx
sudo /sbin/sysctl -w net.ipv4.tcp_keepalive_time=200 net.ipv4.tcp_keepalive_intvl=200 net.ipv4.tcp_keepalive_probes=5
It works for me now.

Happened to me because I selected "No" in the "Publicly accessible" option.
Go to the instance details and choose "Modify"

I had the same problem as gabi-lee. I selected "No" in the "Publicly accessible" option. Consequently, despite having inbound security wide open, I was unable to connect.
"Amazon Error setting/closing connection"
To fix, choose your redshift cluster and then click the "cluster" drop down.
Select modify and then change publicly accessible.
It took about a minute or so to take effect.

The recommendation posted above by Regalcom worked for me. Thanks!

I was able to connect after editing SG of default VPC.

@Regalcom, thank you - that worked!

Thanks ghost, this worked for me
Here's what worked for me.
On the Redshift dashboard, on the left panel, click on "Security".
Go to "Security Groups" tab.
Click on "default" on the list.
Click "Add Connection Type".
On the modal, choose "CIDR/IP" on the "Connection Type".
Your IP address would be automatically populated on the input field.
Click "Authorize".

Go to Security Group, select default user, then edit the inbound rules, add Redshift to Source from Anywhere, worked for me.
=>
Redshift | TCP | 5439 | <IP_ADDRESS>/0

i hade the same problem go to redshift cluster modify and let public access on

Here's what worked for me.

On the Redshift dashboard, on the left panel, click on "Security".
Go to "Security Groups" tab.
Click on "default" on the list.
Click "Add Connection Type".
On the modal, choose "CIDR/IP" on the "Connection Type".
Your IP address would be automatically populated on the input field.
Click "Authorize".

I had the same problem as gabi-lee. I selected "No" in the "Publicly accessible" option. Consequently, despite having inbound security wide open, I was unable to connect.
"Amazon Error setting/closing connection"
To fix, choose your redshift cluster and then click the "cluster" drop down.
Select modify and then change publicly accessible.
It took about a minute or so to take effect.

Thank you! I got connected as soon as i enabled public access. Thanks again.

Thank you! I got connected as soon as i enabled public access. Thanks again.

I had the same problem as gabi-lee. I selected "No" in the "Publicly accessible" option. Consequently, despite having inbound security wide open, I was unable to connect.
"Amazon Error setting/closing connection"
To fix, choose your redshift cluster and then click the "cluster" drop down.
Select modify and then change publicly accessible.
It took about a minute or so to take effect.

Thank @whitsonk! I got connected as soon as i enabled public access. Thanks again.

Methods to follow if you come across [Amazon][500150] Error setting/closing connection:
1.Sign-in into AWS Console
2.Search in AWS console for 'Security groups' - You will get 'Security groups (ECS feature)'
3.Select Inbound rules tab < select edit inbound rules (Add Rule: Type=Redshift, Source=MyIP)
4.Also in Redshift- Open your Cluster < Actions < Modify publically accessible setting < Enable < Save Changes
Also download the latest JDBC driver (JAR) from Amazon Redshift < Clusters
Connect Redshift with SQLWorkbench/J.


[BUG] AttributeError: pakcage_dir typo breaks editable installs

setuptools version
64.0.0
Python version
3.9
OS
ubuntu
Additional environment information
https://github.com/pypa/setuptools/commit/7ae5a6903c83eb43a96ad02c874990d02ee92672 introduced the typo
Description
.
Expected behavior
.
How to Reproduce
.
Output
 Traceback (most recent call last):
        File "/tmp/pip-build-env-msencw5_/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 138, in run
          self._create_wheel_file(bdist_wheel)
        File "/tmp/pip-build-env-msencw5_/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 283, in _create_wheel_file
          self._install_namespaces(unpacked, dist_info.name)
        File "/tmp/pip-build-env-msencw5_/overlay/lib/python3.9/site-packages/setuptools/command/editable_wheel.py", line 171, in _install_namespaces
          src_root = Path(self.project_dir, self.pakcage_dir.get("", ".")).resolve()
        File "/tmp/pip-build-env-msencw5_/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 105, in __getattr__
          raise AttributeError(attr)
      AttributeError: pakcage_dir

Thank you very much for reporting this @agroszer.

Unsure if related, but since this morning all of our packages have began breaking with the following:
distutils.errors.DistutilsSetupError: Error: setup script specifies an absolute path:

          /var/folders/r7/ggtlz3qs465bzrr2zjtf1h380000gp/T/tmpforaj_f1.build-temp/plethora_signals.egg-info/SOURCES.txt

      setup() arguments must *always* be /-separated paths relative to the
      setup.py directory, *never* absolute paths.

      error: Support for editable installs via PEP 660 was recently introduced
      in `setuptools`. If you are seeing this error, please report to:

      https://github.com/pypa/setuptools/issues

      Meanwhile you can try the legacy behavior by setting an
      environment variable and trying to install again:

      SETUPTOOLS_ENABLE_FEATURES="legacy-editable"
      [end of output]

Hi @synchronizing, probably a different problem, could you open a new issue with a reproducer?
Meanwhile we investigate/fix the problem you can try using SETUPTOOLS_ENABLE_FEATURES="legacy-editable" or relying on a non-editable installation.

Both the env, and non-editable installations work. The issue started this morning, however, and was not present pre-64.0.0. Will open new issue, thank you.

In my case, it was indeed an editable dep breaking my testing environment that led me to find the typo in #3497

Sorry for the inconvenient @leonixyz.
The tests for original behaviour in the develop command were scarce because of the deprecation, so the particular code path ended up untested (it does not help that we have some issues with the coverage report).
Thank you for providing the fix. If that is OK with you, I was planning to wait and see if we can also investigate and fix the second problem reported by @synchronizing quickly before cutting a bug fix release. If the second issue is difficult to solve, I will split them in separated releases.

@abravalheri looks like a bunch of other issues arose.... please merge and release

@agroszer, sorry for the delay. Due to the time the CI takes to run, it is good to group all the fixes together.
Please let me know v64.0.1 solves your problem. I can re-open the ticket if necessary.

thanks that fixes it


Gateway - getting price query failed error after restarting gateway connection

Gateway - after restart of the gateway Client failed to get access to the token list for amm_data_feed_example.py script
Steps To Reproduce

Clone and install Source client + gateway
gateway generate certs
./gateway-setup.sh
yarn start --passphrase=password, make sure gateway is ONLINE
connect quickswap and uniswap polygon mainnet
start the script amm data_feed example
check status and review script working
stop the gateway
yarn start --passphrase=password again and make sure gateway is ONLINE
start the same script again

Actual:
review that bot Failed to get data from AmmDataFeed
Expected:
Client are able to start the script successfully again after restart of the gateway
Loom Video:
https://www.loom.com/share/c456e9e18ff04e47af9f5153b1393118
logs_amm_data_feed_example.log
logs_gateway_app.log

2023-07-19 15:01:18,451 - 439205 - hummingbot.data_feed.amm_gateway_data_feed - ERROR - Error getting data from AmmDataFeed[uniswap_polygon_mainnet]Check network connection. Error: Error on POST https://localhost:15888/amm/price Error: {'message': 'Price query failed: Token not supported: WMATIC', 'httpErrorCode': 500, 'errorCode': 1013, 'stack': 'Error: Price query failed: Token not supported: WMATIC\n    at /home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:74:23\n    at Generator.throw (<anonymous>)\n    at rejected (/home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:6:65)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)'}
2023-07-19 15:01:18,452 - 439205 - hummingbot.data_feed.amm_gateway_data_feed - ERROR - Error getting data from AmmDataFeed[quickswap_polygon_mainnet]Check network connection. Error: Error on POST https://localhost:15888/amm/price Error: {'message': 'Price query failed: Token not supported: LINK', 'httpErrorCode': 500, 'errorCode': 1013, 'stack': 'Error: Price query failed: Token not supported: LINK\n    at /home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:74:23\n    at Generator.throw (<anonymous>)\n    at rejected (/home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:6:65)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)'}
2023-07-19 15:01:18,770 - 439205 - hummingbot.connector.exchange.binance.binance_api_order_book_data_source.BinanceAPIOrderBookDataSource - INFO - Subscribed to public order book and trade channels...
2023-07-19 15:01:18,819 - 439205 - hummingbot.core.data_type.order_book_tracker - INFO - Initialized order book for BTC-USDT. 1/1 completed.
2023-07-19 15:01:19,001 - 439205 - hummingbot.strategy.script_strategy_base - WARNING - binance is not ready. Please wait...
2023-07-19 15:01:19,481 - 439205 - hummingbot.data_feed.amm_gateway_data_feed - ERROR - Error getting data from AmmDataFeed[quickswap_polygon_mainnet]Check network connection. Error: Error on POST https://localhost:15888/amm/price Error: {'message': 'Price query failed: Token not supported: WMATIC', 'httpErrorCode': 500, 'errorCode': 1013, 'stack': 'Error: Price query failed: Token not supported: WMATIC\n    at /home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:74:23\n    at Generator.throw (<anonymous>)\n    at rejected (/home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:6:65)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)'}
2023-07-19 15:01:19,527 - 439205 - hummingbot.data_feed.amm_gateway_data_feed - ERROR - Error getting data from AmmDataFeed[uniswap_polygon_mainnet]Check network connection. Error: Error on POST https://localhost:15888/amm/price Error: {'message': 'Price query failed: Token not supported: LINK', 'httpErrorCode': 500, 'errorCode': 1013, 'stack': 'Error: Price query failed: Token not supported: LINK\n    at /home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:74:23\n    at Generator.throw (<anonymous>)\n    at rejected (/home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:6:65)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)'}
2023-07-19 15:01:19,580 - 439205 - hummingbot.client.ui.hummingbot_cli - ERROR - Unhandled error in prompt_toolkit: Error on POST https://localhost:15888/amm/price Error: {'message': 'Price query failed: Token not supported: LINK', 'httpErrorCode': 500, 'errorCode': 1013, 'stack': 'Error: Price query failed: Token not supported: LINK\n    at /home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:74:23\n    at Generator.throw (<anonymous>)\n    at rejected (/home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:6:65)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)'}
NoneType: None
2023-07-19 15:01:19,580 - 439205 - hummingbot.client.ui.hummingbot_cli - ERROR - Unhandled error in prompt_toolkit: Error on POST https://localhost:15888/amm/price Error: {'message': 'Price query failed: Token not supported: WMATIC', 'httpErrorCode': 500, 'errorCode': 1013, 'stack': 'Error: Price query failed: Token not supported: WMATIC\n    at /home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:74:23\n    at Generator.throw (<anonymous>)\n    at rejected (/home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:6:65)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)'}
NoneType: None
2023-07-19 15:01:19,581 - 439205 - hummingbot.client.ui.hummingbot_cli - ERROR - Unhandled error in prompt_toolkit: Error on POST https://localhost:15888/amm/price Error: {'message': 'Price query failed: Token not supported: AAVE', 'httpErrorCode': 500, 'errorCode': 1013, 'stack': 'Error: Price query failed: Token not supported: AAVE\n    at /home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:74:23\n    at Generator.throw (<anonymous>)\n    at rejected (/home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:6:65)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)'}
NoneType: None
2023-07-19 15:01:19,581 - 439205 - hummingbot.client.ui.hummingbot_cli - ERROR - Unhandled error in prompt_toolkit: Error on POST https://localhost:15888/amm/price Error: {'message': 'Price query failed: Token not supported: LINK', 'httpErrorCode': 500, 'errorCode': 1013, 'stack': 'Error: Price query failed: Token not supported: LINK\n    at /home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:74:23\n    at Generator.throw (<anonymous>)\n    at rejected (/home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:6:65)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)'}
NoneType: None
2023-07-19 15:01:19,581 - 439205 - hummingbot.client.ui.hummingbot_cli - ERROR - Unhandled error in prompt_toolkit: Error on POST https://localhost:15888/amm/price Error: {'message': 'Price query failed: Token not supported: WMATIC', 'httpErrorCode': 500, 'errorCode': 1013, 'stack': 'Error: Price query failed: Token not supported: WMATIC\n    at /home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:74:23\n    at Generator.throw (<anonymous>)\n    at rejected (/home/nikita/gwstaging/dist/src/connectors/uniswap/uniswap.controllers.js:6:65)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)'}
NoneType: None

@vic-en

Observed that this error would occur too when suddenly gateway connection has been stop and start again.

Setup simple AMM arb then start the strategy
Close the gateway connection by either stopping it or exit terminal
After successfully terminated the gateway connection, start it again until Client display gateway service is online
Start the AMM arb strategy again, observe behavior

2023-07-20 18:55:58,679 - 1207 - GatewayEVMAMM - INFO - Network status has changed to NetworkStatus.NOT_CONNECTED. Stopping networking...
2023-07-20 18:56:00,004 - 1207 - hummingbot.core.gateway.gateway_http_client - WARNING - Call to https://localhost:15888/amm/price failed. See logs for more details.
2023-07-20 18:56:00,004 - 1207 - GatewayEVMAMM - WARNING - Cannot connect to host localhost:15888 ssl:default [Connect call failed ('<IP_ADDRESS>', 15888)]
2023-07-20 18:56:00,004 - 1207 - hummingbot.core.gateway.gateway_http_client - WARNING - Call to https://localhost:15888/amm/price failed. See logs for more details.
2023-07-20 18:56:00,004 - 1207 - GatewayEVMAMM - WARNING - Cannot connect to host localhost:15888 ssl:default [Connect call failed ('<IP_ADDRESS>', 15888)]
2023-07-20 18:56:00,004 - 1207 - hummingbot.core.gateway.gateway_http_client - WARNING - Call to https://localhost:15888/amm/price failed. See logs for more details.
2023-07-20 18:56:00,004 - 1207 - GatewayEVMAMM - WARNING - Cannot connect to host localhost:15888 ssl:default [Connect call failed ('<IP_ADDRESS>', 15888)]
2023-07-20 18:56:00,005 - 1207 - hummingbot.core.gateway.gateway_http_client - WARNING - Call to https://localhost:15888/amm/price failed. See logs for more details.
2023-07-20 18:56:00,005 - 1207 - GatewayEVMAMM - WARNING - Cannot connect to host localhost:15888 ssl:default [Connect call failed ('<IP_ADDRESS>', 15888)]
2023-07-20 18:56:06,030 - 1207 - hummingbot.core.gateway.gateway_http_client - WARNING - Call to https://localhost:15888/amm/price failed. See logs for more details.
2023-07-20 18:56:06,031 - 1207 - GatewayEVMAMM - WARNING - Error on POST https://localhost:15888/amm/price Error: {'message': 'Price query failed: Token not supported: WMATIC', 'httpErrorCode': 500, 'errorCode': 1013, 'stack': 'Error: Price query failed: Token not supported: WMATIC\n    at /home/rapcmia/github/gateway/gateway-stg/dist/src/connectors/uniswap/uniswap.controllers.js:74:23\n    at Generator.throw (<anonymous>)\n    at rejected (/home/rapcmia/github/gateway/gateway-stg/dist/src/connectors/uniswap/uniswap.controllers.js:6:65)\n    at processTicksAndRejections (node:internal/process/task_queues:96:5)'}
2023-07-20 18:56:06,060 - 1207 - hummingbot.core.gateway.gateway_http_client - WARNING - Call to https://localhost:15888/amm/price failed. See logs for more details.

While this happened, other gateway commands seems to be working ok and able to connect other wallets. However when you try to start a script or strategy for gateway this issue occurs after the prompt yes Our workaround is to relaunch both gateway and client
logs_amm-uniswap-polygon.log

Facing the same issue. Any updates?

Currently for workaround:
Source (bot Client source + gateway source):
need to ./gateway-setup.sh step again:
https://docs.hummingbot.org/gateway/installation/#run-setup-script

Currently for workaround: If not working on Source (bot Client source + gateway source): need to ./gateway-setup.sh step again: https://docs.hummingbot.org/gateway/installation/#run-setup-script

I did some testing by running a script and strategy, stopping Gateway, and then re-starting. Initially, I had some issues but after I ran ./gateway-setup.sh, I was able to stop and restart Gateway / client successfully for both the AMM Data Feed script and the AMM-ARB strategy.
Users using Gateway from source should run ./gateway-setup.sh after each release since configs for connectors may change.

Investigate this issue by running a set of user scenarios and this had me reproduce the issue on a 3/5 attempts
https://github.com/hummingbot/gateway/assets/73840223/065bba06-3018-4a05-a423-ba19f48f9377

Setup source client + gateway and make sure gateway-setup.sh else wont go Gateway: Online
Setup a Uniswap-Ethereum-Goerli for this test but should happened to other DEX connectors too ✅
Setup AMM-arb and start the strategy, after few minutes try to disconnect your gateway
After being disconnected, try to reconnect again. On other instances it reconnects ok but sometimes would return amm/price failed

logs_amm-uniswap-goerli.log
logs_gateway_app.log.2023-07-28.log

Ran tests on different nodeURLs ankr and infura and still issue occurs. On the screen recording below, observed that its running the amm_data_feed_examples and when gateway has disconnected and reconnected the error on price query occurred
Steps to reproduce:

Setup both latest development for source and gateway
Run amm_data_feed_example script
While running for couple of minutes, stop gateway then start again
Observe the issue on log panel

https://github.com/hummingbot/gateway/assets/73840223/31554f45-774e-4cdf-914f-edf52238ca35
Another observation, when the issue occurred we stop/start both gateway and client without restarting and seem to work ✅
https://github.com/hummingbot/gateway/assets/73840223/7ced1e12-9bce-48c4-b8f2-57b70fa397fb
We have also ran test this by disconnecting from the network for both the script and amm-arbitrage and still works as expected. Still Looking for other possible scenarios
logs_amm_data_feed_example.log
logs_gateway_app.log.2023-08-01.log

Description: Fix gateway connection issue described above
Acceptance criteria: gateway Instance work after restart, QA tests passed, PR including tests, etc
Bounty: P2
Bounty Amount: 50,000 HBOT
Status: OPEN

Reproduced on the latest development
https://www.loom.com/share/68653d8735ab47d6be84d65914d5cd03
However noticed that there's a workaround that you could try to do the Source gateway restart with the delay / Restart the client and gateway with the delay (not start immediately) and it could help to fix the issue and start the script successfully

Heyy @nikspz, I've tried to fix this issue in the above PR. Could you take a look and check if it's eligible for the bounty?
Thanks :)

@OjusWiZard please fill up & submit the AML Policy Form via this link: https://forms.gle/p3zhnSpkP2eZnj9m9
This is a one-time process for each ETH Address used to receive HBOT bounty.

Hey @carlitogetaladajr, I've filled out the form, thanks!

Thanks for your participation. The bounty has been sent to @OjusWiZard . The fix has been deployed on development branch and hummingbot version 1.21.0.


Unable to import net.serenitybdd.junit.runners.SerenityRunner;

I am not able to follow through the course as I am stuck at lesson "Writing Your First Serenity BDD Test Case"
The issue is it failed to recognize SerenityRunner.

this was resolved when i manually updated the serenity junit version with the version number in pom.xml file.

net.serenity-bdd
serenity-junit
3.3.4
test


support for ZCU1xx

LK"I
Hi ,
Is this repository supports ZCUxx chips?
like you described here (just replace the Zynq base release with 2018.1-zcu106-release.tar.xz :
https://www.digikey.com/eewiki/display/linuxonarm/Zynq-7000

@HarelAshwal not yet, any cheap ZCUxx boards/kits?

LK"I
Don't have a clue...
I'm going to use this board... (9K$....)
https://www.xilinx.com/products/boards-and-kits/zcu111.html


could I set right style when I specify menuPosition={'right'}

I specify menuPosition={'right'} of sideMenu, and all children element disappeared.
then I notice that side menu bespread whole screen while I can't see them in my viewport, so I set styles below.
container: {
  flex: 1,
  left: window.width * 1 / 3,
  width: window.width * 2 / 3,
}

now I could layout correctly, but my question is, why shouldn't I set right of styles, like below
// not work
container: {
  flex: 1,
  right: 0,
  width: window.width * 2 / 3,
}

thanks for your time :)

I think right: 0 would also works, but the width is still to big to see your elements. Try to use textAlign: 'right' or alignItems: 'flex-end'.

@Kureev  thanks for your reply, I try your idea but also not work, maybe some other reason cause right: 0 work incorrectly.
in any case, left: window.width * 1 / 3 seem normally, thanks for your smart react-native-side-menu :)

There is still a room for improvements, but I don't have enough time to work on it :(

Can you explain me how to reproduce a bug?

I'm not sure you could reappear the same bug as mine, but I'll try my best to select relevant code below.
a little messy :)
App.js
<SideMenu
        menu={<Menu />}
        isOpen={this.state.isOpen}
        onChange={(isOpen) => this.updateMenuState(isOpen)}
        menuPosition={'right'}  // I change default position here
        disableGestures={true}>  // here I disable geatures
  <CustomNavigator style={{ flex: 1, position: 'absolute', height: 64, top: 0, left: 0, }} />
  <CustomView style={{ flex: 1, marginTop: 64, }}>
  <Swiper style={{ flex: 1, }}>  // from leecade/react-native-swiper/
    <ScrollView style={{ flex: 1, }}> // some custom component </ScrollView>
  </Swiper>
  </CustomView>
</SideMenu>

Menu.js
const window = Dimensions.get('window');

<View style={{ flex: 1, }}>
  <View style={styles.container}>
  </View>
</View>

container: {
    flex: 1,
    position: 'absolute',
    left: window.width * 1 / 3,  // right: 0  can't work :(
    top: 0,
    width: window.width * 2 / 3,
    height: window.height,

},
although above code work normally now, but I also want to find out why right: 0 not  layout incorrectly.
regards.
thanks for your patience :)

Thanks for your example! I'll do my best to figure out why it doesn't work for you :+1:

a quick fix for this could be to go to index.js and edit the SideMenu's render function like this:
render() {
    if (this.props.menuPosition == 'right') {
        var offsetStyle = {left: deviceScreen.width - this.props.openMenuOffset};
    }
    else {
        var offsetStyle = {right: deviceScreen.width - this.props.openMenuOffset};
    }
    const menu = <View style={[styles.menu, offsetStyle]}>{this.props.menu}</View>;

    return (
        <View style={styles.container} onLayout={this.onLayoutChange.bind(this)}>
            {menu}
            {this.getContentView()}
        </View>
    );
}

I think this one can be closed.


Typescript in project reference cannot be debugged in typescript (with map files)

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.88.1
OS Version: Windows 11 Pro 23H2
TSC version: 5.4.2

Steps to Reproduce:

Attached ZIP contains directory with 2 TS projects: 'lib' is configured to compile with composite true; 'proj' is configured to reference 'lib'
If the import in 'proj' imports from 'lib/src' (importing the .ts file), 'proj' will not run: 'Cannot find module'
If the import in 'proj' imports from 'lib/dist/src', 'proj' will run. VS code allows setting a breakpoint in the imported 'simple.ts' file line 2, but it is not hit.
Setting a breakpoint at 'proj' simplecaller.ts line 7 and stepping into the reference project puts you in 'simple.js', despite the 'simple.js.map' file being present in the directory.

projreftest.zip

I'm not the only one with this issue: see #165664  which got bot-closed

Sigh. Project references, while a very useful feature of TS (and been around a while!) are basicallly useless in VS Code at this point. No way to reasonably debug across them...


Run validations when type check failed

Not a big issue, but could be annoying
For example let's assume we have a active_interaction object used as a form object.
class SomeForm < ActiveInteraction::Base
  integer :value
  string :comment

  validates_presence_of :comment

  def execute
  end
end

Currently
When user enters incorrect number, he gets an error for value but no error for comment. Once the value error is fixed user submits the form and now gets error on comment. Which looks strange as on the previous submit there were no errors for comment.
How it should be
In this use case we need to present all errors at once, including type and validation errors.
Currently don't know any work around for it. And would be nice to have such feature by default.
Thoughts?

See #354, #300, and #223. This is the expected behavior:

When you run this interaction, two things will happen. First ActiveInteraction will type check your inputs. Then ActiveModel will validate them. If both of those are happy, it will be executed.

https://github.com/orgsync/active_interaction/tree/v3.5.2#validations

This has come up several times which means it might be worth revisiting. I'm already doing a major error overhaul in v4.0.0.

As workaround it is possible to use everything as interface to skip type validation, right?


Reference to input element

IMPORTANT: Before reporting anything here please report it on the main project at https://github.com/mdbassit/Coloris That is the main repository where bugs should be fixed and we will port them to this project once the original Coloris project does a release.
If your issue is NPM or TypeScript related then it is OK to report that here.
Is that possible to support passing element as an input on top of query Selector for el? In case of shadow dom, passing the query selector might not work.

This should be a feature request over at https://github.com/mdbassit/Coloris, we only package the code and add the TypeScript typings.
Currently the library really does expect a selector, and also seems to use event delegation. Perhaps it might be easier to specify the base element (= the shadow root) against the selector is resolved, but that can be discussed over at the main repo.


Set Save button on top, enable it only when devfile changed

What type of PR is this:
/kind feature
What does this PR do / why we need it:
Which issue(s) this PR fixes:
Fixes #6978
Fixes #7003
PR acceptance criteria:

[ ] Unit test

[ ] Integration test

[ ] Documentation

How to test changes / Special notes to the reviewer:

/override OpenShift-Integration-tests/OpenShift-Integration-tests
Flaky test
[FAILED] [4.728 seconds]
odo create/delete/list/set namespace/project tests delete project when force-deleting a valid project [BeforeEach] should successfully delete the project synchronously with --wait
  [BeforeEach] /go/odo_1/tests/integration/cmd_namespace_test.go:74
  [It] /go/odo_1/tests/integration/cmd_namespace_test.go:104


Add offset additionally to multiplier to make it possible to support devices which report Voltages

Is your feature request related to a problem? Please describe.
I'm trying to add some entities which report voltages instead of battery percentage to my battery state list. Unfortunately the multiplier is not enough as a battery is empty way above 0V
Describe the solution you'd like
Add a offset value which gets subtracted before multiplication with the multiplier. So for 2 AA battery devices which have a working voltage range of about 2V = empty to 3V = full, you could set offset = 2 and multiplier = 100.

Check KString/thresholds function in v3.0.0 version


Limit linkerd instances search to a particular Kubernetes namespace?

Hi,
I'm trying to deploy linkerd-viz in a Kubernetes cluster where I have rights on a single namespace, and I'm getting the following errors:
Failed to list *v1.Service: User \"system:serviceaccount:my-namespace:viewer\" cannot list all services in the cluster"
Failed to list *v1.Pod: User \"system:serviceaccount:my-namespace:viewer\" cannot list all pods in the cluster"
Failed to list *v1.Endpoints: User \"system:serviceaccount:my-namespace:viewer\" cannot list all endpoints in the cluster"

It seems that Prometheus is trying to list services / pods in the whole cluster. Is there a way to have it restrict itself to the namespace my-namespace only? I was thinking that using a __meta_kubernetes_namespace meta label could do the trick, but I'm unsure whether that will change the API call that Prometheus does, or just filter the services afterwards.
Note that I run linkerd-viz is run under a viewer service account that can list services / pods inside my namespace.
Thanks!

Hi @christophetd,
I think may be correct that using __meta_kubernetes_namespace will only be a post-filter, but won't affect the initial query.
If you want to try that anyway, I think you'd use something like:
  relabel_configs:
  - source_labels: [__meta_kubernetes_namespace]
    action: keep
    regex: my-namespace

FWIW, it looks like the feature you want may have been implemented recently in Prometheus:
https://github.com/prometheus/prometheus/pull/2642
Let us know what you find.

Thanks for the pointer, it seems that the feature has been implemented starting from Prometheus v1.7.0. If that's fine for you, I can send out a PR to update it to this version (v1.4.0 currently used is from Nov 2016, it could use some update anyway)

I opened a PR to update to Prometheus v1.7.0, see #45

This is awesome, thanks @christophetd. We'll take a look at the PR.

Addressed in https://github.com/linkerd/linkerd-viz/pull/45. Thanks @christophetd !

could you share how its configured in linkerd plz?


Release Google.Cloud.Video.LiveStream.V1 version 1.0.0

Changes in this release:
No API surface changes; just dependency updates and promotion to GA.

Created release for Google.Cloud.Video.LiveStream.V1-1.0.0


Deprecate automated postman test

This has been failing for a while and we should remove this job. We now have sufficient integration tests and other automated tests that we can safely deprecate.
https://github.com/stargate/data-api/actions/workflows/postman-docker.yaml

Postman was used as post-deployment tests. Without it, we will not have any automated tests to run after deployment.
Propose a possible solution: we could add a config in the current Integration tests. After the deployment, we could let the ITs run against different environments.


Overwrite Existing / Skip on Generation Update

If there are multiple layers with the same name during the generation, just the first one will be used to create a Shared Style. Later on, if you add a layer with the same name as an already existing Shared Style, the style itself will be overridden, and all the related instances updated.

@lucaorio Tested fairly extensively with:

all new styles
new styles and existing styles
object names overlapping with existing style names
combinations of multiples of object names and style names
Works as expected/discussed in all situations. Good work 👍

@ShaneNZ Excellent! Going to merge it to master tonight. I wish I had more time to spend on open source stuff haha Thanks!

Know what you mean - you got there fast than I did :)
On 7 March 2017 at 09:51, Luca Orio<EMAIL_ADDRESS>wrote:

@ShaneNZ https://github.com/ShaneNZ Excellent! Going to merge it to
master tonight. I wish I had more time to spend on open source stuff haha
Thanks!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/lucaorio/sketch-styles-generator/pull/4#issuecomment-284528359,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACaFVqaEIOBoz_ReXDxmo3z4aCBCPWD8ks5rjHHsgaJpZM4MTybi
.


[Q]: How long does the "bash run.sh --stage -1 --stop-stage 3" take to finish?

It's been running for good 4 hours on the rtx 4090 under wsl2.
Currently its at
Extracting and storing features:  77%|████████████████████████████████████████████████████████████████▎                   | 157105/205044 [1:11:35<18:37, 42.88it/s]

The answer is 5 hours :)


KAFKA-14569: Migrate Connect's integration test EmbeddedKafkaCluster from ZK to KRaft mode

https://issues.apache.org/jira/browse/KAFKA-14569
The existing EmbeddedConnectCluster (used in Connect integration tests) uses a backing EmbeddedKafkaCluster which internally also spins up an EmbeddedZookeeper.
This patch migrates the EmbeddedKafkaCluster to run in KRaft mode, leveraging the existing KafkaClusterTestKit from core.
Connect / MirrorMaker integration tests which setup a Kafka cluster with SASL_PLAINTEXT or  SSL listeners needed to be updated to take into account the controller listeners as well and also update the authorizer used (from kafka.security.authorizer.AclAuthorizer to org.apache.kafka.metadata.authorizer.StandardAuthorizer, see KIP-801) since the old authorizer relied on ZooKeeper.
The existing EmbeddedKafkaCluster had some logic to restart brokers and have them listening on the same ports as earlier (in order to verify Connect's functionality when its backing Kafka cluster goes down and then comes back up). This was refactored to move the responsibility of using a fixed port in the broker's listeners config to the tests themselves.
Some changes to KafkaClusterTestKit in order to allow externally configuring listeners configurations and other minor improvements.

Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Removing the stale label because I plan to re-visit this one.

@yashmayya Are you still working on this to get it over the finish line or is it okay for me to take over?

@mdedetrich apologies for the late response, I didn't get notified for your comment oddly enough. Please feel free to take over, thanks!

I've published https://github.com/apache/kafka/pull/16599, which is just a squash+rebase of this PR.


Termux

Make it work on termux or make it work on mobile

Pillow and Ultralytics package can't be installed on termux

Make it work on termux or make it work on mobile

How exactly? If the required libraries can't be installed on Termux how can I make it work on Termux??


Begin connection pool implementation

Not done yet, but I'd like feedback on the initial implementation. It's dead-simple, since we don't need to expose the pool publicly. What do you think?

@tannernelson Now it creates new connections on the fly and only when necessary. Good call.
One big issue remains: When the number of concurrent requests exceeds the pool limit, a deadlock occurs. See the while loop at line 118. The problem is that while attempting to block the caller until a connection becomes available, I'm also blocking the Database thread, which prevents connections from being recycled.
We need a way to put the caller on hold without blocking the Database thread, while still synchronizing access to all resources. It shouldn't be hard, but I thought I'd share the update so far to get feedback anyway.

@collinhundley thanks for this! We're going to postpone this for a 1.1 tag to get 1.0 up and running on release. Once we get all of those out the door, we'll start working on this.
Thanks as always for some awesome content!

@LoganWright no problem. I need it for my own project so I’ll keep hacking on my fork. It should be ready to merge by 1.1.

PS, this will be merged into the new connection-pool branch. The implementation is finished (I think), so feel free to review it.


Add bhavabhuthi's details to the team

I have run it in local as discussed and I can see no errors. CC: @PixeledCode


Does it support tailwindUI

Hey
awesome new library! I like the direction this is going with integrating with tailwind. I wonder whether it also supports tailwind UI. THe question could be more general like supporting tailwind plugins both those that add utilities and those who add components (i.e https://github.com/tailwindcss/custom-forms )
excited to test this out!

Hi @oorestisime and thanks for the feedback 😄
You should be able to use this library to to build what you see in Tailwind UI, as it is based on the same CSS. Though it would not be a copy/paste operation, as the API is different. Also the plugins hook into their configuration tool I believe, so that would not work.
That said! It would be possible to build like a component library reflecting Tailwind UI where you would consume actual components instead of copy/paste code from their website 😄

Please reopen if any more questions 😄


Postmaster::Queue::Index (Old 231)

In gitlab by @A-Lawrence on Aug 31, 2015, 01:08
null

In gitlab by @A-Lawrence on Aug 31, 2015, 01:14
Added the Postmaster::Queue::index page.  Resolves issue #488512 (231).
ÔåÆ <<cset 52e888ceec83>>
ALawrence66 - 2015-02-20T02:06:09.453658+00:00


Authentication with ASP 5 and Identity/SignInManager

It seems from the limited documentation that this is intended to sit on top of and work with the Indentity framework, but there doesnt look to be an interface to create an IdentityUser as a result of logging in with Authentication methods. I'm not sure if this is a bug or something I'm missing. Taking a look at the Account controller from the default ASP 5 .NET project in VS, the method that expects to handle the successful login has the user logged in but not where it expects it.
public async Task<IActionResult> ExternalLoginCallback(string returnUrl = null)
{
            if (HttpContext.User.Identities.Any(identity => identity.IsAuthenticated))
            {
                _logger.LogDebug("Users have been logged in by an oauth/facebook/google provider");
            }
            var info = await _signInManager.GetExternalLoginInfoAsync();
            // Always null
            if (info == null)
            {
                return RedirectToAction(nameof(Login));
            }
}

Is there a trick to get these two systems to work together or is it a bug that they authenticate on different levels?

You create users via the UserManager.   The template flow creates a user in the confirmation method, see: https://github.com/aspnet/Templates/blob/dev/src/Rules/StarterWeb/IndividualAuth/Controllers/AccountController.cs#L201

In theory yes, but it never gets to that point. It bails out before forwarding there, https://github.com/aspnet/Templates/blob/dev/src/Rules/StarterWeb/IndividualAuth/Controllers/AccountController.cs#L165
Where it needs to get to here in order to get forwarded to the ExternalLoginConfirmation. https://github.com/aspnet/Templates/blob/dev/src/Rules/StarterWeb/IndividualAuth/Controllers/AccountController.cs#L192

No, the external login identity its looking for specifically comes from the authenticationScheme defined in the identity options, by default its something like "Microsoft.AspNet.Identity.External".  HttpContext.User is not relevant to SignInManager at all

So basically something needs to sign in using the authenticationScheme that this method is expecting if you are trying to get the template flow to work.  Note this external cookie flow is purely a mechanism that the templates created, you are free to replace this/roll your own at the end of the day

Ok, thanks. Just digging in from the templates it made it seem like they were interconnected. Good to know.


fix(README): update links to expedition.dev

:tada: This PR is included in version 1.6.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Update CatBoost estimators to return self in fit rather than underlying model for consistency

Updates CatBoost estimators to return self in fit rather than underlying model for consistency. Our other estimators return self, catboost should too.
This fell out of work I've been doing w/ components and pipelines returning Woodwork and general cleanup of EvalML 😛

@chukarsten I think you're right in that our other tests don't explicitly check that, for example, the return type of fit is self. Feature importance is a little less obvious but I think in most cases we have test_feature_importance which checks using numpy's np.testing.assert_almost_equal method whether or not our feature importance method returns the same values as using the external library directly, for example. That check will yell if its not comparing two numpy arrays and therefore helps us check if the return value is the expected numpy array :d

@chukarsten I think you're right in that our other tests don't explicitly check that, for example, the return type of fit is self. Feature importance is a little less obvious but I think in most cases we have test_feature_importance which checks using numpy's np.testing.assert_almost_equal method whether or not our feature importance method returns the same values as using the external library directly, for example. That check will yell if its not comparing two numpy arrays and therefore helps us check if the return value is the expected numpy array :d


Refactor Makefile for older version of pandoc

Older versions of pandoc are not capable of using wkhtmltopdf under the
hood, so the html file and then the pdf file has to be manually created
using pandoc and then wkhtmltopdf, respectively.

Solved by #1


Bump RSpec for Tests

Hi,
Is there any specific reason why RSpec is kept at such old versions? I have seen #181  and am wondering if that is still the issue. I am trying to package dm-* gems for Debian and RSpec 1 has been out of Debian for quite some time (AFAIK, it is not available in any major OSs). So, can you please update the Tests to run using the later versions of RSpec?

The project is mostly unmaintained at this point.
PRs welcome, but you'll have to keep the test matrix all running.
We don't update the tests because, for the purposes of end-users, the tests are not required to use the gem.
It may happen at some point, but it is not a small undertaking. Sorry, I cannot guarantee anything.


improve behavior of null strings in symbols, arguments

Reported by Ansley B Manke on 2 Jan 2004 19:48 UTC
-- when a null argument is passed to a script (and probably other times as well)
the symbol is difficult to work with.  See examples in
~ansley/ans_ferret/users/furue/sub_subst.jnl
Ideas:

when an empty string is detected as part of a symbol or argument substitution
then the substitution string is interpreted as the single character NULL (binary
zero).  This will presumably appear as a string of length 1 to TM_LENSTR1  and
will get substituted as a NULL character wherever the substitution is used (also
check how TM_LENSTRsees it).
after symbol substitutions have been completed the string is de-nulled (nulls
removed)

Migrated-From: http://dunkel.pmel.noaa.gov/trac/ferret/ticket/757

Comment by<EMAIL_ADDRESS>on 17 Jun 2005 18:14 UTC
Further examples in Bugzilla #652

Attachment from<EMAIL_ADDRESS>on 20 Jun 2005 16:28 UTC
examples of this bug
REPLACE THIS TEXT WITH UPLOADED FILE ./attachments/TRAC_757_GIT_357/sub_subst.jnl


Add a xAxisLabel and yAxisLabel for Bar Chart

Expected
We will allow users to add labels to the axis on the bar chart, the same way we do for the step chart.

This feature will be greatly appreciated. Looking forward to it

@iamkk11 it should be available now in the new release 2.9.11. Let us know if there is an issue.

Hi @dalerasrorov-eb  its not working see below error message.
×
Error: Method not supported by Britechart: xAxisLabel yAxisLabel
my package.json is:
"britecharts": "^2.9.1",
"britecharts-react": "^0.4.1",

@iamkk11 Sorry to hear that. I tried to use the latest britecharts bundle and had those new functions available with no issue. Try doing yarn install first and try again. If that doesn't work, removing node_modules folder: rm -r node_modules and then yarn install. I think that the updates didn't go through due to cache or other reasons. Let me know if those didn't work. Thanks!

@dalerasrorov-eb  i did npm start -- --reset cache to no avail.. I am using npm

It works well for me: https://codepen.io/Britecharts/pen/yEwpeg?editors=1010
Just to make sure @iamkk11, you are trying to add labels in the bar chart, right?


Add strictness flag which causes LSODA to exit on warning

Add flags to the Python wrapper which tell LSODA to terminate whenever the
` 'lsoda -- warning..internal t = %g and h_ = %g are such that in the machine, t + h_ = t on the next step"
warning is issued.
Currently, the user is only told about this warning through stderr. Allowing the user to set numbalsoda to be more strict will allow for better unit testing in code using numbalsoda.
To do this, a flag is added to the numba function lsoda(), which defaults to the original behaviour. The same flag is added to LSODA::lsoda, LSODA::update and lsoda_wrapper. When this flag is set to true and the above warning is issued, the solver will terminate with istate* = -2.

It looks like you didn't pass exit_on_warning to lsoda_update: https://github.com/joeyshuttleworth/numbalsoda/blob/bfe92d1df2b9dc55ddb898e0d071da1d8b55e9eb/src/wrapper.cpp#L39

Can you fix this?

Whoops, I've fixed that. I also added a test which shows the desired behaviour.

Thanks!


Default measurement for annealing for the first demo implementation.

This PR brings the proposal for the default square measurement for annealing needed for the first demo.
WIP.

Base branch should be https://github.com/qilimanjaro-tech/qililab/pull/767 ?

Base branch should be #767 ?

It is, I branched out from yours.

Made some comments

addressed, thanks!


"Recea" is listed in the drop-down menu

Description:
When user writes "Recea" the location appears in the drop-down menu next to the comune and the county witch is a part according to the law
Precondition:
The website is up an running.

Step 1
Write in the search bar "Recea"

Expected results
The location appears in the drop-down menu as "Recea, Argeș"


DVO demo: real Ring project

Description
DVO demo: real Ring project
Type of change

Demo

Testing steps
N/A
Checklist

[ ] make before_commit passes
[ ] updated documentation wherever necessary
[ ] added or modified tests if necessary
[ ] updated schemas and validators in insights-data-schemas in case of input/output change

Codecov Report
Patch has no changes to coverable lines.
:loudspeaker: Thoughts on this report? Let us know!.


remove sigstore-review-bot

Summary

Release Note

Documentation

should we terminate that bot as well? i mean delete the account

yes, i can do that but the permissions need to be removed while the account still exists


Suggestion for adding bitswap to substrate

I can fix the build failures, but wanted to get some early feedback first. This is intended to superceed the PR by @expenses https://github.com/paritytech/substrate/pull/6795 in a way that it is useful for sunshine-protocol to use this feature effectively.
ipfs-embed now supports plugging in an external networking stack. [0] is the corresponding PR to use the NetworkService of a substrate light client as the backend for ipfs-embed in the sunshine client.
all the changes outside of sc-networking come from adding a generic parameter to the NetworkService that implements the MultihashDigest trait from tiny-multihash. This is used by bitswap to verify the hashes of ipfs blocks. (We require this as we have a custom hasher based on sp-trie that checks that the content identifier matches the trie root).

adds providers, provide and unprovide methods to NetworkService for registering content in the dht.

adds a BootstrapComplete event to NetworkService, and starts bootstrapping the dht. (not sure why this wasn't required before)

adds bitswap send, send-all, want and cancel methods to NetworkService for making and handling bitswap requests.

when a peer is discovered it calls the connect method on the bitswap NetworkBehaviour.

uncomments the role checking causing light client peers being dropped prematurely.

[0] https://github.com/sunshine-protocol/sunshine-core/pull/5

This can be rebased on top of #6795 as a follow-up.

@tomaka what are your thoughts on supporting light client to light client network protocols?

wanted to get some early feedback first

I'm not exactly sure what you would like to get feedback on.
If the question is whether the feature should go in Substrate, judging by the number of :rocket: emojis I guess the answer is yes.
If the question is whether this should be implemented using a NetworkBehaviour, well that's what matches our architecture so yes.
I can give a review of small things to tweak, but I suppose you're not at this point yet.

@tomaka what are your thoughts on supporting light client to light client network protocols?

What do you mean by that? Light clients can connect and can theoretically talk to each other. In practice there isn't anything that a light client could ask to another light client at the moment.
If you're asking in the context of the DHT, our general assumption at the moment is that light clients are short-lived and that they're more or less parasites for all networking-related purposes (cc #3303).

I'm not exactly sure what you would like to get feedback on

Whether or not this belongs in substrate and if a network behaviour is the way to go. Question answered, thanks.

What do you mean by that? Light clients can connect and can theoretically talk to each other. In practice there isn't anything that a light client could ask to another light client at the moment.

The intention of this PR is to change that.

I can give a review of small things that could be tweaked, but I suppose you're not at this point yet.

I think it's ready for review, just waiting for ci to turn green.

I gave a review in #6795
I'd prefer if possible to document how bitswap works from a high-level perspective so that people can actually understand how to use it, rather than just throwing an API in the air and implicitly asking users to look at the code of libp2p-bitswap to understand what to do.

Reposting relevant review comment: https://github.com/paritytech/substrate/pull/6795#discussion_r495795259

Closed b/c of staleness.


feat(StopRedesign): Route Filter Pills

Summary of changes
Added filter pills to filter the departure/routes list on the new stop redesign page

Asana Ticket: Filter Routes or Departures by Mode

General checks

[ ] Are the changes organized into self-contained commits with descriptive and well-formatted commit messages? This is a good practice that can facilitate easier reviews.
[ ] Testing. Do the changes include relevant passing updates to tests? This includes updating screenshots. Preferably tests are run locally to verify that there are no test failures created by these changes, before opening a PR.
[ ] Tech debt. Have you checked for tech debt you can address in the area you're working in? This can be a good time to address small issues, or create Asana tickets for larger issues.

New UI, or substantial UI changes

[ ] Cross-browser compatibility is less of an issue now that we're no longer supporting IE, but changes still need to work as expected in Safari, Chrome, and Firefox.
[ ] Are interactive elements accessible? This includes at minimum having relevant keyboard interactions and visible focus, but can also include verification with screen reader testing.
[ ] Other accessibility checks such as sufficient color constrast, or whether the layout holds up at 200% zoom level.

New endpoints, or non-trivial changes to current endpoints

[ ] Have we load-tested any new pages or internal API endpoints that will receive significant traffic? See load testing docs
[ ] If this change involves routes, does it work correctly with pertinent "unusual" routes such as the combined Green Line, Silver Line, Foxboro commuter rail, and single-direction bus routes like the 170?

I don't know why it feels slow for you.  I am seeing it load in about a second.  It shouldn't be anything other than html and the site that is loading

I couldn't come up with a good way to separate the pill buttons into their own component.  Since what pills display depends on the departures array data, and the state of the pills affects the display of the departures array data.
It seems like their state is too closely linked to create a clean separation from the 2 pieces of the page.

@kotva006 Yeah you're right, you'll need the departures and filtered departures in the top-level component. There's a pattern for that – you can pass the unfiltered departures and the filtering function into the filter-pills-component as props. Something like <DepartureFilters departures={departures} onModeChange={setFilteredDepartures}  /> should do the trick, give it a try!


html2pdf selector

Are you using the latest version of jsPDF?
Yes
Have you tried using jspdf.debug.js?
Yes
Hello everyone,
I need to generate some data in a table from my html document to a pdf. The problem is I have to do it dynamically and not just by parsing the entire document.
Like that :
html2pdf(document.querySelector(table1)) > to render the table on the 1st page
html2pdf(document.querySelector(table2)) > to render the table2 on the 2nd page (...)
(I know it's not a good syntax since it will create 2 pdf documents...)
The problem is that html2pdf function remember the place of the element on the document and generate the element at the same place. I would like to do a simple append, like fromHTML is able to do, but fromHTML is very limited in term of styling...
Second problem, if the document you parse is to high, it will not create a new page with the rest of the document but crop it, another thing that fromHTML can do.
So what I would like to do:

Add pages like I was able to do with fromHTML:
pdf.fromHTML(...)
pdf.addPage()
pdf.fromHTML(...)
Generate a part of the document at the beginning of the pdf and not at his actual place in the html
Don't crop if not enough place but create a new page with the rest of the content

So is there a way to do it currently or will it be possible in the near future?
Thanks in advance for your help :)

Thanks for the suggestion. Yeah that sounds like something we should do. There's a new .html() API in the works to replace the addHTML and fromHTML plugins.

Cool hope to see more in the near future, I will continue to use fromHTML until it comes, thanks for your answer.


Add "types" path to exports

Closes/fixes #620
Checklist

[ ] This PR makes changes to the public API
[X] I have included links for closing relevant issue numbers

thanks for contributing