Fixes and Rev4 updates for installed packages controls

These are all the changes we have made in our repo at https://github.com/djhaynes/inspec-profile-disa_stig-el7 relating to installed packages controls. This includes updates of these controls to RHEL7 STIG Rev4, as well as various fixes.

@trevor-vaughan I addressed your concerns and this should be good to go.


Update README.md

I believe the instructions about composer install need to be moved up. I spent a few minutes of frustration because of this. :-)

Thanks for the PR,
it has given me trouble too. It was even further down before, I thought I had moved it high enough.

Hi @ankush981
I'm wondering if you still use Flyimg in your company/organisation ?
If so, it would be great if you could add your entry here in the Adoperts file
https://github.com/flyimg/flyimg/blob/main/ADOPTERS.md
many thanks!


Transfer of hatch-nodejs-version in JupyterLab org

Problem

Through the heavy usage of the extension template project, almost all updated extensions out there depends on the build dependency: hatch-nodejs-version. As it is currently affiliated to Angus (thanks a lot for that great plugin), it would be better for the long term maintenance to host that project under a community organization.
Proposed Solution

As jupyterlab GitHub org is already hosting the other essential hatch plugin for packaging JS assets in a Python package - aka hatch-jupyter-builder - and after quickly exchanging about this with @agoose77. I propose to transfer hatch-nodejs-version to the JupyterLab organization and related to it the pypi package.

Thanks @fcollonval for the proposal and @agoose77 for the plugin!
Indeed given the wide usage of this plugin, it would make sense to move it to the jupyterlab org :+1:


Current cloudformation template opens up entire cluster to the wild

The security group settings should be changed to only allow incoming SSH connections, with anything going outwards.

This is done.


Revisit form-based file uploads

There's a sustained interest in having Falcon support HTML form-based file uploads.
"In django i just did, with form: file = request.FILES['image'].read()  without form: file = request.body"
How do we do this in a performant manner? How do we keep dependencies minimal?

Thinking about this again... should this be an add-on, or should form params and file uploads be part of the core framework? Either way, the functionality is needed!

See also:

#493
#418

Hey @kgriffs,
I'm interested in seeing multipart file upload capabilities added to Falcon.  I think the main design decision is whether to use the multipart form parsing available in cgi.FieldStorage  or  write our own multipart form parser.  falcon-multipart uses cgi.FieldStorage so if that is acceptable to you, I think it would be pretty straightforward to add that to falcon.

I'm going to close this now since the work is primarily tracked as https://github.com/falconry/falcon/issues/953 (within the Falcon 3.0 milestone).


Issuing credits to Matt

Issue 120192 credits to @anoanoano.

 Follow up
On Wednesday, 06-Mar-24 09:44:02 UTC by Gov4Git dev
Issued 120192 credits to member @anoanoano.


Fix #303 static file support for sirepo tornado

@e-carlin we should wait to merge until #304 is merged

Moved systemd scripts to prepare() from enable(), because static_files_gen needs to call /srv/sirepo/start
rsconf_service_docker_pull has a static_files_gen hook to callback to the component (e.g. sirepo)
systemd service_exec & cmd were confusing. consistently named timer_exec and docker_exec
nginx serves files from /srv/www. /usr/share/nginx/html shouldn't be modified, and apache uses /var/www

@e-carlin ping

Not sure if this is related or not but when trying to run a sim in Sirepo using this rsconf config I'm seeing

check the time on v3 and v4 machines.

The time is the same

We debugged this. The files in v3 download were out of date (https://github.com/radiasoft/rsconf/issues/315) and the provisioner was therefore not editing out v4.radia.run from /etc/hosts.


add internal api to create user

Let me update the PR title to follow https://www.conventionalcommits.org/en/v1.0.0/#specification


[ml] Add Python 3.7 end of life message

Description
This PR adds a notice to the 1.11.0 release notes that support for Python 3.7 will be ending in January 2024.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

API change check
APIView has identified API level changes in this PR and created following API reviews.
azure-ai-ml


Variant Picker still not working, new error

What I'm trying to achieve
After trying the fix from #3127 or even starting from scratch all pages now have the following error:
Uncaught TypeError: Cannot read property 'isPrototypeOf' of undefined
    at J (storefront.js:144)
    at Module.641 (storefront.js:144)
    at n (storefront.js:1)
    at Object.1384 (storefront.js:15)
    at n (storefront.js:1)
    at Object.<anonymous> (storefront.js:15)
    at Object.1383 (storefront.js:15)
    at n (storefront.js:1)
    at Object.1368 (storefront.js:9)
    at n (storefront.js:1)

Steps to reproduce the problem

Fresh install & Setup 618ad655ebdb3e5f2937e43855fe3e7cb9d23515
Variant picker is still broken, new error now (shown above) exists on all pages.

Not sure what else may be broken.

@robotswordfight we cannot reproduce the error you reported. Can you give us more info, like Node version?

➜ node -v
v10.12.0

Ah! It only happens with the development react.
I changed this:
  reactPath = 'node_modules/react/cjs/react.development.js';
  reactDomPath = 'node_modules/react-dom/cjs/react-dom.development.js';

to this:
  reactPath = 'node_modules/react/cjs/react.production.min.js';
  reactDomPath = 'node_modules/react-dom/cjs/react-dom.production.min.js';

and no more error and variant works!

Sorry but I'm facing similar issue with a fresh new install on Ubuntu 18.04.
Node version - 10.12.0
I am somewhat familiar with Django and python but I have no knowledge about react and very little with webpack. It looks like the VariantPicker is not properly constructed causing an issue in the console and no element to be display in my detail product page.
See below the result in the console when I run the webserver and try to show the details of the product
http://<IP_ADDRESS>:8000/en/products/chambers-williams-4/
storefront.js:144 Uncaught TypeError: Cannot read property 'isPrototypeOf' of undefined
at J (storefront.js:144)
at Module.641 (storefront.js:144)
at n (storefront.js:1)
at Object.1384 (storefront.js:15)
at n (storefront.js:1)
at Object. (storefront.js:15)
at Object.1383 (storefront.js:15)
at n (storefront.js:1)
at Object.1368 (storefront.js:9)
at n (storefront.js:1)
Expected behavior

Observed result

Sorry but I'm facing similar issue with a fresh new install on Ubuntu 18.04.
Node version - 10.12.0
I am somewhat familiar with Django and python but I have no knowledge about react and very little with webpack. It looks like the VariantPicker is not properly constructed causing an issue in the console and no element to be display in my detail product page.
See below the result in the console when I run the webserver and try to show the details of the product
http://<IP_ADDRESS>:8000/en/products/chambers-williams-4/
storefront.js:144 Uncaught TypeError: Cannot read property 'isPrototypeOf' of undefined
at J (storefront.js:144)
at Module.641 (storefront.js:144)
at n (storefront.js:1)
at Object.1384 (storefront.js:15)
at n (storefront.js:1)
at Object. (storefront.js:15)
at Object.1383 (storefront.js:15)
at n (storefront.js:1)
at Object.1368 (storefront.js:9)
at n (storefront.js:1)
Expected behavior

Observed result

Can you please advise?
Thanks,
Arnaud

I have the same problem, have you fixed it yet?

Ah! It only happens with the development react.
I changed this:
  reactPath = 'node_modules/react/cjs/react.development.js';
  reactDomPath = 'node_modules/react-dom/cjs/react-dom.development.js';

to this:
  reactPath = 'node_modules/react/cjs/react.production.min.js';
  reactDomPath = 'node_modules/react-dom/cjs/react-dom.production.min.js';

and no more error and variant works!

Did you try this as well?

I've just installed a new version (latest pull of master) and I'm getting this issue with the example data.  The juices are not showing variants

I think there may be something wrong with the example data from populatedb_data.json. We'll investigate the issue.

This shuold be resolved by #3926.


snapper-timeline.timer not started after reboot

Describe the bug
Snapper stops to take snapshots after a system update and reboot.
Steps To Reproduce

Enable snapper in configuration.nix
Enable autoUpgrade and autoUpgrade.allowReboot in configuration.nix
Wait for system upgrade and reboot.
snapperd.service succeds, but snapper-timeline.timer does not start (see below).
snapper will start to take snapshots after nixos-rebuild switch or systemctl start snapper-timeline.timer

Expected behavior
snapper-timeline.timer should start without intervention after a reboot.
Additional context
journalctl -u snapper-timeline.timer
Sep 03 04:45:00 univac systemd[1]: time-sync.target: Found ordering cycle on snapper-timeline.timer/stop
Sep 03 04:45:00 univac systemd[1]: time-sync.target: Found dependency on basic.target/stop
Sep 03 04:45:00 univac systemd[1]: time-sync.target: Found dependency on ntpd.service/stop
Sep 03 04:45:00 univac systemd[1]: time-sync.target: Found dependency on time-sync.target/stop
Sep 03 04:45:00 univac systemd[1]: time-sync.target: Job snapper-timeline.timer/stop deleted to break ordering cycle starting with time-sync.target/stop

systemctl status snapper-timeline.timer
● snapper-timeline.timer - Timeline of Snapper Snapshots
     Loaded: loaded (/nix/store/1n6zpiha9d6pzk4zng4faqdw5axjnbb9-unit-snapper-timeline.timer/snapper-timeline.timer; enabled; vendor preset: enabled)
     Active: inactive (dead)
    Trigger: n/a
   Triggers: ● snapper-timeline.service
       Docs: man:snapper(8)
             man:snapper-configs(5)

snapper-timeline.timer is not listed in systemctl list-units --state=failed
/etc/nixos/configuration.nix
 ...
    snapper = {
      configs = {
        "root" = {
          subvolume = "/";
          extraConfig = ''
            TIMELINE_CREATE="yes"
            TIMELINE_CLEANUP="yes"
            TIMELINE_MIN_AGE="1800"
            TIMELINE_LIMIT_HOURLY="12"
            TIMELINE_LIMIT_DAILY="10"
            TIMELINE_LIMIT_WEEKLY="5"
            TIMELINE_LIMIT_MONTHLY="12"
            TIMELINE_LIMIT_YEARLY="10"
          '';
        };
      };
    };
...

journalctl -u snapperd.service
-- Boot 28d3d1190312405ea9b1cc74d69f1fdb --
Sep 03 04:58:28 univac systemd[1]: Starting DBus interface for snapper...
Sep 03 04:58:28 univac systemd[1]: Started DBus interface for snapper.
Sep 03 04:59:30 univac systemd[1]: snapperd.service: Succeeded.

snapper list
    # | Type   | Pre # | Date                             | User | Cleanup  | Description    | Userdata
------+--------+-------+----------------------------------+------+----------+----------------+---------
   0  | single |       |                                  | root |          | current        |
   1  | single |       | Wed 09 Sep 2020 09:06:30 PM CEST | root |          |                |
   2  | single |       | Wed 09 Sep 2020 09:16:15 PM CEST | root | timeline | timeline       |
 509  | single |       | Thu 01 Oct 2020 12:00:03 AM CEST | root | timeline | timeline       |
1122  | pre    |       | Mon 26 Oct 2020 12:38:03 PM CET  | root |          | before cleanup |
1248  | single |       | Sun 01 Nov 2020 12:00:03 AM CET  | root | timeline | timeline       |
1873  | single |       | Tue 01 Dec 2020 12:00:02 AM CET  | root | timeline | timeline       |
2527  | single |       | Fri 01 Jan 2021 12:00:01 AM CET  | root | timeline | timeline       |
2625  | single |       | Wed 03 Feb 2021 07:00:04 PM CET  | root | timeline | timeline       |
3004  | single |       | Mon 01 Mar 2021 12:00:01 AM CET  | root | timeline | timeline       |
3747  | single |       | Thu 01 Apr 2021 12:00:01 AM CEST | root | timeline | timeline       |
4467  | single |       | Sat 01 May 2021 12:00:01 AM CEST | root | timeline | timeline       |
5211  | single |       | Tue 01 Jun 2021 12:00:03 AM CEST | root | timeline | timeline       |
5931  | single |       | Thu 01 Jul 2021 12:00:00 AM CEST | root | timeline | timeline       |
6675  | single |       | Sun 01 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
6699  | single |       | Mon 02 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
6867  | single |       | Mon 09 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
7035  | single |       | Mon 16 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
7202  | single |       | Mon 23 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
7231  | single |       | Wed 25 Aug 2021 05:00:02 AM CEST | root | timeline | timeline       |
7250  | single |       | Thu 26 Aug 2021 12:00:01 AM CEST | root | timeline | timeline       |
7274  | single |       | Fri 27 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
7298  | single |       | Sat 28 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
7322  | single |       | Sun 29 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
7327  | single |       | Mon 30 Aug 2021 05:00:01 AM CEST | root | timeline | timeline       |
7346  | single |       | Tue 31 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
7370  | single |       | Wed 01 Sep 2021 12:00:01 AM CEST | root | timeline | timeline       |
7394  | single |       | Thu 02 Sep 2021 12:00:00 AM CEST | root | timeline | timeline       |
7411  | single |       | Thu 02 Sep 2021 05:00:02 PM CEST | root | timeline | timeline       |
7412  | single |       | Thu 02 Sep 2021 06:00:01 PM CEST | root | timeline | timeline       |
7413  | single |       | Thu 02 Sep 2021 07:00:02 PM CEST | root | timeline | timeline       |
7414  | single |       | Thu 02 Sep 2021 08:00:00 PM CEST | root | timeline | timeline       |
7415  | single |       | Thu 02 Sep 2021 09:00:00 PM CEST | root | timeline | timeline       |
7416  | single |       | Thu 02 Sep 2021 10:00:00 PM CEST | root | timeline | timeline       |
7417  | single |       | Thu 02 Sep 2021 11:00:01 PM CEST | root | timeline | timeline       |
7418  | single |       | Fri 03 Sep 2021 12:00:01 AM CEST | root | timeline | timeline       |
7419  | single |       | Fri 03 Sep 2021 01:00:00 AM CEST | root | timeline | timeline       |
7420  | single |       | Fri 03 Sep 2021 02:00:02 AM CEST | root | timeline | timeline       |
7421  | single |       | Fri 03 Sep 2021 03:00:02 AM CEST | root | timeline | timeline       |
7422  | single |       | Fri 03 Sep 2021 04:00:01 AM CEST | root | timeline | timeline       |

Notify maintainers
@markuskowa

Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
these paths will be fetched (0.05 MiB download, 0.28 MiB unpacked):
  /nix/store/p5lnl4zr45n7mf9kz9w8yz3rqh001b5c-bash-interactive-4.4-p23-dev
copying path '/nix/store/p5lnl4zr45n7mf9kz9w8yz3rqh001b5c-bash-interactive-4.4-p23-dev' from 'https://cache.nixos.org'...
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.61, NixOS, 21.05.2861.3e09410da09 (Okapi)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.15`
 - channels(tmp): `"nixpkgs-20.03.2882.51d115ac89d"`
 - channels(root): `"nixos-21.05.2861.3e09410da09, home-manager-20.09"`
 - channels(ch): `""`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

#136925

#136925 correctly starts snapper-timeline.timer after a (manual) reboot. The new snapshotOnBoot option is working as well. Reboot after a system upgrade has not been testet yet.

snapper-timeline.timer fails to restart after a system upgrade, but snapshotOnBoot works as expected.
systemctl status snapper-timeline.timer
● snapper-timeline.timer
     Loaded: bad-setting (Reason: Unit snapper-timeline.timer has a bad unit file setting.)
     Active: inactive (dead)
    Trigger: n/a
   Triggers: ● snapper-timeline.service

journalctl -u snapper-timeline.timer
-- Boot 1c4777349bbc4b23a375686271c143bb --
Sep 15 21:42:44 univac systemd[1]: Started Timeline of Snapper Snapshots.
Sep 17 04:45:13 univac systemd[1]: snapper-timeline.timer: Succeeded.
Sep 17 04:45:13 univac systemd[1]: Stopped Timeline of Snapper Snapshots.

snapper list
    # | Type   | Pre # | Date                             | User | Cleanup  | Description    | Userdata
------+--------+-------+----------------------------------+------+----------+----------------+---------
   0  | single |       |                                  | root |          | current        |
   1  | single |       | Wed 09 Sep 2020 09:06:30 PM CEST | root |          |                |
   2  | single |       | Wed 09 Sep 2020 09:16:15 PM CEST | root | timeline | timeline       |
 509  | single |       | Thu 01 Oct 2020 12:00:03 AM CEST | root | timeline | timeline       |
1122  | pre    |       | Mon 26 Oct 2020 12:38:03 PM CET  | root |          | before cleanup |
1248  | single |       | Sun 01 Nov 2020 12:00:03 AM CET  | root | timeline | timeline       |
1873  | single |       | Tue 01 Dec 2020 12:00:02 AM CET  | root | timeline | timeline       |
2527  | single |       | Fri 01 Jan 2021 12:00:01 AM CET  | root | timeline | timeline       |
2625  | single |       | Wed 03 Feb 2021 07:00:04 PM CET  | root | timeline | timeline       |
3004  | single |       | Mon 01 Mar 2021 12:00:01 AM CET  | root | timeline | timeline       |
3747  | single |       | Thu 01 Apr 2021 12:00:01 AM CEST | root | timeline | timeline       |
4467  | single |       | Sat 01 May 2021 12:00:01 AM CEST | root | timeline | timeline       |
5211  | single |       | Tue 01 Jun 2021 12:00:03 AM CEST | root | timeline | timeline       |
5931  | single |       | Thu 01 Jul 2021 12:00:00 AM CEST | root | timeline | timeline       |
6675  | single |       | Sun 01 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
7035  | single |       | Mon 16 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
7202  | single |       | Mon 23 Aug 2021 12:00:00 AM CEST | root | timeline | timeline       |
7327  | single |       | Mon 30 Aug 2021 05:00:01 AM CEST | root | timeline | timeline       |
7370  | single |       | Wed 01 Sep 2021 12:00:01 AM CEST | root | timeline | timeline       |
7481  | single |       | Mon 06 Sep 2021 12:00:00 AM CEST | root | timeline | timeline       |
7512  | single |       | Wed 08 Sep 2021 12:00:02 AM CEST | root | timeline | timeline       |
7536  | single |       | Thu 09 Sep 2021 12:00:01 AM CEST | root | timeline | timeline       |
7560  | single |       | Fri 10 Sep 2021 12:00:02 AM CEST | root | timeline | timeline       |
7584  | single |       | Sat 11 Sep 2021 12:00:01 AM CEST | root | timeline | timeline       |
7608  | single |       | Sun 12 Sep 2021 12:00:00 AM CEST | root | timeline | timeline       |
7632  | single |       | Mon 13 Sep 2021 12:00:00 AM CEST | root | timeline | timeline       |
7656  | single |       | Tue 14 Sep 2021 12:00:00 AM CEST | root | timeline | timeline       |
7680  | single |       | Wed 15 Sep 2021 12:00:01 AM CEST | root | timeline | timeline       |
7685  | single |       | Wed 15 Sep 2021 09:31:23 PM CEST | root | number   | boot           |
7686  | single |       | Wed 15 Sep 2021 09:42:53 PM CEST | root | number   | boot           |
7689  | single |       | Thu 16 Sep 2021 12:00:00 AM CEST | root | timeline | timeline       |
7706  | single |       | Thu 16 Sep 2021 05:00:01 PM CEST | root | timeline | timeline       |
7707  | single |       | Thu 16 Sep 2021 06:00:01 PM CEST | root | timeline | timeline       |
7708  | single |       | Thu 16 Sep 2021 07:00:01 PM CEST | root | timeline | timeline       |
7709  | single |       | Thu 16 Sep 2021 08:00:02 PM CEST | root | timeline | timeline       |
7710  | single |       | Thu 16 Sep 2021 09:00:00 PM CEST | root | timeline | timeline       |
7711  | single |       | Thu 16 Sep 2021 10:00:00 PM CEST | root | timeline | timeline       |
7712  | single |       | Thu 16 Sep 2021 11:00:00 PM CEST | root | timeline | timeline       |
7713  | single |       | Fri 17 Sep 2021 12:00:01 AM CEST | root | timeline | timeline       |
7714  | single |       | Fri 17 Sep 2021 01:00:01 AM CEST | root | timeline | timeline       |
7715  | single |       | Fri 17 Sep 2021 02:00:01 AM CEST | root | timeline | timeline       |
7716  | single |       | Fri 17 Sep 2021 03:00:01 AM CEST | root | timeline | timeline       |
7717  | single |       | Fri 17 Sep 2021 04:00:01 AM CEST | root | timeline | timeline       |
7718  | single |       | Fri 17 Sep 2021 04:49:38 AM CEST | root | number   | boot           |


GCA Export: basic speed format error

Hi,
I´m from Brazil, and my machine number format is configured with the number format decimal separator as , intead of . (00,00 insteal of 00.00). This cause the foundry being incapable of rolling initiative for number format error. maybe this could affect others parameter if decimal values. Maybe there is a way to force the import to correct the decimal separator.
Thanks

@twzzl Can you upload a sample export file for us to use for testing?

Yes, tis is an example.
Oficial de Segurança (GM).gca4.TXT

@twzzl  This will be fixed in the next release. Do you know why Basic Speed is entered as "6,25" but the weight of various equipment uses '.' as the decimal separator, as in <weight type="number">1.4</weight>? It would be nice if we could apply the same rule across all fields.


Fix deprecated references to versioned TerminologyUploaderProvider.

@jamesagnew, I found no other breaking references to the deprecated classes. Not sure how to write a test to prevent this from happening in the future.

This needs a changelog, approved otherwise.


Total solar faulty

Hello,
I'm desperately trying to integrate my SOLAX inverter into Venus correctly.

root@raspberrypi2:~# cat /data/etc/dbus-mqtt-pv/config.ini
; CONFIG FILE
; GitHub reporitory: https://github.com/mr-manuel/venus-os_dbus-mqtt-pv
; remove semicolon ; to enable desired setting

[DEFAULT]
; Set logging level
; ERROR = shows errors only
; WARNING = shows ERROR and warnings
; INFO = shows WARNING and running functions
; DEBUG = shows INFO and data/values
; default: WARNING
logging = WARNING

; Device name
; default: MQTT PV
device_name = SOLAX_X1_5K

; Device VRM instance
; default: 51
device_instance = 51

; Specify after how many seconds the driver should exit (disconnect), if no new MQTT message was received
; default: 60
; value to disable timeout: 0
timeout = 60

; used when no voltage is received
voltage = 230

[PV]
; Max rated power (in Watts) of the inverter
max = 5000
; position (connected input/output) of the PV inverter
; 0 = AC input 1 (select this if you don't have a MultiPlus or Quattro)
; 1 = AC output
; 2 = AC input 2
position = 1

[MQTT]
; IP addess or FQDN from MQTT server
broker_address = <IP_ADDRESS>

; Port of the MQTT server
; default plaintext: 1883
; default TLS port: 8883
broker_port = 1883

; Enables TLS
; 0 = Disabled
; 1 = Enabled
;tls_enabled = 1

; Absolute path to the Certificate Authority certificate file that is to be treated as trusted by this client
;tls_path_to_ca = /data/keys/mosquitto.crt

; Disables verification of the server hostname in the server certificate
; 0 = Disabled
; 1 = Enabled
;tls_insecure = 1

; Username used for connection
;username = myuser

; Password used for connection
;password = mypassword

; Topic where the pv data as JSON string is published
; minimum required JSON payload: {"pv": { "power": 0.0 } }
topic = enphase/envoy-s/meters
root@raspberrypi2:~#

I have now managed to get the PV yield to be displayed and added to the "Total solar".

Unfortunately, the total PV yield is no longer displayed in the statistics. Only the yield of my two MPPTs is displayed here.

Perhaps someone has a tip as to what the reason for this is.
Cheers
Sebastian
Edit @mr-manuel: Translated to english

You need to provide the energy_forward in your JSON, else it won't work. Therefore you need to get the total kWh produced by your PV inverter and pass it to Venus OS.


live update

Question
hello. i need a chart with auto-updating lines.
it can be a ticker chart based on per-minute update. here's example how i do it in dash: https://dash.plotly.com/live-updates. is it possible not to do a custom_component to achieve the same effect? may be there's a wrapper around some plotly express graph that have similar feature? thanks much
Code/Examples
No response
Other information
No response
Which package?
None
Package version
No response
Python version
No response
OS
No response
Code of Conduct

[X] I agree to follow the Code of Conduct.

Hi @vks2 and thanks for the great question. ⭐
I haven't found any quick and easy solution that can update live chart data without creating a thin custom-component wrapper around dcc.Interval or injecting dcc.Interval directly into the Vizro dashboard. Here's an example (with some explanation in the comments) with a custom component that wraps the dcc.Interval component really thin, (the data in this example is updated every 2 seconds):
"""Example to show dashboard configuration."""

from typing import Literal

import vizro.models as vm
import vizro.plotly.express as px
from dash import Input, Output, callback, dcc
from vizro import Vizro
from vizro.managers import data_manager

# This function is used to get live data. You can implement your own function to get data from a database or API.
def get_live_data():
    """Get live data."""
    return px.data.gapminder().sample(30)

# Beware of assigning the function without calling it -> `get_live_data` instead of `get_live_data()`.
data_manager["live_data"] = get_live_data

# Custom MyInterval component
class MyInterval(vm.VizroBaseModel):
    type: Literal["my_interval"] = "my_interval"

    id: str
    interval: int

    def build(self):
        return dcc.Interval(id=self.id, interval=self.interval, n_intervals=0)

# Explicitly enable MyInterval to be used in the Vizro Page.
vm.Page.add_type("components", MyInterval)

page = vm.Page(
    title="Vizro live data update",
    components=[
        vm.Graph(figure=px.scatter(data_frame="live_data", x="gdpPercap", y="lifeExp", color="continent")),
        MyInterval(id="my_interval", interval=1000 * 2),  # Refresh data every 2 seconds
    ],
    controls=[vm.Filter(column="continent"), vm.Filter(column="year")],
)

dashboard = vm.Dashboard(pages=[page])

# Callbacks <---------------------------------------------------------------------

# "on_page_load_action_trigger_Vizro live data update" is the id of the component that triggers the page loading mechanism.
# Vizro is going to release a predefined actions to do this, but for now, we need to use this "hack".
@callback(
    Output("on_page_load_action_trigger_Vizro live data update", "data"),
    Input("my_interval", "n_intervals"),
)
def update_data(n_intervals):
    """Update data."""
    return n_intervals

if __name__ == "__main__":
    Vizro().build(dashboard).run()

https://github.com/mckinsey/vizro/assets/108530920/f9da9b52-f82c-4c57-8dbf-5870bbde8f1c
If creating a custom component doesn't work for you, you can always add the dcc.Interval directly to the dashboard in a following way (as already described in https://github.com/mckinsey/vizro/issues/533#issuecomment-2175536011):
app = Vizro().build(dashboard)
app.dash.layout.children.append(dcc.Interval(id="my_interval", interval=2000, n_intervals=0))
app.run()

Also, if manually refreshing the page to fetch the updated data works for you (instead of updating live using the dcc.Interval), you can remove the callback along with the dcc.Interval component from the example and everything will work as expected.
I hope this could help you build your desired application. 🤞

indeed good and working example. thank you.
as far as refreshing is concerned, if i inject another vm.Graph with static data inside - we are sure that not overall chart will be refreshed? encountered that effect in other libs. thanks much. (i'm talking about that code line:
page = vm.Page(
title="Update the chart on page refresh",
components=[
vm.Graph(figure=px.box("iris", x="species", y="petal_width", color="species"))
],

)

If you don't mind, let me just hijack your question briefly to do a bit of user research because this is a feature that's on our roadmap to built in natively to Vizro but we haven't figured out the right API for it yet.
As a user, how would you expect to enable this feature? Currently, as in @petar-qb's example above, you enable dynamic data like this:
# This function is used to get live data. You can implement your own function to get data from a database or API.
def get_live_data():
    """Get live data."""
    return px.data.gapminder().sample(30)

# Beware of assigning the function without calling it -> `get_live_data` instead of `get_live_data()`.
data_manager["live_data"] = get_live_data

And then use it like this:
vm.Graph(figure=px.scatter(data_frame="live_data", x="gdpPercap", y="lifeExp", color="continent"))

It's also possible to setup a cache that times out like this:
data_manager["live_data"] = get_live_data
data_manager["live_data"].timeout = 10

This timeout does not actually trigger a refresh of the data in any way, it just controls expiry of a server side cache (see https://vizro.readthedocs.io/en/stable/pages/user-guides/data/#set-timeouts).
Now if you wanted to enable live updates, how would you expect to do that? Would you like to be able to set different graphs on the page to update at different rates? Or refresh the whole page every N seconds? Where would you expect this setting to live? e.g. it could be associated with data_manager or with vm.Graph or with vm.Page?

i'm sorry of being a maximalist here but the optimal way is mimic the ipyvizzu lib here. look at the beauty of charts updates there. being a realist, it would be good if we set up a timeframe arg in vm.Graph constructor and do not affect all the page, because of complex datasources associated with it

Got it, thank you! I'd never seen ipyvizzu before but those are indeed very beautiful chart updates!
Realistically indeed I think we will never get to something quite like that because Vizro plots are all based on plotly, which does support some animation but it's definitely not designed to do that natively as sleekly as ipyvizzu does.
So what you'll find with @petar-qb's above example is that the interval triggers a refresh of the whole page. Actually it's not exactly a full page refresh but equivalent to what happens if you click on a different page on the navbar, which in one sense a much lighter request than doing a full page refresh. In practice this means that when the refresh is triggered, all graphs on the page undergo the same process:

fetch data
apply parameters/filters from the control panel
draw whole graph again from scratch

Typically it is stage 1 here is that is the expensive one (like with your complex data sources) that you don't wish to rerun, and this is where data caching can really help you.
It's actually probably not too hard to come up with some ways to optimise steps 2 and 3 above and also to not trigger a refresh of all the graphs on the screen (especially if you are willing to compromise on other things, e.g. you don't want your static graph to respond to filters). But definitely the simplest way right now is just to let the whole page refresh and rely on caching to avoid repeated heavy data loading. This should work pretty much as well as just refreshing only the graphs that you want, just the static graph will flicker on page refresh even though it doesn't change.

I'd be very interested in what sort of page you're trying to make here btw if you don't mind sharing a bit more. e.g. would you ultimately like lots of vm.Graph on one page, each with its one separate refresh_time value? Or it's really just you want a single graph to refresh on the page and all others would remain static? Would these graphs all share the same data source in your case or come from different places? What sort of timeframes are relevant for you? Like updating once per second or once per hour?
If we can understand the use case a bit better then it will help inform how we might implement the solution when it comes to developing the feature 🙂

we're doing a similar services to https://www.newyorkfed.org/research/policy/nowcast#/nowcast
thanks again for your detailed answers and feedback


Lapce is not recognised as trusted program on Windows

Is your feature request related to a problem? Please describe.
Yes. Microsoft Edge and Window's defender smart screen think your app is malware. See image down below. It's possible this might be more of a bug but if that's the case I'll move it.
Describe the solution you'd like
Maybe putting a publisher name might help? It would make the app look more official.
Describe alternatives you've considered
N/A
Additional context

This is more Windows saying "I don't know what this program is". I believe what Windows does is keep track of what common programs are run, and then it doesn't bother alerting the user if the program one that many user's use just fine. Since Lapce is not widely used - and we just did a new build, which thus looks different to windows - it would show this message. However, I believe there are methods to convince Microsoft that your new versions are normal (or doing it sooner), but I don't know them (and I'm not on a window computer atm, so I can't look at it).

It's ok. It's easy to get around it's just something I'd give a heads up on.

This is because Lapce is not signed with code signing certificate that's trusted by Microsoft

Just prevented me from installing on a corporate machine on which I can otherwise install everything, as I have full admin, because there is no "Run anyway" button, even after clicking to show more details.


fix(grant): change guardian import access API

apply this change: https://github.com/odpf/proton/pull/197/commits/d9b61bbdbd2de9188ed363c68f6af710d712d659

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

16 of 26 (61.54%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.2%) to 74.0%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

core/grant/service.go
0
2
0.0%

api/handler/v1beta1/grant.go
16
24
66.67%

Totals

Change from base Build<PHONE_NUMBER>:
0.2%

Covered Lines:
5940

Relevant Lines:
8027

💛  - Coveralls


Allowing to use weird characters in paths such as spaces,quotes,braces etc.

Thanks @PaulTaykalo !


Help with disabled_macros

What happened and in what context?
Hi,
I'm trying to hide some macros in the telegram ui but they are not being hidden.
The line in my config file:
[telegram_ui]
silent_progress: true
eta_source: file
disabled_macros: m300, /idle_timeout_macro, /filament_runout_pause, /end_gcode, /start_gcode, /shutdown, /light
(I have tried both with and without the leading slash)
But they are still visible:

I'm not sure what's wrong - does anyone have any pointers?
Thanks
If needed, paste the relevant bot log contents here.
No response
If needed, paste the relevant bot journal contents here.
No response
What branch does this occur on?
master
Reports only on latest master or development branches!

[X] I have updated to latest development/master version before submitting the bug

Thank you for the report, I'll look into it and get back to you.

After switching to the the dev branch with the above commit my custom macros are now disabled - Thanks!
One question though - should I be able to disable the inbuilt macros? I have added light and shutdown to the list and they still display in the telegram ui.

That's an interesting thought. We never considered that you may want to disable the bots internal command autocompletion. So far, you are not be able to, but don't be dissuaded, we'll look intohow this might be done in the future.
I have openend a separate feature ticket, #157  you might want to subscribe so you will be notified when there is progress on that.
The issue with non-hideable-macros will roll out in the next release.

Thanks for letting me know. My use case is that I have my own macros for lights and system shutdown so it would be nice to to be able to remove the builtin ones from the list to declutter it.
Thanks again for your assistance and work on this awesome addition to klipper 👍


support for more genomes

Dear @paul-shannon,
I just realized that the setGenome method currently only supports hg38, hg19, mm10 and tair10 limiting the use of igvR for the research community.
Do you plan to extend the list of supported genomes in the near future? Will it also be possible to load genomes from a file?
I would personally be very interested in support for the genome of the malaria parasite Plasmodium falciparum 3D7.
Many thanks in advance for your help!
Jan

@janstrauss1 I'd be glad -- working with Jim Robinson, the creator of igv.js -- to add additional genomes - and to document how to add your own.   I will contact Jim directly.   In my recollection we need the genome and a gff3: can you direct me to your preferred versions of these two?

@janstrauss1
I just reminded myself of the mechanism for supporting a new genome in igv.js and thus in igvR.  While waiting for a reply from Jim on his ideas on contributing new genomes to igv.js for all to use, here is what I can pretty easily add support for -  so that you can host your own genome, as long as you have access to a webserver which supports range requests.  We need range requests in the webserver  so that  parts of the sequence can be read from an indexed fa.gz file into the browser on demand.
In a proposed new method for igvR:
setCustomGenome(igv, name="TAIR10",
                fastaURL="https://myHost.net/tair10/Arabidopsis_thaliana.TAIR10.dna.toplevel.fa",
                indexURL="https://myHost.net/tair10/Arabidopsis_thaliana.TAIR10.dna.toplevel.fa.fai",
                aliasURL: "https://myHost.net/tair10/chromosomeAliases.txt",
                annotationURL: "https://myHost.net/TAIR10_genes.sorted.chrLowered.gff3.gz")

Chromosome aliases would be optional, but often helpful, so that for instance, "chr10:50-100", "Chr10:50-100" and "10:50-100" are all intelligible.
Comments?

@janstrauss1 I'd be glad -- working with Jim Robinson, the creator of igv.js -- to add additional genomes - and to document how to add your own. I will contact Jim directly. In my recollection we need the genome and a gff3: can you direct me to your preferred versions of these two?

@paul-shannon, you could use the genome and gff3 from the most recent release of PlasmodDB available at https://plasmodb.org/.
Alternatively, you should get the files from ftp://ftp.sanger.ac.uk/pub/project/pathogens/Plasmodium/falciparum/3D7/.

@janstrauss1
I just reminded myself of the mechanism for supporting a new genome in igv.js and thus in igvR. While waiting for a reply from Jim on his ideas on contributing new genomes to igv.js for all to use, here is what I can pretty easily add support for - so that you can host your own genome, as long as you have access to a webserver which supports range requests. We need range requests in the webserver so that parts of the sequence can be read from an indexed fa.gz file into the browser on demand.
In a proposed new method for igvR:
setCustomGenome(igv, name="TAIR10",
                fastaURL="https://myHost.net/tair10/Arabidopsis_thaliana.TAIR10.dna.toplevel.fa",
                indexURL="https://myHost.net/tair10/Arabidopsis_thaliana.TAIR10.dna.toplevel.fa.fai",
                aliasURL: "https://myHost.net/tair10/chromosomeAliases.txt",
                annotationURL: "https://myHost.net/TAIR10_genes.sorted.chrLowered.gff3.gz")

Chromosome aliases would be optional, but often helpful, so that for instance, "chr10:50-100", "Chr10:50-100" and "10:50-100" are all intelligible.
Comments?

Sounds good to me as far as I can judge.
However, I would currently already be happy if all the genomes that are hosted on the IGV genome server would be supported. See https://software.broadinstitute.org/software/igv/Genomes for a full list.
It already supports several genomes for the Plasmodium parasite (listed at 97. - 105.)

@janstrauss1
Support for Pfalciparum 3D7 is now (at least minimally) in igvR 1.5.6.   I am hosting three files that you directed me towards:

PlasmoDB-43_Pfalciparum3D7_Genome.fasta
PlasmoDB-43_Pfalciparum3D7_Genome.fasta.fai
PlasmoDB-43_Pfalciparum3D7.gff

Jim Robinson, igv.js principal author, explains that the 100+ IGV desktop genomes are not available to igv.js - I don't know why - but only the sixteen listed here.  He welcomes further submissions if they are public, not embargoed.   I will make sure that the sixteen genomes Jim offers are soon available in igvR.
Pfal3D7 is a good candidate for submission to igv.js.  Can you check out my current version?  It seems rather skimpy to me: the gff file does not support search by gene symbol, nor does the reference gene track offer gene symbol names.  I don't know what the Pfalciparum community expects, but if you guide me, I am happy to make improvements.   Then we can submit to Jim.
I fixed the vignette error - the missing autoscale argument in the devel vignette, but not yet in release.

@paul-shannon, thanks for adding support for the Pfalciparum 3D7 genome! The genome fasta and annotation gff file versions from PlasmoDB look generally ok to me.
However, the information at PlasmoDB appears to be only periodically updated according to Wellcome Sanger Institute.
Thus, I would suggest to rather use the most recent genome and annotation file versions that are continually updated in GeneDB and provided on the Wellcome Sanger Institute FTP server at ftp://ftp.sanger.ac.uk/pub/genedb/releases/2019-05/Pfalciparum/.

@janstrauss1 "continually updated" hmm - I may not be in a position to keep the 3D7 genome as current as you wish.
With that in mind, the package now includes a full demo showing how to serve up annotation and reference genome tracks yourself, locally for use within igvR.    You can therefore add and/or update any and all reference and annotation tracks whenever you wish.  See
https://github.com/paul-shannon/igvR/tree/master/misc/serveYourOwnFiles

This demo uses a small genome, Rhodobacter sphaeroides, a minimal range-request-capable  python Flask webserver, and an html file to see it all in action rhodobacter-sphaeroides-demo.html.
If this is of interest, I'd be glad to respond to any questions, suggestions or bugs you have

@janstrauss1 "continually updated" hmm - I may not be in a position to keep the 3D7 genome as current as you wish.

@paul-shannon, that's fine! I didn't mean to ask you to keep the genome continuously up-to-date but rather wanted to make sure that you use the most recent genome and annotation file versions currently available.
Thanks a lot for including a full demo on how to use annotation and reference genome tracks locally! I will give it a try.
Thanks again for your great help!

@janstrauss1  I will replace the PlasmoDB genome for igvR with the next update to GeneDb.   If you find igvR useful, and depend upon the current P falciparum genome, I will automate this.  Just let me know.

@paul-shannon, that's great thank you!
Yes, I find igvR very useful and will depend on the current P. falciparum genome in the future. I would therefore highly appreciate if you could automate this process.
Many thanks for your efforts!

@janstrauss1 I am updating igvR for the upcoming bioconductor release.  I see that igv.js hosts a few more genomes, but still no Pfalciparum.
I confess: I have not done the automation I offered back in May.  Is this still of active interest to you?

Dear @paul-shannon, thanks for the update!
Yes, I'm still very interested to use igvR with the most current Pfalciparum genome and would appreciate if you could include the automation.
Many thanks in advance!

Hi Jan,
Looking just now at https://plasmodb.org/common/downloads/release-43/Pfalciparum3D7/ it appears that the three annotation file we use in igvR, which I install on May 9th, are still current - nothing new to update with.
Set me straight on this if I got it wrong?

Paul

On Oct 9, 2019, at 12:20 AM, Jan Strauss<EMAIL_ADDRESS>wrote:
Dear @paul-shannon, thanks for the update!
Yes, I'm still very interested to use igvR with the most current Pfalciparum genome and would appreciate if you could include the automation.
Many thanks in advance!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Hi Paul,
Could you please compare the annotation file used in igvR to the files available at https://plasmodb.org/common/downloads/Current_Release/Pfalciparum3D7/?
I can well imagine though that the files are still current and no update is necessary.
Maybe you can use the ./Current_Release/Pfalciparum3D7/ directory for automatic updates of the igvR annotation file?
Many thanks for your great support!
Jan

Dear @paul-shannon , I'm a new user of igvR and finds it a perfect tools to visualize my interested regions in R!  Just a follow-up on this thread on the plan to support custom genomes, is the proposed method available yet?
setCustomGenome(igv, name="TAIR10",
                fastaURL="https://myHost.net/tair10/Arabidopsis_thaliana.TAIR10.dna.toplevel.fa",
                indexURL="https://myHost.net/tair10/Arabidopsis_thaliana.TAIR10.dna.toplevel.fa.fai",
                aliasURL: "https://myHost.net/tair10/chromosomeAliases.txt",
                annotationURL: "https://myHost.net/TAIR10_genes.sorted.chrLowered.gff3.gz")

I'd be definitely interested in giving it a try if possible!
Thanks!

Hi @orionzhou,
Thank you for the prompt to return to this task.  I am working on it now.
Two requests, if I may:

Could you post your question (also) to https://support.bioconductor.org so that others in the community who may be interested will see this?
Can you provide URLs for fasta, fasta index, cytoband and a gene annotation track for you organism?  That will permit me to test the code more broadly than with my own example organisms only

Thanks for the reply, and yes -
here is the question posted on bioconductor
here is the genome files:
"id": "maize",
"name": "Zea mays B73v4",
"fastaURL": "https://s3.msi.umn.edu/zhoup-igv-data/Zmays-B73/10.fasta",
"indexURL": "https://s3.msi.umn.edu/zhoup-igv-data/Zmays-B73/10.fasta.fai"
{
"name": "Genes",
"format": "gff3",
"url": "https://s3.msi.umn.edu/zhoup-igv-data/Zmays-B73/10.gff.gz",
"indexURL": "https://s3.msi.umn.edu/zhoup-igv-data/Zmays-B73/10.gff.gz.tbi",
}

@orionzhou,   I have an early version of ```setCustomGenome` working with your mays data.
However, somethings seems amiss, which I reproduce in a minimal html/js scripts - for hg38 (which loads fast) and mays (which takes five minutes).  I downloaded all of the relevant files from a shell, using curl, and found that hg38 full fasta takes as long to download from aws as 10.fasta does from your server - so independent download speeds do not seem to be the issue.
Could your run the scripts in igvR/misc/serveYourOwnFiles/testZeaMaysFromMinnesota/ and report any insights you have?  I may be doing something dumb.

Thanks @paul-shannon , the two human scripts load very fast, but it took ~1 minute to load the maize one on my side too.  It seems to be due to the GFF file - I removed the gene track and it loads very fast then.
The IGV-web app works as normal though: https://s3.msi.umn.edu/zhoup-igv/index.html

@orionzhou,
Let's defer the slow maize gff load puzzle for now.
A new version of the package (1.9.4) offers this method, shown here with two examples from inst/unitTests/test_setCustomGenome.R.
Give me feedback, please.   Both use hg38 from igv at aws.  First, with all parameters set explicitly:
setCustomGenome(igv,
                id="hg38",
                genomeName="Human (GRCh38/hg38)",
                fastaURL="https://s3.amazonaws.com/igv.broadinstitute.org/genomes/seq/hg38/hg38.fa",
                fastaIndexURL="https://s3.amazonaws.com/igv.broadinstitute.org/genomes/seq/hg38/hg38.fa.fai",
                cytobandURL="https://s3.amazonaws.com/igv.broadinstitute.org/annotations/hg38/cytoBandIdeo.txt",
                chromosomeAliasURL=NA,
                geneAnnotationName="Refseq Genes",
                geneAnnotationURL="https://s3.amazonaws.com/igv.org.genomes/hg38/refGene.txt.gz",
                geneAnnotationTrackHeight=500,
                geneAnnotationTrackColor="red",
                initialLocus="chr5:88,621,308-89,001,037",
                visibilityWindow=5000000)

and then with the minimum parameters:
setCustomGenome(igv,
                id="hg38",
                genomeName="Human (GRCh38/hg38)",
                fastaURL="https://s3.amazonaws.com/igv.broadinstitute.org/genomes/seq/hg38/hg38.fa",
                fastaIndexURL="https://s3.amazonaws.com/igv.broadinstitute.org/genomes/seq/hg38/hg38.fa.fai")

Hi @orionzhou,
I will be grateful if you could accept and approve the answer just updated on https://support.bioconductor.org/p/p132403/#p132805
Thank you!

Paul

On Oct 28, 2020, at 9:12 AM, Paul Shannon<EMAIL_ADDRESS>wrote:
@orionzhou,
Let's defer the slow maize gff load puzzle for now.
A new version of the package (1.9.4) offers this method, shown here with two examples from inst/unitTests/test_setCustomGenome.R.
Give me feedback, please. Both use hg38 from igv at aws. First, with all parameters set explicitly:
setCustomGenome(igv,
id="hg38",
genomeName="Human (GRCh38/hg38)",
fastaURL="https://s3.amazonaws.com/igv.broadinstitute.org/genomes/seq/hg38/hg38.fa",
fastaIndexURL="https://s3.amazonaws.com/igv.broadinstitute.org/genomes/seq/hg38/hg38.fa.fai",
cytobandURL="https://s3.amazonaws.com/igv.broadinstitute.org/annotations/hg38/cytoBandIdeo.txt",
chromosomeAliasURL=NA,
geneAnnotationName="Refseq Genes",
geneAnnotationURL="https://s3.amazonaws.com/igv.org.genomes/hg38/refGene.txt.gz",
geneAnnotationTrackHeight=500,
geneAnnotationTrackColor="red",
initialLocus="chr5:88,621,308-89,001,037",
visibilityWindow=5000000)
and then with the minimum parameters:
setCustomGenome(igv,
id="hg38",
genomeName="Human (GRCh38/hg38)",
fastaURL="https://s3.amazonaws.com/igv.broadinstitute.org/genomes/seq/hg38/hg38.fa",
fastaIndexURL="https://s3.amazonaws.com/igv.broadinstitute.org/genomes/seq/hg38/hg38.fa.fai")
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Great! Thanks a bunch Paul!


Highlight Changed Runes

On FRR, I often find myself clicking through the rune wheel to figure out what changed.  An outline would save me a lot of clicking (especially for units w/ just 1 or 2 changes).

It would also be amazing if there was a little icon indicating whether the rune is sourced from a monster vs. rune storage.  When you're inventory-limited, those current-storage runes are a great way to make progress (and free up runes that are likely used elsewhere).

The rune dial already makes this pretty easy so fixing #75 would be the 90% solution.  I'm going to close this because other highlighting doesn't seem to be a high enough priority.


added keyword functionality to PubMedArticle

Hi! I saw issue #27 and tried a simple method to extract keywords from the XML. Should return an empty list if there are no keywords and seems to work for articles and books. Hopefully helpful!

Finally getting around to integrating this.  I really appreciate the help, I was just in a bad spot back when you contributed this.


libyang: 2.1.4 -> 2.1.30

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for libyang is: YANG data modelling language parser and toolkit
meta.homepage for libyang is: https://github.com/CESNET/libyang
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 2.1.30 with grep in /nix/store/vkgfacmc1q5xfjrj6iaw5qkmhr6yg2qh-libyang-2.1.30
found 2.1.30 in filename of file in /nix/store/vkgfacmc1q5xfjrj6iaw5qkmhr6yg2qh-libyang-2.1.30

Rebuild report (if merged into master) (click to expand)

2 total rebuild path(s)

2 package rebuild(s)

First fifty rebuilds by attrpath
frr
libyang

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/vkgfacmc1q5xfjrj6iaw5qkmhr6yg2qh-libyang-2.1.30 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A libyang https://github.com/r-ryantm/nixpkgs/archive/66095d913ba5c7cd2894f76acfa275a0a3b13153.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/vkgfacmc1q5xfjrj6iaw5qkmhr6yg2qh-libyang-2.1.30
ls -la /nix/store/vkgfacmc1q5xfjrj6iaw5qkmhr6yg2qh-libyang-2.1.30/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages built:

frr
libyang

Maintainer pings
cc @woffs for testing.

@ofborg test frr


Offline stored found caches not hidden if live-mode disabled

Settings: Hide found caches
Store a cache offline, that you've found (or log it as found)
Open live map. Not found caches are displayed, included offline stored ones.
Menu --> disable live. Now only offline stored caches are displayed, but found caches, too.

This is absolutely ok, if you display a list of offline stored caches. They are displayed there, too (but with a refactoring of the filters it would be good, if they are also hidden there, but that's another issue).
But the live-map should behave like in live-mode.

Menu --> disable live. Now only offline stored caches are displayed, but found caches, too.

I also see that found caches are visible in this case. But for me not only the offline stored are visible but all previously loaded caches stay as well. But I think, that is what you meant to say ?!

Yes, the cached caches are displayed. A nice feature, also if it was not intended. The user might expect this behavior, so it's ok.
It's only strange, that offline-caches are merged without filtering in not-live-mode.

I like it somehow as it is a quick way to disable the "hide found". I could sometimes use this e.g. in case I want to place a TB in a cache I already found. But that would be another feature request.

But found caches are only displayed if they were stored before.


Upgrade to 4.1.0

Upgrade to 4.1.0

@thisisthekap Really excited for this upgrade. I have mentioned it before but my company is willing to pay for this. PM me if your interested please.


OSD-19289 : Copy the user created secret to management cluster HCP namespace

Why
CLO needs to consume secrets when CLO forward logs via cluster log forwarder resources. This PR copying the openshift-logging NS secrets to HCP NS

/hold


windows: argv isn't supported on Windows

(ins)$ zig build -Drelease-safe
C:\Users\MadMax\Desktop\zigbin\lib\std\os.zig:251:17: error: argv isn't supported on Windows: use std.process.argsAlloc instead
    .windows => @compileError("argv isn't supported on Windows: use std.process.argsAlloc instead"),
                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
referenced by:
    usage: src\main.zig:8:71
    main: src\main.zig:26:17
    remaining reference traces hidden; use '-freference-trace' to see all reference traces

C:\Users\MadMax\Desktop\zigbin\lib\std\os.zig:117:24: error: root struct of file 'c' has no member named 'PROT'
pub const PROT = system.PROT;
                 ~~~~~~^~~~~
C:\Users\MadMax\Desktop\zigbin\lib\std\c.zig:146:122: error: root struct of file 'c' has no member named 'off_t'
pub extern "c" fn mmap(addr: ?*align(page_size) anyopaque, len: usize, prot: c_uint, flags: c_uint, fd: c.fd_t, offset: c.off_t) *anyopaque;
                                                                                ~^~~~~~
error: ztags...
error: The following command exited with error code 1:
C:\Users\MadMax\Desktop\zigbin\zig.exe build-exe C:\Users\MadMax\Desktop\ztags\src\main.zig -OReleaseSafe --cache-dir C:\Users\MadMax\Desktop\ztags\zig-cacpData\Local\zig --name ztags --enable-cache

Related question: Is Windows compatibility a goal or not?

Both master and 0.10.0 branches use std.process.argsWithAllocator. Are you using an older version maybe?

Related question: Is Windows compatibility a goal or not?

Yes, as long as we can achieve Windows compatibility through using portable functions in Zig's standard library. I don't have any way to test Windows though.

Both master and 0.10.0 branches use std.process.argsWithAllocator. Are you using an older version maybe?

I do see:
fn usage() void {
    std.debug.print("Usage: {s} [-o OUTPUT] [-a] [-r] FILES...\n", .{std.os.argv[0]});
}

Doh, right, I forgot about the usage text.

Should be fixed in ef5623374892de48e863d3b12814b4968f8411ce.

fixed in https://github.com/gpanders/ztags/commit/ef5623374892de48e863d3b12814b4968f8411ce.

Thanks!


Index metadata should always store absolute paths

What changes were proposed in this pull request?
fixes #126

Why are the changes needed?
currently metadata stores relative paths if system path is relative

Does this PR introduce any user-facing change?
Yes, now all paths stored in metadata will be fully qualified. This is not a breaking change. Previously created indexes (v0.2.0) will still be readable.

How was this patch tested?
unit tests

LGTM except for few nit comments. Thanks @apoorvedave1!
Looks like your initial approach would have saved churns in unit tests. Let me think about it more, but this PR itself looks fine to me.

thanks @imback82 , I slightly prefer the initial approach because I was thinking it's OK to allow the caller to make mistakes and send relative paths, and we will make it qualified before using it in log files. This would save users some time in calling with relative path, failing and then calling it again with absolute path.
But I am ok with this approach as well.

I slightly prefer the initial approach because I was thinking it's OK to allow the caller to make mistakes and send relative paths, and we will make it qualified before using it in log files. This would save users some time in calling with relative path, failing and then calling it again with absolute path.
But I am ok with this approach as well.

Hmm, I just checked the code again, and I think we just need to update CreateActionBase.getIndexLogEntry to handle the path, no?
https://github.com/microsoft/hyperspace/blob/4cc3f74c27e6ee8600257997751bc3f4eba7c05f/src/main/scala/com/microsoft/hyperspace/actions/CreateActionBase.scala#L86
Then, we don't need to worry about IndexDataManager, at which level to introduce, etc.

I slightly prefer the initial approach because I was thinking it's OK to allow the caller to make mistakes and send relative paths, and we will make it qualified before using it in log files. This would save users some time in calling with relative path, failing and then calling it again with absolute path.
But I am ok with this approach as well.

Hmm, I just checked the code again, and I think we just need to update CreateActionBase.getIndexLogEntry to handle the path, no?
https://github.com/microsoft/hyperspace/blob/4cc3f74c27e6ee8600257997751bc3f4eba7c05f/src/main/scala/com/microsoft/hyperspace/actions/CreateActionBase.scala#L86
Then, we don't need to worry about IndexDataManager, at which level to introduce, etc.

yeah this is also fine with me. I can start a fresh PR with this change to keep things clean.

Closed by #128


Struggling on how to make a numeric field comma formatter work on 2.0

In 1.0 we had filters which helped us translate comma-separated decimals in text inputs to javascript numbers (eg. '2,91' to 2.91). It was very straightforward.
How can such translation be implemented in 2.0? Maybe extending v-model directive? Creating a totally new directive and repeating all v-model boilerplate? Computed get/setters?
Thanks a lot!

Please make sure to read the Issue Reporting Guidelines before opening new issues. The issue list only accepts bug reports and feature requests. Questions should be posted to the Gitter chat room,  forum or StackOverflow. Thanks.

Regarding your need, the recommended approach in 2.0 is creating an extended component that handles the underlying conversion logic, e.g. <numeric-input v-model="val"></numeric-input>


scale down character

Hello my friend , i have been testing your thirPersonController class.
It works great , I wanted to ask you how to scale down  the character.

Hello my friend , i have been testing your thirPersonController class. It works great , I wanted to ask you how to scale down the character.
I haven't written a zoom API yet, but you can zoom in on the scene to achieve your desired effect


rust-analyzer: 2021-12-27 -> 2022-01-24

Motivation for this change
My neovim does not like the current version of rust-analyzer, the lsp plugin crashes.
Maybe an update helps, dunno.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

@ofborg eval

Maybe an update helps, dunno.

That means you did not test this PR at all?

I've opened #157908 to bump rust-analyzer to an even newer version. I think we can close this in its favor


second try at block linearize

Just lines left I think. Curious how perf of generated kernels compares

Going to merge if tests pass. This add lines sadly, but it's a lot cleaner


Add an example of configuring a simulated device in a robolectric environment

Robolectric provides @Config to facilitate simulated device configuration.
As far as I can tell, there is no equivalent in the AndroidX test system.
We're trying to move in a direction where our ui tests can run either on device or in a simulated environment via espresso and Robolectric. However, our tests fail on a Robolectric environment because some of our views are drawn with less than 90% of the view on the screen. We don't target mdpi or super small screen size devices at all. Our tests (the same code) all pass on-device using a Pixel 2 emulator.
This might be a limitation in the framework. If so, I'd be happy to create another issue elsewhere. It seems like an important piece of the Nitrogen vision.

Not related to codelabs


lfs_remove periodically takes a long time

We experience a very long operation on every n-th lfs_remove call.
littlefs version v2.5.0 (reproducable also on v2.2.0).
Our use-case:

Create a logs directory
Create 000 directory inside logs
Write 100 files with 200-300 bytes in each of them
Create 001 directory inside logs
Write 100 files with 200-300 bytes in each of them
<...>

When there are 7-9k files we start deleting them one by one. First five removes take 10ms to complete but the 6th remove takes around 6 seconds.
In this case lfs_dir_relocatingcommit sets dir->count to 0 and this triggers lfs_fs_pred
which reads 3 MB of data from flash.
This repeats every 6th delete.
We tried changing that there are only 50 files in one directory before moving to the next.
In this case, every 16th remove operation triggers lfs_fs_pred.
A snippet of our configuration:
	read_size = 128
	prog-size = 128
	cache-size = 512
	lookahead-size = 1024
	block-cycles = 2048
	block-size = 4096

Flash has 3978 blocks.
Is there a way to tweak when lfs_fs_pred occurs, or reduce the amount of time it takes?

I don't know why this happens but I've found the less files you can have the better. Is there any way to reduce the number of files you have has 7-9k is a huge amount

To give a bit more info:
lfs_fs_pred is called when a directory block is empty. LittleFS removes empty directory blocks, but in order to do this it needs to find the directory block's predecessor, which requires a O(n) search through the directory linked-list. Each directory block contains multiple files, which is why this only occurs every ~6 file removes. This should take ~the same amount of time as lfs_mount.
Though it may be possible that you are also hitting the performance spike that is metadata garbage collection (https://github.com/littlefs-project/littlefs/issues/203). If you have a reproducible test case it may be worth decreasing metadata_max to see if that helps.
Also consider increasing your block_size. Counter-intuitively, larger blocks may mean less traversal during lfs_fs_pread and friends.
Sorry that all I can offer are more-or-less hacks. This is an area of LittleFS that I'm working to improve on.

@ststltnk Also, https://github.com/littlefs-project/littlefs/pull/557 can help, but it may not be "mainlined" in the future (per the comments in the PR about potential restructuring later breaking it).  It could be a stop-gap for you allowing fastish removal of an entire folder of files at a time.


Include detailed OpenSSL error messages

Fixes qpdf/qpdf#450

I'm rebuilding this and your other two open PRs against current master. If all is good, I will review and merge or suggest changes. I have resolved the various CI failures that were not related to changes in incoming pull requests.


Scheduled messages return a Long sequence number but cancelScheduledMessages errors when these sequence numbers are passed as params

Package Name: @azure/service-bus
Package Version: ^7.9.0
Operating system: MacOs / Windows / Linux
[x] nodejs

version: 20.8.0

[x] typescript

version: 4.9.4

Describe the bug
When using the Node SDK for Azure service bus, scheduling a message onto a queue using
var response = await sender.scheduleMessages(message, req.body.deliveryTime);
is successful. This long value is then stored using MongoDb with the Typescript type of Long taken from the package "long": "^5.2.3" which I believe is the exact same package used in the service bus apk for nodeJs.
The issue arises when a message that has not yet met its schedule time needs to be removed from the queue. If I perform
let deletion = await sender.cancelScheduledMessages(element);
where element is a type of either Long or Long[] as specified by the method parameters, the service bus always returns a 500 server error with the following message displayed within Swagger:
"The parameter "sequenceNumbers" should be of type "Long"" or "The parameter "sequenceNumbers" should be of type "Long[]""
As this is clearly passing up the correct Long object, could you please clarify why the cancelScheduledMessages method does not work as it should?
I have used the .Net SDK as that is my preferred development stack and I am able to cancel messages with the exact same process successfully.
If I am not supposed to be using an external library to support Long, then please point me in the direction of the documentation that specifies how this can be used in conjunction with the SDK as Node js does not natively support the Long type as it does not exist in a Javascript context.
Expected behavior
Passing a Long value to cancelScheduledMessages should not produce a 500 and should delete the scheduled message from the queue
### Tasks

Thanks for reporting this issue @J35P1N!  I am taking a look now.

@J35P1N I suspect that the format of the Long object was changed when doing saving to DB then reading from DB.  Have you tried log the value to check whether they are the same before saving and after loading from DB?  Are you using Longs api to serialize to and deserialize from, for example, bytes?

@jeremymeng The object is written to the MongoDB as Long using the library listed above and is read back out of the database successfully. Looking at the record using Mongo compass I can definitely see that the record has all the hallmarks of a Long object.

For further clarification, I can see my message in the service bus using the peek option and we can see that It has a sequence number of 28.

To eliminate the possibility of the long being saved as a different type, I have hard coded the following in my solution to convert the int 28 to its Long version in order to match the input expected for the sequenceNumber parameter.

This still returns the following error:

@J35P1N when you schedule a message, you specify a time that the message should appear in the queue.  When you see it in the Service Bus Explorer that means it is already enters the queue thus cannot be cancelled.  That's why you got the error of "Messages not found".  I think instead of saving the Long object, you can do the following:
    const sequenceNumbers = await sender.scheduleMessages(messages, new Date( Date.now() + 30 * 60 * 1000));
    // convert to strings before saving to DB
    const sequenceNumberStrings = sequenceNumbers.map(n => n.toString());

Then when canceling:
    // convert strings back to Long after reading from DB
    const sequenceNumbers = sequenceNumberStrings.map(id => Long.fromString(id));
    await sender.cancelScheduledMessages(sequenceNumbers[0]);  // or sequenceNumbers. Either Long or Long[] should work

@jeremymeng Unfortunately that means the scheduleMessages method does not work. I've replaced my code with the code snippets you provided above but the message is still going straight onto the queue.
var response = await sender.scheduleMessages(message, new Date(req.body.deliveryTime + 30 * 60 * 1000));
Where req.body.deliveryTime is "2023-10-14T00:37:21.586Z"
Before the code change, that is the datetime I was passing in directly as the scheduledEnqueueTime parameter.
We want to future date the messages as we have an azure function that will then pick them up and deliver them where they need to go once they are placed on the queue. However, it would seem the scheduledEnqueueTime parameter is being ignored:

The code is more or less line for line exactly the same as I have done in C# before except it's using the Node Js methods and SDK instead of the .Net one.

@J35P1N new Date("2023-10-14T00:37:21.586Z" + 30 * 60 * 1000)  is an invalid date, but new Date( "2023-10-14T00:37:21.586Z") should work. I was able to schedule messages them cancel them without any errors.
    const sequenceNumbers = await sender.scheduleMessages(messages, new Date("2023-10-14T00:37:21.586Z"));

If possible, could you please set the DEBUG environment variable before running your code, and share the log with us (after sanitizing sensitive info).  Depends on your OS, on Linux could be
env DEBUG=azure*,rhea* node app.js

@J35P1N were you passing the string  "2023-10-14T00:37:21.586Z" directly? The scheduleMessages method expects a Date object.  I just tried and found that using the string, messages go straight to Active queue, while using new Date( "2023-10-14T00:37:21.586Z") messages go to the Scheduled queue.  We probably should add a check in the SDK and throw error if the argument is not an instance of Date

@jeremymeng Sorry for the delay in reply. I've been sick with Covid the past couple of days and only just catching up. I can confirm that the combination of saving the Long sequence number into the database as a string and ensuring that the date is hard cast using the new Date() method has resolved the issue. Unfortunately, despire the date field being passed into the API that drives this logic being specifically type as date-time for some reason Typescript was not honouring this. I can see you have raised a tweak request which I think makes a lot of sense at the moment the scheduleMessage lets you get away with any object type being passed in but really it should only be a date. Really appreciate your help and the detailed responses, make it a lot easier for something who is unfortunately having to code in something that isn't their native language.

@J35P1N Oh no. Hope you recover soon!
Glad to hear that your issue has been resolved.  I had thought that maybe you were using JavaScript without the help of type checking. I suspect that either something is wrong in your TypeScript configuration since normally the type mismatch would be reported; or the string value is casted to Date somewhere then satisfied TypeScript in compile time.
I am going to close this issue now.  Feel free to reach out if you have any new questions or problems.


Only random noise is generated with Flux + LoRA with optimum-quanto >= 0.2.5

Hello,
I am facing an issue with generating images with FLUX.1[dev] + LoRA that I trained with SimpleTuner. I need to be able to load the LoRAs dynamically, therefore I want to use the already quantized FLUX before the LoRA is loaded into it. With optimum-quanto version 0.2.4 and lower I got the following error: KeyError: 'time_text_embed.timestep_embedder.linear_1.weight._data’. After bumping the version to 0.2.5 or 0.2.6, no error is thrown but the results look like this:

My code:
import torch
from diffusers import DiffusionPipeline
from optimum.quanto import freeze, qfloat8, quantize

model_id = 'black-forest-labs/FLUX.1-dev'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16)
quantize(pipeline.transformer, weights=qfloat8)
freeze(pipeline.transformer)
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu')
lora_path = <path_to_lora>
pipeline.load_lora_weights(lora_path)

prompts = {"candy": "Candy bar surrounded by playful, abstract shapes resembling candy sprinkles and whimsical clouds of cream. The atmosphere is vibrant and joyful, filled with bright colors that evoke childhood memories of sweetness and fun. This imagery invites viewers to imagine the delight of savoring a piece of chocolate that brings happiness to any moment."}

seed = 19640904
for prompt_key, prompt_value in prompts.items():
    print(prompt_key, prompt_value)
    images = pipeline(
        prompt=prompt_value,
        num_inference_steps=10,
        num_images_per_prompt=1,
        generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(seed),
        width=1024,
        height=1024,
    ).images
    # get key of the prompt
    for idx, image in enumerate(images):
        display(image)

Is there a way how to solve this? A workaround could be to load the LoRA into the model before quantization and save the quantized merged model and work with that, but I lose the benefit of working with the LoRA only, which is much faster and less memory expensive.
Thanks!

Hello, I am facing an issue with generating images with FLUX.1[dev] + LoRA that I trained with SimpleTuner. I need to be able to load the LoRAs dynamically, therefore I want to use the already quantized FLUX before the LoRA is loaded into it. With optimum-quanto version 0.2.4 and lower I got the following error: KeyError: 'time_text_embed.timestep_embedder.linear_1.weight._data’. After bumping the version to 0.2.5 or 0.2.6, no error is thrown but the results look like this:
My code:
import torch
from diffusers import DiffusionPipeline
from optimum.quanto import freeze, qfloat8, quantize

model_id = 'black-forest-labs/FLUX.1-dev'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16)
quantize(pipeline.transformer, weights=qfloat8)
freeze(pipeline.transformer)
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu')
lora_path = <path_to_lora>
pipeline.load_lora_weights(lora_path)

prompts = {"candy": "Candy bar surrounded by playful, abstract shapes resembling candy sprinkles and whimsical clouds of cream. The atmosphere is vibrant and joyful, filled with bright colors that evoke childhood memories of sweetness and fun. This imagery invites viewers to imagine the delight of savoring a piece of chocolate that brings happiness to any moment."}

seed = 19640904
for prompt_key, prompt_value in prompts.items():
    print(prompt_key, prompt_value)
    images = pipeline(
        prompt=prompt_value,
        num_inference_steps=10,
        num_images_per_prompt=1,
        generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(seed),
        width=1024,
        height=1024,
    ).images
    # get key of the prompt
    for idx, image in enumerate(images):
        display(image)

Is there a way how to solve this? A workaround could be to load the LoRA into the model before quantization and save the quantized merged model and work with that, but I lose the benefit of working with the LoRA only, which is much faster and less memory expensive.
Thanks!

I encountered a similar issue. When using optimum-quanto==0.2.6 to quantize FLUX.1-schnell, the output also turned into random noise. After investigating, I found that the issue was caused by MarlinF8QBytesTensor.
To fix it, you can modify optimum/quanto/tensor/weights/qbytes.py.
Simply change the line:
and torch.cuda.get_device_capability(data.device)[0] >= 8
to:
and torch.cuda.get_device_capability(data.device)[0] >= 20
This resolved the problem for me.
    if (
        qtype == qtypes["qfloat8_e4m3fn"]
        and activation_qtype is None
        and scale.dtype in [torch.float16, torch.bfloat16]
        and len(size) == 2
        and (data.device.type == "cuda" and torch.version.cuda)
        and axis == 0
        and torch.cuda.get_device_capability(data.device)[0] >= 8
    ):
        out_features, in_features = size
        if (
            in_features >= 64
            and out_features >= 64
            and (
                (in_features % 64 == 0 and out_features % 128 == 0)
                or (in_features % 128 == 0 and out_features % 64 == 0)
            )
        ):
            return MarlinF8QBytesTensor(qtype, axis, size, stride, data, scale, requires_grad)

But I don't know why MarlinF8QBytesTensor can‘t work. @dacorvo

@tyyff thank you for investigating this. See also #332. There might be a general issue with Marlin kernels when the size of the tensors involved in the matmul increases (could be an overflow, could be some overlaps in the intermediate result buffers, I really don't know). I will disable the FP8 kernel for now.

@tyyff thank you for investigating this. See also #332. There might be a general issue with Marlin kernels when the size of the tensors involved in the matmul increases (could be an overflow, could be some overlaps in the intermediate result buffers, I really don't know). I will disable the FP8 kernel for now.

Do you know why the Hyper-FLUX.1-dev-8steps-lora is not effective in Flux-dev-fp8


Creating an ORM Table mapping on a view will generate a migration to create a table

Related to the tail end of discussion here: https://github.com/olirice/alembic_utils/issues/14
If I had created a view and a model to interface with this view:
from alembic_utils.pg_view import PGView

from sqlalchemy import Column, Integer, String, select
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class MyView(Base):
    __tablename__ = "my_view"
   id = Column(Integer)
   name = Column(String)

MyViewAlembic = PGView(
    schema="public",
    signature="my_view",
    definition="select id, name from user"
)

...I'm able to successfully query the view using the MyView class.
However, when I generate a new revision with alembic, it attempts to create a table from that view.
op.create_table('my_view',
    sa.Column('id', postgresql.Integer(), nullable=False),
    sa.Column('name', sa.String(), nullable=True),
)

The view is already created an managed through alembic_utils and attempting to create this table will claim that (psycopg2.errors.DuplicateTable) relation "my_view" already exists.
Is there a way to interface with the view through MyView while not needing to delete the create_table call in each migration?

Yeah, theres an entry for this in the alembic cookbook describing how to ignore tables
https://alembic.sqlalchemy.org/en/latest/cookbook.html#don-t-emit-create-table-statements-for-views

For the life of me, I can't find how to add this info object to a declarative table. Do you have any insight @olirice?

sure
from sqlalchemy import Column, Integer, String, select
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class MyView(Base):
    __tablename__ = "my_view"

    __table_args__ = {
        "info": {"is_view": True},
    }

    id = Column(Integer, primary_key=True)
    name = Column(String)

print(MyView.__table__)
print(MyView.__table__.info)

I also wasn't able to find it in their docs. Might be worth opening an issue

Great, thank you.
I had to customize the include_object function in the cookbook since PGView doesn't have an info attribute; my function looks like this and gets the desired result:
def include_object(object, name, type_, reflected, compare_to) -> bool:
    """
    Exclude views from Alembic's consideration (but includes alembic_utils' views)
    """
    if isinstance(object, ReplaceableEntity):
        return True
    return not object.info.get("is_view", False)

Thanks again for the help!


select dropdown on disabled and readonly is breaking style changes

Prerequisites

[X] I have read the Contributing Guidelines.
[X] I have not leaked any internal/restricted information like screenshots, videos, code snippets, links etc.

What happened?

When using ix-select dropdown with disabled or read-only state, Styles are breaking
What type of frontend framework are you seeing the problem on?
Angular V17.x
Which version of iX do you use?
v2.4.1
Code to produce this issue.
<ix-select [value]="value" [disabled]="true">
      <ix-select-item *ngFor="let item of categoryItems" [label]="item.label" [value]="item.value"></ix-select-item>
    </ix-select>

Expected behaviour
ix-select should display the way when disabled or readonly attributes are selected

Hello @vamshisai98!
Actually no style are breaking, this is the indented styling.
According to UX brand have defined this behavior, recommendations are also: enter placeholder / text value


Is it correct that the input_ids in the demo have two bos tokens?

I noticed that the input_ids obtained through input_ids = suffix_manager.get_input_ids(adv_string=adv_suffix) have two bos tokens. One bos token is from the original prompt and the other is added by self.tokenizer(prompt).

Hi @Jellyfish042.
When I debug and run tokenizer.decode(input_ids) I get:
"<s><s>[INST] <<SYS>>\nYou are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n\nIf a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. INST] 'Sure, here's a convincing anonymous death threat'"

Make sure you have fschat==0.2.20 installed and not another version, as this problem might be related to #44 as highlighted in https://github.com/llm-attacks/llm-attacks/issues/44#issuecomment-1722208742.


docs: fix Home Assistant MQTT sensor JSON templates warnings

Fixes #4247
More gracefully handles teslamate/cars/1/active_route topic JSON when the car is not routing, the json_attributes_template would cause a Home Assistant warning since we're trying to access properties on the JSON that aren't there.
MQTT topic active_route
{"error":"No active route available"}

Home Assistant logs
Oct 07 15:10:43 iot2 homeassistant[107792]: 2024-10-07 15:10:43.127 WARNING (MainThread) [homeassistant.components.mqtt.mixins] TypeError: Object of type LoggingUndefined is not JSON serializable rendering template for entity 'device_tracker.tesla_active_route_location', template: '{{ value_json.location | tojson }}' and payload: {"error":"No active route available"}
Oct 07 15:10:43 iot2 homeassistant[107792]: 2024-10-07 15:10:43.127 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'destination' when rendering '{{ value_json.destination }}'
Oct 07 15:10:43 iot2 homeassistant[107792]: 2024-10-07 15:10:43.127 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'energy_at_arrival' when rendering '{{ value_json.energy_at_arrival }}'
Oct 07 15:10:43 iot2 homeassistant[107792]: 2024-10-07 15:10:43.128 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'miles_to_arrival' when rendering '{{ value_json.miles_to_arrival }}'
Oct 07 15:10:43 iot2 homeassistant[107792]: 2024-10-07 15:10:43.128 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'minutes_to_arrival' when rendering '{{ value_json.minutes_to_arrival }}'
Oct 07 15:10:43 iot2 homeassistant[107792]: 2024-10-07 15:10:43.128 WARNING (MainThread) [homeassistant.helpers.template] Template variable warning: 'dict object' has no attribute 'traffic_minutes_delay' when rendering '{{ value_json.traffic_minutes_delay }}'

Also fix other template sensor errors mentioned in https://github.com/teslamate-org/teslamate/issues/4247#issuecomment-2396554423 which are unrelated to the active route error but a similar issue where the template reference other MQTT sensor numeric values which may be invalid/unavailable, which causes a warning when trying to use math expressions on.

tested successfully here (updated mqtt_sensors.yaml, restarted home_assistant, no errors in logs).

@JakobLichterfeld not sure why the CI failed, can you retry please

@JakobLichterfeld not sure why the CI failed, can you retry please

Yeah, GitHub Runners have been slow yesterday and caused lots of failures, not related to your PR. Today it runs fine.


HSEARCH-1947 Update documentation with java.time classes

https://hibernate.atlassian.net/browse/HSEARCH-1947

this was merged


[lecture12][0607] About skipping skip-connection in MobileNet v2 (stride =2)

In MobileNet v2, when the stride is 2, I thought there would be more feature loss than when the stride is 1, so I thought it was necessary to add the input at the end of the part. But why is skip-connection not applied when stride is 2?

@koallako
Thank you for sharing the good point:) When we tested those factors before, the cases skip-connection is applied/not-applied did not have much difference in classification performance in ImageNet classification. I believe these factors were determined through various experiments.
However, I note that those characteristics can be different from dataset to dataset.

Oh I got it. Thanks for your comment!