esphome impure, causes glibc errors

Describe the bug
Running esphome from nixpkgs on nixos2111 results in glibc errors.
Steps To Reproduce
Steps to reproduce the behavior:

nix run  nixpkgs#esphome compile --debug flipper_switch.yaml

flipper_switch.yaml is basically this config https://www.esphome-devices.com/devices/athom-smart-plug-tp29/
(esphome on nixos2111 was broken for me as well)
Expected behavior
Compiles firmware
Screenshots
acquiring global GC lock '/nix/var/nix/gc.lock'
acquiring read lock on '/nix/var/nix/temproots/986796'
acquiring write lock on '/nix/var/nix/temproots/986796'
downgrading to read lock on '/nix/var/nix/temproots/986796'
using cache entry '{"name":"flake-registry.json","type":"file","url":"https://github.com/NixOS/flake-registry/raw/master/flake-registry.json"}' -> '{"etag":"\"8cb00b8710ab079546c48c7eb41c3b06f603b6a81be44cade2a89ae4d661e5fc\"","url":"https://raw.githubusercontent.com/NixOS/flake-registry/master/flake-registry.json"}', '/nix/store/157arm6mc317qax77qw2jx668fyihx88-flake-registry.json'
acquiring global GC lock '/nix/var/nix/gc.lock'
looked up 'flake:nixpkgs' -> 'github:NixOS/nixpkgs/nixpkgs-unstable'
acquiring write lock on '/nix/var/nix/temproots/986796'
downgrading to read lock on '/nix/var/nix/temproots/986796'
using cache entry '{"name":"source","type":"file","url":"https://api.github.com/repos/NixOS/nixpkgs/commits/nixpkgs-unstable"}' -> '{"etag":"W/\"d9d551f5b9145e1805f02833ef8ed1d3acf9d720ca444c1f481f9bc0e213003c\"","url":"https://api.github.com/repos/NixOS/nixpkgs/commits/nixpkgs-unstable"}', '/nix/store/g87krsh2rb9ims6zmk14lnwwajca0w02-source'
HEAD revision for 'https://api.github.com/repos/NixOS/nixpkgs/commits/nixpkgs-unstable' is 17b62c338f2a0862a58bb6951556beecd98ccda9
acquiring write lock on '/nix/var/nix/temproots/986796'
downgrading to read lock on '/nix/var/nix/temproots/986796'
using cache entry '{"rev":"17b62c338f2a0862a58bb6951556beecd98ccda9","type":"git-tarball"}' -> '{"lastModified":1653738054,"rev":"17b62c338f2a0862a58bb6951556beecd98ccda9"}', '/nix/store/nbph6bdglyhaid9cfbwiwby0fpy6ga6y-source'
got tree '/nix/store/nbph6bdglyhaid9cfbwiwby0fpy6ga6y-source' from 'github:NixOS/nixpkgs/17b62c338f2a0862a58bb6951556beecd98ccda9'
checking access to '/nix/store/nbph6bdglyhaid9cfbwiwby0fpy6ga6y-source/flake.nix'
evaluating file '/nix/store/nbph6bdglyhaid9cfbwiwby0fpy6ga6y-source/flake.nix'
old lock file: {
  "nodes": {
    "root": {}
  },
  "root": "root",
  "version": 7
}
computing lock file node ''
new lock file: {
  "nodes": {
    "root": {}
  },
  "root": "root",
  "version": 7
}
using cached string attribute 'legacyPackages.x86_64-linux.esphome.type'
using cached string attribute 'legacyPackages.x86_64-linux.esphome.drvPath'
using cached string attribute 'legacyPackages.x86_64-linux.esphome.outPath'
using cached string attribute 'legacyPackages.x86_64-linux.esphome.outputName'
using cached string attribute 'legacyPackages.x86_64-linux.esphome.name'
building of '/nix/store/93rvj30xcn83glg53jgaq69wjr24ypjq-esphome-2022.5.1.drv!out' from .drv file: created
building of '/nix/store/93rvj30xcn83glg53jgaq69wjr24ypjq-esphome-2022.5.1.drv!out' from .drv file: woken up
querying info about missing paths...
starting pool of 7 threads
entered goal loop
building of '/nix/store/93rvj30xcn83glg53jgaq69wjr24ypjq-esphome-2022.5.1.drv!out' from .drv file: init
building of '/nix/store/93rvj30xcn83glg53jgaq69wjr24ypjq-esphome-2022.5.1.drv!out' from .drv file: loading derivation
acquiring write lock on '/nix/var/nix/temproots/986796'
downgrading to read lock on '/nix/var/nix/temproots/986796'
building of '/nix/store/93rvj30xcn83glg53jgaq69wjr24ypjq-esphome-2022.5.1.drv!out' from .drv file: have derivation
acquiring write lock on '/nix/var/nix/temproots/986796'
downgrading to read lock on '/nix/var/nix/temproots/986796'
building of '/nix/store/93rvj30xcn83glg53jgaq69wjr24ypjq-esphome-2022.5.1.drv!out' from .drv file: done
building of '/nix/store/93rvj30xcn83glg53jgaq69wjr24ypjq-esphome-2022.5.1.drv!out' from .drv file: goal destroyed
INFO Reading configuration flipper_switch.yaml...
INFO Detected timezone 'Europe/Berlin'
INFO Generating C++ source...
INFO Compiling app...
/nix/store/bcwyhqvik6jhcklib33v6kykdl4zp9d4-bash-5.1-p8/bin/sh: symbol lookup error: /usr/lib/libc.so.6: undefined symbol: _dl_audit_symbind_alt, version GLIBC_PRIVATE
Traceback (most recent call last):
  File "/nix/store/5jngq8dlfvnh75ydhsx1pdz0yr28zw94-platformio-6.0.1/lib/python3.9/site-packages/platformio/project/helpers.py", line 71, in get_default_projects_dir
    assert IS_WINDOWS
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/nix/store/5jngq8dlfvnh75ydhsx1pdz0yr28zw94-platformio-6.0.1/bin/.platformio-wrapped", line 6, in <module>
    from platformio.__main__ import main
  File "/nix/store/5jngq8dlfvnh75ydhsx1pdz0yr28zw94-platformio-6.0.1/lib/python3.9/site-packages/platformio/__main__.py", line 21, in <module>
    from platformio import __version__, exception, maintenance
  File "/nix/store/5jngq8dlfvnh75ydhsx1pdz0yr28zw94-platformio-6.0.1/lib/python3.9/site-packages/platformio/maintenance.py", line 22, in <module>
    from platformio import __version__, app, exception, fs, telemetry
  File "/nix/store/5jngq8dlfvnh75ydhsx1pdz0yr28zw94-platformio-6.0.1/lib/python3.9/site-packages/platformio/app.py", line 60, in <module>
    "value": get_default_projects_dir(),
  File "/nix/store/5jngq8dlfvnh75ydhsx1pdz0yr28zw94-platformio-6.0.1/lib/python3.9/site-packages/platformio/project/helpers.py", line 81, in get_default_projects_dir
    subprocess.check_output(["xdg-user-dir", "DOCUMENTS"])
  File "/nix/store/vzqny68wq33dcg4hkdala51n5vqhpnwc-python3-3.9.12/lib/python3.9/subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/nix/store/vzqny68wq33dcg4hkdala51n5vqhpnwc-python3-3.9.12/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['xdg-user-dir', 'DOCUMENTS']' returned non-zero exit status 127.

Additional context
Add any other context about the problem here.
Notify maintainers
@globin
@elseym
@hexa
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
output here

I think the problem is actually in platformio:
DEBUG Running add_pin_initial_states_array in esphome.components.esp8266.gpio (num 31)
DEBUG Adding global: const uint8_t ESPHOME_ESP8266_GPIO_INITIAL_MODE[16] = {OUTPUT, 255, 255, INPUT_PULLUP, INPUT_PULLUP, INPUT_PULLUP, 255, 255, 255, 255, 255, 255, OUTPUT, OUTPUT, OUTPUT, 255};
DEBUG Adding global: const uint8_t ESPHOME_ESP8266_GPIO_INITIAL_LEVEL[16] = {1, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 255, 1, 0, 0, 255};
DEBUG  -> finished
DEBUG Running add_arduino_global_workaround in esphome.core.config (num 33)
DEBUG Adding global: #define yield() esphome::yield()
DEBUG Adding global: #define millis() esphome::millis()
DEBUG Adding global: #define micros() esphome::micros()
DEBUG Adding global: #define delay(x) esphome::delay(x)
DEBUG Adding global: #define delayMicroseconds(x) esphome::delayMicroseconds(x)
DEBUG  -> finished
INFO Compiling app...
DEBUG Running:  platformio run -d .esphome/build/flipperswitch -v
/nix/store/bcwyhqvik6jhcklib33v6kykdl4zp9d4-bash-5.1-p8/bin/sh: symbol lookup error: /usr/lib/libc.so.6: undefined symbol: _dl_audit_symbind_alt, version GLIBC_PRIVATE
Traceback (most recent call last):
  File "/nix/store/5jngq8dlfvnh75ydhsx1pdz0yr28zw94-platformio-6.0.1/lib/python3.9/site-packages/platformio/project/helpers.py", line 71, in get_default_projects_dir
    assert IS_WINDOWS
AssertionError

During handling of the above exception, another exception occurred:

cc @oxzi @makefu

Might this be related to #173599? At least both are mentioning xdg-user-dir.

Unfortunately I cannot reproduce the error based on the YAML from the posted URL.
$ git describe
22.05-pre-48384-g4a88391c998
$ nix-shell -I nixpkgs=$(pwd) -p esphome --pure --run "esphome compile foo.yml"
. . .
INFO Successfully compiled program.

Im getting a similar glibc error when attempting to use Android Studio:
com.intellij.openapi.vcs.VcsException: /run/current-system/sw/bin/setsid: symbol lookup error: /usr/lib/libc.so.6: undefined symbol: _dl_audit_symbind_alt, version GLIBC_PRIVATE

After looking for a fix I found a few similar issues being reported: https://github.com/rust-lang/rust-analyzer/issues/12410 and https://github.com/haskell/haskell-language-server/issues/2928.
The last latter mentions a channel mismatch, although rebuilding my system (without any channels, just a flake) with my local nixpkgs checkout from which I build Android Studio didn't seem to change anything. Tried adding xdg-user-dir to the environment as well since #173599 was mentioned, but no luck.

@IvarWithoutBones What glibc version did you use to fix this? Could you please provide instructions on how to fix this?

@IvarWithoutBones What glibc version did you use to fix this? Could you please provide instructions on how to fix this?

I made my system use glibc from my local nixpkgs checkout so its consistent with the one in the package as follows:
~/nix/nixpkgs > nixos-rebuild switch --override-input nixpkgs .
~/nix/nixpkgs > reboot
~/nix/nixpkgs > nix build -Lf . esphome
~/nix/nixpkgs > ./result/bin/esphome

If you update your system to use the same version of glibc as the nixpkgs flake registry entry it should work as well, so I expect this problem might go away with a regular system upgrade as well.

Still having this issue in PlatformIO.


[Feature Request] Remove only blank lines

[Feature Request] I would really love it if there could be an option to remove double newlines (eg. remove blank lines)
Thanks

Thank you for the feature request! I have added this feature in v1.0.5. Please update the plugin and let me know what you think.


feat(miner rpc): support txn_add_gateway

Issue

Link: https://github.com/NebraLtd/hm-pyhelper/issues/91
Summary: Use RPC to return add_gateway_txn in this format..

How
Invoke RPC endpoint, then deserialize result to
construct the expected payload:

base64 decode
deserialize txn wrapper blockchain_txn_pb2
unwrap the blockchain_txn_add_gateway_v1_pb2 txn
parse needed fields

In order to parse the resulting addresses, a leading 0 byte representing the version must be added.

Leading 0 removed on miner: https://github.com/helium/libp2p-crypto/blob/master/src/libp2p_crypto.erl#L493-L494
Leading 0 added on app: https://github.com/helium/helium-js/blob/8d5cb76e156fb80de6fc80f239b43e3872c7b7d7/packages/crypto/src/Address.ts#L64
Called from: https://github.com/helium/helium-js/blob/8d5cb76e156fb80de6fc80f239b43e3872c7b7d7/packages/transactions/src/utils.ts#L16
Called from: https://github.com/helium/helium-js/blob/8d5cb76e156fb80de6fc80f239b43e3872c7b7d7/packages/transactions/src/AddGatewayV1.ts#L85
Called from: https://github.com/helium/hotspot-app/blob/86fa04d3e1ae86886309943cffa9fc5fbb32326d/src/utils/transactions.ts#L51

This also confirms that this response can be signed by the owner. ~The only doubt I still have is if, and at what point the payer (us) would need to sign this. I have asked on Discord for clarification.~
Via discord:

The app sends the partial transaction to the onboarding server. The onboarding server does the final signature and the app submits it to the blockchain

Screenshots

References

Related to: https://github.com/NebraLtd/hm-pyhelper/issues/91
Part of: https://github.com/NebraLtd/product-management/issues/5

Checklist

[x] Tests added
[x] Cleaned up commit history (rebase!)
[x] Documentation added
[x] Thought about variable and method names

This work is currently blocked by upstream issues:

RPC responses do not contain payer: https://github.com/helium/miner/issues/1118#issuecomment-1017708533. They closed this but said they would address. Worse case, we'll have to revert to dbus, which is working correctly.
Inconsistent fees: https://github.com/helium/miner/issues/1376. Not a blocker strictly speaking, but we should understand what's going on.

@marvinmarnold I closed #1118 because it was for a different issue that has been fixed long ago. Can you reference/file a different bug which describes the particular issue you're having? Thanks!

Update: Now able to deserialize addresses by adding the version \x00 byte. Currently blocked by https://github.com/helium/miner/pull/1378. This (probably) resolves both the issues below.

https://github.com/helium/miner/issues/1377
https://github.com/helium/miner/issues/1376

I'm sure this is intentional for readability, would it be better if we condensed the 3 lines using hm_pyhelper.miner_json_rpc.exceptions

@SebastianMaj you mean like this? https://github.com/NebraLtd/hm-pyhelper/pull/99/commits/469ca9ddb92b903d0f53fa2cdec0ddf44c9384af


mkcert as local user getting Not: not found

I'm running mkcert as a local Docker container user in order to get it installed into the NSS DB so that it will work with headless Chrome (and Selenium) in a Docker container. I've used this approach successfully in a different container, but in this one (selenium/standalone-chrome-debug) I'm hitting a strange error that I'm not quite sure how to diagnose or debug.
Command: output
$ TRUST_STORES=nss mkcert -install
/usr/local/bin/mkcert: 1: /usr/local/bin/mkcert: Not: not found

From what I can tell, the tool is getting unexpected output and trying to execute that output. When I tried another invocation of this (with sudo -E in case permissions were an issue), it failed on line 26 of sh.
It's running Ubuntu:
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:        18.04
Codename:       bionic

I install mkcert in the Dockerfile with:
ENV MKCERT_VERSION=v1.4.0
RUN sudo su -c 'apt-get update && \
  apt-get install -y curl openssl libnss3-tools && \
  curl -sSL https://github.com/FiloSottile/mkcert/releases/download/$MKCERT_VERSION/mkcert-$MKCERT_VERSION-linux-amd64 -o /usr/local/bin/mkcert && \
  chmod +x /usr/local/bin/mkcert && \
  apt-get purge -y curl && apt-get clean && \
  chmod +x /selenium2-startup.sh'

And I see paths to the expected binaries with which, etc.

Ah. The contents of /usr/local/bin/mkcert are Not Found, so I guess the URL changed from last time I used it. That'll do it! Closing this.

P.S. link was fine, my switch to single quotes when porting my Dockerfile from headless Chrome to Selenium was the culprit. Obviously a literal $MKCERT_VERSION won't work.

I'm running mkcert as a local Docker container user in order to get it installed into the NSS DB so that it will work with headless Chrome (and Selenium) in a Docker container. I've used this approach successfully in a different container, but in this one (selenium/standalone-chrome-debug) I'm hitting a strange error that I'm not quite sure how to diagnose or debug.
Command: output
$ TRUST_STORES=nss mkcert -install
/usr/local/bin/mkcert: 1: /usr/local/bin/mkcert: Not: not found

From what I can tell, the tool is getting unexpected output and trying to execute that output. When I tried another invocation of this (with sudo -E in case permissions were an issue), it failed on line 26 of sh.
It's running Ubuntu:
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 18.04.2 LTS
Release:        18.04
Codename:       bionic

I install mkcert in the Dockerfile with:
ENV MKCERT_VERSION=v1.4.0
RUN sudo su -c 'apt-get update && \
  apt-get install -y curl openssl libnss3-tools && \
  curl -sSL https://github.com/FiloSottile/mkcert/releases/download/$MKCERT_VERSION/mkcert-$MKCERT_VERSION-linux-amd64 -o /usr/local/bin/mkcert && \
  chmod +x /usr/local/bin/mkcert && \
  apt-get purge -y curl && apt-get clean && \
  chmod +x /selenium2-startup.sh'

And I see paths to the expected binaries with which, etc.

You should try "./mkcert -install"


Refactor core for a more streamlined API for aggregators/controllers

Description
This is a major refactor of the core library, with the goals to provide:

A clearer plugin-architecture for user- defined helpers and aggregators
a new controller accommodating stragglers and policies for how to handle them
Improvements to the db-backend  schema and tracing
Simplification of deployments
Many code and docs improvements

It's a bit confusing that we have /network and common/net

I think I would move up (towards root) everything in fedn/fedn so that one would have "cli" on the same level as "network" etc. This is to avoid the ugly full paths such as fedn/fedn/fedn/network.

any reason why reducer don't have it's own folder? Combiner, Clients etc have it...

No particular reason.


Fix created directories permissions

During a build, the created directories (build, shared source, ...)
were created using 0644 permissions instead of 0755.
What's really weird is that it used to work in VirtualBox VMs, but
fails (which is to be expected) in VMWare VMs.

:+1:


Add NonderterministicFastCommit to speed up migrations when ordering isn't required

Updates #394
This PR adds NonderterministicFastCommit to improve speed, bytes/op, and allocs/op compared to FastCommit.
NonderterministicFastCommit commits changes in nondeterministic order. It can be used by migration program when ordering isn't required.
FastCommit vs NonderterministicFastCommit
Casual micro benchstat output (from my busy desktop):
                             │  before.txt  │              after.txt               │
                             │    sec/op    │    sec/op     vs base                │
StorageFastCommit/10-12         89.72µ ± 4%   57.50µ ±  3%  -35.92% (p=0.000 n=10)
StorageFastCommit/100-12        118.9µ ± 1%   116.0µ ±  4%        ~ (p=0.436 n=10)
StorageFastCommit/1000-12       4.086m ± 5%   2.397m ± 25%  -41.35% (p=0.000 n=10)
StorageFastCommit/10000-12     12.629m ± 4%   9.857m ±  3%  -21.95% (p=0.000 n=10)
StorageFastCommit/100000-12    102.73m ± 0%   72.26m ±  1%  -29.66% (p=0.000 n=10)
StorageFastCommit/1000000-12     1.544 ± 2%    1.141 ±  2%  -26.09% (p=0.000 n=10)
geomean                         6.661m        4.848m        -27.21%

                             │  before.txt  │              after.txt              │
                             │     B/op     │     B/op      vs base               │
StorageFastCommit/10-12        28.92Ki ± 0%   28.05Ki ± 0%  -3.00% (p=0.000 n=10)
StorageFastCommit/100-12       286.4Ki ± 0%   278.6Ki ± 0%  -2.71% (p=0.000 n=10)
StorageFastCommit/1000-12      3.009Mi ± 0%   2.901Mi ± 0%  -3.58% (p=0.000 n=10)
StorageFastCommit/10000-12     28.65Mi ± 0%   27.79Mi ± 0%  -2.98% (p=0.000 n=10)
StorageFastCommit/100000-12    278.8Mi ± 0%   271.1Mi ± 0%  -2.75% (p=0.000 n=10)
StorageFastCommit/1000000-12   2.923Gi ± 0%   2.821Gi ± 0%  -3.49% (p=0.000 n=10)
geomean                        9.101Mi        8.820Mi       -3.09%

                             │ before.txt  │             after.txt              │
                             │  allocs/op  │  allocs/op   vs base               │
StorageFastCommit/10-12         219.0 ± 0%    205.0 ± 0%  -6.39% (p=0.000 n=10)
StorageFastCommit/100-12       1.980k ± 0%   1.875k ± 0%  -5.30% (p=0.000 n=10)
StorageFastCommit/1000-12      19.23k ± 0%   18.23k ± 0%  -5.22% (p=0.000 n=10)
StorageFastCommit/10000-12     191.1k ± 0%   181.1k ± 0%  -5.24% (p=0.000 n=10)
StorageFastCommit/100000-12    1.918M ± 0%   1.816M ± 0%  -5.30% (p=0.000 n=10)
StorageFastCommit/1000000-12   19.15M ± 0%   18.15M ± 0%  -5.22% (p=0.000 n=10)
geomean                        62.31k        58.91k       -5.45%

[ ] Targeted PR against main branch
[x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work
[x] Code follows the standards mentioned here
[x] Updated relevant documentation
[x] Re-reviewed Files changed in the Github PR explorer
[x] Added appropriate labels

Looks like validation and storage health checks passed tonight with this PR added to atree migration program at:

(locally modified) https://github.com/onflow/flow-go/tree/feature/atree-inlining-cadence-v0.42

Need to take another look at logs to confirm.

Would it be possible to get a diff of this new function and the existing deterministic version of it? It would make it easier to review, by seeing how they differ.
If the functions only differ in a small way, maybe it's possible to extract the shared code and only make the parts that differ configurable?

@turbolent  Good suggestions!  However, in this case there are more changes than unchanged code so a diff, etc. isn't as helpful as it normally would be.
The changes are mostly about eliminating overhead and parallelizing as much as possible.
I outlined steps in both functions below so we can see the differences at higher level.
FastCommit() commits deltas in deterministic order:

sort non-temp slab IDs by address and index
send non-temp slab IDs to job queue
(in goroutines) read and encode slabs from deltas

job queue (input): slab ID
result queue (output): encoded slab and possible error

collect all encoded results from result queue in temporary map
commit deleted and encoded results in order to base storage

In order to commit in deterministic order, the commit step has to wait until all slabs are encoded, and there are also overheads from sorting slab IDs and collecting encoded results in temporary map.
NonderterministicFastCommit() commits deltas in nondeterministic order:

send non-temp and non-empty slab to job queue
(in goroutines) read and encode slabs

job queue (input): slab ID and slab
result queue (output): encoded slab and possible error

commit deleted slabs to base storage
commit encoded slabs from result queue to base storage

In nondeterministic approach, the commit steps are in parallel with encoding and there isn't any temp map to collect data.
Even though goroutine encoding step is similar, FastCommit() reads slab from deltas map while NonderterministicFastCommit() reads slab from job queue.  Although I can modify FastCommit() to be the same in this aspect, it doesn't seem worthwhile since that would use more memory without making FastCommit() faster.

@fxamacker Thank you for the description! 👌  I'll take another look today

@fxamacker Does this also have to get ported to the 1.0 branch?

@turbolent I will port it for completeness, but it is only needed if we use the optimization for Cadence 1.0 migration without atree inlining.


Fix Multipart content view parsing

Description
On get_message_content_view, the content type wasn't including the boundary, and was only setting the MIME type. This made the multipart content view unusable, as the boundary was required on parsing. To fix the issue, we assign the full content type instead.
This wasn't triggered by any previous tests because they would test against the multipart parser directly, and not the generic parser.
Checklist

[X] I have updated tests where applicable.
[x] I have added an entry to the CHANGELOG.

I'm not sure why the check is failing?

https://github.com/mitmproxy/mitmproxy/actions/runs/7878070568/job/21495523975?pr=6653#step:5:74
Test is failing because test_multipart is not covering line 17. Only test_api does. We are (a bit too) strict here...
Cheap fix is to list http_message directly as an arg and then not have any branches: https://github.com/mitmproxy/mitmproxy/blob/1a02ebb89f6765d827f2fe0086dfe5960eb6e093/mitmproxy/contentviews/query.py#L8-L12
Thanks for the quick RTT! 🍰

This PR should be good now!


Explicit SCSI-1 support: Add support for .hd1 image files

rascsi shall support a new image type ".hd1". This type is identical with ".hds" except for the SCSI level and response code format being SCSI-1 instead of SCSI-2.
The alternative approach of being able to (having to ...) configure the SCSI and response code levels would force additional configuration settings on the users and would require much more changes in the core, and also in the clients. Most users do not have the expert knowledge required for these settings anyway and would run into issues with regular platforms when playing with these settings.
To be done after merging the SASI removal changes.
Also see https://github.com/akuker/RASCSI/issues/799.

@Pacjunk In order to speed things up a bit I created a branch testing_scsi1. This branch contains all pending changes (SASI removal) plus the changes required to support the .hd1 extension. Can you please test whether the testing_scsi1 branch resolves the remaining issues with your VAXstation?

@Pacjunk There is a branch feature_scsi1_image_file now, which will be the branch used for the upcoming PR.

@akuker You have changed the c-cpp.yml file in this branch. I'm not sure whether this was intentional, because there is also a ci_integration branch.

@akuker You have changed the c-cpp.yml file in this branch. I'm not sure whether this was intentional, because there is also a ci_integration branch.

I was trying to keep it from spamming your email inbox as much :) Since I enabled the action, it's going to fail every time you commit.

@akuker I see, thank you. I was not aware was this file was doing, this why I was asking. I assume we have to ensure that it is not merged into develop later?

You are correct. Feel free to take a look at what I've got at : https://github.com/akuker/rascsi_ci_integration/actions
I'll merge it into the real repo later today

@akuker I will most likely have a look at the actions in the future. I'm not familiar with GitHub actions because I use GitLab for my own projects that use automation. GitLab was the better choice for me because I needed a free Docker repository for them. GitHub did not provide a free private repository when I set up my projects. As far as I know they still do not provide that.

@uweseimet OK, I have tested on the new branch!  The web interface is broken, so I had to learn how to use rasctl (not a bad thing!). Anyway, tested with an hds file and an hd1 file. The hds file returned levels 2/2 and the hd1 file returned levels 1/1 so that bit worked OK.
However, I noticed that my system would now boot off the hds file (SCSI-2) as well. After looking at the trace logs (more rasctl commands!), it appears that the code is no longer preventing page code 0, so my system boots fine off the SCSI-2 device. Was this something missed during your major code rearrangement? - not that I'm complaining :-)

@Pacjunk :-) I'm afraid this is a bug, mode page 0 should still be rejected. There is no unit test yet for this. I cannot immediately see from the code what's wrong, maybe you can? Adding a log with the relevant MODE SENSE would help.
Regarding non-existent mode pages in general it would be possible to extend rascsi, so that you can specify static content for a mode page on the command line, by providing the sequence of bytes to be returned. But I would not really like to add this functionality. The clients would also have to offer this, making it even harder to understand all available settings. In addition, you would have to know which data the device has to return for a particular page. For vendor-specific pages you usually do not know that.

@Pacjunk After having a closer look at the code I would expect the condition in mode_page_device.cpp:50:
	LOGTRACE("%s Requesting mode page $%02X", __PRETTY_FUNCTION__, page);

	// Mode page data mapped to the respective page numbers, C++ maps are ordered by key
	map<int, vector<BYTE>> pages;
	AddModePages(pages, page, changeable);

	if (pages.empty()) {
		LOGTRACE("%s Unsupported mode page $%02X", __PRETTY_FUNCTION__, page);
		throw scsi_error_exception(sense_key::ILLEGAL_REQUEST, asc::INVALID_FIELD_IN_CDB);
	}

to be true (page list is empty) and add a trace log item and throw an exception, which would result in an error condition to be reported.

@Pacjunk I just noticed I was wrong and there already is a unit test for page 0. It's the first in mode_pages_test.cpp. It tests that an exception is raised if page 0 is requested.

Attached is part of the log. That "Requesting mode page" message isn't even there!
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.743] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Selection() Selection Phase Target ID=2
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() Command Phase
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[0]=$1A
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[1]=$00
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[2]=$00
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[3]=$00
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[4]=$0C
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[5]=$00
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] void ScsiController::Execute() Execution phase command $1A
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [debug] ++++ CMD ++++ void ScsiController::Execute() Executing command $1A
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [debug] bool Dispatcher<T>::Dispatch(T*, DWORD) [with T = ModePageDevice; DWORD = unsigned int] Executing ModeSense6 ($1A)
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual void ScsiController::DataIn() Going into Data-in Phase
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] void ScsiController::Send() Sending handhake with offset 0, length 12
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] void ScsiController::Send() Move to next phase datain (6)
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual void ScsiController::Status() Status Phase $00
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] void ScsiController::Send() Sending handhake with offset 0, length 1
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] void ScsiController::Send() Move to next phase status (8)
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual void ScsiController::MsgIn() Message In phase
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual void ScsiController::MsgIn() Transitioning to Send()
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] void ScsiController::Send() Sending handhake with offset 0, length 1
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual void ScsiController::MsgIn() Transitioning to Send()
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] void ScsiController::Send() Move to next phase msgin (9)
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual void ScsiController::BusFree() Bus free phase

Can you please investigate?Am 04.09.2022 14:17 schrieb Pacjunk @.**>:
Attached is part of the log. That "Requesting mode page" message isn't even there!
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.743] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Selection() Selection Phase Target ID=2
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() Command Phase
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[0]=$1A
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[1]=$00
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[2]=$00
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[3]=$00
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[4]=$0C
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] virtual void ScsiController::Command() CDB[5]=$00
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [trace] void ScsiController::Execute() Execution phase command $1A
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [debug] ++++ CMD ++++ void ScsiController::Execute() Executing command $1A
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.744] [debug] bool Dispatcher::Dispatch(T, DWORD) [with T = ModePageDevice; DWORD = unsigned int] Executing ModeSense6 ($1A)
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual void ScsiController::DataIn() Going into Data-in Phase
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] void ScsiController::Send() Sending handhake with offset 0, length 12
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] void ScsiController::Send() Move to next phase datain (6)
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual void ScsiController::Status() Status Phase $00
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] void ScsiController::Send() Sending handhake with offset 0, length 1
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.745] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] void ScsiController::Send() Move to next phase status (8)
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual void ScsiController::MsgIn() Message In phase
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual void ScsiController::MsgIn() Transitioning to Send()
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] void ScsiController::Send() Sending handhake with offset 0, length 1
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual BUS::phase_t ScsiController::Process(int) Initiator ID is 6
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual void ScsiController::MsgIn() Transitioning to Send()
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] void ScsiController::Send() Move to next phase msgin (9)
Sep  4 17:40:57 raspberrypi RASCSI[433]: [2022-09-04 17:40:57.746] [trace] virtual void ScsiController::BusFree() Bus free phase
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

@Pacjunk Hmm, the previous comment is strange. It resulted from a mail reply, but some github did not deal well with it ;-).
Anyway, what I mean is that there appears to be a contradition: Either mode page 0 is returned even though it shouldn't be, and in this case the trace log should show the respective MODE SENSE command and how it is processed. Or the trace log shows nothing, and in this case there wouldn't have been any MODE SENSE command at all, also not for page 0.
Note sure what the resolution is of this contradiction is, but this is what I would like you to investigate. Please always use the latest sources from the feature branch.

In disk.cpp:
int length = (int)cdb[4];  --> The modesense6 requests length 12 (in my trace)
size = 12;  --> Length of the mode parameter block
size += super::AddModePages(cdb, &buf[size], length - size); --> length-size = 12 - 12 = 0
In mode_page_device.cpp:33:
int ModePageDevice::AddModePages(const DWORD *cdb, BYTE *buf, int max_length)
{
	if (max_length <= 0) {
		return 0;
	}

In the request my machine sends, max_length will be 0. This is <=0 so the test will exit the function immediately. That is why we don't see the "requesting mode page" message.
Zero is a valid value, so maybe the test should be <0 instead of <=0. Then again, is it really possible for max_length to be negative??  If not, drop the whole "if" statement.

@Pacjunk I added an explicit test for mode page 0 to my external test sute, which is running on my Atari. The mode page part (see below) also does not find anything wrong. It can read all available mode pages, and trying to read the non-existing mode page 0 fails because there is none. From my perspective this is consistent with the unit tests and the implementation and is the correct behavior.
If there is still anything wrong on your side, feedback would be great.
  MODE SENSE
    Reading all mode pages with MODE SENSE (6)
      Received 106 data bytes
        Available pages list: 1, 3, 4, 8, 48
        Page 3: Format device page (current, not savable)
          Tracks per zone: 8
          Alternate sectors per zone: 0
          Alternate tracks per zone: 0
          Alternate tracks per logical unit: 0
          Sectors per track: 25
          Data bytes per physical sector: 512
          Interleave: 1
          Track skew factor: 11
          Cylinder skew factor: 20
          Soft sector formatting (SSEC): 0
          Hard sector formatting (HSEC): 1
          Removable (RMB): 0
          Surface (SURF): 0
        Page 4: Rigid disk drive geometry page (current, not savable)
          Number of cylinders: 23714
          Number of heads: 8
          Starting cylinder-write precompensation: 0
          Starting cylinder-reduced write current: 0
          Drive step rate: 0
          Landing zone cylinder 0
          Rotational offset: 0
          Medium rotation rate: 7200
        Page 8: Caching page (current, not savable)
          Read cache disable (RCD): 0
          Write cache enable (WCE): 0
          Multiplication factor (MF): 0
          Write retention priority: 0
          Demand read retention priority: 0
          Disable pre-fetch transfer length: -1
          Minimum pre-fetch: 0
          Maxium pre-fetch: -1
          Maximum pre-fetch ceiling: -1
    Reading all mode pages with MODE SENSE (10)
      Received 110 data bytes
        Available pages list: 1, 3, 4, 8, 48
    Checking for support of vendor-specific mode page 0
      Mode page 0 is not supported

@Pacjunk My comment overlaps with yours. I will check.

@Pacjunk I see. I don't want to remove the statement because AddModePages is called from different locations and max_value is a signed value, and I want to keep is a part of defensive programming.
But I just commited your suggested change regarding the max_length check to the feature branch. Can you please verify?

@Pacjunk By the way, a user of my hard disk software sent me information that shows that BlueSCSI drives are SCSI-1, not SCSI-2:
 INQUIRY
    Calling with valid data
      Device type: Direct Access
      Removable media support: No
      Device name: 'QUANTUM'
      Firmware revision: '1.1'
      SCSI/SPC version: SCSI-1 CCS
      Response data format: SCSI-1 CCS
      Additional length: $1F

This can explain some differences between RaSCSI and BlueSCSI. Even with .hd1 files there might be cased where rascsi is not SCSI-1 compliant. If such a case is found a new ticket should be created. The current change only deals with INQUIRY values and nothing else.

That should do it, but I can't really test for a day or so  (sorry about the earlier delay - I was away). I'll let you know.
Yes, I know bluescsi returns SCSI-1 for the inquiry - that's why I never had the problem with that system. My system worked fine with the .hd1 image file, so it only seems to care about the inquiry results.

@Pacjunk I don't expect that regarding SCSI-1 anybody is going to test more right now. What I meant is that if things are fine for you now I can raise a PR, because this ticket is for .hd1 only anyway. If somebody should find anything wrong with SCSI-1 in the future, she/he can create a new ticket. WRT to SCSI-1 (other than INQUIRY) the current code does not differ from previous releases anyway. So any problem would very likely have been there before.
Please confirm that we are fine for the final PR.

OK. I'm rebuilding now, but won't be able to test until tomorrow night. I'll let you know in 24 hours.  Thanks,

@Pacjunk OK, thank you. If a different issue than the one this ticket is directly dealing with should pop up, a new ticket should be created, so that we can more this .hd1 change.
After testing, please also let me know whether https://github.com/akuker/RASCSI/issues/799 is resolved with this ticket, or whether there is still something left in https://github.com/akuker/RASCSI/issues/799 that needs to be addressed. Hopefully not ;-).

@uweseimet OK, things seem to be operating as expected.
  If I boot the .hd1 file, then everything works fine. Once the system is booted I can access both drive types as the page 0 problem only occurs at boot. I'm happy for the solution to progress  (not so much that a solution is required in the first place, but hey, that's life!). Thanks for your work on this.
@rdmark Does the web front end need any tweaking to support the .hd1 image files?  I can't look myself as the web server seems to be broken in this branch.

@Pacjunk Thank you for your feedback and the time spent on testing! I am going to finalize the PR.

@Pacjunk Please remember to comment on the status of https://github.com/akuker/RASCSI/issues/799. Is there anything left in https://github.com/akuker/RASCSI/issues/799 to be done after merging the .hd1 support?

No, should be able to close that one.

If the web UI won’t start I guess there must be some assumption in the code
that isn’t valid anymore. I can take a look when I’m back from vacation in
two weeks, unless someone else beats me to it. ;)
Also at the very least the hd1 image type should be added to the image file
creation drop down.
On Wed, Sep 7, 2022 at 10:13 AM Pacjunk @.***> wrote:

@rdmark https://github.com/rdmark Does the web front end need any
tweaking to support the .hd1 image files? I can't look myself as the web
server seems to be broken in this branch.
—
Reply to this email directly, view it on GitHub
https://github.com/akuker/RASCSI/issues/814#issuecomment-1239059289, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAQCYM4HT6V37VSNWX7RS6LV5BFDJANCNFSM57XGULDA
.
You are receiving this because you were mentioned.Message ID:
@.***>


All tests are broken

Re write the tests due to them breaking.
I re wrote the class definition of Game and need to re-write the tests to pass a Player object to the classes rather than a str object. Otherwise the class fails as str doesn't have a setCounter method.... Whoops D:

Fixed all the testing proceedures!


Implement nextUrl for OpenID Authentication

Description
Adds support for nextUrl in OpenID Authentication. This change also supports nextUrl for multi authentication.
If a user's session times out, they will resume where they previously left off instead of being directed back to the home page.
This change is modeled off of similar work done for SAML:
Related PRs:

https://github.com/opensearch-project/security-dashboards-plugin/pull/1039

Category
Enhancement
Issues Resolved
https://github.com/opensearch-project/security-dashboards-plugin/issues/1141
Testing
Tested using a keycloak IdP in local development. Opening this PR up as a draft as functional tests are written for this change to verify the new behavior.
I'm pushing the function test repo to update to Cypress12 which has official support for cross-origin testing: https://github.com/opensearch-project/opensearch-dashboards-functional-test/issues/801
Check List

[x] New functionality includes testing
[x] New functionality has been documented
[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

It will be picked back up once https://github.com/opensearch-project/security-dashboards-plugin/pull/1579 is merged

Btw I think we need to update the branch with the latest changes

I'll open up a manual backport for this.

The backport for this is blocked by the backport for: https://github.com/opensearch-project/security-dashboards-plugin/pull/1579


correctly set anti-deadlock timer after the Initial space was dropped

Depends on #2452.
See https://github.com/quicwg/base-drafts/pull/3556.

Codecov Report

Merging #2457 into master will decrease coverage by 0.02%.
The diff coverage is 90%.

@@            Coverage Diff             @@
##           master    #2457      +/-   ##
==========================================
- Coverage   85.65%   85.63%   -0.02%
==========================================
  Files         120      120
  Lines        9206     9207       +1
==========================================
- Hits         7885     7884       -1
- Misses        975      976       +1
- Partials      346      347       +1

Impacted Files
Coverage Δ

internal/ackhandler/sent_packet_handler.go
75.32% <90%> (-0.45%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b9ab1d1...65568a2. Read the comment docs.


As seen in #12123, 48.5 is actually centered, rather than 49. (49 is shifted to the right by half a unit, making it not centered)
-271.5 for middle scroll would perfectly center middle scroll, tested using a similar method to #12123 , but checking how far from the center the notes are but split across the middle of the strum
Time bar and time text are obviously centered though lol.

448 kbps of pussé

As seen in #12123, 48.5 is actually centered, rather than 49. (49 is shifted to the right by half a unit, making it not centered)
-271.5 for middle scroll would perfectly center middle scroll, tested using a similar method to #12123 , but checking how far from the center the notes are but split across the middle of the strum
Time bar and time text are obviously centered though lol.

I didn't ask but thx lmao

👍

timeBarBG is already centered


Remove the defaults for textBaseline

We can't know what the correct default is. It depends entirely on the application's locale.
cc @tvolkert @ramnavan
Reverts https://github.com/flutter/flutter/pull/60586, https://github.com/flutter/flutter/pull/61425. See also https://github.com/flutter/flutter/issues/58053.

@Hixie - changes makes sense to me. added a minor suggestion to the test.

applied suggested fix, will land on green


Produce symbol_url on cache-hit

See discussion here: https://github.com/luser/rust-minidump/pull/259#issuecomment-952559889
Reiterating the options:

somehow saving the url into the cache alongside the cached file
just assuming a cache hit would have the same url we would have generated

I personally like the simplicity of 2, but it only really works on the assumption of one url provider (which tbh, I think is always true in practice). 1 is the "right" solution.
If we do 1, I'm inclined to extend the breakpad symbol format with a new CACHEINFO entry or something that the parser can slurp up.

Ah that's perfect, thanks!


What difference between registered external service and unregistered ones in HTTPS protocol?

Hi, I am using istio1.0.0 release and I set the google external service in the following doc. I send HTTPS request to both https://www.google.com and https://www.bing.com . And I noticed that both requests were 200 OK.
I want to know that what difference between both requests while I only set the google external service since both requests seems to be successful. Is my current situation correct?  Will Istio-proxy log the https requests?
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
  name: google
  namespace: istio-test
spec:
  hosts:
  - www.google.com
  ports:
  - number: 443
    name: https
    protocol: HTTPS
  resolution: DNS
  location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: google
  namespace: istio-test
spec:
  hosts:
  - www.google.com
  tls:
  - match:
    - port: 443
      sni_hosts:
      - www.google.com
    route:
    - destination:
        host: www.google.com
        port:
          number: 443
      weight: 100

root@sleep-776b7bcdcd-lbnkb:/# curl -I https://www.google.com
HTTP/1.1 200 OK
Date: Mon, 03 Sep 2018 11:47:35 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2018-09-03-11; expires=Wed, 03-Oct-2018 11:47:35 GMT; path=/; domain=.google.com
Set-Cookie: NID=138=TJdfDa2QxS2j2bxF8PqU959FSC29qlb7E9VlaVg9nEWFM48p2rgox0Ud7xPYFF9kUbaTQKFF8LJeQx7H5Ozk2HWoq_wpdGg8KWp8x0PF3JgKyWx2H9VINCOOTP0JIVf7; expires=Tue, 05-Mar-2019 11:47:35 GMT; path=/; domain=.google.com; HttpOnly
Transfer-Encoding: chunked
Alt-Svc: quic=":443"; ma=2592000; v="44,43,39,35"
Accept-Ranges: none
Vary: Accept-Encoding

root@sleep-776b7bcdcd-lbnkb:/# curl -I https://www.bing.com
HTTP/1.1 200 OK
Cache-Control: private, max-age=0
Content-Length: 0
Content-Type: text/html; charset=utf-8
Vary: Accept-Encoding
P3P: CP="NON UNI COM NAV STA LOC CURa DEVa PSAa PSDa OUR IND"
Set-Cookie: SRCHD=AF=NOFORM; domain=.bing.com; expires=Thu, 03-Sep-2020 11:47:54 GMT; path=/
Set-Cookie: SRCHUID=V=2&GUID=D79C4C88C59B442590DB76058AD822ED&dmnchg=1; domain=.bing.com; expires=Thu, 03-Sep-2020 11:47:54 GMT; path=/
Set-Cookie: SRCHUSR=DOB=20180903; domain=.bing.com; expires=Thu, 03-Sep-2020 11:47:54 GMT; path=/
Set-Cookie: _SS=SID=217FB8A33A2562232743B4FC3BF5637D; domain=.bing.com; path=/
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-MSEdge-Ref: Ref A: 35078CF6BC4D4907AEE53843B44F6532 Ref B: ASHEDGE0910 Ref C: 2018-09-03T11:47:54Z
Set-Cookie: _EDGE_S=F=1&SID=217FB8A33A2562232743B4FC3BF5637D; path=/; httponly; domain=bing.com
Set-Cookie: _EDGE_V=1; path=/; httponly; expires=Sat, 28-Sep-2019 11:47:54 GMT; domain=bing.com
Set-Cookie: MUID=2BA3A21DD5BA681514C0AE42D46A696D; path=/; expires=Sat, 28-Sep-2019 11:47:54 GMT; domain=bing.com
Set-Cookie: MUIDB=2BA3A21DD5BA681514C0AE42D46A696D; path=/; httponly; expires=Sat, 28-Sep-2019 11:47:54 GMT
Date: Mon, 03 Sep 2018 11:47:54 GMT

And I'm sure the includeIPRange arg for istio-init container in this POD is *.

Name:           sleep-776b7bcdcd-lbnkb
Namespace:      istio-test
Node:           <IP_ADDRESS>/<IP_ADDRESS>
Start Time:     Mon, 03 Sep 2018 19:33:22 +0800
Labels:         app=sleep
                pod-template-hash=3326367878
Annotations:    sidecar.istio.io/status={"version":"250a1f462190fb065274655b8c50a870c89365a5f3c6afbf9ddf46f768e09beb","initContainers":["istio-init"],"containers":["istio-proxy"],"volumes":["istio-envoy","istio-certs...
Status:         Running
IP:             <IP_ADDRESS>
Controlled By:  ReplicaSet/sleep-776b7bcdcd
Init Containers:
  istio-init:
    Container ID:  docker://25aa3e9e9dbbf73b04b800df5576149ac06693bd2ce1f137257b791d680dba03
    Image:         docker.io/istio/proxy_init:1.0.0
    Image ID:      docker-pullable://istio/proxy_init@sha256:345c40053b53b7cc70d12fb94379e5aa0befd979a99db80833cde671bd1f9fad
    Port:          <none>
    Host Port:     <none>
    Args:
      -p
      15001
      -u
      1337
      -m
      REDIRECT
      -i
      *
      -x

      -b

      -d

    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Mon, 03 Sep 2018 19:33:23 +0800
      Finished:     Mon, 03 Sep 2018 19:33:23 +0800
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:        20m
      memory:     400Mi
    Environment:  <none>
    Mounts:       <none>

I found that istio-1.0.0 with no mtls enabled will only manage http request, after i use istio with mtls enabled, only registered external service can be visited in POD in https protocol


Edit over SSH

Issue Type
Feature Request
Description
I would like an action, "Open Remote File (SFTP)", that would open a file on a remote system, using SFTP (SSH FTP).
This would

Open an editor pane with the contents of the file
Update the file on the remote system whenever I save the editor pane

This would not

Show anything in the explorer pane or any other side pane, beyond the editor entry in open editors

Why this is not a duplicate?
Technically, this is a duplicate of #23076. However, that issue was incorrectly closed as a duplicate of allow reading from stdin (#7320/#6161). This was incorrect as "remote editing by SSH" is very different from "allow reading from stdin".
This is distinct from #27423 as that issue is "open remote folder via SSH", not file. From a UI perspective, opening a file is much simpler, as the only UI needed is an editor pane and a few commands. A remote folder requires the contents of that folder to be displayed somewhere, such as the explorer pane.
VS Code Info
VS Code version: Code 1.20.1 (f88bbf9137d24d36d968ea6b2911786bfe103002, 2018-02-13T15:34:36.336Z)
OS version: Windows_NT x64 10.0.16299

Kind of a dupe of #29194 but once that (monster) item is done extension will be able to implement this. There is already a experimental implementation for ftp doing this (see https://github.com/jrieken/ftp-sample). So, ultimately closing as 'extension-candidate' but the API hasn't been finalised yet.

@jrieken Does the same hold true for #27423? That is, being closed as an extension candidate pending the remote API? Because it sounds like the report API is a remote FS API, which sounds like it would support directories.

report API is a remote FS API, which sounds like it would support directories.

Yup, that's what it will be

We just announced remote development with VS Code, check out the blog post for details


Using Immutable.Record?

The way things work now I don't think there is a good way to store/load Record types in state since toJS<->fromJS looses the Record type, i.e. state.save -> state.load does not work as expected. Am I missing something?

http://facebook.github.io/immutable-js/docs/#/fromJS check reviver


I wish /docs/user-guide/environment-guide/ didn't return an HTML document as text when clicking on the 'pod definition' link in the "Query the Service" section

I wish /docs/user-guide/environment-guide/ didn't return an HTML document as text when clicking on the 'pod definition' link in the "Query the Service" section
If you search for the text 'pod definition' and click the link, it takes you to this page:
http://kubernetes.io/docs/user-guide/environment-guide/show-rc.yaml
Which appears to be interpreted as text by Chrome, but contains HTML.

I think the link you refer to is at the bottom of http://kubernetes.io/docs/user-guide/environment-guide/
Same thing happens to me; the file in question is really at https://github.com/kubernetes/kubernetes.github.io/blob/master/docs/user-guide/environment-guide/show-rc.yaml
Also the containers/ link earlier on that page goes to http://kubernetes.io/docs/user-guide/containers/ which isn't appropriate.

Thanks for pointing this out. The page will soon be converted to a new task topic. We'll fix the strange link at that time.


Provide a way for a script to see all of the qualified variables available for a qualifier

This will be useful for script developers, especially model and authorization policy developers, to see what qualified variables are actually available within an execution context.

The qualified variable available changes depending on the context, so this will require some thought. Many variables only exist once they are observed.


prevent caching of index.html, add cache-control header for other req…

When upgrading the RaspiBlitz, some users get an error with the webUI since the cached index.html in the browser tries to load resources which don't exist anymore, see e.g. https://github.com/cstenglein/raspiblitz-web/issues/633
The JS & CSS filenames have the hash attached to them, so they are no problem, but the index.html always has the same filename, thus the browser wants to use the cached version.
This PR adds Cache-Control header to prevent any caching of the index.html (no-store) and tells the browser for other resources, that it should revalidate them (no-cache, must-revalidate). Don't be confused with no-cache - it does caching, it's a misnomer, see https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control

@rootzoll would like to have this included in the next version please


Fix color codes

Fix color codes
There were issues with some messages because SimpleCloud used '&' instead of '§' to use color codes and that didn't work. I replaced it to '§' so it works now.

Old messages with wrong characters:

New messages with right characters:

As you can see, the messages are now formatted correctly.

heloooooo c:


Handle error when sequentially submitting request for the sponsor fails

Context: https://github.com/api3dao/airnode/pull/734#discussion_r767515048 (Github doesn’t show comments from resolved conversations, so below is the copy of the context from github discussion)
The comment was about this line: https://github.com/api3dao/airnode/blob/ddeeb7cc56f0b58c393bc55a74eb51976e601345/packages/airnode-node/src/evm/fulfillments/index.ts#L53
The comment:

This is a very good point. We should:
Log the message that the request failed to be submitted
Handle this in the call site (line 120) and return early (maybe log something like: Skipping further ${numOfRemainingRequests} for the sponsor wallet since they would fail because of the wrong nonce anyway).
This error is not handled anywhere up to the processProviderTransaction and that one should not throw.
The error is caught by processEvmProviderRequests but the error message (Unable to process provider:${state.settings.name} requests) will be misleading since some of the requests may be processed successfully.

No, I forgot to assign you. Thanks for the reminder.
@amarthadan I think we should really automate this.

Actually, I am not sure how to test this and I'd suggest moving to DONE. Do you have an idea how to test this @amarthadan ?

@amarthadan I think we should really automate this.

I just checked issues, it's already planned :)


ci-cd failure at testing

Hi team! I noticed our ci-cd action is failing at this line. I failed to notice this when I approved the PR and will try to troubleshoot to correct this. But if you have any insights into what's happening please let me know! Thanks
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}

Something weird is happening at the testing- when run outside of an environment file I am getting an error that the datexplore module cannot be found. When run without the environment file,  the old test failures from way back in milestone 2 seem to have reappeared- this is from when we lost some changes to Jordan's tests and had to revert. I have attached screenshots of both error messages.
If we do not figure out the environment errors by tomorrow I will go to Tiff's office hour to ask her about this.


Created ISpecks

Description
Currently there are 2 speck implementation, 1 local and 1 server-side. This was okay for the hackathon, but the current implementation has quite the performance penalties.
I've implemented an ISpeck interface to allow for more flexibility in server/client side handling of them.
In the local implementation, it de-serializes and caches the geometry to prevent constant, slow de-serialization.
In the server implementation it has all of the information it had before, but with a few different constructors.
Type of change
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

@cwensley, @rfeathers068 I can't quite work out why this doesn't work. And I don't think I'm debugging it correctly. Everything builds, but the specks aren't passed around correctly. Would love a hand with this one, either fixing, or suggesting a better implementation. 😋

One thing I noticed, which Curtis may have to help with is the Database Migration didn't update for me so that might be part of the issue as well.

One thing I noticed, which Curtis may have to help with is the Database Migration didn't update for me so that might be part of the issue as well.

I had a feeling this might play a part

Hey @clicketyclackety, a few things that I can see about this PR:

Why is ServerSpeck in SpeckLib but LocalSpeck is in Crash?  I would expect ServerSpeck to be in Crash.Server unless there's a reason for that?  Ideally we should just use the same class on both sides and if we need other information on either side we can wrap it in an outer class.
I'm not sure you can use an Interface to pass things around with SignalR.. what class would it create on the server if it knows nothing about LocalSpeck?  It does not have any knowledge to create a ServerSpeck or LocalSpeck.
Any database model changes need to be reflected with a dotnet ef add migration <name_of_migration> in the Client.Server\ folder.  I should add a VS Code task for that and some instructions to the docs.
The ServerSpeck class requires a public default constructor for it to be used by the Entity Framework database, otherwise there's no way for the database calls to create instances of the class.

That's all I can see for now, I hope this helps!

Hey @cwensley,
Thanks for the notes. I was able to get things running again with a few changes;

ServerSpeck gets used by Crash to convert into LocalSpeck. LocalSpeck uses GeometryBase and I don't want the server to have to use Rhino dlls. If you can suggest a better way to separate things, I'd love that.
That is really handy to know. Makes total sense. I've moved to inheritance
Handy command. Would love a script for it. For now I've just been cleaning the directories and I updated the initial migration
I've ensured default constructors and getters/setters. Everything is currently public.

I think likely you guys may spot some more things. But it does now work nicely! Much nicer to be able to connect to the server and retrieve your work in progress stuff 😊

@cwensley if you could take a look at the new implementation, that'd be fab 😊


feature: Context aware rule engine evaluation operation

Rule-engine is defined as list of rules. Every rule is defined as tree of condition. So, we have three possible execution points to have context cancel check

rule-engine level - check for context cancellation on rule-engine. Evaluate is called. this is too early to check

condition level - single condition evaluation is much lighter as compare to context cancellation check.

Therefore, context check is added at rule evaluation level as option 3.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

40 of 40 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.4%) to 93.75%

Totals

Change from base Build<PHONE_NUMBER>:
0.4%

Covered Lines:
465

Relevant Lines:
496

💛  - Coveralls


Add a command for reading and modifying (i.e., bumping) the version in the pyproject.toml

:white_check_mark: Both Rye and Poetry have this, and it's quite useful both...

to avoid manually incrementing the version number (and potentially making a mistake there with the digits),
and to have this field on the pyproject.toml file serve as the single source of truth for a project's version, easily accessible (via uv) in CI workflows, etc.

:stop_sign: The latter is where my personal interests lie: I have a CI workflow that tags and deploys releases when the version is bumped, currently using Poetry for it. I don't really want to pull in an additional tool to parse the project specs, so it's currently keeping me from considering migrating to uv.
:scroll: My proposal is replacing the current functionality of the version subcommand with this, since uv's version can already be accessed via the -V | --version option, and other project management currently exist as "root" subcommands.
:information_source: I'm willing to give this one a go myself if maintainers greenlight the proposal.

It's most convenient to me to have git tag based versioning:
https://github.com/mtkennerly/poetry-dynamic-versioning
I would love to have this functionality.

Also very interested in this feature. This kind of read/update version command seems very common (not only Poetry but npm, etc) and it'd make migration to uv from other tools that much more straightforward.

Would maintaining version identifiers outside of pyproject.toml be considered out of scope for uv? For instance, updating something like a version in a src/package/init.py?

@JonZeolla  I've used PDM's dynamic versioning in the past (which is essentially the same as the Hatch example you provide) but have found it kind of annoying in practice. IMO it's preferable to have the version in pyproject.toml be the source of truth. That version can be exposed in the Python package using importlib.metadata.version:
[tool.poetry]
name = "my-package"
version = "1,2,3"

# my_package/__init__.py

from importlib.metadata import version

__version__ = version("my_package")

Personally I also like the idea of basing the version on the current git tag when using together with CI/CD.
In that case it would be great if there was a command to set the version in pyproject.toml to the value of the git tag.

Covering what bump-my-version (and previously bumpversion) do would cover that (including customizing the git tag and commit message format)
An interesting addition would be to have a way to trigger a custom
script to bump the CHANGELOGs like a hook to a python script.
An even better but probably too opinionated option could be to add a CHANGELOG management system (like changie) but it might be out of scope of uv?

For others who need this.
uvx --from=toml-cli toml set --toml-path=pyproject.toml project.version $VERSION

Consider how poetry does it: https://python-poetry.org/docs/cli#version
My workflow looks like this:
# Show current version
poetry version

# Bump version
poetry version minor

# Add a git tag with a matching version to what's in pyproject.toml
git tag "v$(poetry version -s)"

Poetry tries to be clever to avoid needing an option to specify whether you are auto-bumping or manually setting a version:

The new version should be a valid PEP 440 string or a valid bump rule: patch, minor, major, prepatch, preminor, premajor, prerelease

This assumes that no-one will ever want to use one of those strings as a version, but that seems like a reasonably safe assumption.

With hatch even when you have the version set as dynamic and red from an example __about__.py module and you bump hatch version $(git describe --broken --tags --always --dirty) (that what I do on build in CI/CD) it will actually change the python module. Which is kinda nice.

With hatch even when you have the version set as dynamic and red from an example __about__.py module and you bump hatch version $(git describe --broken --tags --always --dirty) (that what I do on build in CI/CD) it will actually change the python module. Which is kinda nice.

Using uvx hatch version ... is mostly fine, but I can't use it when it's a stub only package because there is no *.py.

I'd love to see something like this as well - I currently use python-semantic-version but that uses SemVer rather than PEP440 - which causes a few minor issues such as not being able to parse tags that are valid PEP440 but not SemVer git tags (i.e. tags we added before migrating to PSV). It also has a nice CHANGELOG generation module, and the ability to generate release or dev version based on branch name (main vs feature). Version bumping rules are based on commit message parsing (i.e. if message starts with feat: bump minor version, if fix: bump patch etc.). As part of the PSV build process, all files that are modified (i.e. pyproject.toml, version.py) are staged, committed and pushed. The commit is also tagged with the version. You can then filter by the commit message in your CI pipeline to prevent infinite recursion.

As a workaround, I use this in my project:
sed -i -e "s/0.0.0/${GITHUB_REF#refs/*/}/" pyproject.toml

https://github.com/astral-sh/packse/blob/70abfe8f64a9746452c02cb514942f879c7eaccc/.github/workflows/release.yaml#L34-L36

As a workaround, I use this in my project:
sed -i -e "s/0.0.0/${GITHUB_REF#refs/*/}/" pyproject.toml

https://github.com/astral-sh/packse/blob/70abfe8f64a9746452c02cb514942f879c7eaccc/.github/workflows/release.yaml#L34-L36

Exactly what I did, I just wasn't comfortable w/ it

Is there a reason why this won't/can't be implemented?

The uv version command already exists, so I need to come up with a design to either phase that out in favor this functionality or choose a new command name.

@zanieb according to the docs there are already other ways to get the uv version.
E.g.
uv --version      # Same output as `uv version`
uv -V             # Will not include the build commit and date
uv pip --version  # Can be used with a subcommand

@sam57719 that doesn't mean it's not a breaking change to switch the functionality. There is downstream code that will break if we change it.

The uv version command already exists, so I need to come up with a design to either phase that out in favor this functionality or choose a new command name.
Basically, that part isn't trivial and we're doing a lot of other things.

Maybe we could just have a way to bump by updating the .toml file?
something like ...
uv build --override-config "project.version=x.x.x"
Maybe someone already thought of that, I'm new to uv 🤷♂

Related: https://github.com/astral-sh/uv/issues/6440

Here's a workaround that may help wait for the feature: uv tree --depth 0 shows the tree of version dependencies of ... just our package:
$ uv tree --depth 0
Resolved 66 packages in 1ms
mass-driver v0.18.0a1

Noting the "Resolved 66 packages in 1ms" is coming on stderr (debug message), so with a bit of fanciful shell:
$ uv tree --depth=0  2> /dev/null | awk '{print $2}'
v0.18.0a1

Not perfect, but it's a start!
PS: Still would love the feature, and personally believe the breakage/change of semantics of uv version is worth it (given uv -V and `uv self exist), but I understand this requires careful thought.

That uv tree -q removes the actual tree output itself is probably a bug and now reported #8379

Hi everyone, I'm new to this and have a question about the version bump functionality. Since the existing uv version command poses challenges for integrating version bumping, would it make sense to introduce a new subcommand like uv project? This subcommand could handle project-related tasks, including version management. For example:
uv project --bump-patch
uv project --bump-minor
uv project --bump-major
Would this approach help avoid conflicts with the existing uv version command and provide a dedicated space for expanding project management features in the future?

Maybe the top level command could be something like:

uv bump python <version> - increase the minimum required python version to the given version (if the given version is lower or equal to the current minimum an error is given)
uv bump python --minor - increase the minimum required python version to the next highest minor version
uv bump python --major - increase the minimum required python version to the next highest major version
uv bump project <version> - increase the project version to the given version (if the given version is lower or equal to the current version an error is given)
uv bump project --patch - increase the project version to the next highest patch version
uv bump project --minor - increase the project version to the next highest minor version
uv bump project --major - increase the project version to the next highest major version

Bumpver would be a good project for inspiration. I submitted https://github.com/mbarkhau/bumpver/issues/239, but it was closed as a duplicate of https://github.com/mbarkhau/bumpver/issues/193, which seems like it's a ways off.

Bumpver would be a good project for inspiration. I submitted https://github.com/mbarkhau/bumpver/issues/239, but it was closed as a duplicate of https://github.com/mbarkhau/bumpver/issues/193, which seems like it's a ways off.

Looks a bit older compared to: https://github.com/callowayproject/bump-my-version but I assume uv maintainers already have good references 😃

Retitled to try to improve discoverability. We get a lot of duplicates of this one.

So, imagine we have dozens of poetry based projects that use Jenkinsfiles configured to use poetry version to build versioned prod docker images. And in order to get this version, we either go back to the good old setuptools days or use some python -c 'import toml ....  hacks, which obviously nobody will do. This basically means uv cannot be used as a build tool for us 😿 which is a shame.
A workaround with pyproject-info could work but if you have dependency groups in your pyproject.toml, the toml is no longer PEP 621 compliant, and it fails.
uv ability to resolve project versions is very important, in our case it is a blocker for its adoption unfortunately.

So, imagine we have dozens of poetry based projects that use Jenkinsfiles configured to use poetry version to build versioned prod docker images. And in order to get this version, we either go back to the good old setuptools days or use some python -c 'import toml ... hacks, which obviously nobody will do. This basically means uv cannot be used as a build tool for us 😿 which is a shame. A workaround with pyproject-info could work, but if you have dependency groups in your pyproject.toml, the toml is no longer PEP 621 compliant, and it fails. uv ability to resolve project versions is very important, in our case it is a blocker for its adoption, unfortunately.

I use this as a workaround:
version_number=$(uvx --from=toml-cli toml get --toml-path=pyproject.toml project.version)

Personally I do this with a version of Python that includes tomllib:
version=$(python -c 'import tomllib;fp=open("pyproject.toml","rb");print(tomllib.load(fp)["project"]["version"]);fp.close()')

It's most convenient to me to have git tag based versioning:
https://github.com/mtkennerly/poetry-dynamic-versioning
I would love to have this functionality.

This is basically the only thing which blocks me porting all my projects over to UV right now.

The uv version command already exists, so I need to come up with a design to either phase that out in favor this functionality or choose a new command name.
Basically, that part isn't trivial and we're doing a lot of other things.

My 2 cents on the issue:
I don't think there are many places where there is a dependency on a uv version, but even if there are and this will be something to phase out/deprecate I think there is a pretty straightforward way of doing it.

The -V, --version args should always point to an actual uv version.
Then at first, the project version management is under a certain prefix, e.g uv version project <args>

So that users can do uv version project minor, uv version project major, etc (the command design could be anything, I just assumed the most common would be the poetry one)

Later on, after deprecation notices the project prefix could be dropped and the command will be just uv version

Ideally, everyone who used an old uv version to check the uv version would switch to uv --version.

It's just something I thought of and any part could be changed/tweaked to better fit the desired outcome.
But anyway I think this is a major one and should be considered since it would make uv better and will allow for an easy drop-in replacement from any poetry based project.


[BugFix] Avoid sampling statistics using count as an alias to duplicate the column name in the table

What type of PR is this：

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Which issues of this PR fixes ：

Fixes #18685
Problem Summary(Required) ：

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto backported to target branch

[ ] 3.0
[ ] 2.5
[ ] 2.4
[ ] 2.3

[FE PR Coverage Check]
:disappointed: fail : 0 / 2 (00.00%)
file detail

path
covered_line
new_line
coverage
not_covered_line_detail

:large_blue_circle:
com/starrocks/statistic/StatisticsCollectJob.java
0
2
00.00%
[129, 131]

run starrocks_image_update

run starrocks_admit_test

@Mergifyio backport branch-3.0

@Mergifyio backport branch-2.5


Pin apps in application switcher

Hello,
First of all, thank you for your hard work, this is awesome!
I would like to be able to pin applications in the application switcher (or in a new type of item) in order to have the following behavior:

Show the launched and pinned apps in the list
On click on a pinned app: if the app is not launched: launch the app
On click a pinned app: if the app is launched: switch to the app
Have an option to pin/unpin apps in the list

What do you think?
In the meantime I guess I will write custom launchers, probably using wmctrl, but it will not be optimal.
Cheers,
Jerome.

While this is a nice idea, I won't have the time to implement this right now. So if anybody want's to do it.... just feel free :)

As Gnome-Pie is now in low-maintanance-mode, I won't have the time to implement this. If you're using GNOME Shell and if you are still interested in this project, you could try out Fly-Pie which is a successor project of Gnome-Pie. Fly-Pie does not yet support the behavior you're looking for, but we can discuss such features over there!


Add kyuubi-spark-monitor module in nightly.yml

Description:
Error:  Failed to execute goal on project kyuubi-spark-sql-engine: Could not resolve dependencies for project org.apache.kyuubi:kyuubi-spark-sql-engine:jar:1.3.0-SNAPSHOT: Could not find artifact org.apache.kyuubi:kyuubi-spark-monitor:jar:1.3.0-SNAPSHOT in Apache Snapshots Repository (https://repository.apache.org/snapshots/) -> [Help 1]
Add Kyuubi-spark-monitor module in nightly.yml

Codecov Report

Merging #772 (5ffb54f) into master (82fd58f) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##             master     #772   +/-   ##
=========================================
  Coverage     79.27%   79.27%
  Complexity       10       10
=========================================
  Files           130      130
  Lines          5090     5090
  Branches        657      657
=========================================
  Hits           4035     4035
  Misses          694      694
  Partials        361      361

Impacted Files
Coverage Δ

...g/apache/kyuubi/engine/spark/KyuubiSparkUtil.scala
90.90% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 69d4197...5ffb54f. Read the comment docs.

thanks, merging to master for v1.3.0


Allow tab content to be in a different part of the DOM

The current implementation requires tabs to be inline within the tabset. The jquery implementation is much less restrictive, allowing the tab-content to be placed anywhere within the DOM.
In general, is there any way to extend the behavior of the angular bootstrap directives rather than rewriting them when there's a similar restriction?

@richburdon - If I understand your question correctly, there's no restriction on where the content (template) for your tabs is defined.  I've defined them in completely separate files and done something like this:
<data-tabset class="tabset" data-justified="true">
  <data-tab data-ng-repeat="tab in tabs"
             data-active="tab.active">
    <data-tab-heading>
      {{ ::tab.heading }}
    </data-tab-heading>

  </data-tab>
</data-tabset>

I think he means that he would like the tab content to be completely decoupled from the directive.
I do not believe we will support this, as it would require stateful knowledge of where it is in the DOM, and the best way to do so is with a parent element, hence the existence of tabset.

That makes sense.  @richburdon don't forget that jQuery, and thus jQuery UI, is a DOM manipulation framework.  Most of what jQuery does is via selectors.  Whereas with Angular, the items are right there (directives or otherwise).

@richburdon, do you require further assistance here?  If not, I'm going to close this.  Thanks.

Closing as won't support as per discussion.


Added  Phonebook Project in C

Hi @mrbhatt2348 , if you faced problem on anything or creating PR again, just comment below. You are just one step away from your successful PR. Add the demo video with readme.md (for this just drag and drop the video inside readme.md and its done). It will be merged 100%


Unknown Board Revision c03112 - Raspberry PI 4 with 4GB

Hi, I try to run Johnny Five on my new Raspberry PI 4 c03112. #119 mentioned a new revision number — apparently there is another one. How can I force it in the mean time to use c03111?
pi@raspberrypi:~/servo_pointer $ yarn start
yarn run v1.19.1
$ sudo node ./server.js
Unknown board revision c03112, assuming Raspberry Pi Zero/2/3 pinout. Unless you are running a compute module or very old RPi this should work fine. Please report this board revision in a GitHub issue at https://github.com/nebrius/raspi-board.
1574610392096 Available Raspi IO
1574610392100 Connected Raspi IO
1574610392106 Repl Initialized
>> /home/pi/servo_pointer/node_modules/j5-io/dist/core.js:69
        throw new Error(`Unknown pin "${pin}"`);
        ^

const five = require("johnny-five");
const Raspi = require("raspi-io").RaspiIO;

const board = five.Board({
  io: new Raspi()
});

board.on("ready", () => {
  const led = new five.Led("P1-P12");
  led.blink();
});

Hi @D1no. The message printed is a warning only, and should work just fine with revision c03111. I'll get it added to the list.
The crash your experiencing is unrelated to the warning. The issue you're experiencing is because P1-P12 is not a valid pin. Did you mean P1-12?

Ok I'll need to check that when I play around again. In the last version I had locally I needed to patch add the revision number to the node module.


taxonRank categories are not standardised

I did a count of the different taxonrank fields, checking the current names in the taxonrank column

taxonRank
kingdom
count

Cultivar
Plantae
7

Form
Plantae
3

Generic  hybrid
Plantae
3

Genus
Plantae
26410

Nothosubspecies
Plantae
6

Species
Plantae
3545162

Species aggregate
Plantae
2489

Species group
Plantae
22731

Species hybrid
Plantae
15259

Subspecies
Plantae
155616

Variety
Plantae
12024

Is there a more rigid name convention or vocabulary that we could adopt?

Yes, we should use this vocabulary: http://rs.gbif.org/vocabulary/gbif/rank_2015-04-24.xml It uses lowercase names + no spaces. Here's a mapping of what it should be:

[ ] Cultivar → cultivar
[ ] Form → form
[ ] Generic  hybrid → not part of vocab, would like to see examples
[ ] Genus → genus
[ ] Nothosubspecies → would like to see examples
[ ] Species → species
[ ] Species aggregate → would have to see examples
[ ] Species group → would have to see examples
[ ] Species hybrid → would have to see examples
[ ] Subspecies → subspecies
[ ] Variety → variety

@stijnvanhoey can you provide some examples for the ones above?


delete operator doesn't work on objects with a metatable

Playground

Seems to work now
https://typescripttolua.github.io/play/#code/MYewdgzgLgBAZiEAuGBvGAjAhgJwPwpgCuAthgKY4wC+MAvGprigAw0DcAUBOVCb1ihYMAG3IAKBCAA0aagEounAA44AlmCiTEAOmw55K9ZvEATcmKjl4u-YqMatUvbnlA


Provide working location/connectivity examples

I'm getting the invalid location error for the provided location/connectivity examples:
Dulles_IE7.DSL
Frankfurt.Dial
China.custom
Dulles:Chrome.DSL

e.g. https://www.webpagetest.org/runtest.php?url=https:\/\/docs.webpagetest.org&k={my_api_key}&location=Frankfurt.Dial.
I added some examples that do work, e.g. https://www.webpagetest.org/runtest.php?url=https:\/\/docs.webpagetest.org&k={my_api_key}&location=ec2-eu-central-1.Dial.

Good catch. Thanks again! :)

Awesome, glad I could help, @tkadlec !
I just noticed we should probably clarify how to use the location param a bit more: https://github.com/WPO-Foundation/webpagetest-docs/pull/43


Error when _cas property is added to Loopback models after calling save()

After updating a property of a Loopback model instance and calling .save, the following error is passed to the callback:
ValidationError: The '<ModelName>' instance is not valid. Details: '_cas' is not defined in the model (value: undefined)
This occurs when the Loopback model is configured with strict set to true, where strict "specifies whether the model accepts only predefined properties or not." (source)
Why is the check-and-set flag included as a property? And if so, is there an option to avoid this?
A workaround is to add a _cas property to the model's json spec.
Alternatively, there is no error & no need to change the model json spec if the following line is removed:
https://github.com/Wiredcraft/loopback-connector-couchbase3/blob/master/lib/couchbase3.js#L397

This seems like a better fix, in-line with the _type property: https://github.com/Wiredcraft/loopback-connector-couchbase3/commit/99b7610b2e7df2367ae3134a913d886433efe2cd

Well it's a feature Couchbase provides, so I'd like to give people a chance to use it, though I don't have a real use case, or maybe we can have an option? Also I guess adding a _cas doesn't hurt does it?

Fair enough, closing.


add GZIP compression

closes #3

Pull Request Test Coverage Report for Build 219

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 89.674%

Totals

Change from base Build 211:
0.0%

Covered Lines:
165

Relevant Lines:
184

💛  - Coveralls


COLOR is broken

Godot 3.0, Godot 3.1
Windows 7
Issue:
When you try to use COLOR on a light shader, returns a white rectangle
for instance LIGHT = COLOR;
I was trying to port the isometric 2D lighting shader from 2.1 to 3.1, but COLOR is broken.
1- create a new shader
2- void light() {
LIGHT =COLOR;
}

COLOR is different between GLES3 and GLES2. For example this shader on a Sprite:
shader_type canvas_item;

void fragment() {
	COLOR;
}

In GLES3 this will render as white, in GLES2 this will render as the sprite's texture. I don't know if this is intended or which one is meant to be correct. For GLES3 you have to set COLOR like eon-s mentioned.

i see what you mean, but now the shader ignores completly the light and draws the sprite normaly (without light), maybe is something to do with the GLES3 like you say, but turning on the GLES2 on editor gives me an error of drives (this doesn´t happend on Godot 2.x) rare

Well i guess that Godot 3.1 GLES3 and GLES2 are pretty broken by the moment :/

Yeah, shaders are still to be tested on GLES2, there are many cases for these to fail (like with loops).
Lot of checks and warnings will have to be added...

I think this was fixed by #23339. Both GLES2 and GLES3 now assign the vertex color to COLOR and require the user to explicitly call texture(TEXTURE, UV) if they want to read from the sprites texture.

Indeed, thanks.


[Service Bus] Long term fix for duplication of module references by rush + node-resolve rollup plugin

The fix deployed for #3119 involves removing duplicate references to Buffer module after they have been generated.
This issue is to investigate why the duplication is happening, and deploy a long term fix for same issue that would not require us to post-process and remove the duplicates.

[ ] Investigate what is causing duplication of the Buffer module on disabling the dedupe flag on node-resolve plugin
[ ] Identify suitable fix and deploy

Following are some notes that could help with further investigation from Eng Sys side -
Qs to answer:

Why is duplication happening?
Why is duplication bad?
How does de-dupe in rollup work?
How is the build step making a difference to rollup output?

Things that can be tried:

If allowing dupe vs de-dupe creates any different outputs with using just npm
Check role of amqp-common by not having rush refer to local instance and pull it from registry instead.

https://www.npmjs.com/package/rollup-plugin-node-resolve/v/4.2.0  See the Usage section for:
// Force resolving for these modules to root's node_modules that helps
      // to prevent bundling the same package multiple times if package is
      // imported from dependencies.
      dedupe: [ 'react', 'react-dom' ], // Default: []

Changelog for this:
https://diff.intrinsic.com/rollup-plugin-node-resolve/4.1.0/4.2.0
https://github.com/rollup/rollup-plugin-node-resolve/releases/tag/v4.2.0
Going through the above links, this seems like the duplication of module references was expected by their code change when they moved from version 4.1.0 to 4.2.0 for rollup-plugin-node-resolve. They did expect users to use dedupe
@ramya0820  @mikeharder

Next step: let's create a minimal repro of this (a depends on b, and both a and b depend on c.  c is a third party dependency) and then experiment with fixes in an isolated situation.
Aside: While we currently use rollup, one possible investigation path would be to switch to webpack or pika.  This represents a bunch of work to switch though, so this would need to wait until after the previews are released.

Also, please refer to thread on #2809 for more context on what was attempted for EventHubs' browser tests to work with Rush.
For rollup to work with npm 4 more de-dupe entries were required. This is opposite of what was needed for Service Bus, where npm didn't require it and rush did. For Event Hubs, we haven't been able to get a short-term fix either (yet).

Looks like something changed with rollup even with Service Bus since I last worked on it.
As I'm working on testing few things related to #3471 / #2809, noticed that even for Service Bus, needing the same additional de-dupe entries to make it work with npm. I don't think the short term workaround for Service Bus works either. We'll need to likely skip running browser tests on CI altogether for now, unless it's with npm I think or some other bundler as discussed earlier today.
cc @ramya-rao-a @bterlson @kurtzeborn @mikeharder @bterlson @KarishmaGhiya

I have created a minimal repro here : https://github.com/KarishmaGhiya/module-duplication-rollup-rush

Update: @ramya0820 and I were able to repro the duplication of buffer issue. Here's the latest commit of the PR to the repro here : https://github.com/KarishmaGhiya/module-rollup-buffer/pull/3/commits/e04a2dd75c8db06f36a56f0f368f6744d7c70cb1
This is the PR -
https://github.com/KarishmaGhiya/module-rollup-buffer/pull/3/commits
cc: @mikeharder @bterlson

Some more notes and things to do next are:

The code we have so far works with npm. And duplication of Buffer in rolled up output was noted (though we only used npm). However, the tests still pass inspite of duplicated entries in rolled up file.
Replicate workspace and run only rush commands and see if behavior is same.

If the tests do not fail with Rush, then we haven't been able to exactly reproduce the problem.
And we may need to look into usage of Buffer again, and take a closer look at implementation of Rhea.
I think Rhea extends and exposes its own version/wrapper of Buffer as 'Buffer', but need to double check on this.

@KarishmaGhiya
Looks like https://github.com/KarishmaGhiya/module-rollup-buffer/pull/3/commits is not configured to run with rush as the standard rush commands - reset-workspace update build are failing with multiple instances of
 ERROR  No package.json (or package.yaml, or package.json5) was found

@ramya0820  The rush reset-workspace is not a standard rush command, it has a script that is configured with our main azure-sdk-for-js repo. I have just the minimal rush commands configured. The build should work.

Update:
Tests are not failing with rush, had created a separate workspace and cloned the code from PR with above change.
Thus, the issue has not been reproduced taking the current setup of
lib-aaa -> Buffer
lib-bee -> Buffer
We might have to likely try and mimic the concept of exposing global variable in the minimal package ‘C’ repro, as opposed to using ‘Buffer’ package which is a bit more complicated.
Or, use Buffer and try to mimic the way it's being used by Service Bus, Rhea, Rhea-promise and amqp-common.
Agree? Any thoughts on which direction we should go next?
@bterlson @mikeharder @ramya-rao-a

While we don't fully understand why the dedupe is needed, we're pretty sure it's related to how pnpm installs dependencies in a tree rather than flattened, which I believe is exactly what the dedupe option was added for:
https://github.com/rollup/rollup-plugin-node-resolve/pull/201
As long the generated browser bundle looks correct and passes tests, I think it's fine to use the dedupe option.

I did some more investigation today; found that there is a bug in the resolve package used by rollup-plugin-node-resolve where it will preserve symlinks in certain situations, even when preserveSymlinks is set to false. A fix is being worked on as we speak, and it may address all of our duplicate buffer issues without needing the dedupe hack (it will certainly address one duplicate, and hopefully the second 🤞).

Re-opening for tracking until https://github.com/browserify/resolve/issues/196 is resolved

Confirming that with browserify/resolve#196 and the dedupe config removed, there are no duplicate copies of buffer in the bundle!