RocksDB scanning Write CF returns keys out of order

Bug Report

What version of TiKV are you using?
https://github.com/tikv/tikv/pull/6513 (https://github.com/tikv/tikv/pull/6513/commits/13139a246882c7a1d6fc9b101820477210d3e3ce) ， which adds some logs based on https://github.com/tikv/tikv/pull/6308 (https://github.com/tikv/tikv/commit/a498b9d63bdcdf3054ae543579277fb8952d5e69) inorder to locate the problem.
The merge-base on master branch is https://github.com/tikv/tikv/commit/2c97d8706c0def76409a184d50c8ba10a904a3cc
Steps to reproduce
In a raw kv cluster, generate about 10G random raw keys in write cf with this program: https://gist.github.com/MyonKeminta/6a9d60111bbe2f6f96b091ac474d2fc4
./rawkv-util-cf --mode=rand-gen --pd-addr=<IP_ADDRESS>:2379 --start-key="31" --end-key="3130303030303030303030303030" --concurrency=16 --key-max-len=32 --cf=write

Then this may happen in rawkv backup and raftstore snapshot generation.
What did you expect?
Rawkv backup or snapshot generation succeeds normally.
What did happened?
Generating snapshot randomly fails:
[2020/02/03 20:22:15.338 -05:00] [INFO] [peer_storage.rs:794] ["requesting snapshot"] [request_index=0] [peer_id=279] [region_id=104]
[2020/02/03 20:22:15.339 -05:00] [INFO] [snap.rs:707] ["scan snapshot of one cf"] [size=0] [key_count=0] [cf=default] [snapshot=/data1/br/deploy/data/snap/gen_104_10_215979_(default|lock|write).sst] [region_id=104]
[2020/02/03 20:22:15.339 -05:00] [INFO] [snap.rs:707] ["scan snapshot of one cf"] [size=0] [key_count=0] [cf=lock] [snapshot=/data1/br/deploy/data/snap/gen_104_10_215979_(default|lock|write).sst] [region_id=104]
[2020/02/03 20:22:15.765 -05:00] [ERROR] [io.rs:82] ["keys scanned from snap out of order"] [current_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A] [last_key=7A311C39017D2D9DD3FEEDAE4B186FAF9FF802313EB566C44596E20B669481760B]
[2020/02/03 20:22:15.765 -05:00] [ERROR] [io.rs:90] ["sst writer put error"] [current_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A] [last_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A]
[2020/02/03 20:22:15.766 -05:00] [ERROR] [region.rs:268] ["failed to generate snap!!!"] [err="snap failed \"[src/raftstore/store/worker/region.rs:230]: [src/raftstore/store/snap/io.rs:76]: Io Storage Engine Invalid argument: Keys must be added in order\""] [region_id=104]
[2020/02/03 20:22:16.301 -05:00] [INFO] [peer.rs:738] ["failed to schedule peer tick"] [err="sending on a disconnected channel"] [tick=CHECK_PEER_STALE_STATE] [peer_id=1962] [region_id=1959]
[2020/02/03 20:22:17.335 -05:00] [WARN] [peer_storage.rs:771] ["failed to try generating snapshot"] [times=1] [peer_id=279] [region_id=104]
[2020/02/03 20:22:17.335 -05:00] [INFO] [peer_storage.rs:794] ["requesting snapshot"] [request_index=0] [peer_id=279] [region_id=104]
[2020/02/03 20:22:17.336 -05:00] [INFO] [snap.rs:707] ["scan snapshot of one cf"] [size=0] [key_count=0] [cf=default] [snapshot=/data1/br/deploy/data/snap/gen_104_10_215979_(default|lock|write).sst] [region_id=104]
[2020/02/03 20:22:17.336 -05:00] [INFO] [snap.rs:707] ["scan snapshot of one cf"] [size=0] [key_count=0] [cf=lock] [snapshot=/data1/br/deploy/data/snap/gen_104_10_215979_(default|lock|write).sst] [region_id=104]
[2020/02/03 20:22:17.776 -05:00] [ERROR] [io.rs:82] ["keys scanned from snap out of order"] [current_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A] [last_key=7A311C39017D2D9DD3FEEDAE4B186FAF9FF802313EB566C44596E20B669481760B]
[2020/02/03 20:22:17.776 -05:00] [ERROR] [io.rs:90] ["sst writer put error"] [current_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A] [last_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A]
[2020/02/03 20:22:17.778 -05:00] [ERROR] [region.rs:268] ["failed to generate snap!!!"] [err="snap failed \"[src/raftstore/store/worker/region.rs:230]: [src/raftstore/store/snap/io.rs:76]: Io Storage Engine Invalid argument: Keys must be added in order\""] [region_id=104]
[2020/02/03 20:22:19.336 -05:00] [WARN] [peer_storage.rs:771] ["failed to try generating snapshot"] [times=2] [peer_id=279] [region_id=104]
[2020/02/03 20:22:19.337 -05:00] [INFO] [peer_storage.rs:794] ["requesting snapshot"] [request_index=0] [peer_id=279] [region_id=104]
[2020/02/03 20:22:19.338 -05:00] [INFO] [snap.rs:707] ["scan snapshot of one cf"] [size=0] [key_count=0] [cf=default] [snapshot=/data1/br/deploy/data/snap/gen_104_10_215979_(default|lock|write).sst] [region_id=104]
[2020/02/03 20:22:19.338 -05:00] [INFO] [snap.rs:707] ["scan snapshot of one cf"] [size=0] [key_count=0] [cf=lock] [snapshot=/data1/br/deploy/data/snap/gen_104_10_215979_(default|lock|write).sst] [region_id=104]
[2020/02/03 20:22:19.770 -05:00] [ERROR] [io.rs:82] ["keys scanned from snap out of order"] [current_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A] [last_key=7A311C39017D2D9DD3FEEDAE4B186FAF9FF802313EB566C44596E20B669481760B]
[2020/02/03 20:22:19.770 -05:00] [ERROR] [io.rs:90] ["sst writer put error"] [current_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A] [last_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A]
[2020/02/03 20:22:19.771 -05:00] [ERROR] [region.rs:268] ["failed to generate snap!!!"] [err="snap failed \"[src/raftstore/store/worker/region.rs:230]: [src/raftstore/store/snap/io.rs:76]: Io Storage Engine Invalid argument: Keys must be added in order\""] [region_id=104]
[2020/02/03 20:22:21.338 -05:00] [WARN] [peer_storage.rs:771] ["failed to try generating snapshot"] [times=3] [peer_id=279] [region_id=104]
[2020/02/03 20:22:21.339 -05:00] [INFO] [peer_storage.rs:794] ["requesting snapshot"] [request_index=0] [peer_id=279] [region_id=104]
[2020/02/03 20:22:21.340 -05:00] [INFO] [snap.rs:707] ["scan snapshot of one cf"] [size=0] [key_count=0] [cf=default] [snapshot=/data1/br/deploy/data/snap/gen_104_10_215979_(default|lock|write).sst] [region_id=104]
[2020/02/03 20:22:21.340 -05:00] [INFO] [snap.rs:707] ["scan snapshot of one cf"] [size=0] [key_count=0] [cf=lock] [snapshot=/data1/br/deploy/data/snap/gen_104_10_215979_(default|lock|write).sst] [region_id=104]
[2020/02/03 20:22:21.777 -05:00] [ERROR] [io.rs:82] ["keys scanned from snap out of order"] [current_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A] [last_key=7A311C39017D2D9DD3FEEDAE4B186FAF9FF802313EB566C44596E20B669481760B]
[2020/02/03 20:22:21.777 -05:00] [ERROR] [io.rs:90] ["sst writer put error"] [current_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A] [last_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A]
[2020/02/03 20:22:21.778 -05:00] [ERROR] [region.rs:268] ["failed to generate snap!!!"] [err="snap failed \"[src/raftstore/store/worker/region.rs:230]: [src/raftstore/store/snap/io.rs:76]: Io Storage Engine Invalid argument: Keys must be added in order\""] [region_id=104]
[2020/02/03 20:22:23.341 -05:00] [WARN] [peer_storage.rs:771] ["failed to try generating snapshot"] [times=4] [peer_id=279] [region_id=104]
[2020/02/03 20:22:23.341 -05:00] [INFO] [peer_storage.rs:794] ["requesting snapshot"] [request_index=0] [peer_id=279] [region_id=104]
[2020/02/03 20:22:23.342 -05:00] [INFO] [snap.rs:707] ["scan snapshot of one cf"] [size=0] [key_count=0] [cf=default] [snapshot=/data1/br/deploy/data/snap/gen_104_10_215979_(default|lock|write).sst] [region_id=104]
[2020/02/03 20:22:23.342 -05:00] [INFO] [snap.rs:707] ["scan snapshot of one cf"] [size=0] [key_count=0] [cf=lock] [snapshot=/data1/br/deploy/data/snap/gen_104_10_215979_(default|lock|write).sst] [region_id=104]
[2020/02/03 20:22:23.772 -05:00] [ERROR] [io.rs:82] ["keys scanned from snap out of order"] [current_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A] [last_key=7A311C39017D2D9DD3FEEDAE4B186FAF9FF802313EB566C44596E20B669481760B]
[2020/02/03 20:22:23.772 -05:00] [ERROR] [io.rs:90] ["sst writer put error"] [current_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A] [last_key=7A311C390155DDA659A89F159E323768D6D2077FE4CCD5250D44D454E2AF41482A]
[2020/02/03 20:22:23.773 -05:00] [ERROR] [region.rs:268] ["failed to generate snap!!!"] [err="snap failed \"[src/raftstore/store/worker/region.rs:230]: [src/raftstore/store/snap/io.rs:76]: Io Storage Engine Invalid argument: Keys must be added in order\""] [region_id=104]
[2020/02/03 20:22:25.343 -05:00] [WARN] [peer_storage.rs:771] ["failed to try generating snapshot"] [times=5] [peer_id=279] [region_id=104]
[2020/02/03 20:22:25.875 -05:00] [FATAL] [lib.rs:503] ["[region 104] 279 unexpected error: Store(Other(\"[src/raftstore/store/peer_storage.rs:788]: failed to get snapshot after 5 times\"))"] [backtrace="stack backtrace:\n   0: tikv_util::set_panic_hook::{{closure}}\n             at components/tikv_util/src/lib.rs:502\n   1: std::panicking::rust_panic_with_hook\n             at src/libstd/panicking.rs:475\n   2: rust_begin_unwind\n             at src/libstd/panicking.rs:375\n   3: std::panicking::begin_panic_fmt\n             at src/libstd/panicking.rs:326\n   4: raft::raft::Raft<T>::prepare_send_snapshot\n             at /home/jenkins/agent/workspace/tikv_ghpr_build_release/tikv/<::std::macros::panic macros>:9\n   5: raft::raft::Raft<T>::send_append\n             at /rust/git/checkouts/raft-rs-841f8a6db665c5c0/2432781/src/raft.rs:636\n   6: raft::raft::Raft<T>::step_leader\n             at /rust/git/checkouts/raft-rs-841f8a6db665c5c0/2432781/src/raft.rs:1873\n   7: raft::raft::Raft<T>::step\n             at /rust/git/checkouts/raft-rs-841f8a6db665c5c0/2432781/src/raft.rs:1245\n   8: tikv::raftstore::store::fsm::peer::PeerFsmDelegate<T,C>::on_raft_message\n             at /rust/git/checkouts/raft-rs-841f8a6db665c5c0/2432781/src/raw_node.rs:356\n   9: tikv::raftstore::store::fsm::peer::PeerFsmDelegate<T,C>::handle_msgs\n             at /home/jenkins/agent/workspace/tikv_ghpr_build_release/tikv/src/raftstore/store/fsm/peer.rs:279\n  10: <tikv::raftstore::store::fsm::store::RaftPoller<T,C> as batch_system::batch::PollHandler<tikv::raftstore::store::fsm::peer::PeerFsm,tikv::raftstore::store::fsm::store::StoreFsm>>::handle_normal\n             at /home/jenkins/agent/workspace/tikv_ghpr_build_release/tikv/src/raftstore/store/fsm/store.rs:686\n      batch_system::batch::Poller<N,C,Handler>::poll\n             at /home/jenkins/agent/workspace/tikv_ghpr_build_release/tikv/components/batch-system/src/batch.rs:290\n  11: batch_system::batch::BatchSystem<N,C>::spawn::{{closure}}\n             at /home/jenkins/agent/workspace/tikv_ghpr_build_release/tikv/components/batch-system/src/batch.rs:361\n      std::sys_common::backtrace::__rust_begin_short_backtrace\n             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/libstd/sys_common/backtrace.rs:136\n  12: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}\n             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/libstd/thread/mod.rs:469\n      <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/libstd/panic.rs:318\n      std::panicking::try::do_call\n             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/libstd/panicking.rs:292\n      std::panicking::try\n             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8//src/libpanic_unwind/lib.rs:78\n      std::panic::catch_unwind\n             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/libstd/panic.rs:394\n      std::thread::Builder::spawn_unchecked::{{closure}}\n             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/libstd/thread/mod.rs:468\n      core::ops::function::FnOnce::call_once{{vtable.shim}}\n             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/libcore/ops/function.rs:232\n  13: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once\n             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/liballoc/boxed.rs:1022\n  14: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once\n             at /rustc/0de96d37fbcc54978458c18f5067cd9817669bc8/src/liballoc/boxed.rs:1022\n      std::sys_common::thread::start_thread\n             at src/libstd/sys_common/thread.rs:13\n      std::sys::unix::thread::Thread::new::thread_start\n             at src/libstd/sys/unix/thread.rs:80\n  15: start_thread\n  16: __clone\n"] [location=/rust/git/checkouts/raft-rs-841f8a6db665c5c0/2432781/src/raft.rs:541] [thread_name=raftstore-5-1]

Rawkv Backup randomly fails:
[2020/02/03 20:19:37.319 -05:00] [ERROR] [endpoint.rs:237] ["keys scanned from cursor out of order"] [current_key=31182A0104C26E561B7FF2FB1E136455C917EB0351C7895E4736EE224F4AB196] [last_key=31182A011A602C2CE0CF1082E07A2E0D2CD9B90D1032FC01ABE49E150290A96A]       [2020/02/03 20:19:37.320 -05:00] [ERROR] [writer.rs:239] ["write sst failed."] [last_key=31182A011A602C2CE0CF1082E07A2E0D2CD9B90D1032FC01ABE49E150290A96A] [key=31182A0104C26E561B7FF2FB1E136455C917EB0351C7895E4736EE224F4AB196]
[2020/02/03 20:19:37.320 -05:00] [ERROR] [endpoint.rs:252] ["backup raw kv build sst failed"] [error="EngineTrait(Engine(\"Invalid argument: Keys must be added in order\"))"]
[2020/02/03 20:19:37.320 -05:00] [ERROR] [endpoint.rs:652] ["backup region failed"] [error="EngineTrait(Engine(\"Invalid argument: Keys must be added in order\"))"] [end_key=] [start_key=] [region="id: 152 start_key: 31161009EA1E94A13E04FA9933B58505BC16B2D01351 end_key: 311896343F2014849C0930A6658F0C8750D4541E82A8CEFCD2A627ED509066F8 region_epoch { conf_ver: 17 version: 6 } peers { id: 153 store_id: 1 } peers { id: 330 store_id: 5 } peers { id: 366 store_id: 6 }"]
[2020/02/03 20:19:37.321 -05:00] [INFO] [endpoint.rs:669] ["backup finished"] [summary="Statistics { lock: CfStatistics { processed: 0, get: 0, next: 0, prev: 0, seek: 0, seek_for_prev: 0, over_seek_bound: 0, flow_stats: FlowStatistics { read_keys: 0, read_bytes: 0 } }, write: CfStatistics { processed: 0, get: 0, next: 1136712, prev: 0, seek: 3, seek_for_prev: 0, over_seek_bound: 0, flow_stats: FlowStatistics { read_keys: 1136712, read_bytes: 286915488 } }, data: CfStatistics { processed: 0, get: 0, next: 0, prev: 0, seek: 0, seek_for_prev: 0, over_seek_bound: 0, flow_stats: FlowStatistics { read_keys: 0, read_bytes: 0 } } }"] [take=9.742272771s]
[2020/02/03 20:19:37.321 -05:00] [WARN] [mod.rs:89] ["handle task BackupTask { start_ts: TimeStamp(0), end_ts: TimeStamp(414391774082498562), start_key: \"31\", end_key: \"3130303030303030303030303030\", is_raw_kv: true, cf: \"write\" }"] [takes=9.743s]

I'm not sure if this is related to our special configurations for Write CF. If not, it might be a bug of RocksDB.
@Little-Wallace @zhangjinpeng1987 PTAL.
cc. @overvenus @3pointer

Will check.

The issue is due to https://github.com/facebook/rocksdb/issues/6666. In summary, rocksdb read wrong result from ingested sst, which is generated by tikv snapshot. Subsequent scan on the data will appear to be out of order, which is actually a wrong result being returned in the middle. The issue does not affect txnkv API, as txnkv key encoding avoids the buggy case.
Investigated with @Little-Wallace and @Connor1996 .

https://github.com/tikv/tikv/pull/7420 has fix this issue.


RDBC-628

https://issues.hibernatingrhinos.com/issue/RDBC-628/PHP-Client-select-fields-additional-assemblies

@alxsabo Env variables are not passed to PRs by design (they will be passed to runs made on branch). In other repositories (e.g. C#) we do have an attribute to handle those kinds of tests. https://github.com/ravendb/ravendb/blob/v5.4/test/Tests.Infrastructure/LicenseRequiredFactAttribute.cs

In Java we have this https://github.com/ravendb/ravendb-jvm-client/blob/v5.4/src/test/java/net/ravendb/client/infrastructure/DisableOnPullRequestCondition.java


New version: prefix-dev.pixi version 0.32.1

Pull request has been created with WinGet Releaser v2 :rocket:
Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

Executable C:\Users\validator\AppData\Local\pixi\bin\pixi.exe returned exit code: 2

(Automated response - build 896.)


Add CI2Go by @ngs

[x] One project per pull request
[x] Insert the project in alphabetical order
[x] Avoid iOS or open-source in description as it is assumed
[x] Use this commit title format if applicable: Add app-name by @github-username
[x] Use approved format for your entry (use 🔶 for Swift projects)

Thx


Style config file breaks vuetify styles on SSR since v0.18.0

Since v0.18.0, adding the configFile to customize the vuetify/settings scss variables breaks the vuetify styling on server side.
When loading the app, we can see the components have no style, and then it gets loaded on client side correctly.
Reverting the version to v0.17.x fixes the issue.
Reproduction on Stackblitz: https://stackblitz.com/edit/github-6wavaj?file=package.json
// nuxt.config.ts
export default defineNuxtConfig({
  ssr: true,
  compatibilityDate: '2024-11-01',
  devtools: { enabled: true },

  modules: ['vuetify-nuxt-module'],

  css: ['./assets/styles/vuetify-globals.scss'],

  vuetify: {
    moduleOptions: {
      disableVuetifyStyles: true,
      styles: {
        configFile: './assets/styles/vuetify-components.scss', // <-- remove this and the SSR styles work
      },
    },
  },

  features: {
    inlineStyles: true,
  },
});

Hi @Kapcash
I think you need to set:
features: {
    inlineStyles: false,
  },

As described here: https://nuxt.vuetifyjs.com/guide/server-side-rendering.html#vuetify-sass-variables

Additional notes:

setting disableVuetifyStyles to false breaks the build. The vuetify styles are not found.
Whether we set features.inlineStyles to true or false doesn't change the output, since the module overwrites this value automatically:
https://github.com/vuetifyjs/nuxt-module/blob/8f93bcbdef2f0474931e06464789eade2b876762/src/utils/configure-nuxt.ts#L29
Switching from @use 'vuetify' to @forward 'vuetify' doesn't change anything.
Removing the css input that loads the vuetify styles breaks the build.


Non-sudo installation would be nice :-)

I'd like to be able to run https://github.com/facebook/fblualib/blob/master/install_all.sh without root ideally, at least for installing thrift, folly et al.  The apt-get bits seems pretty normal/ok/necessary to use sudo.
One possible implementation could be to provide --prefix option to the script, and if that directory is not writable, then use sudo to run the make install commands.  Or simply have an option --elevated-install, defaults to true, as well as the --prefix option.

Seems I dont need this any more, and no updates for ages
=> closing


Fetching logs from outside Openshift cluster into EFK in Openshift v.3.6

fluentd version : 12.39
environment	: RHEL
We have an EFK Stack configured in our Openshift 3.6 cluster that enables centralized logging for our cluster. Our Openshift cluster resides on Openstack platform. We have a requirement where we need to fetch logs from few instances(VM) in Openstack into the EFK cluster present in Openshift. By achieving this, we will be able to view logs from a few tools that are deployed in Openstack VM's too in the Kibana present on Openshift. Is there a procedure or a way through which we can achieve this requirement?
Our fluentd uses the following image :
registry.access.redhat.com/openshift3/logging-fluentd:v3.6
Help would be appreciated !!
Thanks.

Achieved this using mux tool in Openshift.


Add  a function that solves disjoint net

Description of the feature
Validation in AEDt does not include strick disjoint net making SIwave AC analysis not working. I overcome this we need to implement a function that solves disjoint net
Steps for implementing the feature
Already discussed with Massimo
Useful links and references
No response

added in #2044 and released in v0.6.32


User id crash

User 11 not 10 when signed in as guest user on Android

Very odd
This is a crash in the system level Tile service (the quick tile in  the notification drop down)
I don't control that code since its part of the operating system. Not sure what we can fix here.
Do you still get the crash even if you do not use the quick tile?

Ah I see your comment about being signed in as a guest.
I assume this is an Android OS thing, if you are not the user that installed the app.
Honestly I've never dealt with multi user setups on Android. I assume you added the tile as a Non Guest user and are attempting to use it as a Guest.
What if you add the Tile as the Guest User first?

Does this still happen on the newly released version 38?


[CRASH] Crashed on startup seems that it's missing a xaml for some reason(??)

Hard crash experienced
Error details :

                        OS: Win32NT
                   Version: Microsoft Windows NT 10.0.26100.0
           OS Architecture: True
          APP Architecture: True
                  Language: Unknown
               APP Version: 3.0.1
                Executable: C:\Program Files\WingetUI\wingetui.exe

Crash Message: Nie można znaleźć tekstu skojarzonego z tym kodem błędu.

Cannot locate resource from 'ms-appx:///Microsoft.UI.Xaml/Themes/themeresources.xaml'. [Line: 0 Position: 0]

Crash Traceback:

This is rather strange, because this problem didn't occur one day, and the next day it popped up without updating anything as far as I recon.

Hello,
Please test this custom build and let me know if it works: https://drive.google.com/file/d/1NO8jhMDBPmMGvlnaW5MXv5vjYjvmFJLa/view?usp=sharing
Thanks!

Hello, everyone,
WingetUI 3.0.2 has been published. It fixes

Crashes with .NET Sdk 8.0.3 (no launch at all)
Crashes with .NET Sdk 8.0.4 (Cannot locate resource from 'ms-appx:///Microsoft.UI.Xaml/Themes/themeresources.xaml)
Crashes with the App Sdk (This app requires whe Windows App runtime) when it was already installed
Crashes when elevating WingetUI from non-admin users

WingetUI 3.0.2 can be downloaded directly from here. If the crashes persist, please reply here and I will reopen the issue.


get_entity_from_xml_record_entity() in hookup.py

@glah: "I found a potential logical error in function get_entity_from_xml_record_entity(): You only test, whether the first element has some content and then assume that all elements have contents. This is not necessarily the case with XML documents. Do you think, we should refactor this code, too?"

Ok, in this case, probably an and clause would be more appropriate:
if (len(record.metadata.find_all(entity)) > 0 and # check if there is the entity
    len(record.metadata.find_all(entity)[0].contents) > 0): # check if there is some actual content for the entity
   # ...


Groovy parser does not correctly handle nested parenthesis

When we attempt to parse:
def foo(Map map) {
    ((map.containsKey("foo"))
            && ((map.get("foo")).equals("bar")))
}

def timestamp(int hours, int minutes, int seconds) {
    (hours) * 60 * 60 + (minutes * 60) + seconds
}

What we actually get is an IllegalStateException out of the parser.
Reproduced in BinaryTest.extraParensAroundInfixOperator()

Half of this has already been fixed through

https://github.com/openrewrite/rewrite/pull/4751

Other half has been fixed today:

https://github.com/openrewrite/rewrite/pull/4801


Malformed JSON string in synopsis example

I seem to get a strange error on one of our servers (basically a test server seems to work, but our live one doesn't), both are centos 6.6 with perl 5.14.2. I can't think of anything obvious different.
Nothing gets displayed, but after 10 secs or so, the following error pops up in the console, and process still running...

malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "Message is too long")

Code is exactly the same as Synopsis. Not quite sure how to debug this further.

Perhaps response from slack is not json string.
I fixed to print unchanged response if not json string.
I hope this change help to to solve problems.


docs(compiler-hooks): adds examples, formatting and missing descriptions

Adds some examples and improves formatting and hooks descriptions.

Do you mind reviewing this? :) @EugeneHlushko @montogeek

Thanks for reviewing @EugeneHlushko, I've commited your suggestions. Let me know if there's anything else to improve!
Also, if you like those stuff I have some more in #2852 😇

Thanks!

Please mention me in PR so i get notified at some point

Thanks to you for reviewing!


Question about using freesurfer .annot files

Hi Abagen experts,
I'm new to the field of neuroimaging/imaging transcriptomics so hopefully this question doesn't sound too dumb.
I want to use .annot file from freesurfer recon-all outputs as an input to abagen.
I have the following code:

import abagen
left_annot = '/Users/xuansean/Desktop/BR2072/label/lh.aparc.annot'
left_atlas = abagen.annot_to_gifti(left_annot)
expression = abagen.get_expression_data(left_atlas)

but this gives me multiple errors:

Traceback (most recent call last):
File "/Users/xuansean/opt/anaconda3/lib/python3.9/site-packages/abagen/images.py", line 562, in coerce_atlas_to_dict
for donor, atl in atlas.items()
AttributeError: 'GiftiImage' object has no attribute 'items'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/xuansean/opt/anaconda3/lib/python3.9/site-packages/abagen/images.py", line 352, in check_atlas
atlas = check_img(atlas)
File "/Users/xuansean/opt/anaconda3/lib/python3.9/site-packages/abagen/images.py", line 218, in check_img
raise TypeError('Provided image must be an existing filepath or a '
TypeError: Provided image must be an existing filepath or a pre-loaded niimg-like object
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/xuansean/MRI_pediatric/abagen/abagen_test.py", line 15, in
expression = abagen.get_expression_data(left_atlas)
File "/Users/xuansean/opt/anaconda3/lib/python3.9/site-packages/abagen/allen.py", line 385, in get_expression_data
atlas, group_atlas = images.coerce_atlas_to_dict(atlas, donors, atlas_info)
File "/Users/xuansean/opt/anaconda3/lib/python3.9/site-packages/abagen/images.py", line 572, in coerce_atlas_to_dict
atlas = check_atlas(atlas, atlas_info)
File "/Users/xuansean/opt/anaconda3/lib/python3.9/site-packages/abagen/images.py", line 355, in check_atlas
atlas, info = check_surface(atlas)
File "/Users/xuansean/opt/anaconda3/lib/python3.9/site-packages/abagen/images.py", line 265, in check_surface
if len(atlas) != 2:
TypeError: object of type 'GiftiImage' has no len()

Initially I thought maybe it was due to the fact that my .annot files are not in fsaverage5 resolution, so I tried to down-sampled it using freesurfer commands but didn't work out as well.
What am I missing here?
Thank you in advance for any help!

Hi, thank you for using abagen!
I would recommend checking the user guide for detailed walk-through, specifically, the page on defining a parcellation.
If you are using aparc, you can directly follow the guide where it demonstrates how to use Desikan-Killiany atlas.
Please feel free to let us know if you need more help.

Thank you for the reply @liuzhenqi77!
Right now I'm just trying to go through the guide with the default Desikan-Killiany atlas. My code is just the same in the guide:
import abagen
atlas = abagen.fetch_desikan_killiany()
print(atlas['image'])
print(atlas['info'])
expression = abagen.get_expression_data(atlas['image'], atlas['info'])
But I got an error message as the following:
Traceback (most recent call last):

File "/Users/xuansean/MRI_pediatric/abagen/abagen_demo.py", line 15, in
expression = abagen.get_expression_data(atlas['image'], atlas['info'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/xuansean/anaconda3/lib/python3.11/site-packages/abagen/allen.py", line 444, in get_expression_data
microarray = probes_.collapse_probes(flatten_dict(files, 'microarray'),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/xuansean/anaconda3/lib/python3.11/site-packages/abagen/probes_.py", line 757, in collapse_probes
micro = micro.set_axis(symbols, axis=1, inplace=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: DataFrame.set_axis() got an unexpected keyword argument 'inplace'

Before the error there are just numerous of

FutureWarning: DataFrame.groupby with axis=1 is deprecated. Do frame.T.groupby(...) without axis instead.

I updated pandas and other needed packages but still got the same error. Really appreciate your help!

This should have been fixed by the latest GitHub commits. Please reopen if more help is needed.


Feat: Support specifying publishVersion when creating or updating oam applications

Description of your changes

Support specifying publishVersion when creating or updating oam applications via API
update version in metadata.yaml from v1.8.0-rc.3 to v1.8.0-rc.4

I have:

[x] Read and followed KubeVela's contribution process.
[x] Related Docs updated properly. In a new feature or configuration option, an update to the documentation is necessary.
[x] Run yarn lint to ensure the frontend changes are ready for review.
[x] Run make reviewableto ensure the server changes are ready for review.
[x] Added backport release-x.y labels to auto-backport this PR if necessary.

Special notes for your reviewer

Because the requirement to specify publishVersion usually only exists during API calls, supporting users to specify this parameter on the front-end can actually lead to unnecessary understanding costs, so we didn't add support for specifying relevant parameters on the UX front-end.
The VelaUX version indicated in the metadata.yaml file is 1.8.0-rc3, but in reality, the available version has already reached 1.9.3. Should I also change this to 1.9.3 or 1.9.4 (instead of v1.8.0-rc.4)?

Additionally, as I urgently need this feature in my production environment, I would like to release a new version of VelaUX

Since this PR only contains API changes, does it mean that we can not specify publishVersion in the frontend?

Yes, I am considering this: if users are supported to fill in PublishVersion on a web page, then for web users, they need to understand what PublishVersion is, why uniqueness needs to be guaranteed, and its relationship with Revision (currently, without specifying PublishVersion, the default value of PublishVersion is the same as Revision), This will bring two problems: firstly, it brings some learning costs to web (usually beginner) users; Secondly, they are easy to fill in duplicates (considering scenarios where multiple people are simultaneously deploying the same application on a web page), and the server needs to support pre-validation support.
Of course, if necessary, I can also add front-end support.

@iambocai I think this change don't impact on the UI. LGTM after passed the CI checks.

@iambocai I think this change don't impact on the UI. LGTM after passed the CI checks.

thanks for your reply, I saw some test faild in ci, but all of them seems to be environment problems, may be we should fix them in another commit？
VelaUX APIServer Test / server-e2e-tests (v1.21) (pull_request) —— No matching K3s versions were found.

the avaliable k3s versions started at v22, but we request v21.

@iambocai I think this change don't impact on the UI. LGTM after passed the CI checks.

thanks for your reply, I saw some test faild in ci, but all of them seems to be environment problems, may be we should fix them in another commit？
VelaUX APIServer Test / server-e2e-tests (v1.21) (pull_request) —— No matching K3s versions were found.  the avaliable k3s versions started at v22, but we request v21.

I have upgrade the minor  server-e2e-tests k8s version to 1.22 and passed the check, but ci still expect v1.21 test's result, may be we should modify git settings too?


feat: initialized application sidebar

This pull request includes several changes to the codebase, focusing on adding new components, updating existing ones, and improving the overall structure and functionality of the application. The most important changes include the addition of new UI components, updates to the routing structure, and enhancements to the sidebar functionality.
New Components and Enhancements:

src/components/ui/input.jsx: Added a new Input component from shadcn
src/components/ui/skeleton.jsx: Added a new Skeleton component from shadcn
src/components/ui/tooltip.jsx: Added a new Tooltip component  from shadcn

Routing and Layout Updates:

src/App.jsx: Updated routing to rename prior Home to Landing page and nested routes for the App section. [1] [2]
src/layouts/AppLayout.jsx: Added AppLayout component with sidebar integration.

Sidebar Functionality:

src/components/layouts/sidebar.jsx: Implemented a comprehensive sidebar component with collapsible items, dropdown menus, and footer actions. All of these can be customized via a dictionary atop of the component for easy customization.
src/hooks/use-mobile.jsx: Included a hook to detect mobile screen sizes for responsive design (from shadcn).

Dependency Updates (Added by shadcn):

package.json: Added new dependencies for @radix-ui/react-collapsible and @radix-ui/react-tooltip.

These changes collectively improve the user interface, enhance the routing capabilities, and provide a more responsive and interactive experience for the application.
https://github.com/user-attachments/assets/f75b0e78-0fdc-49eb-a2c8-61d9fd601601

I apologise for merge the PR prematurely, reviewing the changes once merged I have noticed the following problems:  In dark mode, some buttons don't fit the theme. Specifically:
1. The user information

2. The theme toggle

3. The theme button in the main page

4. The inbox options button

It is important to note that all of these buttons do display well when hovered over

I need you to give me more context. This does not happen to me

I tried in firefox, chrome and brave on desktop
And on chromium, brave, and firefox on mobile


Build on OpenBSD 7.0 fails - undefined cxx_throw test_eh_personality

A fresh build, fresh configured with "ccmake ." yields:
[ 12%] Linking C executable CXXExceptions_legacy_optimised
ld: error: /home/multix/code/gnustep-svc/libobjc2/Build/libobjc.so: undefined reference to cxx_throw() [--no-allow-shlib-undefined]
ld: error: /home/multix/code/gnustep-svc/libobjc2/Build/libobjc.so: undefined reference to test_eh_personality [--no-allow-shlib-undefined]
cc: error: linker command failed with exit code 1 (use -v to see invocation)

This still happens on 7.2, something gets detected wrong. If I turn off OLDABI_COMPAT I get the same error, in a different file
ld: error: ../libobjc.so.4.6: undefined reference to cxx_throw() [--no-allow-shlib-undefined]
ld: error: ../libobjc.so.4.6: undefined reference to test_eh_personality [--no-allow-shlib-undefined]
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** Error 1 in . (Test/CMakeFiles/CXXExceptions_optimised.dir/build.make:117 'Test/CXXExceptions_optimised')
*** Error 2 in . (CMakeFiles/Makefile2:2853 'Test/CMakeFiles/CXXExceptions_optimised.dir/all')
*** Error 2 in /home/multix/code/gnustep-cvs/libobjc2/Build (Makefile:166 'all': make -s -f CMakeFiles/Makefile2 al```

The CMake has now been reworked and has a bunch of bug fixes.  Can you try again?


duplicate retry attempts

Issue
We have a simple test app to evaluate the retry feature of redis-node.
The test app writes to redis every x seconds. Then, we run a script on which every 90s, it will enable/disable port 6379, which lets us to check the robustness of the reconnection logic in the library.
We found that quite often (maybe 50% of the time) after 6s to 10s, we will get a socket ECONNRESET during a reconnection, which spawns another reconnect, duplicating the number of retry attempts. Logs below:
Although it does not appear to be causing any issues (so far), just wanted to check that this double retry will not cause any issues in the library's code flow.
Feb 15 00:38:59 info [16248] redis attempt [12] total_retry_time [6600] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 00:38:59 info [16248] redis attempt [1] total_retry_time [0] errMsg [Redis connection to my-redis:6379 failed - read ECONNRESET]
Feb 15 00:38:59 info [16248] redis attempt [2] total_retry_time [100] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 00:38:59 info [16248] redis attempt [3] total_retry_time [300] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 00:38:59 info [16248] redis attempt [4] total_retry_time [600] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 00:39:00 info [16248] redis attempt [5] total_retry_time [1000] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 00:39:00 info [16248] redis attempt [13] total_retry_time [7800] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]

Environment

Node.js Version: 10.16.3

Redis Version: 4.0.14

Platform: Azure Redis

Just another quick update, the test app picked up a more severe case, with 7 x ECONNRESET events, which made the total retries jumped from 1 to 8.
Feb 15 01:06:41 info [16248] redis attempt [12] total_retry_time [6600] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:41 info [16248] redis attempt [12] total_retry_time [6600] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:42 info [16248] redis attempt [1] total_retry_time [0] errMsg [Redis connection to my-redis:6379 failed - read ECONNRESET]
Feb 15 01:06:42 info [16248] redis attempt [1] total_retry_time [0] errMsg [Redis connection to my-redis:6379 failed - read ECONNRESET]
Feb 15 01:06:42 info [16248] redis attempt [1] total_retry_time [0] errMsg [Redis connection to my-redis:6379 failed - read ECONNRESET]
Feb 15 01:06:42 info [16248] redis attempt [1] total_retry_time [0] errMsg [Redis connection to my-redis:6379 failed - read ECONNRESET]
Feb 15 01:06:42 info [16248] redis attempt [2] total_retry_time [100] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:42 info [16248] redis attempt [2] total_retry_time [100] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:42 info [16248] redis attempt [2] total_retry_time [100] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:42 info [16248] redis attempt [13] total_retry_time [7800] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:42 info [16248] redis attempt [2] total_retry_time [100] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:42 info [16248] redis attempt [13] total_retry_time [7800] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:42 info [16248] redis attempt [3] total_retry_time [300] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:42 info [16248] redis attempt [3] total_retry_time [300] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:42 info [16248] redis attempt [3] total_retry_time [300] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:42 info [16248] redis attempt [3] total_retry_time [300] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [4] total_retry_time [600] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [4] total_retry_time [600] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [4] total_retry_time [600] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [4] total_retry_time [600] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [1] total_retry_time [0] errMsg [Redis connection to my-redis:6379 failed - read ECONNRESET]
Feb 15 01:06:43 info [16248] redis attempt [5] total_retry_time [1000] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [2] total_retry_time [100] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [5] total_retry_time [1000] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [1] total_retry_time [0] errMsg [Redis connection to my-redis:6379 failed - read ECONNRESET]
Feb 15 01:06:43 info [16248] redis attempt [5] total_retry_time [1000] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [5] total_retry_time [1000] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [2] total_retry_time [100] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [3] total_retry_time [300] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:43 info [16248] redis attempt [3] total_retry_time [300] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:44 info [16248] redis attempt [14] total_retry_time [9100] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:44 info [16248] redis attempt [6] total_retry_time [1500] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:44 info [16248] redis attempt [6] total_retry_time [1500] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:44 info [16248] redis attempt [4] total_retry_time [600] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:44 info [16248] redis attempt [6] total_retry_time [1500] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:44 info [16248] redis attempt [14] total_retry_time [9100] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]
Feb 15 01:06:44 info [16248] redis attempt [6] total_retry_time [1500] errMsg [Redis connection to my-redis:6379 failed - connect EACCES x.x.x.x:6379]

Please disregard this reported issue. There is a bug with the test app, which is creating a new redis client on connection timeout, but the previous client was still alive.


Dodata vest za privatizaciju severtransa

👷 Deploy Preview for balkantransport processing.
🔨 Explore the source changes: b9bb5a108d66652475e4731508ed86c8bc711650
🔍 Inspect the deploy log: https://app.netlify.com/sites/balkantransport/deploys/615b18e542c43b0008693359


How to run the same tests multiple times with different values?

Subject of the issue
Suggestion for running multiple tests expressed in Issue #2078 does not work as I expected.  Should the method shown under Steps to reproduce work to run a test multiple times with different values?
In my real tests, I am passing val to a class initializer that is setup in a let statement to test various conditions for the initializer parameters that should produce the same results.
Your environment

Ruby version: 2.5.5
rspec-core version: 3.8.2

Steps to reproduce
I tried to use the original suggestion by @myronmarston in Issue #2078 but it is not working as I had hoped.  Here is a simplified test following that pattern...
RSpec.describe "Dynamic tests" do
  [1, 2].each do |val|
    let(:my_val) { val }

    it "runs some test logic with #{val} expecting 1" do
      puts "Test: runs some test logic with #{val} expecting 1"
      puts "  val=#{val}"
      puts "  my_val=#{my_val}"
      expect(my_val).to eq 1
    end
    it "runs some test logic with #{val} expecting 2" do
      puts "Test: runs some test logic with #{val} expecting 2"
      puts "  val=#{val}"
      puts "  my_val=#{my_val}"
      expect(my_val).to eq 2
    end
  end
end

Expected behavior
Test: runs some test logic with 2 expecting 1
  val=2
  my_val=2
F
Test: runs some test logic with 1 expecting 1
  val=1
  my_val=1
.
Test: runs some test logic with 2 expecting 2
  val=2
  my_val=2
.
Test: runs some test logic with 1 expecting 2
  val=1
  my_val=1
F

Note that my_val and val should always have the same value.
Actual behavior
Test: runs some test logic with 2 expecting 1
  val=2
  my_val=2
F
Test: runs some test logic with 1 expecting 1
  val=1
  my_val=2
F
Test: runs some test logic with 2 expecting 2
  val=2
  my_val=2
.
Test: runs some test logic with 1 expecting 2
  val=1
  my_val=2
.

Note that my_val is NOT set based on the value of val.  It is always 2.

I solved my own question.  In case someone else also runs into this, the solution is to wrap the let statement and all tests inside a context block...
RSpec.describe "Dynamic tests" do
  [1, 2].each do |val|
    context 'with multiple values' do
      let(:my_val) { val }

      it "runs some test logic with #{val} expecting 1" do
        puts "Test: runs some test logic with #{val} expecting 1"
        puts "  val=#{val}"
        puts "  my_val=#{my_val}"
        expect(my_val).to eq 1
      end
      it "runs some test logic with #{val} expecting 2" do
        puts "Test: runs some test logic with #{val} expecting 2"
        puts "  val=#{val}"
        puts "  my_val=#{my_val}"
        expect(my_val).to eq 2
      end
    end
  end
end

Now produces...
Test: runs some test logic with 1 expecting 2
  val=1
  my_val=1
F
Test: runs some test logic with 1 expecting 1
  val=1
  my_val=1
.
Test: runs some test logic with 2 expecting 1
  val=2
  my_val=2
F
Test: runs some test logic with 2 expecting 2
  val=2
  my_val=2
.

As you can see, val and my_val have the same value for all tests, which is what would be expected.

I solved my own question. In case someone else also runs into this, the solution is to wrap the let statement and all tests inside a context block...
RSpec.describe "Dynamic tests" do
  [1, 2].each do |val|
    context 'with multiple values' do
      let(:my_val) { val }

      it "runs some test logic with #{val} expecting 1" do
        puts "Test: runs some test logic with #{val} expecting 1"
        puts "  val=#{val}"
        puts "  my_val=#{my_val}"
        expect(my_val).to eq 1
      end
      it "runs some test logic with #{val} expecting 2" do
        puts "Test: runs some test logic with #{val} expecting 2"
        puts "  val=#{val}"
        puts "  my_val=#{my_val}"
        expect(my_val).to eq 2
      end
    end
  end
end

Now produces...
Test: runs some test logic with 1 expecting 2
  val=1
  my_val=1
F
Test: runs some test logic with 1 expecting 1
  val=1
  my_val=1
.
Test: runs some test logic with 2 expecting 1
  val=2
  my_val=2
F
Test: runs some test logic with 2 expecting 2
  val=2
  my_val=2
.

As you can see, val and my_val have the same value for all tests, which is what would be expected.

Thanks! I had the same issue and the workaround fixed it

Just to clarify - it's not a workaround, it's just Ruby.
  [1, 2].each do |val|
    let(:my_val) { val }

expands to:
  let(:my_val) { 1 }
  let(:my_val) { 2 }

and the last one wins.
Shared examples are usually a better solution.

Just to clarify - it's not a workaround, it's just Ruby.
  [1, 2].each do |val|
    let(:my_val) { val }

expands to:
  let(:my_val) { 1 }
  let(:my_val) { 2 }

and the last one wins.
Shared examples are usually a better solution.

This isn't exactly true. [1, 2].each is creating two separate context blocks, which unfortunately will have the same description when run. In the first context block, my_val will equal 1, and in the second, it will equal 2. That's why the tests are intermittently passing/failing.

is creating two separate context

Just [1, 2].each { |i| let(:val) { i } } doesn't.
But
[1, 2].each do |i|
  context "when i is #{i}" do
    let(:val) { i }
    it { … }

does.
I personally prefer shared examples. And there’s a cop that will raise an offence if you define examples inside iterators (can’t recall if example groups, too).


Use Set/FoldDict vs. Dict

@andersk Thanks for the review!  I'll try to address your findings tomorrow.  One quick thing is that Set uses size, not length, which kinda makes sense if you think about it:
> x = new Set([1,2,3])
Set { 1, 2, 3 }
> x.size
3
> x.length
undefined

@andersk I think I've addressed all your comments.  Can you give it another review?

I've added some commits here.  The IntDict commit is pretty invasive, because I am greedy about wanting maps to be keyed by ints and not have to convert them to strings.
I think Anders will be happy with the zjsunit-related commits. :)

Addressed all the feedback and re-pushed.  There are a couple places where I disagree on inlining helper functions.  We can always do that as a follow-up.  I fixed all the unnecessary two-liners that caused a let as well as other things pointed out, like moving some conversions earlier in the code flow.
I also made the code more defensive.  So "5" will work as a key; we'll just whine about it.

Also added a few more commits.

Apparently the primary source of stringified stream_ids is data-stream-id attributes. We can use elem.data("stream-id") instead of parseInt(elem.attr("data-stream-id"), 10), since .data already converts to number where appropriate.

I really wish we didn't put so much data in the DOM, but that's how the settings code work.
I'll fix it to use data('stream-id') (and double-check that it does convert to int).

@andersk Regarding the case of whether we fail hard for an undefined key in FoldDict, you are advocating for the principle of failing fast and failing clearly, which is obviously a good principle to follow in general.
I'll make the case for why being a little bit resilient here to bad data is a better approach here, and we'll see if @timabbott agrees.  Yesterday, we introduced a little bit of code to czo that relied on the invariant that every user has a non-falsy full name.  This is a variant we actually enforce, but apparently exactly one of our 13,000+ users had a full_name matching the empty string.  It was some user from 2017 who's probably never been back to the site.
That one piece of bad data took down the whole search feature.
The analogy here would be something one rogue message having a null topic for some reason, and it takes down an entire message view.  Instead of just the one piece of data failing a get call, and we see the blueslip error in gmail.

Tim, everything up to but not including the first IntDict commit should be ready to merge.  Whatever we decide on FoldDict return a TypeError can wait, since that's a behavior change that we'll want to do as a separate commit.

One of the things that came up a couple times in this review is that we have lots of code like this:
person.user_id || person.id

This code comes from the fact that some functions could potentially operate on "pseudo" person objects like the message's display_recipient field.  In person, we call user_ids "user_id", whereas in display_recipient we call user_ids "id".  And then there are occasionally non-person objects like the pseudo object we create for an "all" mention in typeahead.
I didn't address all of these, but there are some functions that are clearly just dealing with message.display_recipient data, so I fixed those.

OK, I merged the block of commits from this leading up to the introduction of IntDict, since those all looked good.

What's the thinking on the IntDict approach?  I'm OK with having an explicit class for a dictionary that requires all keys to be integers, since the extra validation that would offer seems helpful.
For the .data() commits, I seem to recall that a couple years ago we replaced all use of .data(user-id) with e.g. .attr("data-user-id") in part because anything set using .data('user-id', foo) will not be visible to .attr("data-user-id") while the opposite isn't true, which is very confusing.  (https://stackoverflow.com/questions/13275812/how-to-get-set-and-select-elements-with-data-attributes for details on how this jQuery feature works), so we aimed to ban use of the .data() setter pattern from the codebase.
I'm OK with using that for getters to get the benefits of the type being right, but I'd like to be careful here since I know we've spent many hours in the past discovering and debugging data synchronization issues that turned out to be because using a .data() setter to override a data-topic-name after a message is edited doesn't do what one thinks.  (I think this was a 5x bigger problem before we migrated a lot of elements to use user/stream IDs rather than names in widgets, so the context has changed) .

The issue is that native Map, rather than stringifying all keys like Dict, considers string and number keys to be distinct. So if we’re replacing Dict with Map, we can no longer get away with as much of the string | number confusion we used to have. A separate validating IntDict class may be necessary at this stage to root out these bugs. But hopefully we’ll be able to remove it in favor of Map when that rooting is done.
As for elem.attr("data-foo") vs. elem.data("foo"), I was trying to be clear that we should do all of them consistently, but Steve keeps changing just the instance I specifically commented on… It seems to me that elem.data("foo") is strictly cleaner than writing parseInt(elem.attr("data-foo"), 10) everywhere. If we’re consistent about it, we don’t have synchronization bugs, and we also don’t have bugs where we forget to parse to number.

I agree with the points Anders is making on IntDict, and I'll add that the new class is faster (there are commented-out benchmarks that you can run that show a 2x speedup) and presumably take less memory (I didn't measure that, but it seems likely).  And, to repeat a point that Anders makes, it gives us a migration path toward using pure Maps in some places.
For the data issue I did indeed miss a few places during the sweep.  But from a consistency standpoint, we use attr in lots of places.  Even if some of our original reasons for replacing data with attr no longer apply, it would be beyond the scope of this PR to fix all those, and I don't really see any clear win there.  I obviously understand the code's more compact, but I like the consistency of having the more explicit data-user-id attribute in all places--templates, selectors, and lookups.  Also, doing the type conversion yourself, instead of jQuery, can be more typesafe in places.
So I'm gonna go back to parseInt.

@andersk If you can convince Tim that his concerns about data are unfounded at this point, it's probably more efficient for us to just sweep all of these at once after this PR is merged.  There are 25 cases of data-user-id, and 19 cases of data-stream-id in static/js, and there are probably some unit test changes that would need to happen.

benchmarks (all microseconds):

old Dict - 2.2 per set/get
IntDict w/string conversions - 1.2 per set/get
IntDict - 1 per set/get

So, even with the Map, using int keys gives another 20% speedup, which gets magnified a lot, since we do lots of set/get in the app.
Moving to a pure Map gives 0.8 microseconds, so we're still paying a 20% overhead for IntDict's wrappers.

If we’re consistent about it, we don’t have synchronization bugs, and we also don’t have bugs where we forget to parse to number.

Yeah, though one thing to be mindful of is that we do put data-foo elements in our HTML templates, so it's not just the JavaScript that needs to be consistent for a given widget.

I'm happy with the strategy here and didn't see any bugs, so I've merged this.  We should be sure to do a checkup on blueslip warnings from chat.zulip.org in a few days to see if this new warning appears
I'm guessing that there's some associated conversion we missed that will result in a bug, but this is as good a time as any to potentially introduce regressions in service of making the codebase more robust long-term.
Thanks @showell and @andersk for all the work on this!


Seed data bug

The Student model has a uniqueness validator on the local_id attribute. That attribute was being generated with rand(1000) in the seed data file so from time to time you'd get a non-unique value. That's fixed here. https://github.com/studentinsights/studentinsights/issues/159
I also added pry because I needed to set a breakpoint to find the issue, and I think pry is quite popular so others might be wanting it.

@eudaimonious nice fix, thanks! :)

This looks good, and adding pry is great too.  The Travis build is failing on adding a Service model here, similar to the second comment in https://github.com/studentinsights/studentinsights/issues/159.  I must have mucked this up when updating the demo data generators.
Either way we should merge this, since it fixed the root problem in the Student, but I'll see if I can take a look at the Service bug tomorrow before Tuesday.
Thanks again, this is awesomely helpful! :)


Refactor requirements to be cross-platform

Moved from conda's auto-generated requirements.txt which was OSX
specific to a manually created environment.yml file which is much
simpler and cross platform. Also allows conda and pip requirements to be
defined in the same file. Fixes #16.
Signed-off-by: Aymen Qader<EMAIL_ADDRESS>

How did you create this shortlist of dependencies and validate that it encompasses the dependencies of the scripts? Some of the packages listed in the current requirements.txt are not present in the environment.yml file.

First of all, jazakallahkhair for taking this task on. I have one quick question before we can land this to ensure we don't have dependency issues moving forward.
How did you create this shortlist of dependencies and validate that it encompasses the dependencies of the scripts? Some of the packages listed in the current requirements.txt are not present in the environment.yml file.

Sorry, I should have been more clear in the pull request. I've gone through the modules required by the scripts, and added them to the environment.yml as needed. This gives a much shorter list than the output of conda list -e which outputs all the packages installed regardless of whether they were installed as dependencies for other packages.
When you run conda env create -f environment.yml and then run conda list -e, the list is very similar barring some platform-specific variation in packages.
I did miss off scikit-learn, but I've just added it and checked the imports for all the scripts so it should encompass everything now.


Add saraos.tech preset

Hey there!
Just adding the template we use for saraos.tech events 🙂

Thanks a lot 🎉


iPXE bootstrap - 404

Pretty sure this is recent:
$ curl -IL http://releases.rancher.com/os/latest
HTTP/1.1 301 Moved Permanently
Date: Thu, 05 Apr 2018 21:25:07 GMT
Connection: keep-alive
Cache-Control: max-age=3600
Expires: Thu, 05 Apr 2018 22:25:07 GMT
Location: https://releases.rancher.com/os/latest
Server: cloudflare
CF-RAY: ****

HTTP/2 404
date: Thu, 05 Apr 2018 21:25:07 GMT
content-type: application/xml; charset=UTF-8
content-length: 192
set-cookie: ****
x-guploader-uploadid: ****
access-control-allow-origin: *
access-control-expose-headers: Content-Type
expires: Thu, 05 Apr 2018 21:25:07 GMT
cache-control: private, max-age=0
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
server: cloudflare
cf-ray: ****

This is producing error https://ipxe.org/3c00e103 in iPXE.

This seems to have progressed to the point where the non-HTTPS redirect URL (http://releases.rancher.com/os/latest) is now 404ing as well:
$ curl -IL http://releases.rancher.com/os/latest
HTTP/1.1 404 Not Found
Date: Fri, 06 Apr 2018 15:29:22 GMT
Content-Type: application/xml; charset=UTF-8
Content-Length: 192
Connection: keep-alive
Set-Cookie: ****
X-GUploader-UploadID: ****
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Content-Type
Expires: Fri, 06 Apr 2018 15:29:22 GMT
Cache-Control: private, max-age=0
Server: cloudflare
CF-RAY: ****

Hi, @vancluever
Can you please share your iPXE file?
I have a test setup with Virtualbox, here are my iPXE file:
#!ipxe
dhcp
set base-url http://xxxxx/rancheros/ipxe
kernel ${base-url}/vmlinuz rancher.state.autoformat=[/dev/sda] rancher.state.dev=LABEL=RANCHER_STATE rancher.state.wait printk.devkmsg=on console=tty1 rancher.autologin=tty1 rancher.cloud_init.datasources=[url:${base-url}/cloud-config.yaml] rancher.password=rancher
initrd ${base-url}/initrd
boot

@niusmallnan that iPXE file is different than the one documented on https://rancher.com/docs/os/v1.2/en/running-rancheros/server/pxe/. If it's been updated, can you all update the docs?
FYI: We have a local build on our TFTP server, so that's not a problem. However, I want to use upstream sources whenever possible.
Thanks!

When I set the base-url to http://releases.rancher.com/os/latest, I didn't get a 404 error.
In fact it needs vmlinuz  and initrd.
vmlinuz: http://releases.rancher.com/os/latest/vmlinuz
initrd: http://releases.rancher.com/os/latest/initrd
There is no fixed iPXE file, depending on which kernel parameters you need to set.

@niusmallnan right - I it slipped my mind that the base URL is just a variable and the full URL should have been checked. Sorry!
In any case, I just switched the sources back and the iPXE error is resolved, so I'm going to close this now.
Thanks for your help!
PS: The iPXE error code is ENOTSUP_ALGORITHM, which happens as well when I switch base-url to the HTTPS source. I noticed that it was redirecting to HTTPS when this error came up and then it stopped, so maybe it has to do with that. I'm using the iPXE bundled with Ubuntu 16.04 and given the version I'm guessing it's using deprecated crypto, which would explain the error.


MB-13859-weight-ticket-status-and-reason-cleanPR

Jira ticket for this change
Summary
After the service member submits their closeout documentation, including at least one weight ticket record, the Services Counselor will review the documentation and provide a status of approve/exclude/reject in the final PPM calculations.
This ticket adds status and reason columns to the DB table for Weight Tickets, and it adds those fields to the models as well. Finally the swagger file is updated with the new fields.
Files and Folders touched:

[migrations/app/migrations_manifest.txt](https://github.com/transcom/mymove/pull/9259/files#diff-3bfbe8b71332fe4dc24d752cb37d3463e9d60e2d6250b20e3c5da7f196040995)
migrations/schema
internal.yaml
ghc.yaml
internalapi/internal/model_to_payload.go
handler/weight_ticket.go
handler/weight_ticket_test.go
swagger-def/definitions

Setup to Run Your Code

💻 You will need to use 2 separate terminals to test this locally.
Terminal 1
Run the migration if the new migration has not run locally.
make db_dev_migrate

Terminal 2
Make sure changes to the yaml are reflected in the autogen files.
make swagger_generate

Additional steps

Check that the DB has the 2 new columns
Check that the validations for the updated model pass.

Verification Steps for Author
These are to be checked by the author.

[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

Verification Steps for Reviewers
These are to be checked by a reviewer.

Backend

[ ] Code follows the guidelines for Logging
[ ] The requirements listed in Querying the Database Safely have been satisfied.

Database
Any new migrations/schema changes:

[ ] Follows our guidelines for Zero-Downtime Deploys
[ ] Have been communicated to #g-database
[ ] Secure migrations have been tested following the instructions in our docs

Messages

:book:
:link: MB-13859

  Generated by :no_entry_sign: dangerJS against 2198f24c5c124a30a92de6d6d1085edef4879fb0


Backup locations are invalid after plugin name changes

What steps did you take and what happened:
Running on master, the provider name gcp is no longer valid inside of a backupstoratelocation.
However, velero.io/gcp is.
 GOOGLE_APPLICATION_CREDENTIALS=~/credentials-velero-gke v server --default-volume-snapshot-locations gcp:gcp-default --log-level debug
and the following BackupStorageLocation definition:
apiVersion: velero.io/v1
kind: BackupStorageLocation
metadata:
  creationTimestamp: 2019-04-12T16:14:17Z
  generation: 4
  labels:
    component: velero
  name: default
  namespace: velero
  resourceVersion: "1851"
  selfLink: /apis/velero.io/v1/namespaces/velero/backupstoragelocations/default
  uid: 05d00579-5d3e-11e9-80f1-0242aa5bd293
spec:
  config: {}
  objectStorage:
    bucket: <mybucket>
    prefix: ""
  provider: gcp
status:
  lastSyncedRevision: a56d8e68-7017-4fb6-9a80-ec09433ae517
  lastSyncedTime: 2019-04-12T16:16:20.160871498Z

results in
An error occurred: some backup storage locations are invalid: error getting backup store for location "default": unable to locate ObjectStore plugin named gcp
and the server halting.
apiVersion: velero.io/v1
kind: BackupStorageLocation
metadata:
  creationTimestamp: 2019-04-12T16:14:17Z
  generation: 4
  labels:
    component: velero
  name: default
  namespace: velero
  resourceVersion: "1851"
  selfLink: /apis/velero.io/v1/namespaces/velero/backupstoragelocations/default
  uid: 05d00579-5d3e-11e9-80f1-0242aa5bd293
spec:
  config: {}
  objectStorage:
    bucket: <mybucket>
    prefix: ""
  provider: velero.io/gcp
status:
  lastSyncedRevision: a56d8e68-7017-4fb6-9a80-ec09433ae517
  lastSyncedTime: 2019-04-12T16:16:20.160871498Z

allows the server to start up as expected.
What did you expect to happen:
At least for the in-tree plugins, the user should only have to specify the provider name, not the velero.io domain prefix.
Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
Environment:

Velero version (use velero version):

Client:
        Version: master
        Git commit: abee09aa2d64813f9971a0103dcca6d5f4ab370a

It looks like this is coming from this line. Need to think on how best to address this, since a) we want qualified plugin names and b) we don't want users to type all the qualified names out every time.

Ah, yeah, this is obvious in retrospect. Obviously :) One option is in the GetObjectStore, GetVolumeSnapshotter, etc. methods in manager.go, we check for a domain prefix on name, and if none is specified, we automatically prefix with velero.io/.  This means the plugins are actually still registered as velero.io/aws, etc., but the user doesn't have to specify velero.io/ in their provider field.

Yeah, that was kind of what I was thinking. However, for non-Velero plugins, it means users still have to provide the fully qualified name, so it makes a distinction.  I also thought about just prepending velero.io to the provider in the install command, but that means any YAML provided would have to add it manually, too...
I'm inclined to do what you said for 1.0, though. It means however the BSL was made, we can still fetch it just fine for most users. Partners will need to update their docs, though.

Yeah, I'd prefer not to break existing installations for the built-in providers on upgrading to 1.0, so retaining the ability to just specify aws, etc. is nice. But yeah, this is definitely a breaking change for external plugin authors + users that we'll need to publicize.

Ok, I'll get a patch in for this.


Friendlist

can you code it so friends that dont have you added can still be shown on your mini map just like konduit/osbuddy/runeloader

https://github.com/runelite/runelite/wiki/Rejected-or-Rolled-Back-Features
Under not currently being considered

Duplicate of #951


Parsing correctly pURLs and prefixing targets.

Prefixing physical (files) targets with file://
Correctly lookup for pURLs which were previously ignored
Snyk-python and snyk-node pass more context about the language so that we can improve not well formatted pURLs

Pull Request Test Coverage Report for Build<PHONE_NUMBER>4
Details

78 of 121 (64.46%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.2%) to 40.501%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/sarif/sarif.go
78
121
64.46%

Totals

Change from base Build<PHONE_NUMBER>4:
0.2%

Covered Lines:
3445

Relevant Lines:
8506

💛  - Coveralls

Closed in favour of https://github.com/smithy-security/smithy/pull/476.


Not generating anything other than jsonSchema

I had this problem, so I asked for the supabase ts file (input) from a coworker, and only one line was different.
export interface Database { was export type Database = { for me, and after changing it to interface, the output file was generated properly.

I confirm this too.
There should be an update.

sorry, was on vacation. thanks @AndyLeezard for fixing this!

@psteinroe  Any chance we could get an updated release with this fix? Thanks!

sorry, had to update some versions in the release action for it to work again. new release is now out! thanks for the contribution everyone.


Want to add a new game

Adding a new style memory match game using html,css

Hey @MohanaSrinivas !
Thank you for raising an issue 💗
You can self assign the issue by commenting /assign in comment 😀
Make sure you follow CODE OF CONDUCT and CONTRIBUTING GUIDELINES 🚀
Don’t Forget to ⭐ our GameZone🎮
Make sure you join our Discord🕹

please open my closed repository

Hey @MohanaSrinivas ! Thank you so much for your raising the issue💗
It’s all yours, you can come anytime again and make some contributions! 🚀
Alone, we can do little, but together we can do so much! 😇


doc: security: Create a vulnerabilities report

In addition to having security vulnerability fixes reported within each
release note page, consolidate all of them in a new vulnerabilities
document.
This gives us two advantages: 1. The vulnerabilities can easily be
referenced in a single place, which is useful for someone trying to
cross reference against CVE lists, and 2. It allows a release to be made
with just CVE numbers when issues are under embargo, and the details can
be added to this vulnerabilities page.  The release notes will be locked
to a tag, and updates will not be visible.
Signed-off-by: David Brown<EMAIL_ADDRESS>

As discussed in today's TSC call.


Few improvements

:tada: This PR is included in version 1.13.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Lessons Learned - Jan 6, 2022

I did this procedure today, and this is what I learned:

Files stored to github.com/my-name/my-project take about a minute to get over to my-name.github.io/my-project, so make sure you add some text to the files to make sure you have your changes.

The business of storing the web-pages in the "gh-pages" branch and your manifest.json and .bin files in a sub-folder of the "main" branch was very confusing.  I got messages about "json has illegal characters" which actually meant is could not find the json and was getting a 404 error.  I eventually moved everything to the "main" branch and got rid of the sub-folder, and go the "settings - pages" of github to say I am using the "main" branch.  This may have been a result of not understanding point 1.

The procedure here mentions this crucial line

<EMAIL_ADDRESS>but after more time than I am willing to admit, and read through the page https://esphome.github.io/esp-web-tools/ which tells me to use the line
<EMAIL_ADDRESS>That is a bunch of versions numbers forward in the last few months - maybe something to do with esp32-arduino 2.00.  The previous line gives an error about "error: didn't get enough status bytes" but the new line pops up a little dialog to install or look at a console, and it all worked the first time.
So that is added to the google record of debugging -- and if all goes well you can go to https://github.com/jameszah/ESP32-CAM-Video-Recorder-junior for a one-click install of a video recorder program on a esp32-cam module with download video to browser functionality. 😄
https://jameszah.github.io/ESP32-CAM-VideoCam/

Thanks for the great feedback!

Yeah I have noticed that too, I think behind the scenes the repo gets converted and deployed so it works as a webpage. Including a version number on the webpage is probably a good idea!

Unless I'm mistaken, which should already be covered by the guide in the setting up GitHub pages section. Maybe it could be made clearer though. I just really quickly threw this guide together after using it as I didn't see much info out there on it.

Wow that is a big difference in versions for sure! I must take a look at the new release and update the docs for it.

Sounds like a cool project! I just love the idea of being able to share a relatively complicated project with people and all they have to do is click one button!

Yes you are right.  I was trying to link from gh-pages branch back to main branch, which does not work.

I suppose the merit of of the gh-pages branch is to move a small subset of the project over to github.io rather than all the source code and misc files.  Plus to keep these 1 MB binaries on a separate branch so they don't clog up people downloading a zip of source code.

Some updates here:
https://github.com/esphome/esp-web-tools/issues/288#issuecomment-1238529180
The four file system may appears not to work anymore.

Thanks for updating here, looks like this repo needs to be updated to match the updated info. I'm not realisticly in a position to do it for a few months , but would like to get to it at some point!


Expose product name in API, as env.productName

Is there a way (API) to detect the current release where the extension is running?
I would like to know this info, so I could differentiate my user data, which is stored on local disk instead of context.workspaceState.update.
Thanks in advance.

@alefragnani There no such thing as a project name constant but you can use the ExtensionContext and the extensionPath constant. That's the location your extension which will contain the product specific install location, like .vscode-insider/extensions (https://code.visualstudio.com/docs/extensionAPI/vscode-api#ExtensionContext)

@jrieken Thanks for your suggestion
I will be using it, until the new API be available.

I would like to know this info, so I could differentiate my user data, which is stored on local disk instead of context.workspaceState.update.

We have now the appName in the API which is the product name. I still wonder why you need this and cannot use the workspaceState cos that already scopes by product etc

Hi @jrieken ,
The contents that I'm saving is a JSON file. I did't use workspaceState because I wanted to let the user to have easy access to this file. So, I decided to use the same approach as keybindings.json and settings.json.
Your suggestion (to use ExtensionContext and extensionPath) does the trick, so for me it would be enough. But since you decided to add an API, I pretend to move to that API, when available.
What do you think about it?

It makes sense to store things outside in a readable file. My concern with using the product name is that it's not a 'derive storage location path' api but just a string and its value might change. So, while it is VS Code Alpha today it is not impossible that it becomes Visual Studio Code Alpha in the future.
I would recommend that you generate a storage file name, using the appName, but then store that filename using the ExtensionContext and retrieve it from there. That gives you speaking filenames but isn't prune to breaking like path-math is.
Unsure where exactly you store those files, but we could talk about exposing the directory path in which VS Code store global-global settings, like /Users/jrieken/Library/Application Support/Code - Alpha/User/settings.json

Well, I'm embarassed... that's exactly what I was looking for :disappointed: I should have make it clear right in the first post. I didn't know that you would expose that folder, so I was using it just for convenience, thinking in better alternatives.
I'm sorry, I didn't mean to make you waste your time with this. I will be more careful next time.

no worries. I have created a new request for that won't likely won't make it in April any more as we are in lock down since today...


Fails on the following construct imaps://foo@myemailerserver.lan:password@myemailerserver.lan:993/INBOX

There appears to be a problem with the following construct
imaps://foo@myemailerserver.lan:password@myemailerserver.lan:993/INBOX
which results in
May 09, 2021 6:50:36 AM de.dfki.km.leech.sax.PrintlnContentHandler processErrorData
INFO: ## PrintlnContentHandler ERROR data ##########################
no data entity id known - in the case of a sub-entity, set it inside the metadata at your implementation of getSubDataEntitiesInformation(..)
under the key CrawlerParser.SOURCEID. Otherwise you maybe try to process an unsupported/broken URL, or it is totally strange.
metadata:
errorStacktrace: 'javax.mail.AuthenticationFailedException: failed to connect, no password specified?
at javax.mail.Service.connect(Service.java:400)
at de.dfki.km.leech.parser.ImapCrawlerParser.connect2Server(ImapCrawlerParser.java:121)
at de.dfki.km.leech.io.ImapURLStreamProvider.addFirstMetadata(ImapURLStreamProvider.java:85)
at de.dfki.km.leech.Leech.parse(Leech.java:671)
at de.dfki.km.leech.Leech.parse(Leech.java:487)
at de.dfki.km.leech.Leech.main(Leech.java:110)
It is a requirement of the email server to use<EMAIL_ADDRESS>as the username, aliases cannot be used as they also require the same construct.
The email server works ok with the mail client supplied as part of the operating system.
Debugging the code shows that the construct is altered to
imaps://foo%40myemailerserver.lan:password@myemailerserver.lan:993/INBOX
Running with this construct causes
metadata:
X-Parsed-By: 'org.apache.tika.parser.CompositeParser'
X-Parsed-By: 'de.dfki.km.leech.parser.ImapCrawlerParser'
errorStacktrace: 'org.apache.tika.exception.TikaException: Error while crawling 'imaps://foo@myemailerserver.lan:password@myemailerserver.lan:993/INBOX'
at de.dfki.km.leech.parser.CrawlerParser.parse(CrawlerParser.java:230)
at de.dfki.km.leech.parser.ImapCrawlerParser.parse(ImapCrawlerParser.java:383)
at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
at org.apache.tika.parser.CompositeParser.parse(CompositeParser.java:280)
at org.apache.tika.parser.AutoDetectParser.parse(AutoDetectParser.java:143)
at de.dfki.km.leech.parser.incremental.IncrementalCrawlingParser.parse(IncrementalCrawlingParser.java:170)
at de.dfki.km.leech.parser.filter.URLFilteringParser.parse(URLFilteringParser.java:119)
at de.dfki.km.leech.Leech.parse(Leech.java:675)
at de.dfki.km.leech.Leech.parse(Leech.java:487)
at de.dfki.km.leech.Leech.main(Leech.java:110)
Caused by: java.lang.NullPointerException
at de.dfki.km.leech.parser.ImapCrawlerParser.getSubDataEntitiesInformation(ImapCrawlerParser.java:354)
at de.dfki.km.leech.parser.CrawlerParser.parse(CrawlerParser.java:170)
... 9 more
errorMessage: 'Error while crawling “imaps://foo%40myemailerserver.lan:password@myemailerserver.lan:993/INBOX''
dataEntityId: 'imaps://foo%40myemailerserver.lan:@myemailerserver.lan:993/INBOX'
resourceName: 'imaps://foo%40myemailerserver.lan:@myemailerserver.lan:993/INBOX'
source: 'imaps://foo%40myemailerserver.lan:@myemailerserver.lan:993/INBOX'
dataEntityContentFingerprint: 'INBOX'
Content-Type: 'remotedatasource/imapfolder'
May 09, 2021 6:58:40 AM de.dfki.km.leech.sax.CrawlReportContentHandler crawlFinished
INFO: Crawl finished:
Report: First handled data entity at 09/05/21 06:58, 1 processed entities, duration 0ms
New data entities: 0
Modified data entities: 0
Removed data entities: 0
Unmodified data entities: 0
Double data entities: 0
Error data entities: 1
remotedatasource/imapfolder:1
On a more positive note, tests with files and web pages work.

Today I can manage it to look into this issue :) Sorry for the long delay. URL parsing is realized with javax.mail.URLName, and it seems that your URL with double '@' breaks its rules as you found out..nevertheless your encoding with %40 works inside the leechcrawler code, the username and password are parsed correctly with your example. As an alternative you can also specify the username and password inside an IMAPCrawlerContext that you put inside a ParseContext, leaving usr:pwd inside the URL.
Beyond the credentials, connection to the IMAP Server is realized with javax.mail.Store.connect(..) inside ImapCrawlerParser, which is directly from Sun (and used from LeechCrawler in the recent version):
        <dependency>
            <groupId>com.sun.mail</groupId>
            <artifactId>javax.mail</artifactId>
            <version>1.6.2</version>
        </dependency>

Sadly, if the credentials given to this Store Object are correct and it fails to connect, I have to refer to the javax.mail project.
Regarding the NullPointer Exception - I can not reproduce the behaviour, and the underlying source line give me no hints what could be the the problem. Maybe you can have a closer look during debugging which Object is null, or try with java 16, it shows the null Object now.


Can not start timemachine from docker compose

Container will exit after start.
> docker-compose up
Creating ittimemachine_tm_1
Attaching to ittimemachine_tm_1
tm_1 | Starting Netatalk services:  netatalk.
ittimemachine_tm_1 exited with code 0

my docker-compose file:
tm:
  image: odarriba/timemachine
  hostname: tm.xxx.com
  volumes:
    - /mnt/hdd/data/timemachine:/timemachine
  ports:
    - '548:548'
    - '636:636'
  environment:
    AFP_LOGIN: usr
    AFP_PASSWORD: pwd
    AFP_NAME: Time_Machine
    AFP_SIZE_LIMIT: 2000000

Hi!
Maybe it could be something related to the running params? Like, for example, the -t -i used in the launch command (you can see it in the README), which are necessary when an interactive shell is used to maintain the docker running (like in this case).
Hope this helps?

Closed as there is no news

Hello @odarriba!
I bumped into this issue as well, using ansible. The container has no running foreground process at the end so it is stopped. I know there is a section about autostarting the container in the README, but unfortunately this is not an option for me. I changed the image to use supervisord[1] to start the process and keep a foreground process running. That way, you deploy the container once, and it gets automatically started and running via the docker process, instead of relying on a distribution specific startup script to manually start the container. Would you be interested in a pull request for this?
[1] https://docs.docker.com/engine/admin/using_supervisord/

@saesh Can you please create a pull request, so I can review the changes and improve the image?
Thanks in advance!

Hi!
I have already modified the image (and the README), so applying a restart policy the docker daemon starts the image itself.
Hope it helps, I'm closing the issue unless anything more appears.


ListObject v1 api don't properly follow S3 api

I'm stuck with an issue with 3rd party S3 implementation that follow the spec in a lazy mannier.
When I'm list a directory with more than 1000 file, multiples query should be made. XML return NextMarker as empty and IsTruncated as True.
According to S3 API ref, it's a valid biavior : https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjects.html
{code}
When response is truncated (the IsTruncated element value in the response is true), you can use the key name in this field as marker in the subsequent request to get next set of objects. Amazon S3 lists objects in alphabetical order Note: This element is returned only if you have delimiter request parameter specified. If response does not include the NextMarker and it is truncated, you can use the value of the last Key in the response as the marker in the subsequent request to get the next set of object keys.
{code}
But comportment on api-list.go is not following this behavior.
			// If next marker present, save it for next request.
			if result.NextMarker != "" {
				marker = result.NextMarker
			}

			// Listing ends result is not truncated, return right here.
			if !result.IsTruncated {
				return
			}

This cause infinite loop with only the first 1000 entry listed.
good co

I'm stuck with an issue with 3rd party S3 implementation that follow the spec in a lazy mannier.

The issue here is your 3rd party S3 implementation @alexises - we only use reset the marker - marker always has correct values you should the full code.
                        // If contents are available loop through and send over channel.
                        for _, object := range result.Contents {
                                // Save the marker.
                                marker = object.Key
                                select {
                                // Send object content.
                                case objectStatCh <- object:
                                // If receives done from the caller, return here.
                                case <-ctx.Done():
                                        return
                                }
                        }

                        // Send all common prefixes if any.
                        // NOTE: prefixes are only present if the request is delimited.
                        for _, obj := range result.CommonPrefixes {
                                select {
                                // Send object prefixes.
                                case objectStatCh <- ObjectInfo{Key: obj.Prefix}:
                                // If receives done from the caller, return here.
                                case <-ctx.Done():
                                        return
                                }
                        }

                        // If next marker present, save it for next request.
                        if result.NextMarker != "" {
                                marker = result.NextMarker
                        }

                        // Listing ends result is not truncated, return right here.
                        if !result.IsTruncated {
                                return
                        }

They must have incorrectly implemented the S3 API @alexises


Add development tooling - Ruff (linting)

It would be great to have the bakerydemo follow a code style guide, both for consistency in code and for this project being a demo of how to build a Wagtail project.
It might be worth adding Ruff instead of Flake8 - see https://github.com/wagtail/wagtail/pull/10324

We have Black as of #347 😃

Thanks @laymonage sorry - I double checked the requirements but only checked production requirements :|

Might close this actually, if Wagtail changes to Ruff we can re-open or revisit later.


Switch to PySide6 to make MaD GUI usabel on M1/M2 architecture

see #25
plan is to first merge https://github.com/mad-lab-fau/mad-gui/tree/arne-dataset-plugin into main, then merge this into main

@MalteOlle Let me know if I can help here!


telemetry_async has micropyGPS dependancies

async version needs micropyGPS dependancies removed and as_gps utilized. Code will not work until fixed.

Done.


Beginner problems

I'm new to reactive programming and I have some troubles. I have a device with N sensors (temperature, pressure, time, ...). I think it would be great idea if I represent sensors as streams. So I end up with this code:
function temperatureSensor(pin) {
  let stream = Kefir.emitter()

  if(pin > 12) {
    stream.error(new Error(`Board has no pin ${pin}.`))
  }

  setInterval(() => {
    stream.emit(readTemperature(pin))
  }, 500)

  return stream
}

Kefir.zip([
  temperatureSensor(1),
  temperatureSensor(42) // Invalid pin, should call .error
]).onValue((state) => {
  // do something with sensor values
}).onError(() => {
  console.error('An error occurred!')
})

There is few problems with this code:
1, .onError will never be called (.onError is attached after .error is called). Maybe use property instead of stream, but I don't know how...
2, .zip is not a ideal function, I need something like that:
// Synchronize sensors, call .onValue, then .onValue every time when sensor value change

a:    ----20------------21--------------
b:    -------25--------------------24---
ab:   --------•----------•----------•---
       [20, 25]   [21, 25]   [21, 24]

Thank you for any advice.

Hi, first issue when .onError callback newer called is because you use Kefir.emitter(). General advice here is to simply avoid using it. In almost any case when you use Kefir.emitter() there is a better alternative.
I'd write temperatureSensor something like this:
function temperatureSensor(pin) {
  if (pin > 12) {
    return Kefir.constantError(new Error(`Board has no pin ${pin}.`));
  } else {
    return Kefir.fromPoll(500, () => readTemperature(pin));
  }
}

Instead of .zip it looks like you need .combine. Looks like beginners often use .zip, but I'm personally never used it, not even once :)
Closing, but feel free to ask further questions.

Just another example with .merge + .slidingWindow:
function readT(pin) { return Math.ceil(Math.random() * 100) };

function temperatureSensor(pin) {
  if (pin <= 12) {
    return Kefir.fromPoll(500, function () {
      return readTemperature(pin)
    });
  } else {
    return Kefir.constantError(
      new Error('Board has no pin' + pin + '.')
    );
  }
};

var sensors = [
  temperatureSensor(1),
  temperatureSensor(9),
  temperatureSensor(14)
];
var sLength = sensors.length;
// assuming you want slices with same amount like your observed sensors

Kefir
  .merge(sensors)
  .slidingWindow(sLength, 0)
  .onError(function (error) {
    // had to put .onError first, otherwise the error never popped up
    console.log('Error is', error);
  })
  .onValue(function (state) {
    console.log('Current state is', JSON.stringify(state));
  })

had to put .onError first, otherwise the error never popped up

Ah, yes. To solve this you also need to add .toProperty after .zip, .combine, .merge etc. This described in the docs http://pozadi.github.io/kefir/#current-in-streams
Have to say, the fact, that one need to know all tiny thing like this to use this lib effectively, doesn't make me happy. I am trying to find a way to make it more easy to learn, but haven't get there yet. https://github.com/pozadi/kefir/issues/43 — here is discussion of one of problems like that.

Ah, thank you for this explanation. :+1:

Thank you very much, your responses are really helpful! I see that I choose the right framework.

I have one more question. I have small JSON file where I store all my persistent configuration. I need to do some work, when configuration load and change. It's possible and good idea to create stream from JSON file? Or is there any alternatives?

Yes this is totally possible, and may make sense. The code can look something like this:
var fileChanges = Kefir.fromBinder(function(emitter) {
  subscribeToFileUpdates(filename, emitter.emit);
  return function() {
    unsubscribeFromFileUpdates(filename, emitter.emit);
  };
});

// if you going to read file synchronously,
// and don't care about errors, you can use .map instead of .flatMapLatest
var config = fileChanges.flatMapLatest(function() {
  return Kefir.fromNodeCallback(function(cb) {
    getJsonFromFile(filename, cb);
  });
}).toProperty();

config.onValue(function(configObj) {
  // here you have latest config,
  // or you can combine `config` with other observables
});

Thank you very much again!

No problem :)

Old issue but I ran into a similar problem (.onError not being called with .zip) and it seems like the fix was simply to swap the order of .onError and .onValue. Any ideas why that's necessary?

Need to call toProperty, the merge/zip/combine converts properties -> streams, which means the current value in the stream (the error) is lost because it's been "emitted" before you subscribed.


Refactor Installation Instructions

Type of PR:
Documentation

PR Description:
This PR refactors the installation instructions for the PR-Agent. The instructions have been moved from the README.md file to a new INSTALL.md file. The README.md file now contains links to the different installation methods in the INSTALL.md file.

PR Main Files Walkthrough:

INSTALL.md: This new file contains detailed instructions for five different methods of installing and running the PR-Agent. These methods include using a Docker image, running as a Github Action, running from source, running as a polling server, and running as a Github App.
README.md: The detailed installation instructions have been removed from this file. Instead, it now contains links to the installation instructions in the INSTALL.md file.

@CodiumAI-Agent /describe

@CodiumAI-Agent

@CodiumAI-Agent /review

Preparing review...

PR Analysis

🎯 Main theme: Refactoring of Installation Instructions
📌 Type of PR: Documentation
🧪 Relevant tests added: No
✨ Focused PR: Yes, the PR is focused as it clearly states in the title and description that it is about refactoring the installation instructions. All changes in the PR are related to this task.
🔒 Security concerns: No, this PR is a documentation update and does not introduce any code changes that could potentially lead to security issues.

PR Feedback

💡 General PR suggestions: The PR is well-structured and the changes are clearly explained. The separation of installation instructions into a separate file makes the README cleaner and more focused. However, it would be beneficial to include a brief summary of each installation method in the README, rather than just links, to give users an idea of what each method involves without having to navigate away from the page.

How to use

Tag me in a comment '@CodiumAI-Agent' and add one of the following commands:
/review - Request a review of the latest update to the PR.
/describe - Modify the PR title and description based on the contents of the PR.
/improve - Suggest improvements to the code in the PR. These will be provided as pull request comments, ready to commit.
/ask  - Pose a question about the PR.


[Sample] Mount volumes for ephemeral containers

Problem Statement
Add the sample policy to mount volumes and set default security context for ephemeral containers. To break down the following policy:

the first rule adds the volume to pods with the label ephemeral-debug: true
the second rule adds the default security context to ephemeral containers
the third rule mounts the volume added by the first rule to ephemeral containers

apiVersion : kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: mount-volumes-for-ephemeral-containers
spec:
  background: false
  rules:
  - name: add-volume-to-debug-pod
    match:
      any:
      - resources:
          kinds:
          - Pod
          namespaces:
          - default
          selector:
            matchLabels:
              ephemeral-debug: "true"
          operations:
          - CREATE
    mutate:
      patchStrategicMerge:
        spec:
          volumes:
          - name: ephemeralcontainer-tmp-tcpdump-volume
            emptyDir:
              sizeLimit: 1G
  - name: mutate-ephemeralcontainers-readonlyrootfilesystem
    match:
      any:
      - resources:
          kinds:
          - Pod
          namespaces:
          - default
    preconditions:
       all:
       - key: "{{ request.object.spec.ephemeralContainers[] || `[]` | length(@) }}"
         operator: GreaterThanOrEquals
         value: 1
    mutate:
      foreach:
        - list: "request.object.spec.ephemeralContainers[]"
          patchStrategicMerge:
            spec:
              ephemeralContainers:
              - name: "{{ element.name }}"
                +(securityContext):
                  +(readOnlyRootFilesystem): true
  - name: add-volumemount
    match:
      any:
      - resources:
          kinds:
          - Pod
          namespaces:
          - default
    preconditions:
       all:
       - key: "{{ request.object.spec.ephemeralContainers[] || `[]` | length(@) }}"
         operator: GreaterThanOrEquals
         value: 1
       - key: "{{ request.object.spec.ephemeralContainers[].volumeMounts[?name=='ephemeralcontainer-tmp-tcpdump-volume'] | `[]` | length(@) }}"
         operator: LessThan
         value: "{{ request.object.spec.ephemeralContainers[] || `[]` | length(@) }}"
    mutate:
      foreach:
        - list: "request.object.spec.ephemeralContainers"
          patchStrategicMerge:
            spec:
              ephemeralContainers:
              - name: "{{ element.name }}"
                volumeMounts:
                - mountPath: /tmp
                  name: ephemeralcontainer-tmp-tcpdump-volume

It would be good to add Chainsaw tests for this policy, the test scenarios:

create the policy
create a pod with the label ephemeral-debug: true
verify the volume is added to the pod
create the ephemeral container, for example kubectl debug -it nginx-64ff7dcdf4-5jbmb --image=network-multitool:v1 --target nginx -- bash
verify the security context and volumeMounts are added to ephemeral containers

Solution Description
See above.
Example "Good" Resource
No response
Example "Bad" Resource
No response
Other Comments
No response
Slack discussion
https://kubernetes.slack.com/archives/CLGR9BJU9/p1721654971501889
Troubleshooting

[X] I have read and followed the documentation AND the troubleshooting guide.
[X] I have verified the policy does not exist in the samples library.

/assign