EPIC: ilab should support multiple serving backends

ilab currently uses llama as the main backend to serve the models. llama doesn't have as much extensive GPU and multi GPU support.
in developing some new tooling, custom vllm support needed to be quickly developed. ilab should natively be able to support vllm as a backend and other backends that exist.
something like --backend when running ilab serve chat or generate would make sense since all of these commands have the ability to start a server.
This might need to be an enhancement. I am open to writing one if needed.
cc @jaideepr97

@cdoern since this is specific to the CLI, IMHO no enhancement is nessessary

https://github.com/instructlab/instructlab/pull/1276 is an initial PR for how adding model serving with vllm would look. There is more work to be done there, but reviewers should have an idea for how the ideas below are implemented.
Since we're adding a new backend to serve, finalizing the UX for serving once for feature is where I want to start.
Here is my proposal of how I see the various ilab model serve commands looking:
Bare-bones but model specific command
ilab model serve --model-path <PATH>

Serves the model at .
If the  is the path for a .gguf file then llama-cpp is automatically used as the model serving backend.
If the path is a directory, vllm is automatically used as the backend.
The path provided to vllm can either be to a Huggingface repo or a directory with .safetensors.

Bare-bones command
ilab model serve

This command has the same behavior as the one above but the --model-path is whatever the default model path is in the config. This is the existing behavior of ilab serve today.

Llama-cpp backend specific commands
ilab model serve llama-cpp --model-path <PATH> --llama-cpp-gpu-layers 4 --llama-cpp-max-ctx-size 1024

This command serves a model with llama-cpp.
The existing flags to ilab serve (besides --model-path & --log-file) are specific to the llama-cpp backend and are prefixed to indicate that. This is helpful so that there is no mix up of flags between backends.

vllm backend specific commands
ilab model serve vllm --model-path <PATH> --vllm-chat-template <PATH> --vllm-args "<MORE VLLM SPECIFIC ARGS>"

This command serves a model with vllm.
If the path provided is not a directory, this command would error out.
There are dozens of flags for vllm. I'm hoping to figure out the most popular ones and add them as flags to ilab model serve vllm with --vllm- as a prefix, similar to what was done with llama-cpp backend args above.
Any remaining flags can be specified in --vllm-args.

Next steps
Here are the TODOs I want to finalize and create sub-tasks or sub-issues for in this EPIC that I see as in scope for #1276 or the 6/13 Milestone:

Get consensus on UX and command behavior (Original PR)
Add support for custom vllms installed with custom venvs (I would like to see this as part of the original PR too but it could be split out)
Add testing

How will our current e2e testing and github actions change?

Docs for this feature

Where do they go?

Figure out some of the most important vllm flags for users

Here are TODOs after the 6/13 Milestone:

Serve vllm in background if there is no model being served when ilab model chat is invoked
Support vllm inferencing on cpus[https://docs.vllm.ai/en/latest/getting_started/cpu-installation.html]
Look into vllm not being able to serve merlinite model on the latest release when I checked earlier this week

@russellb @cdoern @n1hility What are your thoughts on the UX and the TODOs for the upcoming release?

A few points at first glance:

I think for a better review this would benefit from a small design doc PR so we can put comments and discuss. Responding to this issue is going to be difficult.
This feels unnatural: ilab model serve llama-cpp, I would imagine something like ilab model serve --backend llama-cpp instead. If additional backends or more complex configurations are added, the command structure might become less clear. Using --backend is immediately clear that --backend is an option, making the command more self-explanatory.
checking on the .gguf extension seems "weak", I'd rather read the first byte of the image to determine if this is .gguf or not.
When using a specific backend, I'd avoid repeating the backend name in the args like --llama-cpp-gpu-layers, which should simply be. --gpu-layers, we know the backend so no need to repeat it.
For VLLM-supported flags, I think we should start with a subset we are comfortable with and progressively introduce more supported flags once we have good CI coverage. I wouldn't worry too much about the number of flags in general.

Per discussion in the community call yesterday, A pull request for a design doc has been opened: https://github.com/instructlab/dev-docs/pull/81
This doc covers the CLI UX and testing. Please move comments and feedback related to the design I proposed above to that PR.

ilab as of 0.18.0 now supports multiple serving backends - llama-cpp was the original and we now also have support for vLLM


docs(contribute): fix incomplete warning

The warning statement about using the --inspect interface is missing the word free. Fixed it.

Thanks!


Clean Up Archive Folder

ProtonMail Bridge
TickTick
Move UniFi to Wiki

Unifi is already moved: https://github.com/iancleary/wiki/blob/b13de0b/docs/rpi-unifi.md#L29

Closed per successful merge of https://github.com/iancleary/personal-ansible/pull/81


community logins bundle alert

Dear Daniel,
Can the app be configure to alert for reaching close (80%) of the API calls consumed for a community logins bundle (2000 API calls) so a new bundle can be purchased before hand.
With much Appreciation,
gitForce

Do you mean number of community logins or API calls associated with those logins? I'm not aware of bundle of API calls that come with community logins. I have a separate issue #4 opened to monitor usage based entitlements which is where the login information is tracked.

Dear Daniel,
To give a general idea.
We use community as the basis for recruiting  candidates to our university
academic staff.
Each candidate needs a license for a very limited number of logins.
That is why we made an advantage of the 2000 logins/per year bundle option.
The limit and actual consumption is shown in company information under
licenses section.
We happened to hit the limit one month earlier :)) and our candidates have
been blocked for several days before we even knew that was the reason.
Then I remembered the wonderful podcast you’ve taken part and wonder if
salesforce limits can handle in its current state something as such.
Kindest regards,
Itay
It would be
בתאריך יום ב׳, 6 ביולי 2020 ב-19:28 מאת Daniel Hoechst <
<EMAIL_ADDRESS>
Do you mean number of community logins or API calls associated with those
logins? I'm not aware of bundle of API calls that come with community
logins. I have a separate issue #4
https://github.com/dhoechst/salesforce-limit-monitor/issues/4 opened to
monitor usage based entitlements which is where the login information is
tracked.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/dhoechst/salesforce-limit-monitor/issues/8#issuecomment-654339252,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADYSZPNYSPH6SAXEW5A75XLR2H3RNANCNFSM4OQYURZA
.

It isn't possible now, but issue #4 should help. I'm not sure how frequently it updates how much is used, though.


Implement incoming layer bitrate estimation and forward the information to interested parties

All done, commits squashed. Closing to avoid accidental, premature merge


DSCS-2582  Additional functionality for tags

@scoen @yetti @ninhnguyen
Please have a look this.

@yetti @ninhnguyen
Some background for Yetrina and Ninh:
This query gets used by a method that sets the amberSession into localMode. That means it only sees what's in memory for the session - it doesn't query the database to find objects. The change ensures that Tags related to the Works or Copies in question are also loaded into memory so they can be found during the localMode.
The reason for setting localMode is to prevent large numbers of database accesses when looping over large numbers of Works or Copies.
Hope that helps (but perhaps as clear as mud).


No sound card generated on macOS

This issue collects information around a sound problem on M1 Macs running UTM 2.0.25 on macOS Big Sur 11.2.3.
It appears that virtual machines on these hosts don't recognize any sound cards, regardless of which sound card is chosen in the VM configuration. (so far all reports received are running various Windows operating systems)
Configuration

UTM Version: 2.0.25
OS Version: macOS Big Sur 11.2.3
Apple Silicon

More information will be added in the comments.
EDIT: the source of this bug has been discovered. See the latest comments.

Issue #2418 is a partial duplicate of this issue. The user reports no sound devices showing in their Windows 10 ARM install, despite trying multiple different sound card settings in the VM configuration.

On Discord, UTM user wiley207 reports the same problem of "no audio device" when using a Windows XP virtual machine on an M1 Mac:

So it seems with the current version of UTM for MacOS, on Big Sur 11.2.3, Windows XP machines created will not have a working sound card

Similarly, another Discord UTM user (with a Spanish expletive as a username) reports no audio in their Windows 10 VM, their host computer is a M1 Mac mini on 11.2.3.

Discord member dionz9 reports "no sound" in their Ubuntu VM on M1 MacBook Air.

The bug seems to have been introduced with d6ccf4d

@conath can you test e1c18e18b5933d1fdc16281769d971a90069c34d ?

Tested using a Windows 7 32 bit guest in a q35 x86_64 machine, running using Hypervisor on an Intel Mac mini. Intel HDA and AC97 work fine!


Bitballon ruby gem is crashing

$ bitballoon deploy
/home/znmeb/.gem/ruby/gems/bitballoon-0.2.4/bin/bitballoon:8:in `block in <top (required)>': undefined method `on' for main:Object (NoMethodError)
	from /home/znmeb/.gem/ruby/gems/slop-4.0.0/lib/slop/options.rb:33:in `initialize'
	from /home/znmeb/.gem/ruby/gems/slop-4.0.0/lib/slop.rb:24:in `new'
	from /home/znmeb/.gem/ruby/gems/slop-4.0.0/lib/slop.rb:24:in `parse'
	from /home/znmeb/.gem/ruby/gems/bitballoon-0.2.4/bin/bitballoon:7:in `<top (required)>'
	from /home/znmeb/bin/bitballoon:23:in `load'
	from /home/znmeb/bin/bitballoon:23:in `<main>'

Thanks for reporting this! The option parsing gem, slop, released a new version with changes to its syntax and we didn't specify a specific version in our gemspec.
I've pushed a new version of the gem that fixes this.

working now - thanks!!


Add support for request helper

Add support for the request() helper.
This is my first significant contribution to phpstan/larastan so all feedback is welcome.

Looks very nice. Does it work?
BTW Larastan's code is checked by PHPStan ;-)

Thank you for your contribution!

I've pushed to the original branch, not sure how GH will handle now that the PR is already merged. Shall I submit another one?

@j3j5 I think you can submit a new PR.


Thread panic when trying to spawn agents on intersection

I've got the following thread panic trying to executing the first point of the "Playing the game" section of the readme:

In sandbox mode, hover over an intersection, right click, and spawn agents. Then you can start the simulation by pressing space or clicking the icon.

HiDPI factor is purportedly 1
load map...
upload textures (14)... 0.0108s
14 textures grouped into 3 arrays (with the same dimensions)
Loading map ../data/system/maps/montlake.bin
Read ../data/system/maps/montlake.bin (4)... 0.0465s
create sim...
create sim took 0.0018s
Map and Sim cost ~10 MB
draw_map...
make DrawRoads (415)... 0.0271s
generate thick roads...
generate thick roads took 0.0028s
prepare DrawLanes (2,181)... 0.0358s
finalize DrawLanes (2,181)... 0.0353s
make DrawIntersections (317)... 0.0264s
generate unzoomed intersections...
generate unzoomed intersections took 0.0009s
make DrawBuildings (3,018)... 0.0074s
upload all buildings...
upload all buildings took 0.0041s
make DrawBusStop (16)... 0.0003s
make DrawAreas (55)... 0.0000s
upload all areas...
upload all areas took 0.0001s
create quadtree...
create quadtree took 0.0715s
static DrawMap consumes 9 MB on the GPU
draw_map... plus 0.0060s
draw_map took 0.2177s
DrawMap cost ~158 MB
load map... plus 0.0201s
load map took 0.2969s

- load map took 0.2969s
  - upload textures (14)... 0.0108s
  - Read ../data/system/maps/montlake.bin (4)... 0.0465s
  - create sim took 0.0018s
  - draw_map took 0.2177s
    - make DrawRoads (415)... 0.0271s
    - generate thick roads took 0.0028s
    - prepare DrawLanes (2,181)... 0.0358s
    - finalize DrawLanes (2,181)... 0.0353s
    - make DrawIntersections (317)... 0.0264s
    - generate unzoomed intersections took 0.0009s
    - make DrawBuildings (3,018)... 0.0074s
    - upload all buildings took 0.0041s
    - make DrawBusStop (16)... 0.0003s
    - make DrawAreas (55)... 0.0000s
    - upload all areas took 0.0001s
    - create quadtree took 0.0715s
    - ... plus 0.0060s
  - ... plus 0.0201s

5 notes:
14 textures grouped into 3 arrays (with the same dimensions)
Loading map ../data/system/maps/montlake.bin
Map and Sim cost ~10 MB
static DrawMap consumes 9 MB on the GPU
DrawMap cost ~158 MB

1 warnings:
Couldn't make geometry for TurnID(LaneID(531), LaneID(192), IntersectionID(52)). look for Pt2D(336.4989, 876.3308) to Pt2D(355.7037, 866.4475) in [Pt2D { inner_x: 336.3221, inner_y: 891.3298 }, Pt2D { inner_x: 336.4989, inner_y: 876.3308 }, Pt2D { inner_x: 345.5252, inner_y: 876.4371 }, Pt2D { inner_x: 355.3143, inner_y: 866.8297 }, Pt2D { inner_x: 366.594, inner_y: 876.7766 }, Pt2D { inner_x: 366.3413, inner_y: 891.7741 }, Pt2D { inner_x: 353.0851, inner_y: 891.5507 }, Pt2D { inner_x: 349.9952, inner_y: 897.5429 }, Pt2D { inner_x: 336.3221, inner_y: 891.3298 }]

- load map took 0.2969s
instantiate scenario...
instantiate scenario took 0.0000s

- instantiate scenario took 0.0000s

- instantiate scenario took 0.0000s
reset simulation...
reset simulation took 0.0046s

- reset simulation took 0.0046s

- reset simulation took 0.0046s
instantiate scenario...
instantiate scenario took 0.0000s

- instantiate scenario took 0.0000s

- instantiate scenario took 0.0000s
thread 'thread 'thread 'thread 'thread '<unnamed><unnamed>' panicked at '<unnamed>' panicked at 'no entry found for key' panicked at '', no entry found for keysrc/libcore/option.rs', :src/libcore/option.rs1190::<unnamed><unnamed>11905' panicked at ':
' panicked at 'no entry found for key5note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
',
src/libcore/option.rs:no entry found for key', no entry found for key', thread 'src/libcore/option.rs1190<unnamed>thread '' panicked at ':<unnamed>no entry found for key1190:' panicked at '', :5no entry found for keysrc/libcore/option.rs
', :src/libcore/option.rssrc/libcore/option.rs:511901190
:thread ':<unnamed>5' panicked at ':
5no entry found for key1190
', :src/libcore/option.rs5:
1190:5
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/libcore/option.rs:378:21
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: "SendError(..)"', src/libcore/result.rs:1165:5
stack backtrace:
   0:     0x55cada4679f4 - backtrace::backtrace::libunwind::trace::hda41dbcdfba36aa0
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.37/src/backtrace/libunwind.rs:88
   1:     0x55cada4679f4 - backtrace::backtrace::trace_unsynchronized::h1a8d6e1f8cb3f5d4
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.37/src/backtrace/mod.rs:66
   2:     0x55cada4679f4 - std::sys_common::backtrace::_print_fmt::h610c4127487e10da
                               at src/libstd/sys_common/backtrace.rs:76
   3:     0x55cada4679f4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h0722dc552e01bd1d
                               at src/libstd/sys_common/backtrace.rs:60
   4:     0x55cada48f1bc - core::fmt::write::h01edf6dd68a42c9c
                               at src/libcore/fmt/mod.rs:1030
   5:     0x55cada4639f7 - std::io::Write::write_fmt::hf15985f193f03c04
                               at src/libstd/io/mod.rs:1412
   6:     0x55cada469f15 - std::sys_common::backtrace::_print::hd8d5d08a1795e743
                               at src/libstd/sys_common/backtrace.rs:64
   7:     0x55cada469f15 - std::sys_common::backtrace::print::hf89a79e3921a2366
                               at src/libstd/sys_common/backtrace.rs:49
   8:     0x55cada469f15 - std::panicking::default_hook::{{closure}}::h3a8f42beb3bb8ae3
                               at src/libstd/panicking.rs:196
   9:     0x55cada469c06 - std::panicking::default_hook::h8f803b0bc31a5c37
                               at src/libstd/panicking.rs:210
  10:     0x55cada46a615 - std::panicking::rust_panic_with_hook::h825f041245da8739
                               at src/libstd/panicking.rs:473
  11:     0x55cada46a1b2 - std::panicking::continue_panic_fmt::hbe0378e33481e81b
                               at src/libstd/panicking.rs:380
  12:     0x55cada46a0a6 - rust_begin_unwind
                               at src/libstd/panicking.rs:307
  13:     0x55cada48ae6a - core::panicking::panic_fmt::h527855ce0bc891f6
                               at src/libcore/panicking.rs:85
  14:     0x55cada48af67 - core::result::unwrap_failed::ha8b77e6004f0ba38
                               at src/libcore/result.rs:1165
  15:     0x55cada2f7c3c - scoped_threadpool::Scope::join_all::h90328822974430c3
  16:     0x55cad9ff3d44 - scoped_threadpool::Pool::scoped::h23d3c0a65b3839aa
  17:     0x55cad9f62db3 - abstutil::time::Timer::parallelize::ha823fecf13b2bbde
  18:     0x55cad9f4d13c - sim::make::spawner::TripSpawner::spawn_all::hc1532ecee05b9d68
  19:     0x55cad9f94edc - sim::sim::Sim::spawn_all_trips::he48301f395ddb241
  20:     0x55cad9dd1786 - game::sandbox::gameplay::spawner::spawn_agents_around::h313367c17e02f941
  21:     0x55cad9dcef57 - game::sandbox::gameplay::spawner::AgentSpawner::new::ha38fdaab740bd5cf
  22:     0x55cad9d774d3 - <game::sandbox::gameplay::freeform::Freeform as game::sandbox::gameplay::GameplayState>::event::h630c9b8defde470f
  23:     0x55cad9d781c0 - <game::sandbox::SandboxMode as game::game::State>::event::h15641abae535d20a
  24:     0x55cad9e0ea12 - <game::game::Game as ezgui::runner::GUI>::event::h9f61dcd2bd50e5d2
  25:     0x55cad9d36d11 - std::panicking::try::do_call::h2d6652771a795e52
  26:     0x55cada4727da - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:80
  27:     0x55cad9df36c9 - ezgui::runner::State<G>::event::h89affbbdff6766dc
  28:     0x55cad9df506f - ezgui::runner::run::h7cacd4f4b053b02f
  29:     0x55cad9dd2bd6 - game::main::h5b516a8028736dc2
  30:     0x55cad9e3fbe3 - std::rt::lang_start::{{closure}}::h9e9cd991840640b1
  31:     0x55cada46a043 - std::rt::lang_start_internal::{{closure}}::ha04574f12d97cbe2
                               at src/libstd/rt.rs:49
  32:     0x55cada46a043 - std::panicking::try::do_call::h7c2a8488f72db90c
                               at src/libstd/panicking.rs:292
  33:     0x55cada4727da - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:80
  34:     0x55cada46abcd - std::panicking::try::hc3a9b5da4250385d
                               at src/libstd/panicking.rs:271
  35:     0x55cada46abcd - std::panic::catch_unwind::hf27600bf8c37809a
                               at src/libstd/panic.rs:394
  36:     0x55cada46abcd - std::rt::lang_start_internal::h409d4f2fe51133b0
                               at src/libstd/rt.rs:48
  37:     0x55cad9dd2ca2 - main
  38:     0x7f2b46f8cb97 - __libc_start_main
  39:     0x55cad9c71afa - _start
  40:                0x0 - <unknown>
thread panicked while panicking. aborting.
fish: 'cargo run --release' terminated by signal SIGILL (Illegal instruction)

+1, also got a crash while starting simulation
(Ubuntu 18.04 x86_64, i7-770HQ, GTX1050)
instantiate scenario...
Read ../data/system/scenarios/montlake/weekday_typical_traffic_from_psrc.bin (1)... 0.0160s
Instantiating weekday_typical_traffic_from_psrc...
load full neighborhood info...
load_all_objects from ../data/input/neighborhoods/montlake...
load_all_objects from ../data/input/neighborhoods/montlake took 0.0000s

- load_all_objects from ../data/input/neighborhoods/montlake took 0.0000s

- load_all_objects from ../data/input/neighborhoods/montlake took 0.0000s
load full neighborhood info took 0.0057s
seed individual parked cars (1,875)... 0.0044s
SpawnTrip (26,975)... 0.0051s
thread '<unnamed>' panicked at 'no entry found for key', src/libcore/option.rs:1034:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
thread '<unnamed>' panicked at 'no entry found for key', src/libcore/option.rs:1034:5
thread '<unnamed>thread '' panicked at '<unnamed>no entry found for key' panicked at '', no entry found for key', src/libcore/option.rssrc/libcore/option.rs::10341034:thread ':5<unnamed>
thread '5' panicked at '
<unnamed>no entry found for key' panicked at '', no entry found for keysrc/libcore/option.rs', :src/libcore/option.rs1034::10345:
5
thread '<unnamed>' panicked at 'no entry found for key', src/libcore/option.rs:1034:5
thread '<unnamed>' panicked at 'no entry found for key', src/libcore/option.rs:1034:5
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: "SendError(..)"', src/libcore/result.rs:999:5
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: "SendError(..)"', src/libcore/result.rs:999:5
stack backtrace:
   0:     0x564e3b832421 - std::sys_common::backtrace::print::h0158bc30f1f08e5f
   1:     0x564e3b83a262 - std::panicking::default_hook::{{closure}}::hd0e5ccf5a951fbfd
   2:     0x564e3b839ff3 - std::panicking::default_hook::ha72e5501b0ffb83c
   3:     0x564e3b83a950 - std::panicking::rust_panic_with_hook::hcca196f1f1162139
   4:     0x564e3b83a4e2 - std::panicking::continue_panic_fmt::hab503b114fbb56f8
   5:     0x564e3b83a3c6 - rust_begin_unwind
   6:     0x564e3b84fbdd - core::panicking::panic_fmt::haa3ad5d410001da3
   7:     0x564e3b6bd325 - core::result::unwrap_failed::hd77977b9fedbe029
   8:     0x564e3b6b6c4a - scoped_threadpool::Scope::join_all::hf48aa041f0a5a517
   9:     0x564e3b378d20 - scoped_threadpool::Pool::scoped::h1708f6e7698ef072
  10:     0x564e3b3ad453 - abstutil::time::Timer::parallelize::h37ac61b85717446f
  11:     0x564e3b3976a9 - sim::make::spawner::TripSpawner::spawn_all::h398387392e763dc9
  12:     0x564e3b2fc51d - sim::make::scenario::Scenario::instantiate::h2684c9bbbcfb5e44
  13:     0x564e3b12a21a - ezgui::event_ctx::EventCtx::loading_screen::h5a0b9e6801f790d4
  14:     0x564e3b163327 - game::sandbox::gameplay::GameplayRunner::initialize::h771063c3288f8090
  15:     0x564e3b0910a7 - game::sandbox::SandboxMode::new::hbd0f7fd9eba658ad
  16:     0x564e3b16361f - game::sandbox::gameplay::change_scenario::h0076a9bb8200f41f
  17:     0x564e3b05c7a2 - core::ops::function::Fn::call::h7dee61cb58e6bf06
  18:     0x564e3b139c05 - <game::game::WizardState as game::game::State>::event::h1b54ddabeb0a2f64
  19:     0x564e3b13805b - <game::game::Game as ezgui::runner::GUI>::event::h4c9b7d472ace1c00
  20:     0x564e3b05c611 - std::panicking::try::do_call::h2fcb73b958ded295
  21:     0x564e3b83e9ca - __rust_maybe_catch_panic
  22:     0x564e3b126bf8 - ezgui::runner::State<G>::event::hf50953513ffec574
  23:     0x564e3b128352 - ezgui::runner::run::hc8eab16c5dbee0ab
  24:     0x564e3b0eee96 - game::main::h085720c1f8e986ac
  25:     0x564e3b18c753 - std::rt::lang_start::{{closure}}::hc0f6c5f1358765f9
  26:     0x564e3b83a363 - std::panicking::try::do_call::hc3f3f274ffe6f0cd
  27:     0x564e3b83e9ca - __rust_maybe_catch_panic
  28:     0x564e3b829089 - std::rt::lang_start_internal::hb1dc547617272b6b
  29:     0x564e3b0eef62 - main
  30:     0x7fed33117b97 - __libc_start_main
  31:     0x564e3b00705a - _start
  32:                0x0 - <unknown>
thread panicked while panicking. aborting.

nonetheless, this project is amazing! looking forward to where it goes in the future

Thanks for reporting! I'm having trouble reproducing either problem, but both are pathfinding breaking.
@mdesantis, from your logs, the spawn agents at intersection works a few times, but then breaks somewhere. Is that right? Can you tell me what IntersectionID you're hovering over when it breaks? (The bottom bar shows this. I'll add this to the logs to make it easier to debug in the future.)
@b-camacho, I'm very confused how this is happening. You're using a binary release? I'll add some better errors to figure this out.

@dabreegster Thanks for looking into it!
I'm compiling from source, commit 77050761. Could not run binary due to https://github.com/dabreegster/abstreet/issues/17, but I doubt these are related.

@b-camacho, would you mind pulling and running again? You should see an error like ".... not in NodeMap". Somehow the contraction hierarchy built to speed up pathfinding isn't including one of the path endpoints.

Sure enough, there it is
spawning agents around IntersectionID(108) (rng seed None)...
thread 'thread '<unnamed>' panicked at 'SidewalkEndpoint(LaneID(569), true) not in NodeMapthread '', <unnamed>map_model/src/pathfind/node_map.rs' panicked at ':SidewalkEndpoint(LaneID(1053), true) not in NodeMap36', :map_model/src/pathfind/node_map.rs13:

Exact repro steps are:

run in either debug or release
open sandbox
click any intersection
choose "spawn agents around this intersection"
Maybe it's relevant that instantiating scenarios causes the same panic.

Here's the full output:
HiDPI factor is purportedly 1.5
load map...
upload textures (14)... 0.3090s
14 textures grouped into 3 arrays (with the same dimensions)
Loading map ../data/system/maps/montlake.bin
Read ../data/system/maps/montlake.bin (4)... 0.4513s
create sim...
create sim took 0.0293s
Map and Sim cost ~10 MB
draw_map...
make DrawRoads (415)... 0.2329s
generate thick roads...
generate thick roads took 0.0215s
prepare DrawLanes (2,181)... 0.1751s
finalize DrawLanes (2,181)... 0.1344s
make DrawIntersections (317)... 0.1905s
generate unzoomed intersections...
generate unzoomed intersections took 0.0082s
make DrawBuildings (3,018)... 0.0691s
upload all buildings...
upload all buildings took 0.0124s
make DrawBusStop (16)... 0.0015s
make DrawAreas (55)... 0.0002s
upload all areas...
upload all areas took 0.0003s
create quadtree...
create quadtree took 0.3998s
static DrawMap consumes 9 MB on the GPU
draw_map... plus 0.0579s
draw_map took 1.3037s
DrawMap cost ~560 MB
load map... plus 0.0558s
load map took 2.1491s

- load map took 2.1491s
  - upload textures (14)... 0.3090s
  - Read ../data/system/maps/montlake.bin (4)... 0.4513s
  - create sim took 0.0293s
  - draw_map took 1.3037s
    - make DrawRoads (415)... 0.2329s
    - generate thick roads took 0.0215s
    - prepare DrawLanes (2,181)... 0.1751s
    - finalize DrawLanes (2,181)... 0.1344s
    - make DrawIntersections (317)... 0.1905s
    - generate unzoomed intersections took 0.0082s
    - make DrawBuildings (3,018)... 0.0691s
    - upload all buildings took 0.0124s
    - make DrawBusStop (16)... 0.0015s
    - make DrawAreas (55)... 0.0002s
    - upload all areas took 0.0003s
    - create quadtree took 0.3998s
    - ... plus 0.0579s
  - ... plus 0.0558s

5 notes:
14 textures grouped into 3 arrays (with the same dimensions)
Loading map ../data/system/maps/montlake.bin
Map and Sim cost ~10 MB
static DrawMap consumes 9 MB on the GPU
DrawMap cost ~560 MB

1 warnings:
Couldn't make geometry for TurnID(LaneID(531), LaneID(192), IntersectionID(52)). look for Pt2D(336.4989, 876.3308) to Pt2D(355.7037, 866.4475) in [Pt2D { inner_x: 336.3221, inner_y: 891.3298 }, Pt2D { inner_x: 336.4989, inner_y: 876.3308 }, Pt2D { inner_x: 345.5252, inner_y: 876.4371 }, Pt2D { inner_x: 355.3143, inner_y: 866.8297 }, Pt2D { inner_x: 366.594, inner_y: 876.7766 }, Pt2D { inner_x: 366.3413, inner_y: 891.7741 }, Pt2D { inner_x: 353.0851, inner_y: 891.5507 }, Pt2D { inner_x: 349.9952, inner_y: 897.5429 }, Pt2D { inner_x: 336.3221, inner_y: 891.3298 }]

- load map took 2.1491s
instantiate scenario...
instantiate scenario took 0.0000s

- instantiate scenario took 0.0000s

- instantiate scenario took 0.0000s
spawning agents around IntersectionID(108) (rng seed None)...
thread 'thread '<unnamed>' panicked at 'SidewalkEndpoint(LaneID(569), true) not in NodeMapthread '', <unnamed>map_model/src/pathfind/node_map.rs' panicked at ':SidewalkEndpoint(LaneID(1053), true) not in NodeMap36', :map_model/src/pathfind/node_map.rs13:
36note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
thread ':<unnamed>13' panicked at '
SidewalkEndpoint(LaneID(691), true) not in NodeMap', map_model/src/pathfind/node_map.rs:36:13
<unnamed>' panicked at 'SidewalkEndpoint(LaneID(998), true) not in NodeMap', map_model/src/pathfind/node_map.rs:36:13
thread '<unnamed>' panicked at 'SidewalkEndpoint(LaneID(1944), false) not in NodeMap', map_model/src/pathfind/node_map.rs:36:13
thread '<unnamed>' panicked at 'SidewalkEndpoint(LaneID(642), false) not in NodeMap', map_model/src/pathfind/node_map.rs:36:13
thread '<unnamed>' panicked at 'SidewalkEndpoint(LaneID(483), false) not in NodeMap', map_model/src/pathfind/node_map.rs:36:13
thread '<unnamed>' panicked at 'SidewalkEndpoint(LaneID(1340), true) not in NodeMap', map_model/src/pathfind/node_map.rs:36:13
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/libcore/option.rs:347:21
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: "SendError(..)"', src/libcore/result.rs:999:5
stack backtrace:
   0:     0x5584287b2a61 - std::sys_common::backtrace::print::h0158bc30f1f08e5f
   1:     0x5584287ba8a2 - std::panicking::default_hook::{{closure}}::hd0e5ccf5a951fbfd
   2:     0x5584287ba633 - std::panicking::default_hook::ha72e5501b0ffb83c
   3:     0x5584287baf90 - std::panicking::rust_panic_with_hook::hcca196f1f1162139
   4:     0x5584287bab22 - std::panicking::continue_panic_fmt::hab503b114fbb56f8
   5:     0x5584287baa06 - rust_begin_unwind
   6:     0x5584287d023d - core::panicking::panic_fmt::haa3ad5d410001da3
   7:     0x5584281bfdca - core::result::unwrap_failed::hafd032a06f103130
                               at /usr/src/rustc-1.37.0/src/libcore/macros.rs:18
   8:     0x5584281c0e95 - core::result::Result<T,E>::unwrap::hbe4130a0022f30a4
                               at /usr/src/rustc-1.37.0/src/libcore/result.rs:800
   9:     0x5584281c4476 - scoped_threadpool::Scope::join_all::hd88dc208fba1b3ce
                               at /home/brian/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped_threadpool-0.1.9/src/lib.rs:219
  10:     0x5584281c467f - <scoped_threadpool::Scope as core::ops::drop::Drop>::drop::h85194363c5c1baad
                               at /home/brian/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped_threadpool-0.1.9/src/lib.rs:248
  11:     0x558426ad014f - core::ptr::real_drop_in_place::h398d5ef157afc50a
                               at /usr/src/rustc-1.37.0/src/libcore/ptr/mod.rs:197
  12:     0x558427191955 - scoped_threadpool::Pool::scoped::h84636df52e3e0468
                               at /home/brian/.cargo/registry/src/github.com-1ecc6299db9ec823/scoped_threadpool-0.1.9/src/lib.rs:182
  13:     0x558426c7b620 - abstutil::time::Timer::parallelize::he7a21c2c5926e1bd
                               at /home/brian/tools/abstreet/abstutil/src/time.rs:325
  14:     0x558426d6a462 - sim::make::spawner::TripSpawner::spawn_all::h212ce524fca724a9
                               at sim/src/make/spawner.rs:191
  15:     0x558426bdb44c - sim::sim::Sim::spawn_all_trips::h5badbb5b96a7b1f3
                               at sim/src/sim.rs:162
  16:     0x55842649b187 - game::sandbox::gameplay::spawner::spawn_agents_around::h55a5844f510fea29
                               at game/src/sandbox/gameplay/spawner.rs:356
  17:     0x5584264996c9 - game::sandbox::gameplay::spawner::AgentSpawner::new::h3b0c505bd74d3ac5
                               at game/src/sandbox/gameplay/spawner.rs:160
  18:     0x5584266729b4 - <game::sandbox::gameplay::freeform::Freeform as game::sandbox::gameplay::GameplayState>::event::h0e25d9bb8c5ebaab
                               at game/src/sandbox/gameplay/freeform.rs:57
  19:     0x5584267e3852 - game::sandbox::gameplay::GameplayRunner::event::h0c2acd4d1ad04e75
                               at game/src/sandbox/gameplay/mod.rs:197
  20:     0x5584264c536b - <game::sandbox::SandboxMode as game::game::State>::event::hc0038cc46a87faf6
                               at game/src/sandbox/mod.rs:95
  21:     0x5584267c36fd - <game::game::Game as ezgui::runner::GUI>::event::he1bf1746280ac84f
                               at game/src/game.rs:44
  22:     0x5584264c1ede - ezgui::runner::State<G>::event::{{closure}}::h03c5b195d8a0f687
                               at /home/brian/tools/abstreet/ezgui/src/runner.rs:84
  23:     0x5584264320b1 - core::ops::function::FnOnce::call_once::hda0cbd78de9397d9
                               at /usr/src/rustc-1.37.0/src/libcore/ops/function.rs:231
  24:     0x55842667d678 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::ha172cc84c455c50b
                               at /usr/src/rustc-1.37.0/src/libstd/panic.rs:315
  25:     0x558426353e2b - std::panicking::try::do_call::heedc7547f4b21649
                               at /usr/src/rustc-1.37.0/src/libstd/panicking.rs:296
  26:     0x5584287bf00a - __rust_maybe_catch_panic
  27:     0x558426353a39 - std::panicking::try::h1acac747bee48db4
                               at /usr/src/rustc-1.37.0/src/libstd/panicking.rs:275
  28:     0x55842667d6f8 - std::panic::catch_unwind::h42b065837ab66a6d
                               at /usr/src/rustc-1.37.0/src/libstd/panic.rs:394
  29:     0x5584264c16ed - ezgui::runner::State<G>::event::h48a2232029bcf607
                               at /home/brian/tools/abstreet/ezgui/src/runner.rs:83
  30:     0x5584264c033b - ezgui::runner::loop_forever::hc8ef5978c7e57a29
                               at /home/brian/tools/abstreet/ezgui/src/runner.rs:402
  31:     0x5584264c2a9e - ezgui::runner::run::hb6bb507e4722275c
                               at /home/brian/tools/abstreet/ezgui/src/runner.rs:302
  32:     0x558426571766 - game::main::hfae87231e2072153
                               at game/src/main.rs:51
  33:     0x55842665d900 - std::rt::lang_start::{{closure}}::h138df2120edec7d1
                               at /usr/src/rustc-1.37.0/src/libstd/rt.rs:64
  34:     0x5584287ba9a3 - std::panicking::try::do_call::hc3f3f274ffe6f0cd
  35:     0x5584287bf00a - __rust_maybe_catch_panic
  36:     0x5584287a96c9 - std::rt::lang_start_internal::hb1dc547617272b6b
  37:     0x55842665d8d9 - std::rt::lang_start::h4d1b110bcc73c83e
                               at /usr/src/rustc-1.37.0/src/libstd/rt.rs:64
  38:     0x55842657191a - main
  39:     0x7f42b8aa5b97 - __libc_start_main
  40:     0x55842635172a - _start
  41:                0x0 - <unknown>
thread panicked while panicking. aborting.
[1]    6395 illegal hardware instruction (core dumped)  cargo run

Ah, those nodes are from e4a34306b0dd956d96f08a32e99210afec45d50c. Since you're running from source, I think when you ran data/grab_seed_data.sh, you got an old set of maps. I updated the data there about an hour ago. If you grab_seed_data.sh and run again, everything should be fine.
Prevention in the future: I need to make sure to upload new seed data as soon as something changes. I'll look into changing the map building pipeline to put checksums of files under version control, so it's very obvious when I need to upload again.

#25 and #26 were the same issue and were fixed by grabbing the new map data, so closing this bug too. I'll make sure to keep the data updated for people building from source. Let me know if you hit any other problems!  (Still looking into the Linux binary compatibility problem)


Add way to hook into entity despawns

What problem does this solve or what need does it fill?
Sometimes, to speed up a system, I cache data in something like a Local<HashMap<Entity, _>>, and avoid heavier calculations if the cache hasn't changed. Much of this state wouldn't make sense as a separate component, and it feels cleaner to do it as a Local. Unfortunately, there's no apparent way to detect if an entity despawns and clean up these local caches.
What solution would you like?
Maybe some sort of entity despawn event. Maybe, for symmetry, it could be like:
enum EntityLifecycle {
    Spawned(Entity),
    Despawned(Entity),
    DespawnedRecursive(Entity), // Or not
}

What alternative(s) have you considered?
I could check whether components used to calculate the cache were removed, but then I'd need to either rerun the system in post-update, or only have a single copy there. That isn't always possible.
I could also externalize the cache, but that feels dirty as it pollutes the module namespace with a) state only relevant to one system and b) a secondary system to clean that state up.
Finally, I could create my own events, but this requires me to manually throw those events whenever I despawn. Given that a) this seems like a general problem and b) Bevy itself knows when it is despawning something, it probably makes more sense for the engine to notify that the entity was despawned.

Have you considered accepting that the cache will only be cleared on the next frame, and thus at the beginning of your system querying components that were removed to clear the cache when necessary?

How do you figure?
If I have Query<(Entity, &Coordinates)>, cache: Local<HashMap<Entity,  (i32, i32)>>, then add my entity to that cache, why would it only
persist for the frame? Besides, the whole point of these caches is to
persist entities across frames, and not react on fractional coordinate
changes where it doesn't make sense.

Oh, never mind, I see what you mean now. Apologies.
I guess I could clean the cache manually, sure. Thanks, that's a good
workaround for now. Though I still think Bevy should generate some sort
of lifecycle event on despawn.

I am glad this can be a good workaround. Could you elaborate on how iterating over those despawn events would differ from iterating over removed components?

My understanding is that iterating over removed components can only be
done in post-update now. So if you have a system with a Local
containing a map of entities to some cached calculated data, you'd have
to run that system once in post-update to catch the removals. That seems
like the kind of footgun that's easy to miss, particularly in cache
cleanup situations where the only indication you'd have looks a lot like
a leak.

My apologies, I was not aware of the post-update restriction. My suggested solution thus does not apply.

Ah, then I guess I misunderstood your suggestion the second time when I
thought I understood it. :)
What I thought you were suggesting was that I iterate through all cached
members and check them against one of my queries via get, removing
them manually if I get an Err. I think that's an OK temporary
workaround, but I still want to advocate for the far more expressive
EventReader<EntityLifecycle> which, IMO, makes it far clearer that I'm
asking the ECS for entity updates and performing some logic based on those.

My understanding is that iterating over removed components can only be
done in post-update now.

This is true, not because there is something special about PostUpdate, but because Commands are applied at the end of Update. If we were to add a new "entity despawn" event, it would still only be available in PostUpdate because commands.despawn(entity) wouldn't be "applied" until the the end of Update.
Some folks are working to make Command application more granular within a stage (we're calling it "stageless" right now), which might solve this problem.
But in the short term, the only way to make "despawns" immediate is to use "exclusive systems" (systems that take a direct &mut World reference and run before normal systems in a stage, after normal systems in a stage and before their commands are applied, or after normal systems in a stage and after their commands are applied).

"Entity Despawn Events"

I do think that this is good thing to add, even though it would have the same limitations the "component removal tracking" has in relationship to Commands. A full entity despawn is semantically different than specific component removals and I imagine there are plenty of legitimate use cases for it. The design for this should probably go through the RFC process as there are a number of ways to implement it (and a number of ways to expose it to the user).

If we were to add a new "entity despawn" event, it would still only be available in PostUpdate

There is a big limitation of RemovedComponents that an event wouldn't have: the RemovedComponents is cleared at the end of the frame while the event can be read after.
So yay for EntityLifecycleEvent!

To be clear, I don't care if the removal is delayed a frame and inherits
event limitations. I'm just looking for a way to clean up Local caches
in a way that doesn't obfuscate my intent, or require me to run a system
multiple times just to catch Removed.

Yeah Events are probably the right call. Something to consider for the design: a global EntityLifecycleEvent for all entities might end up being expensive to consume in some cases because you need to iterate over all entities across all contexts. Sometimes considering them globally will be necessary, but it would be interesting to consider ways to "filter down" events, maybe by storing them per-archetype? Could they be queryable?

This probably can be considered closed with component hooks as a MVP. I'd like to see easier access to the inserted or removed components to make this a bit cleaner to use though.


Fixed markdown include

Description
Fixed the include in the docs.
What's included?

Docs

Test Steps

[ ] View docs

General Tests for Every PR

[ ] npm run serve:prod still works.
[ ] npm run tslint passes.
[ ] npm run stylelint passes.
[ ] npm test passes and code coverage is not lower.
[ ] npm run build:lib still works.

Screenshots or link to StackBlitz/Plunker

Oh wow, good catch! thanks!


not support Apple M1 ?

Please make sure your issue is not addressed in the FAQ.
Please include the following information:

[x] 1.0.0.
[x] macOS 11.2
[x] infer -- javac Hello.java
[ ] The full output in a paste, for instance a gist.
[ ] If possible, a minimal example to reproduce your problem (for instance, some code where
infer reports incorrectly, together with the way you run infer to reproduce the incorrect
report).

output
MAC OS X: Failed pinning to cpu 0, trying 0/2
MAC OS X: Failed pinning to cpu 0, trying 0/2
MAC OS X: Failed pinning to cpu 0, trying 0/2
MAC OS X: Failed pinning to cpu 0, trying 0/2
MAC OS X: Failed pinning to cpu 0, trying 0/2
.....

We just upgraded to OCaml 4.12.0, which supports the M1's architecture. Does building from master resolve the issue?
Your logs seem to be this message from the parmap library, and indeed there seems to be an infinite loop if pinning fails for cpu 0 so maybe that library needs to add M1 support too.

@jvillard I ran into a problem that I think is related to Apple M1 as well.
When I am trying to build Infer (using build-infer.sh) I encounters with the following (and the build fails):
[2262/3757] Building C object projects/openmp/runtime/src/CMakeFiles/omp.dir/z_Linux_asm.S.o
FAILED: projects/openmp/runtime/src/CMakeFiles/omp.dir/z_Linux_asm.S.o
/Library/Developer/CommandLineTools/usr/bin/clang -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Domp_EXPORTS -I/var/folders/rn/sjvwnyfd72vbr786pf8c1msh0000gn/T/clang-setup.jVEkFI/build/projects/openmp/runtime/src -I/Users/harmim/Projects/infer/facebook-clang-plugins/clang/src/download/llvm-project/openmp/runtime/src -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/usr/include/libxml2 -I/var/folders/rn/sjvwnyfd72vbr786pf8c1msh0000gn/T/clang-setup.jVEkFI/build/include -I/Users/harmim/Projects/infer/facebook-clang-plugins/clang/src/download/llvm-project/llvm/include -I/Users/harmim/Projects/infer/facebook-clang-plugins/clang/src/download/llvm-project/openmp/runtime/src/i18n -I/Users/harmim/Projects/infer/facebook-clang-plugins/clang/src/download/llvm-project/openmp/runtime/src/include -I/Users/harmim/Projects/infer/facebook-clang-plugins/clang/src/download/llvm-project/openmp/runtime/src/thirdparty/ittnotify -g -O2 -fPIC -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -Wall -Wcast-qual -Wformat-pedantic -Wimplicit-fallthrough -Wsign-compare -Wno-extra -Wno-pedantic -O3 -DNDEBUG -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -fPIC   -D _GNU_SOURCE -D _REENTRANT -x assembler-with-cpp -MD -MT projects/openmp/runtime/src/CMakeFiles/omp.dir/z_Linux_asm.S.o -MF projects/openmp/runtime/src/CMakeFiles/omp.dir/z_Linux_asm.S.o.d -o projects/openmp/runtime/src/CMakeFiles/omp.dir/z_Linux_asm.S.o -c /Users/harmim/Projects/infer/facebook-clang-plugins/clang/src/download/llvm-project/openmp/runtime/src/z_Linux_asm.S
/Users/harmim/Projects/infer/facebook-clang-plugins/clang/src/download/llvm-project/openmp/runtime/src/z_Linux_asm.S:1752:5: error: unknown directive
    .size __kmp_unnamed_critical_addr,8
    ^

I have found, e.g., here https://github.com/briansmith/ring/issues/1163, https://github.com/denoland/deno/issues/8346 that this kind of error may be related to  Apple M1.

@harmim Can you verify if this is still a problem?

@martintrojer Unfortunately, I don't build Infer on Macbook any more. Maybe someone else can verify it?

@harmim @martintrojer I installed infer on my M1 mac from source on main branch, and it works well.

This is also working for me with Java, any chance of a release from master so that we don't have to compile clang?

@tsonglew Can you please share the binary?

@tsonglew Can you please share the binary?

infer.zip
here it is

@tsonglew, Thanks! I ended up building from source.

If anyone is interested, we'll be maintaining current and upcoming infer binaries for Apple Silicon on Instabug/infer.

If anyone is interested, we'll be maintaining current and upcoming infer binaries for Apple Silicon on Instabug/infer.

@mhashim6
Hello,I use this release infer-osx-v1.1.0.tar.xz,bug it return an error
dyld[25730]: Library not loaded: @rpath/libgmp.10.dylibem/RenderNodes/LOTStrokeRenderer.m   Referenced from: <44DF3598-DAC5-3E2D-B840-4E731D6D4FF3> /opt/infer-osx-v1.1.0/lib/wrappers/clang   Reason: tried: '/opt/infer-osx-v1.1.0/lib/libso/libgmp.10.dylib' (no such file), '/opt/infer-osx-v1.1.0/lib/libso/libgmp.10.dylib' (no such file), '/usr/local/lib/libgmp.10.dylib' (no such file), '/usr/lib/libgmp.10.dylib' (no such file, not in dyld cache)bgmp.10.dylib' (no such file)
I alse build infer with this repository main branch,but alse get error
Error message: clang-15: error: unknown argument: '-ivfsstatcache' Infer needs a working compilation command to run.
Is there any idea for me? Thank you.

If anyone is interested, we'll be maintaining current and upcoming infer binaries for Apple Silicon on Instabug/infer.

@mhashim6 Hello,I use this release infer-osx-v1.1.0.tar.xz,bug it return an error dyld[25730]: Library not loaded: @rpath/libgmp.10.dylibem/RenderNodes/LOTStrokeRenderer.m Referenced from: <44DF3598-DAC5-3E2D-B840-4E731D6D4FF3> /opt/infer-osx-v1.1.0/lib/wrappers/clang Reason: tried: '/opt/infer-osx-v1.1.0/lib/libso/libgmp.10.dylib' (no such file), '/opt/infer-osx-v1.1.0/lib/libso/libgmp.10.dylib' (no such file), '/usr/local/lib/libgmp.10.dylib' (no such file), '/usr/lib/libgmp.10.dylib' (no such file, not in dyld cache)bgmp.10.dylib' (no such file) I alse build infer with this repository main branch,but alse get error Error message: clang-15: error: unknown argument: '-ivfsstatcache' Infer needs a working compilation command to run.  Is there any idea for me? Thank you.

Is it solved?

@pastel001 Have you resolve the "libgmp.10" problem?

I installed libgmp with brew install gmp and was able to run the macOS Infer 1.2.0 binary successfully.


PruneCluster and  Angular CLI

Hi, is there any way to integrate plunecluster with Angular-CLI?.

I am currently looking for a way to achieve this also.
Please @lexsalg  did you eventually found a way around it?


IsHttps returning false and it should be true

Created a default ASP.NET Core Web Api project using VS 2017 (15.2 (26430.6) Enterprise.
Modified the Get action in the ValuesController from the default template:
        // GET api/values
        [HttpGet]
        public IEnumerable<string> Get()
        {
            return new string[] {   "value1",
                                    "value2",
                                    $"scheme:{Request.Scheme}",
                                    $"IsHttps:{Request.IsHttps}"
            };
        }

I enable SSL in the Debug settings
{
  "iisSettings": {
    "windowsAuthentication": false,
    "anonymousAuthentication": true,
    "iisExpress": {
      "applicationUrl": "http://localhost:60915/",
      "sslPort": 44391
    }
  },
  "profiles": {
    "IIS Express": {
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "https://localhost:44391/api/values",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      }
    },
    "WebApplication4": {
      "commandName": "Project",
      "launchBrowser": true,
      "launchUrl": "api/values",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      },
      "applicationUrl": "http://localhost:60916"
    }
  }
}

Run the app and get this:
["value1","value2","scheme:https","IsHttps:True"]
I publish the app and deploy to an on-premise IIS 8.5 server, configure with the latest 1.1.2 runtime and 1.1.0 Windows Server Hosting module.

I configure the site to use SSL and the request returns values using https:// in the URL, but the returned values from the controller show false for the IsHttps and http for the scheme.

Test example is here.

@Tratcher - ideas? What repo should this go to?

BasicMiddleware. It's likely an issue with the UseForwardedHeaders settings / headers.
Can you dump your request headers?

Try this:
            services.Configure<ForwardedHeadersOptions>(options =>
            {
                 options.RequireHeaderSymmetry = false;
            });

This is now the default in 2.0.

Yup, that was the fix.  Question: is this something I should have been aware of?  Should I have known that I had to to do this based on our server config? Because quite honestly, I haven't got a clue as to how or why this fixes those properties (schema and isHttps) on HttpRequest.  Is there anything in the docs about this?

Nevermind.  I found this stack overflow which led me to this github issue and then ultimately this one which explains this issue and it's history.  Thanks for the help guys!


fix the time of copyright

Ⅰ. Describe what this PR does
this pr is to fix the time of copyright
Ⅱ. Does this pull request fix one issue?

fixes #XXXX
Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews

you should add -s when you use command "git commit". please modify the pr


Get the data of the group assigned members to the application.

Background

When assigned in a group, the correct data was not being captured.
The application encountered performance errors due to usage in over 200 environments.

What did it

Limitation of parallelism by concurrency
Acquisition of groups
Group and user caching
Improvement of error handling

Performance result
As a premise

Over 200 apps
over 3,500 users
over 1,900 groups

Initially ( without Group Fetch ) 15min
2nd ( with Group ) 1h50min
3rd ( with Cache ) 15min 🌟


Spider: Clayton Plan Commission And Architectural Review Board

URL: https://www.claytonmo.gov/government/boards-and-commissions/plan-commission-and-architectural-review-board
Calendar URL: https://www.claytonmo.gov/calendar/meetings/-seldept-8/-toggle-all
Spider Name: clay_plan_arb
Agency Name: Clayton Plan Commission And Architectural Review Board
See the contribution guide for information on how to get started

I can take this one

Just to clarify, should I plan to add links to meeting minutes/transcripts from past meetings? This department is not always consistent about uploading meeting minutes, and the last time they did so was for a meeting in June (over 2 months ago).

We scrape the meeting materials' links whenever possible. If they're not available, you can just leave it as an empty list [].


iOS 5 not working

Not work in Ipad 1
Mozilla/5.0 (iPad; CPU OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3

Add the gist requestAnimationFrame polyfill fix this.
https://gist.github.com/paulirish/1579671

I've added a polyfill which should fix this.
Let me know if there still is problems (don't have a iOS 5 device to test with)


Drop support for Swift 4.2

Swift 5.0 was released over two years ago, and Apple now requires apps submitted to the App Store to be built with Xcode 12, so it is extremely unlikely anyone using OneTimePassword still needs to compile it with Swift 4.2.
This PR also updates the Travis CI config to build with modern tools.

The Codecov failure is due to loss of coverage on a code path that is only exercised when running tests on a build from Xcode pre-11. This will be removed in a future PR.


[REQ] Add option to open FFDT if it enabled on current tab

Hi,
Would be great if you add this ability to your add-on.
Thank you

I like this idea, will try to implement if I'll have time.

R.I.P


Fix scalar divide-by-zero panics

There was a gap where BigUint::zero() / 0u32 wouldn't actually trigger
an expected panic, since there were no digits, so it just returned 0.

bors r+


[Action Recognition] Cannot achieve inference speed as mentioned

After days working on OpenVino conversion, I manage to convert VTN models to IR. However, I failed to reproduce the inference speed that you mentioned in the paper.
More specifically, MobilenetV2-VTN got ~25 fps, Resnet got ~4fps.
My inference code works as following:

Gather the the rgb_clip (for instance seq_len=16)
Add it to the whole model (Encoder + Decoder). It is when I compute the speed

Can you give any hints here?

I follow this workflow and I think it is much faster
Resnet34: 35fps
MobilenetV2: 126fps
https://github.com/opencv/open_model_zoo/tree/master/demos/python_demos/action_recognition


Is it possible to use  "mark.js" with "csv-to- html-table" ?

Browser : Edge  100.0.1185.44 IE mode
mark.js v6.2
Hello
I'm a beginner of JavaScript and html
It is a first time to ask a question on git-hub site
"mark.js" is a very good function
Is it possible to use  "mark.js" with "csv-to- html-table" ?
"csv-to- html-table" is to initialize datatables by the csv file (refer to URL : http://derekeder.github.io/csv-to-html-table/ )
It will be very useful to to have both functions work together
I made a following html and it works well on my desktop of the PC under the internet access environment though has to be allowed to active control
Only "csv-to- html-table" works if under no internet access environment (stand alone PC)
However it does not work on the company network server and got a warning messsage
"DataTables warning: table id=table-container-table - Cannot reinitialise DataTable.
For more information about this error, please see http://datatables.net/tn/3"
Datatables option "destroy" is used  to reinitialise the table
Please let me know if I missed anything ,anything is incorrect  or any suggestion

<link href="css/bootstrap.min.css" rel="stylesheet">
<link href="css/dataTables.bootstrap.css" rel="stylesheet">


ENH: expose raw feature categories in shap.plots.bar

Problem Description
The code snippet below will first generate an explainer class and then obtain an Explanation object that if sliced by one row of data and passed to  shap.plots.bar will generate a local bar plot for one prediction:
explainer = shap.TreeExplainer(model, X)
explanation = explainer(X)
shap.plots.bar(explanation[0])

So far so good but the result, in a general case, will produce a completely unexplainable plot without references to real feature names and values :

The feature names can be modified by assigning values to the feature names attribute of the Explanation object:
explanation.feature_names and this will be displayed correctly on the plot.
However, I see no option or possibility to display the feature value in a friendly way that could be meaningful for people that need not be familiar with the encodings and/or feature preparations that might be happening to the X array.
Note, as a an example, other plots have indeed a display override option through the parameter features which allows to pass a display feature values matrix and greatly improved the interpretability of the graph:
shap_values = explainer.shap_values(X)
shap.plots.force(explainer.expected_value, shap_values[0], features=X_display.iloc[0])

Alternative Solutions
No response
Additional Context
No response
Feature request checklist

[X] I have checked the issue tracker for duplicate issues.
[ ] I'd be interested in making a PR to implement this feature

There is indeed this functionality by assigning the attribute 'display_data' of the Explanation class as explained in https://github.com/shap/shap/issues/2498.
Watch out that no nan or None types are present in the array since this will lead to errors. The line below worked for me:
explanation.display_data = X_display.fillna('null').values
My only suggestion would be to improve the docs!


Animation: Jump

Jump up, flip and drop down.

A simple jump is a position translate with autoreverse
let animation = CABasicAnimation(keyPath: "position.y") // or CASpringAnimation(...
animation.autoreverses = true

a flip is already implemented but we could implement one with  old api if necessarry
        if axis == .x {
            transform3D = CATransform3DRotate(self.layer.transform, CGFloat(degreeToRadians(180)), 0.0, 1.0, 0.0)
        } else {
            transform3D = CATransform3DRotate(self.layer.transform, CGFloat(degreeToRadians(180)), 1.0, 0.0, 0.0)
        }
let animation = CABasicAnimation(keyPath: "transform")
animation.repeatCount = 2
animation.toValue =  NSValue(caTransform3D: transform3D)

Yes, if we have the new type for jump, then we can use it in IB. Otherwise, your solution is very straightforward when using in code 👍
I think the CAAnimation API is better for flip , please change it if you think it is better, thanks.


cartItem is not always an object

Convert cartItem when it's an array, for instance when coming from API :
https://github.com/tastyigniter/ti-ext-api/blob/master/apiresources/Orders.php#L56
Also related to this bug: https://github.com/tastyigniter/ti-ext-api/issues/15

I've applied a fix here to deal with it on the API side, so the change to core should not be necessary. https://github.com/tastyigniter/ti-ext-api/pull/16/commits/b87a0b2a8d1a62b8248213bf5e9937c6eaa27fbb

You can close this... lets keep the changes within the API extension. 👍


Fix ecosytem config

Related issues
closes #3842
Which environment this relates to

[x] Test version (https://test.storefrontcloud.io) - this is a new feature or improvement for Vue Storefront. I've created branch from develop branch and want to merge it back to develop
[x] RC version (https://next.storefrontcloud.io) - this is a stabilisation fix for Release Candidate of Vue Storefront. I've created branch from release branch and want to merge it back to release
[ ] Stable version (https://demo.storefrontcloud.io) - this is an important fix for current stable version. I've created branch from hotfix or master branch and want to merge it back to hotfix

Upgrade Notes and Changelog

[x] No upgrade steps required (100% backward compatibility and no breaking changes)
[ ] I've updated the Upgrade notes and Changelog on how to port existing VS sites with this new feature

Contribution and currently important rules acceptance

[x] I read and followed contribution rules

@andrzejewsky Shouldn't you add args instead of replacing script?

@andrzejewsky Shouldn't you add args instead of replacing script?

This. Merged fix still gives me the same error. Adding args instead of replacing script solved the issue:
{
  "apps": [
    {
      "name": "server",
      "max_memory_restart": "1G",
      "instances": "4",
      "exec_mode": "cluster",
      "env": {
        "TS_NODE_PROJECT": "tsconfig-build.json",
        "NODE_ENV": "production"
      },
      "interpreter": "./node_modules/.bin/ts-node",
      "script": "./core/scripts/server.ts",
      "args": "-P ./tsconfig-build.json",
      "node_args": "--max_old_space_size=1024",
      "log_date_format": "YYYY-MM-DD HH:mm:ss",
      "ignore_watch": [
        "core/build/config.json",
        "node_modules"
      ]
    }
  ]
}

Ok so maybe it's worth to check, thanks @kristofers-magebit

@andrzejewsky I'm receiving the same error using the released version of v1.11.0 from the master branch.

@themreza Hey, please try with ./node_modules/.bin/pm2 install typescript and pm2 install typescript

@andrzejewsky Thank you. I can confirm it works fine.

@andrzejewsky ofc we are to fix it once and for all, right?:)

@pkarw actually it's not a bug in the VSF, it's the problem with pm2 - sometimes there is no TS used by pm2 that's why you have to install it - I think I can add a postinstall script that will run mentioned command

Ok, I have thought on this kind of solution actually!

@andrzejewsky It's weird. After restarting the server the same error has reappeared.
Running the pm2 install typescript doesn't seem to fix it anymore.
Using yarn start:inspect works fine, but not yarn start.
By the way, PM2's website suggests not to use this method. Any other workarounds?

We highly don’t recommend to use this in production as it slows down your app. In that case, your app must be bundled i.e. transpiled from the source to get a pre-processed version of your app.
https://doc.pm2.io/en/runtime/integration/transpilers/

@themreza  In that case, I think we need to bundle server.ts and run it without TS that will be the best solution - I'll create an issue for that


Adding custom links in header

I see that the project already has a /about link, which is not linked anywhere directly in the page (or maybe I did not find it?)
Is there an official way to have custom links in the header?
Awesome theme by the way, thank you :smile:

I have also noticed that. Personally would like it at least in the footer

You'll have to override the header.html file. So, copy themes/novela/layouts/partials/header.html to layouts/partials/ (you will have to make this directory if it doesn't exist).
Then, modify this file to add additional links.
For instance, here is what the nav-controls div in my header currently looks like:

      <button id="aboutButton" class="icon-wrapper" type="button" onclick="location.href='/about'">
        {{ partial "icons/ui/about.html" . }}
      </button>

      <button id="blogButton" class="icon-wrapper" type="button" onclick="location.href='/post'">
        {{ partial "icons/ui/blog.html" . }}
      </button>

      <button id="githubButton" class="icon-wrapper" type="button" onclick="location.href='https://github.com/daniellivingston'">
        {{ partial "icons/ui/github-header.html" . }}
      </button>

      <button id="copyButton" class="icon-wrapper">
        {{ partial "icons/ui/link.html" . }}

            copied

        <input id="copyText" style="opacity: 0;" type="text" class="tool-tip" />
      </button>

      <button id="themeColorButton" class="icon-wrapper">

      </button>


Plan how to remove expired deploys and emit event to SSE server

9.5.2023 no longer necessary as we have reduced the size of the auction on disk in 2.0


Small? Error in post_full

Trying to scrape: https://www.instagram.com/explore/tags/satoyamaexperience/
python crawler.py posts_full -t satoyamaexperience -n 1000 -o ./output

Error
Traceback (most recent call last):
  File "crawler.py", line 77, in <module>
    args.debug
  File "crawler.py", line 25, in get_posts_by_user
    return ins_crawler.get_user_posts(username, number, detail)
  File "/Users/philipperemy/PycharmProjects/instagram-crawler/inscrawler/crawler.py", line 100, in get_user_posts
    user_profile = self.get_user_profile(username)
  File "/Users/philipperemy/PycharmProjects/instagram-crawler/inscrawler/crawler.py", line 89, in get_user_profile
    post_num, follower_num, following_num = statistics
ValueError: not enough values to unpack (expected 3, got 0)

Thanks a million in advance :)

@huaying Let me know if you need more info! Btw this repo looks pretty solid!

Actually running this works:
python crawler.py hashtag -t satoyamaexperience -n 1000 -o ./output

Yes, -t only works in hashtag mode  :)


Newest list has showing recent listing price instead old one?

I've bought NFT for 10000 BSX
List it for 333333 BSX
Noticed on landing it's shows at newest list, recent price on second card, where should show old one - 10000 BSX

Case
https://beta.kodadot.xyz/bsx/gallery/1643292083-2

👋

So it should show the latest trade price instead of listing price? What about the fresh mint nft without trade history?


Changing to microservices Architecture in docker

There might be some problems with regard to imports since some of the structure has changed.
The microservices are all in the same port. They are connected to each other through networks (see docker-compose.yml). Any microservices on the same network can reach out to each other using their container name. So all the addresses in localhost need to be changed (the backend is not accessible from the localhost anymore- this also adds to the security of the application).

Good work!


[Merged by Bors] - feat(RingTheory/StandardSmooth): ring isomorphisms are standard smooth

We show that any R-algebra S where algebraMap R S is bijective, is standard smooth by constructing an explicit submersive presentation.
This contribution was created as part of the AIM workshop "Formalizing algebraic geometry"
in June 2024.

Thanks!
bors merge

Thanks for the reviews!


Multiple LwjglAWTCanvas are not rendered simultaneously

Issue details
I have three LwjglAWTCanvas objects, each containing a copy of my game. I added them to a JFrame and want to render them simultaneously. The three canvases are, however, not all rendered "each frame", but rather after another in separate frames. For example:

canvas1 is rendered on frame 1
canvas2 is rendered on frame 2
canvas3 is rendered on frame 3
canvas1 is rendered on frame 4
etc.

What should happen is that all three canvases are rendered each frame. Instead, the framerate of each game is effectively divided by three. When I hide one or two of the canvases through setVisible(), the framerate of the other canvases increases correspondingly.
Reproduction steps/code
Download example project (contains all dependencies)
The code I used is listed below.
package test;

import com.badlogic.gdx.graphics.Color;

import javax.swing.*;
import java.awt.*;

public class Main extends JFrame {

    public static void main(String[] args) {
        new Main();
    }

    public Main() {
        setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        Container container = getContentPane();
        container.setLayout(new BorderLayout());

        var layout = new BoxLayout(container, BoxLayout.X_AXIS);
        container.setLayout(layout);

        var game1 = new Game(Color.BLUE);
        var canvas1 = game1.createCanvas(null);
        container.add(canvas1.getCanvas());

        JButton button = new JButton("Button");
        container.add(button);

        var game2 = new Game(Color.RED);
        var canvas2 = game2.createCanvas(canvas1);
        container.add(canvas2.getCanvas());

        JButton button2 = new JButton("Button");
        container.add(button2);

        var game3 = new Game(Color.GREEN);
        var canvas3 = game3.createCanvas(canvas1);
        container.add(canvas3.getCanvas());

        button.addActionListener(e -> canvas2.getCanvas().setVisible(!canvas2.getCanvas().isVisible()));
        button2.addActionListener(e -> canvas3.getCanvas().setVisible(!canvas3.getCanvas().isVisible()));

        pack();
        setSize(800, 600);
        setVisible(true);
    }
}

package test;

import com.badlogic.gdx.ApplicationAdapter;
import com.badlogic.gdx.Gdx;
import com.badlogic.gdx.backends.lwjgl.LwjglAWTCanvas;
import com.badlogic.gdx.backends.lwjgl.LwjglApplicationConfiguration;
import com.badlogic.gdx.graphics.Color;
import com.badlogic.gdx.graphics.GL20;
import com.badlogic.gdx.graphics.glutils.ShapeRenderer;

import java.util.Random;

public class Game extends ApplicationAdapter {
    private final Color color;
    private final LwjglApplicationConfiguration config;
    private ShapeRenderer renderer;
    private int x;
    private boolean reverse;

    public Game(Color color) {
        this.color = color;
        config = new LwjglApplicationConfiguration();
        config.vSyncEnabled = true;
        config.foregroundFPS = 60;
        config.backgroundFPS = 60;
    }

    @Override
    public void create() {
        super.create();

        renderer = new ShapeRenderer();
        renderer.setAutoShapeType(true);
        Random random = new Random();
        x = 1 + random.nextInt(39);
    }

    @Override
    public void render() {
        super.render();

        Gdx.gl.glClearColor(1, 1, 1, 1);
        Gdx.gl.glClear(GL20.GL_COLOR_BUFFER_BIT);

        renderer.begin(ShapeRenderer.ShapeType.Filled);
        renderer.setColor(color);
        renderer.rect(20 + x, 20, 20, 20);
        renderer.end();

        if (x == 40 || x == 0) reverse = !reverse;
        if (reverse) x--;
        else x++;
    }

    public LwjglAWTCanvas createCanvas(LwjglAWTCanvas sharedCanvas) {
        return new LwjglAWTCanvas(this, config, sharedCanvas);
    }
}

Version of LibGDX and/or relevant dependencies
libGDX 1.9.14 with the LWJGL2 backend
Please select the affected platforms
I only tested it on Windows.

[ ] Android
[ ] iOS
[ ] HTML/GWT
[X] Windows
[ ] Linux
[ ] MacOS

I'm not sure very many people are using LwjglAWTCanvas. You may need to dig into what is going on yourself, sorry.


[Fixed] Allowing malicious code injection #34

Previously the code execution by the spawn process has the root permission so there was threat that anyone can modify the file on the server. Now i have fixed this issue by spawning the process with non root process so it will avoid any kind of changes like removing files, applications, adding file or modifying the existed files.

Addresses #42 (but read suggestions too)


{Misc.} Fix use-maxsplit-arg pylint error

Description
Finish the unfinished tasks of https://github.com/Azure/azure-cli/pull/20192
Fix use-maxsplit-arg pylint error.

Thank you for your contribution! We will review the pull request and get back to you soon.


Add ability to specify a value of a part of version only

Similar to --new_version=X.X.X, it can be --new_version-part=X parameter that tells the value to use as the part of the version. Other than specified part values will be taken from the current version.
# Say current_version=0.1.0
bump2version --new-version-part 3 patch
# will set 0.1.3 version value, where major and minor are taken from the current version

Right now I need to write bash script to handle that.

Closing as duplicate of https://github.com/peritus/bumpversion/issues/50


Persona 5 Strikers PC Assets Decryption

This PR adds support for P5S PC asset decryption.
The decryption algorithms were reversed from the executable and later simplified to readable code (I hope 😳).
Encryption is also implemented, but not used at the moment.
LINKDATA Decryption
P5S PC LINKDATA.BIN entries can either be compressed or encrypted, but never both at the same time.
The encryption algorithm is a form of Mersenne twister, and the key is the index of the entry in LINKDATA.IDX, plus a constant.
LINKDATA.IDX itself is located in the executable.

To decrypt:
Cethlean.DataExporter.exe --linkdata --game "P5SPC" <out_dir> <in_dir>

Example - first entry in LINKDATA.IDX:

RDB Decryption
P5S PC RDB ZLib compressed entries are always encrypted.
Conversely, non-compressed RDB entries are never encrypted.
The encryption algorithm uses XORShift32, and the key is the entry's KTID.

To decrypt:
Cethlean.DataExporter.exe --rdb --filelist filelist-Scramble-rdb.csv <out_dir> <in_dir>

Example - an encrypted shader from system.rdb:

SRST Decryption
P5S PC external .files with an SRST header can be encrypted.
To determine whether a file is encrypted, the game checks offset 0x20 in the KTSR header which immediately follows the 0x10 byte SRST header.
The byte value stored in that offset is the encryption key length, with the key itself following from the next byte - if the value is 0, no decryption is performed.
Only the audio files themselves (KOVS header) are encrypted. The SRST, KTSR and KOVS wrapper headers remain unencrypted, and are used by the game to determine what needs to be decrypted.
The encryption algorithm is Blowfish in ECB mode.
The game decrypts SRST file contents on-the-fly, during stream playback.

To decrypt:
Cethlean.DataExporter.exe --rdb --filelist filelist-Scramble-rdb.csv <out_dir> <in_dir>
All encrypted SRST external files in P5S PC are referenced in RRPreview.rdb.

Example - from the BGM .srst:

Thanks


Add gperftools as a submodule

Description
gperftools were historically used for profiling back when the collector image was still based on ubuntu. Unfortunately, UBI repositories don't provide a precompiled version of gperftools, but nothing prevents us from building it ourselves.
Checklist

[x] Investigated and inspected CI test results
[ ] Updated documentation accordingly

Testing Performed
Manually tested the gperftools API endpoints and the generated dumps.

It doesn't look like P/Z are supported by gperftools, is there any risk as is? should we disable on those archs?

It doesn't look like P/Z are supported by gperftools, is there any risk as is? should we disable on those archs?

Good point, I think we can disable them until we have some time to test them.

Doing a lightweight perf evaluation shows that gperftools idle presence at
runtime doesn't change things much, so we can assume it has small resources
footprint when not actively profiling.
Here is the graph showing CPU usage under a moderate berserker load (arrival:
10, departure: 200, random: true) with without gperf at all, with gperf idling,
and with gperf actively profiling the heap. The raw data is shown with
transparency, and the rolling median is plotted over it to show the smoother
trend. While without gperf & with idling gperf CPU consumption is essentially
the same, of course when profiling we've got visible overhead.

The next one is an event propagation latency (i.e. time between a syscall was
fired in the system, and a corresponding event was received by Collector) under
the same load, with and without gperf. Except one outlier, they're again
essentially the same.

To summarize, there seems to be no reasons to think gperftools enabling can
have a significant impact on the performance, unless actively profiling.


Fix HuberLoss.forward avoid loss of significance

Jenkins, test this please.

Jenkins CI test (for commit 81b139d24cbf174ce3d40309714828374de27834, target branch master) succeeded!


Release snippet slightly incorrect

I needed to use snapshots_deps instead of snapshots_dependencies.
http_archive(
    name = "com_cognitedata_bazel_snapshots",
    sha256 = "a756658bf1425eab7b9104c979ef7af885df3832db38469e200ce7433a7f2570",
    url = "https://github.com/cognitedata/bazel-snapshots/releases/download/0.7.6/snapshots-0.7.6.tar",
)

load("@com_cognitedata_bazel_snapshots//snapshots:dependencies.bzl", "snapshots_deps")

snapshots_deps()

load("@com_cognitedata_bazel_snapshots//snapshots:repositories.bzl", "snapshots_register_toolchains")

snapshots_register_toolchains()

Thanks!

But unfortunately, even with that snippet, it doesn't work for me but results in the following error:
**❯ bazel run snapshots
INFO: Streaming build results to: https://app.buildbuddy.io/invocation/0e608bfc-b43a-4980-b2ab-047ba4aa0473
ERROR: /home/peter/.cache/bazel/_bazel_peter/8a0e5e440c2a644038228219d25b0c32/external/com_google_cloud_go_storage/BUILD.bazel:5:11: in go_library rule @com_google_cloud_go_storage//:storage: alias '@com_google_cloud_go//internal:go_default_library' referring to target '@com_google_cloud_go//internal:internal' is not visible from target '@com_google_cloud_go_storage//:storage'. Check the visibility declaration of the former target if you think the dependency is legitimate
ERROR: /home/peter/.cache/bazel/_bazel_peter/8a0e5e440c2a644038228219d25b0c32/external/com_google_cloud_go_storage/BUILD.bazel:5:11: in go_library rule @com_google_cloud_go_storage//:storage: alias '@com_google_cloud_go//internal/optional:go_default_library' referring to target '@com_google_cloud_go//internal/optional:optional' is not visible from target '@com_google_cloud_go_storage//:storage'. Check the visibility declaration of the former target if you think the dependency is legitimate
ERROR: /home/peter/.cache/bazel/_bazel_peter/8a0e5e440c2a644038228219d25b0c32/external/com_google_cloud_go_storage/BUILD.bazel:5:11: in go_library rule @com_google_cloud_go_storage//:storage: alias '@com_google_cloud_go//internal/trace:go_default_library' referring to target '@com_google_cloud_go//internal/trace:trace' is not visible from target '@com_google_cloud_go_storage//:storage'. Check the visibility declaration of the former target if you think the dependency is legitimate
ERROR: /home/peter/.cache/bazel/_bazel_peter/8a0e5e440c2a644038228219d25b0c32/external/com_google_cloud_go_storage/BUILD.bazel:5:11: in go_library rule @com_google_cloud_go_storage//:storage: alias '@com_google_cloud_go//internal/version:go_default_library' referring to target '@com_google_cloud_go//internal/version:version' is not visible from target '@com_google_cloud_go_storage//:storage'. Check the visibility declaration of the former target if you think the dependency is legitimate
ERROR: /home/peter/.cache/bazel/_bazel_peter/8a0e5e440c2a644038228219d25b0c32/external/com_google_cloud_go_storage/BUILD.bazel:5:11: Analysis of target '@com_google_cloud_go_storage//:storage' failed
WARNING: errors encountered while analyzing target '//:snapshots': it will not be built
INFO: Analyzed target //:snapshots (331 packages loaded, 12453 targets configured).
INFO: Found 0 targets...
INFO: From BazelWorkspaceStatusAction stable-status.txt:
Getting workspace status...
Getting docker config from /home/peter/.docker/config.json
Found docker config at /home/peter/.docker/config.json
Outputting docker config
ERROR: command succeeded, but not all targets were analyzed
INFO: Elapsed time: 11.967s, Critical Path: 0.04s
INFO: 1 process: 1 internal.
ERROR: Build failed. Not running target
INFO: Streaming build results to: https://app.buildbuddy.io/invocation/0e608bfc-b43a-4980-b2ab-047ba4aa0473
FAILED: Build did NOT complete successfully

The build from source seems to  work:
http_archive(
    name = "com_cognitedata_bazel_snapshots",
    sha256 = "a756658bf1425eab7b9104c979ef7af885df3832db38469e200ce7433a7f2570",
    url = "https://github.com/cognitedata/bazel-snapshots/releases/download/0.7.6/snapshots-0.7.6.tar",
)

load("@com_cognitedata_bazel_snapshots//snapshots:dependencies.bzl", "snapshots_deps")

snapshots_deps()

load("@com_cognitedata_bazel_snapshots//snapshots:repositories.bzl", "snapshots_register_toolchains")

snapshots_register_toolchains(
    name = "snapshots",
    from_source = True,
)```


Ensure mappings used in KG-IDG adhere to SSSOM

Incorporate SSSOM (https://github.com/mapping-commons/sssom) as part of KG-IDG metadata.
@justaddcoffee - you had some ideas regarding what this entails

@caufieldjh thanks for the ping. I think the action items here is "use Drug Central CURIEs for drugs in KG-IDG". (This I think is sensible because as @cmungall said, CHEBI IDs will often make distinctions between chemicals that are not important to us.
For example here's the CHEBI entry for acetylsalicylic acid (aspirin) CHEBI:15365 and here's a separate CHEBI entry for acetylsalicylate CHEBI:13719, which is the same drug except in the form that exists at high pH. For our purposes, these are the same thing. Drug Central has just one entry for aspirin, which is nice for our use case.
SSSOM I think maybe can help us do this. The idea would be to construct tooling to make an SSSOM mapping file for all drugs we ingest that maps any drug ID -> Drug Central ID, and then apply this mapping to make KGX applies the mapping.
Another strategy would be to just perform the mapping manually during each transform of the drug database. We'd just put the drug central CURIE in the node ID column, and put any other CURIEs (CHEBI, or whatever) in the xref column.

Great - this is a dupe of #45 then, didn't know if you had plans for SSSOM beyond drug CURIE mapping

I'm in favor of SSSOM for the sake of reusability (e.g., this probably won't be the last time we'll have CHEBI IDs but want to map to drugs, and for that matter perhaps some mappings for existing CHEBI xrefs like CHEMBL) though it seems like adding the xrefs during the transform may be faster to implement on its own and isn't incompatible with generating and using SSSOM maps.


Thoughts on GPS tracking

GPS tracking could make for a useful addition to this app, but it would likely be controversial.
I'm going to use this thread to discuss how this could possibly be implemented in a sane and useful way.

It would have to be optional

At minimum, you would make use of the tracked locations when submitting that you're infected. This could be used to generate some kind of public heat map so people can check exposures in the case where they may not have a phone or for some other reason.

Some locations would potentially be sensitive, so this could be a secondary workflow that runs after submitting your private keys. Each location would presented individually to its owner so they decide whether or not to submit that one.

This could be useful for narrowing out situations, say, where they're idling in traffic beside other people in a situation where transmission is likely not possible.

This is against the terms of use for ExposureNotification. Closing.


add End-to-end best practices

Some applications use webcomponents to develop
https://open-wc.org/testing/ missing this part.

I think you mean end-to-end testing? Indeed we are missing some recommendations in that field. Do you have any recommendations?


arm: add support for softfp in arm vfp assembly files

This PR adds support for softfp in all arm vfp assembly kernel files.
I have done only basic testing of this PR since I don't have an ARM machine at my disposal. All my testing were done on QEMU emulator.
Request owners of following issues to help test this PR and see whether the issue is resolved or not.
#1088 #1145 #1168 #1206 #1208 #1215 #1145 @misbullah @gangm @odieXin @ctgushiwei  @mariasmo @akira-miasato @gangm

@tomaklutfu  Thanks for reviewing.
Yes. Those were wrong. I agree. Had corrected those in my local workspace. But still the GEMM answer was not correct. I didnt explore it much as these files were no longer required after all the changes were made in the assembly files.
Please see that I have deleted those files in the last commit.

@ashwinyes Thank you for the PR.
Do I need to merge #1216 at first?

No #1216 is not required. It can closed.
I would prefer these changes to get tested by someone working on android and get a "Tested" stamp on it. I have only tested it on QEMU emulator.

We will test it this week

ashwinyes & xianyi,
i am trying to understand this request.
so i have been compiling openblas using the following command:
make TARGET=ARMV7 HOSTCC=gcc CC=arm-linux-androideabi-gcc NOFORTRAN=1 NUM_THREADS=32 libs
I believe it defaults to "hardfloat" right?
my understanding is that the PR removes the "hardfloat" by default? and can i use the same above mentioned command or do i need to use the following:
ARM_SOFTFP_ABI=1
and to test, do can i get the latest from develop branch? or should i wait until the PR is merged?
thanks

Let me explain.
softfp and hard float ABI differs in the way the arguments are passed on to functions and how values are returned. If your project has full C files only, then compiler will take care of everything.
But issue is when you have hand written assembly code. The code in OpenBLAS was always assuming hard float ABI was used. So the code expects values to be certain registers according hard fp ABI which will not be the case when you use softfp ABI.
Now coming to your question. arm-linux-androideabi-gcc (also arm-linux-gnueabi-gcc) by default is softfp ABI. When you use this compiler to compile OpenBLAS code, the issue I mentioned above arises.
The PR doesn't remove anything. It just checks whether you are using a softfp or hard fp compliant and adjusts the arguments/return values accordingly.
So, basically you don't need to give any additional make parameters to OpenBLAS with the PR.
To test, you can use the branch https://github.com/ashwinyes/OpenBLAS/tree/develop_arm_softfp until this PR gets merged.
Hope I cleared your doubt.

ashwinyes
thanks for the explanation.
so prior to your changes OpenBLAS was assuming hard fp even though the default for arm-linux-androideabi-gcc  is softfp. Am i correct in my understanding?
and now with your changes , it will compile for only soffp and any other libraries i compile down the stream i shuould assume softfp and not hardfp, correct?
thanks

Yes. You are correct.

thanks

I'll be checking it today regarding #1168 and #1215.

Tested and issue #1168 seems solved with Android NDK r15b, with speedups comparable to what I achieved with the dirty adaptation from #1215/#1216.
However, some changes regarding GCC/Clang support will be needed (NDK dropped GCC support). I should open a new issue for those things, since this PR focuses on the softfp/hard ABI issues.

"fnmacs s3 , s1, s4" is a valid ARM instruction. If clang doesn't recognize, then its more of a clang issue than OpenBLAS issue. But I will try to see if I can workaround it.

Looks like clang is not recognizing pre-UAL mnemonics. We will have to replace with UAL mnemonics.
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0473c/CIHFAHFG.html

@akira-miasato
Addressed the clang issue in https://github.com/xianyi/OpenBLAS/pull/1226

thanks a lot, India guy

thanks a lot, sure enough, India's software industry has good superiority.


fix(select): [select] Adapt to multiple+disabled+showOnly scenarios w…

…ith different themes
PR
PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our Commit Message Guidelines
[x] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: N/A
What is the new behavior?
Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information

Summary by CodeRabbit

New Features

Introduced additional attributes (:disabled, :required) for selection options, enhancing user control in the tiny-select component.
Added new scenarios to demonstrate various configurations of the tiny-select component.

Improvements

Enhanced tag management with new computed properties for better visibility and closure handling of tags in different states.
Updated styling options for improved visual consistency and spacing in disabled states across select and input components.

Bug Fixes

Streamlined the logic for handling disabled states, tags, and class bindings in the Vue component for clearer user interactions.

Before:

After：


doc: comparison against already existing libs.

how does that compare to existing lib doing that already? just to name few...
https://github.com/microsoft/tsyringe
https://github.com/inversify/InversifyJS

You will now find a feature table for needle here.  https://github.com/morganstanley/needle#feature-support
Associated PR here - Adding feature table with support status
Generally getting into direct comparison with other libraries is not considered healthy for OSS projects.  Happy for community to do their own comparisons however.

Maybe to highlight the limitations you faced using the other DI containers would help.


[Bug] Catch any errors caused by parsing json schema

Motivation
Passing json_schema with types unimplemented by Outlines to the /generate endpoint would crash the scheduler and take down the whole server.
Example:
skip invalid json schema: json_schema={"type":"object","properties":{"id":{"type":"string","format":"uuid"},"name":{"type":"string","minLength":2,"maxLength":50},"age":{"type":"integer","minimum":0,"maximum":120},"email":{"type":"string","format":"email"},"tags":{"type":"array","items":{"type":"string"},"minItems":1,"maxItems":5},"role":{"type":"string","enum":["admin","user","guest"]},"settings":{"type":"object","properties":{"theme":{"type":"string","enum":["light","dark"]},"notifications":{"type":"boolean"}},"required":["theme","notifications"],"additionalProperties":false},"joinedAt":{"type":"string","format":"date"}},"required":["id","name","age","email","tags","role","joinedAt"],"additionalProperties":false,"$schema":"http://json-schema.org/draft-07/schema#"}, e=NotImplementedError('Format email is not supported by Outlines')
"email" is a valid JSON schema format but is not implemented. Using that JSON schema would crash the sglang scheduler.
Modifications
Catches `NotImplemented errors thrown by JSON schema creation
Checklist

[x] Format your code according to the Contributor Guide.
[x] Add unit tests as outlined in the Contributor Guide.
[x] Update documentation as needed, including docstrings or example tutorials.

@zolinthecow It is merged. Thanks for the contribution.


fn2 is no a function

Describe the bug
I'm using remix.run and vitest. In version 0.34.2 my test is working correctly. But in version 0.34.3 same test fails. The error is 'fn2 is not a function'. I understand that the error is in a dependency on remix.run, but it happens when upgrading the vitest version.
Please let me know if I should report this on remix.run.
Reproduction
This work: https://codesandbox.io/p/sandbox/form-vitest-0-34-2-qnq6kr?file=/package.json:32,23
This fail: https://codesandbox.io/p/sandbox/form-vitest-0-34-3-rdylty?file=/package.json:32,23
System Info
System:
    OS: Linux 5.15 Ubuntu 20.04.5 LTS (Focal Fossa)
    CPU: (24) x64 13th Gen Intel(R) Core(TM) i7-13700
    Memory: 29.33 GB / 31.19 GB
    Container: Yes
    Shell: 5.0.17 - /bin/bash
  Binaries:
    Node: 18.16.0 - ~/.nvm/versions/node/v18.16.0/bin/node
    npm: 9.7.1 - ~/.nvm/versions/node/v18.16.0/bin/npm
    pnpm: 8.7.5 - ~/.local/share/pnpm/pnpm
  Browsers:
    Chrome: 116.0.5845.179
  npmPackages:
    @vitejs/plugin-react: ^4.0.4 => 4.0.4
    @vitest/coverage-istanbul: ^0.34.4 => 0.34.4
    @vitest/ui: ^0.34.4 => 0.34.4
    vite: ^4.4.9 => 4.4.9
    vitest: ^0.34.4 => 0.34.4

Used Package Manager
npm
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

Hi @eduardodallmannsysmap
We've encountered the same issue in our team while upgrading vitest to latest on Remix 2.
Looks like the issue is happens from the formData parsing with request.formData().
Since you're on Node 18, you can remove the installGlobals from your test setup file, Node 18 already has fetch, Headers, Request, Response and other needed globals.
You can find all globals in Node 18 here: https://nodejs.org/dist/latest-v18.x/docs/api/globals.html
You can see without the globals the tests are all passing: https://codesandbox.io/p/sandbox/form-vitest-0-34-3-forked-vyh4my?file=%2Fpackage.json%3A32%2C22

Thanks @akamfoad
That was the problem, solved here


Build a Toggle Switch

Description
Build the UI of the Toggle Switch component as shown in the image below
Screenshots

@abhishek-900 you are assigned to another issue.
Please check that out


[BUG] Error ocurred while enabling NVIDIA Persistence Daemon

Describe the bug
when trying to change the graphics to nvidia only, an error is shown to enable a service
To Reproduce
Steps to reproduce the behavior:

Run sudo envycontrol -s nvidia
See error

System Information:

Model: Acer aspire 5
Distro: Fedora 39
Kernel: 6.6.9-200
DE/WM and Display Manager (if applicable): Gnome 45 GDM
EnvyControl version: 3.3.1
Nvidia driver version: 545.29.06

Additional context
Add any other context about the problem here. If possible try to reproduce the problem with --verbose flag and attach its output.
Switching to nvidia mode
Enable ForceCompositionPipeline: False
Enable Coolbits: False
Failed to enable unit: Unit file nvidia-persistenced.service does not exist.
ERROR: An error ocurred while enabling service
INFO: Removed file /etc/X11/xorg.conf
INFO: Removed file /etc/modprobe.d/nvidia.conf
INFO: Found Nvidia GPU at 01:00.0
INFO: Found Intel iGPU
INFO: Created file /etc/X11/xorg.conf
# Automatically generated by EnvyControl

Section "ServerLayout"
    Identifier "layout"
    Screen 0 "nvidia"
    Inactive "intel"
EndSection

Section "Device"
    Identifier "nvidia"
    Driver "nvidia"
    BusID "PCI:1:0:0"
EndSection

Section "Screen"
    Identifier "nvidia"
    Device "nvidia"
    Option "AllowEmptyInitialConfiguration"
EndSection

Section "Device"
    Identifier "intel"
    Driver "modesetting"
EndSection

Section "Screen"
    Identifier "intel"
    Device "intel"
EndSection

INFO: Created file /etc/modprobe.d/nvidia.conf
# Automatically generated by EnvyControl

options nvidia-drm modeset=1
options nvidia NVreg_UsePageAttributeTable=1 NVreg_InitializeSystemMemoryAllocations=0

INFO: Found gdm Display Manager
Rebuilding the initramfs...
Successfully rebuilt the initramfs!
Operation completed successfully
Please reboot your computer for changes to take effect!

Hi @pedroborraz, could you send me the output of:
systemctl --all | grep nvidia

Also I need to know how did you installed the Nvidia drivers (eg: akmod-nvidia from rpmfusion)?
Greetings :)

output:
borraz@fedora:~$ systemctl --all | grep nvidia
  nvidia-fallback.service                                                                   loaded    inactive dead      Fallback to nouveau as nvidia did not load
  nvidia-powerd.service                                                                     loaded    inactive dead      nvidia-powerd service

I installed the drivers from the rpmfusion non free repo, from gnome software.
thanks for the quick attention

Looks like the rpmfusion is missing the nvidia-persistenced dependency.
Try installing it with:
sudo dnf install nvidia-persistenced

Let me know if it works ;)

this worked thank you very much, I will close the issue
love u <3


Wipe badges when new uploaded

there are many badges these days which exist for limited time, due to how the upload system works right now it never removes ones which dont exist. this is kind of a hacky solution but it fixes it


LIBCLOUD-650: ex_create_multi_value_record only returns one record

Fixes that bug and also adds tests for that function.
Bug discovery and solution by @jvrplmlmn

@Kami ?

Sorry for the delay (recovering after surgery).
Good catch and merged into trunk.


Move Documentation website to another documentation engine

Currently the documentation uses Jekyll, and it is pretty weak when it comes to product-grade documentation (macros, external sourcing for docs-as-code, etc., etc.). I think it would be preferable to have another documentation engine, preferably Docusaurus or Hugo. I find Docusaurus much more powerful and sustainable if we must stick to Markdown, but IIUIC @tomakehurst prefers Hugo. If we want to have professional documentation with a lot of future scalability, we should probably adopt AsciiDoc and Antora right away. Subject for a discussion.

Engine Evaluation Notes
[MkDocs PoC] - https://github.com/wiremock/components-site/
Antora PoC - in progress

References

https://docusaurus.io/
https://gohugo.io/
https://antora.org/ - for AsciiDoc hence more professional

TL;DR: This week in my head

Antora's quite pleasant to work with

@SimonVerhoeven Yes, indeed. The biggest problem there is need to have Markdown, and it creates quite a lot of issues with Antora. I have a pending discussion about this on Antora Zulip (here). You can find my evaluation notes here too. If  I do not manage to sort Markdown support in a reasonable way, might end up with MkDocs + Multirepo


num_retries doesn't work with env_var

Hi,
I'm using env_var in my profile.yml file, but when trying to set the num_retires attribute with env_var it fails with the following error:
ERROR: Runtime Error
  Credentials in profile "my_profile", target "dev" invalid: '0' is not valid under any of the given schemas

To set the env var I ran this: export DBT_ATHENA_NUM_RETRIES=0
My profile.yml:
my_profile:
  outputs:
    dev:
      database: "{{ env_var('DBT_ATHENA_DB') }}"
      region_name: "{{ env_var('DBT_REGION_NAME') }}"
      s3_staging_dir: "{{ env_var('DBT_S3_STAGING_DIR') }}"
      schema: "{{ env_var('DBT_ATHENA_SCHEMA') }}"
      type: athena
      num_retries: "{{ env_var('DBT_ATHENA_NUM_RETRIES') }}"
  target: dev

A few notes:

When using the 0 directly it works fine: num_retries: 0
The other attributes are assigned as expected using the env_var

The issue is on how you use the "{{ env_var('DBT_ATHENA_NUM_RETRIES') }}" as it's rendered as a string, could you try
{{ env_var('DBT_ATHENA_NUM_RETRIES') }} without quotes?

Already tried, it throws a syntax error.
I also tried:
export DBT_ATHENA_NUM_RETRIES='0'
and
export DBT_ATHENA_NUM_RETRIES="0"

The issue is on your side, you are passing a string to a value that should be an integer. Try this
env_var('DBT_ATHENA_NUM_RETRIES') | int }}

You are right :)
Thanks!