Lock entity not not getting toggled

I just updated to the latest main commit and I'm having issues with getting the Lock entity toggled. I can't toggle it manually either.
When using the HomeKey it doesn't react. When doing manually it hangs on "Unlocking" in the Home app with the following logs in serial monitor:
I (179621) LockMechanism: update > New LockState=0, Current LockState=1
I (546199) LockMechanism: update > New LockState=1, Current LockState=1
I (549684) LockMechanism: update > New LockState=0, Current LockState=1
I (552368) LockMechanism: update > New LockState=1, Current LockState=1
I (554962) LockMechanism: update > New LockState=0, Current LockState=1

Is this just me?

Hi Iona,
The lock is now entirely MQTT-controlled and HomeKey no longer directly interacts with the state instead it's only publishing the authentication data through mqtt on the MQTT_AUTH_TOPIC topic and you can decide from there what the state should be with the following control topics:

MQTT_SET_STATE_TOPIC
MQTT_SET_TARGET_STATE_TOPIC
MQTT_SET_CURRENT_STATE_TOPIC

In case you are not familiar with how the state in HomeKit is handled, know that you have the "target state" which is where you want to go and the "current state" which is where you currently are that way you can have a more fluid flow of changing the state and better indication of success-failure.
The MQTT_SET_STATE_TOPIC is controllling both the target and the current state.
When interacting with the lock from the Home app, the target state will be published via mqtt to the MQTT_STATE_TOPIC topic. Also, when using HK the toggled state will be published to the same topic which i think i'll remove since it will cause unwanted issues if it's not synced with the state that you want and it would complicates things needlessly to try make it work.
If you are a HASS user, it now integrates with HA's Tags which you can use to create some nice automations.
I'm looking to create a wiki here sometime in the future to detail all the things and perhaps even a discord server if there is the need but right now i'm still focusing on making the code more structured and stable.

I see, thank you for the thorough explanation and clarification!
Was there a specific reason to why the direct interaction with the HK lock state was dropped? I was hoping to have the lock as standalone as possible for reliability. That way I wouldn’t have to rely on a separate mqtt broker.
Having mqtt support is really valuable! However, to me, it would make much more sense to have it control the HK lock state independently and then publish mqtt states if you want that synced with HASS or other systems. 🤔
Iona

I just had some difficulties with theset_current_state and set_target_state in HASS automations, but luckily that's what it needed to enable the Home app entity toggle on HomeKey scan 🥳 .
Relay/GPIO control is indeed much anticipated and preferred in my case since that reduces complexity and risks of failure.  Having it battery powered is also on the list but power outages are not that common here.
My use case is with an indoor retrofit BLE battery powered lock that hopefully I'll be able to  tap into it with a relay/optocoupler in order to trigger the lock/unlock sequences. There's also the cleaner alternative of using the BLE API to control it remotely but that would require a bit more code work.
I really appreciate your work on this. 🙂
Iona

Great, glad you figured it out!
GPIO Control will be added more further down the line as i currently have some other things that i need to take care first.
Talking about battery-powered, that is an entire beast on it's own since you have to account for the whole hardware from the ground up, you can't just use any pre-made board since most development boards have a voltage regulator and usb-ttl converter with a high quiescent current and that's just one issue and when i am going to attempt to make it battery-powered, i will have to switch to BLE as Wifi is not really suited for battery applications.
Since you managed to make the lock work, i'll be closing this issue.
If you have any other issues, you can also contact me on Discord @rednblkx

Во-первых, я хочу уточнить, что HomeKey не связан с фактической блокировкой сам по себе, на самом деле это отдельная «Сервис» в HomeKit, и именно так HAP разделяет различные объекты общего аксессуара.
Я в основном удалил "прямое взаимодействие", так как это была просто фиктивная вещь, которую я поставил для тестирования перед введением mqtt, чтобы избежать конфликтов или нежелательного поведения. Однако я собираюсь переработать эту часть, возможно, просто сделать вариант, чтобы она действовала как фиктивный переключатель, как раньше (но все еще имела бы управление MQTT) или, может быть, реализовать какое-то оптимистичное состояние, хотя я думаю, что последнее может оказаться слишком сложным, в любом случае любые предложения приветствуются.
Примечание: я работаю над веб-интерфейсом в ветке разработки, поэтому любые настройки и параметры будут включены туда в будущем после его объединения.
Кроме того, хотя я понимаю ваше беспокойство по поводу необходимости полагаться на брокера mqtt, обратите внимание, что в проекте в настоящее время нет реализации GPIO, и поэтому для обновления своего состояния ему приходится полагаться на внешнюю обратную связь.

Good idea to "bring back" the dummy switch!!! Because the mqtt broker is not needed, if you can do everything by simple automation in the home application without unnecessary bridges and brokers. That's what the matter protocol was invented for after all.


Please buy sussy toolbox (suspicious toolbox buff)

changelog
-gives it a chest rig so you can carry your sussy tools in prime syndi fashion
-doubles its storage cap and ups its damage even more for more sussy robussy
-adds syndi gas mask as well for ultimate sussage
-skub

instead of additions it really could use it's price reduced, like a LOT
like, even to 1 or 2 tc

Tbh it could use both this and have it's price reduced

ok its 2 now, sussy toolbox SSSSS tier now


moveWithSpeed behavior

Thanks for sharing this great project freeing the dobot from the stupid decisions of its creators, I managed to make the arm move after flashing your firmware into the arduino but I am not reall sure how things are supposed to work.
My initial understanding was that the accelerometers were used to get an approximation of the arm position which seems confirmed by what is shown when connecting to the dobot (I am using example-sdk.py as base with most code commented) but as soon as I try to make it move I am lost.
moveWithSpeed seems to take absolute coordinates but if I execute two times the same line the arm moves two times when I expected to only move once and stay in position.
I got even more lost when I try to think of it as relative positioning, it does not either for me, so... I have no clue how it is supposed to work :(
PS: I also tried moveTo but it does not run, I get an undefined method.

Correct, the accelerometers are used to get approximate pose upon firmware initialization. initialization occurs every time Arduino is reset. In case of linux and mac (I have not tested it in windows) Arduino reset every time the serial port is opened (every time to execute an example).
Also correct about absolute coordinates. They are not relative. Well, the base angle is assumed 0 upon initialization as there are no sensors on it. The reference frame can be seen on https://github.com/maxosprojects/open-dobot/blob/master/docs/images/reference-frame.png
So, given the base angle is zero at start the Y coordinate is also assumed zero at start. So the only coordinates to be read from accelerometers are X and Z.
And your another correct assumption, if I got it right, is that the same command repeated two times should move the arm only once. This is because when the first command finishes, the next command should move the arm to the tool to the same location, which has been already achieved  by the first command. Not sure what you mean by line there, I hope I got it right.
Apologies to moveTo. This is kind of leftover. May be used later when command stitching is implemented (when you could issue multiple commands that would be stitched together into one smooth trajectory without intermediate stops).
There, of course, might be a bug there that simply prevents your dobot from reaching the coordinates you asked it to. Could you share the code you are using to move the arm that isn't working as expected? Don't forget to use proper code block formatting.

Thanks for the detailed answer, no need to apologies I was just trying to understand how this works and wondered what the difference was between moveTo and moveWithSpeed :)
I realized later that, judging by the image showing how the axis system was mapped that the rotation was not reported by any sensor but the behavior I got is still odd.
Here is the code I used:
from DobotSDK import Dobot
import time

# Maximum speed in mm/s
speed = 700
# Acceleration in mm/s^2
acceleration = 100

dobot = Dobot('/dev/tty.usbmodemFD131', debug=True)
dobot.moveWithSpeed(190.0, 0.0, 50, speed, acceleration)

What I couldn't figure out, and still can't after your answer is why the arm move each time I run this code without unplugging the usb cable, I tried the calibration script and what is reported seems fine, I get an approximately correct angle for each arms.
I might have done something wrong but can't really see what it could be.
I will do some more test later today maybe I overlooked something, I have other questions but I will ask them on your forum since they are clearly not an issue ^^
PS: I am using the latest (0.6) firmware on the arduino mega.

I just confirmed that the code I pasted above move the arm everytime I run it, I didn't touch the USB connection or the power on the arm controller.
Any idea what might be wrong and what I could look for ?

nevermind, got it...
I plugged the motor in the wrong position which explain the chaotic behavior, when plugged correctly it works much better :)

fyi, according to what I said above the initialization happens every time you run a script because every time the serial port is opened, which triggers Arduino's bootloader to reset the controller. While executing the same command twice in the same script (actually repeating the same line twice IN the script) should not move the arm, executing the script twice may move the arm every time. This happens becuase the accelerometers are not very accuarate and the estimated tool pose may differ between executions as the accelerometers are re-read on every controller reset (as part of the setup() ). This means that even with the motors hooked up correctly you may see a slight move, don't get confused.
Good that you figured the motors out.

I saw the slight move but overall the positions seems now stable, I forgot that the arduino was reset everytime time the serial port was open though, thanks for pointing that out.


Switch to using OpenJDK13

Update the cloud build image to use OpenJDK 13. This fixes the following issue: https://bugs.openjdk.java.net/browse/JDK-8233655, and should allow https://github.com/google/ground-android/pull/617 to be merged.

@dturner
As the last step, could you also please remove the experimental images?

@dturner As the last step, could you also please remove the experimental images?

Done


Can't get it to work on macOS Monterey 12.0.1

Hi Jorge,
Did you already manage to get pinentry-touchid to work with macOS 12.0.1? Or is it maybe already working for you?
I installed it following your guide here on github via homebrew.
For me it isn't and just throws:
sign_and_send_pubkey: signing failed for RSA "cardno:000x 0000xxxx" from agent: agent refused operation
pinentry-mac is working fine.
Here's my .zshrc:
unset SSH_AGENT_PID
if [ "${gnupg_SSH_AUTH_SOCK_by:-0}" -ne $$ ]; then
  export SSH_AUTH_SOCK="$(gpgconf --list-dirs agent-ssh-socket)"
fi

alias pinentry='pinentry-mac'

#export GPG_TTY=$(tty)
#gpg-connect-agent updatestartuptty /bye >/dev/null

I used pinentry-mac without the last two lines and it doesn't seem to make a difference if they are there anyway, so I commented them out again.
Here is my .gnupg/gpg-agent.conf:
enable-ssh-support
use-standard-socket
default-cache-ttl 600
max-cache-ttl 7200
#pinentry-program /usr/local/bin/pinentry-mac
pinentry-program /usr/local/opt/pinentry-touchid/bin/pinentry-touchid

and finally my gpg.conf:
auto-key-retrieve
no-emit-version
default-key 907BB9A8BA3B629D6F7EFA8A04D3A0AB20551091
#keyserver
cert-digest-algo SHA256
no-comments
personal-cipher-preferences AES AES256 AES192 CAST5
personal-digest-preferences SHA256 SHA512 SHA384 SHA224
ignore-time-conflict
allow-freeform-uid
use-agent

As I said - this setup is working fine with pinentry-mac (latest version, installed via homebrew) for about two years now.
Any ideas? Or simply a "12.0.1 is just not supported right now" ? ;)
Thanks on advanced and also for your work (which, hopefully, I'll be able to use someday) ;)

Hi @congoelmex, I opened an issue for a slightly different issue, but like yourself, I can get pinentry-mac to work, not the touchID.
I do see the keychain entry and I tried allowing pinentry-touchid but it still won't prompt me anything...

Hi @congoelmex, Thanks for opening this issue.
Unfortunately I haven't tested pinentry-touchid in macOS Monterey just yet. @Gby56 are you also running in macOS Monterey? I will post here as soon as I have some more info.

+1
I am also struggling to get this working on Monterey. Subscribing myself to any updates here.

I have the same issue. Fresh install with hombrew.
/tmp/pinentry-touchid.log:
2021/11/24 01:49:42 main.go:105: Ready!
2021/11/24 01:49:42 main.go:256: Error calling pinentry-mac: unexpected response: ERR 83918950 Inappropriate ioctl for device <Pinentry>
2021/11/24 01:49:42 main.go:260: pinentry-mac didn't return a password

Just as a data point, this does work for me on Monterey 12.1 (M1 Pro).

This workaround resolved the issue for me: https://github.com/jorgelbg/pinentry-touchid/issues/3#issuecomment-915440261

From this error:
2021/11/24 01:49:42 main.go:256: Error calling pinentry-mac: unexpected response: ERR 83918950 Inappropriate ioctl for device <Pinentry>

it seems that pinentry-touchid is falling back to pinentry/pinentry-curses instead of pinentry-mac. Can you check by running:
❯ pinentry-touchid -check

and also there is a new -fix flag that should automatically fix the symlink for you:
❯ pinentry-touchid -fix
✅ /usr/local/opt/pinentry/bin/pinentry is now pointing to pinentry-mac

Hi @jorgelbg ,
Thanks for the hint, but now there is another issue:
Your documentation states, that I have to save the passphrase in the MacOS Keychain. The Problem is, that I'm not offered the option to do so.
I use a Nitrokey for authenticating againt my SSH servers, so the gpg-key is saved on the Nitrokey itself.
When I try to access the SSH server, I'm prompted to enter the Passphrase but no checkbox to save something in the keychain is shown.
If I try it (for testing purposes) with an gpg-key that is in my gpg-keyring on the harddrive (as opposed to being saved on the Nitrokey), the "Save in keychain" option in the pinentry-mac dialogue is displayed.
Any thoughts on that? ;)

@congoelmex even if you don't see the option to save in the Keychain in the pinentry-mac UI, pinentry-touchid should still create an item in the keychain automatically, as long as no duplicated entry is found. If a duplicated item is found it should be logged in:
$TMPDIR/pinentry-touchid.log

and you could use the commands from https://github.com/jorgelbg/pinentry-touchid/issues/11#issuecomment-1205519449 to find those duplicated entries (if any).

Hi @jorgelbg and thank you for your answer. It still doesn't work.
Maybe it helps, if I share some details:
pinentry-mac version: 1.1.1 (installed bei GPGSuite from gpgtools.org)
pinentry-touchid version: latest version from your git repository
I start with this gpg-agent.conf:
enable-ssh-support default-cache-ttl 600 max-cache-ttl 7200 #pinentry-program /usr/local/opt/pinentry-touchid/bin/pinentry-touchid pinentry-program /usr/local/MacGPG2/libexec/pinentry-mac.app/Contents/MacOS/pinentry-mac debug-level basic log-file /Users/congoelmex.gnupg/gpg-agent.log
There is no entry regarding GnuPG in my macOS-keychain.
The Nitrokey Pro 2 is connected and if I enter
echo 1234 | gpg -as -
I'm presented the pinentry-mac window, where I enter the passphrase to unlock (notice the missing save option):

After that, I check my macOS - keychain: No entry about GnuPG whatsoever...
Now I change gpg-agent.conf to use pinentry-touchid:
enable-ssh-support default-cache-ttl 600 max-cache-ttl 7200 pinentry-program /usr/local/opt/pinentry-touchid/bin/pinentry-touchid #pinentry-program /usr/local/MacGPG2/libexec/pinentry-mac.app/Contents/MacOS/pinentry-mac debug-level basic log-file /Users/malte/.gnupg/gpg-agent.log
After killing the gpg-agent with
gpgconf --kill gpg-agent"gpgconf
I try the gpg encryption again for testing:
echo 1234 | gpg -as -
Now, the pinentry-touchid pops up:

If I enter the passphrase twice, the test message gets encrypted...but I'm not offered to use TouchID...at no point...
And no keychain entry for GnuPG or the like is created..
Just for completeness, here are the results from the commands you suggested:
security dump-keychain | grep GnuPG
outputs nothing, because grep can't find what is not there.. ;)
security find-generic-password -s 'GnuPG'
outputs:
security: SecKeychainSearchCopyNext: The specified item could not be found in the keychain.
Here, the contents of $TMPDIR/pinentry-touchid.log:
2022/08/31 18:31:05 main.go:118: Ready!
nothing else...
I don't really think that it would be helpful to post the contents of "gpg-agent.log" (and I don't want to spam this thread too much), but if you think it could help, I'll gladly post ist (maybe twice - one time with pinentry-mac and one time with pinentry-touchid)...
To be frank, I'm at a loss here and just can't get pinentry-touchid to work with my Nitrokey... :(

should these entries:
2022-08-31 19:41:53 gpg-agent[7351] starting a new PIN Entry
2022-08-31 19:41:53 gpg-agent[7351] DBG: connection to PIN entry established
2022-08-31 19:41:53 gpg-agent[7351] You may want to update to a newer pinentry
from the gpg-agent.log worry me somehow?
Even if I try to install the homebrew version of the agent it still shows version 1.1.1

@congoelmex i am getting exactly the same symptoms as you on Ventura 13.2.1 :( Did you manage to get touchid working?

I was having same issue on Ventura 13.2.1 and 13.3.
But now I am in a state where it keeps asking my for my PIN and then confirm the PIN. So it "works", but instead of using touch-id I now have to provide the PIN twice.
Note: using yubikey

Same issue here, I need to provide the PIN twice and no touch id.


Keep space between vertical stacked cards

Looks nicer, i think!
Default UI

Lovelace vertical stacked

If you use Firefox enable shadowroot and custom elements in experimental features


Fix iOS and Android labels under accessibilityHint

These labels were starting the sentence and made it look off.
Before:
https://reactnative.dev/docs/accessibility#accessibilityhint

After:

Validation

Pull down this branch
Run yarn start
Go to http://localhost:3000/docs/next/accessibility
Verify that the iOS and Android labels look correct

@Simek Thanks for the feedback! I updated the PR with the correct labels.

Thank you for the updates! 🙏
However, it looks like the headers were ensuring the block display of labels, so we need to wrap them with <p> or other block tag, to fix the display:
Maybe also an empty line between <div>s will help parser to distinguish HTML from MDX parts.

I added a space between the labels and text and it looks like that fixed the spacing issue.


feature: Add dragData options to dataset and axis to prevent dragging

feature: Add dragData options to dataset and axis scale to prevent dragging of lines or scales

Very nice, thats a good addition! Thanks a lot for the PR.


Works only with Swift?

I guess this fantastic Project will only works with Swift? Because my whole project is written in Objective c and i don´t have the time to change everything to Swift in the near future.

Swift can be used together with Objective C. Swift is supported from iOS 7 and up!

Yeah but only when i create a new class, and my existing Classes with Objective c have to be re created isn't it?

No, that is not how it works. You can literally just have both Swift and Objective C classes in your project and they can use each other. From Objective C you can just instantiate the TZStackView like this:
TZStackView *stackView = [[TZStackView alloc] initWithArrangedSubviews: @[view1, view2, view3]]

You do have to add an import to your Objective C class like this:
#import "YourProject-Swift.h"

Of course you have to replace "YourProject" with your actual project name. This is a header-file that is auto-generated when you use Swift. The header file will contain all the Swift classes you want to use in Obj-C format. With this trick Obj-C and Swift are compatible.
Let me know if you need help with this some more.

Oh i didn´t know that this is possible, thank you very much, that helps me a lot!!  Let me know if you run into any trouble!


No hierarchy in table of contents

I noticed that there's no hierarchy in the list of table of contents.

While I expect it to be something like:

h2 title

another h2 title

h3 title

another h3 title

Let me know if you're interested in a pull request to update it now that I've finish the code for my own Astro site's table of contents.

That sounds helpful, thanks @chenxsan! We’ll need to make sure the code highlighting the current item still works.


Maps in cargo teleporter.

Also removes the impotent cargo dock and provides some simple crates filled with various useful stuff.

Merge this before #2848 , I was here first. I'm not dealing with map conflicts.


Added boiler plate to server code

The boiler plate includes python requirements for flask, a framework
within which apis can be created and a default HelloWorld route for
developers to follow.

Looks good to me (LGTM)


Cranelift: wasmtime-jit-icache-coherence doesn't compile on FreeBSD

This causes CI to fail for https://github.com/bjorn3/rustc_codegen_cranelift/pull/1295.
error[E0432]: unresolved import `libc`
 --> /.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-jit-icache-coherence-2.0.0/src/libc.rs:3:5
  |
3 | use libc::{syscall, EINVAL, EPERM};
  |     ^^^^ help: a similar path exists: `crate::libc`
  |
  = note: `use` statements changed in Rust 2018; read more at <https://doc.rust-lang.org/edition-guide/rust-2018/module-system/path-clarity.html>

error[E0433]: failed to resolve: use of undeclared crate or module `libc`
 --> /.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-jit-icache-coherence-2.0.0/src/libc.rs:7:30
  |
7 | const MEMBARRIER_CMD_GLOBAL: libc::c_int = 1;
  |                              ^^^^ use of undeclared crate or module `libc`

error[E0433]: failed to resolve: use of undeclared crate or module `libc`
 --> /.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-jit-icache-coherence-2.0.0/src/libc.rs:8:51
  |
8 | const MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE: libc::c_int = 32;
  |                                                   ^^^^ use of undeclared crate or module `libc`

error[E0433]: failed to resolve: use of undeclared crate or module `libc`
 --> /.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-jit-icache-coherence-2.0.0/src/libc.rs:9:60
  |
9 | const MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED_SYNC_CORE: libc::c_int = 64;
  |                                                            ^^^^ use of undeclared crate or module `libc`

FreeBSD isn't an officially-supported platform but we can take patches on a best-effort basis. I'm happy to take a look if someone wants to make a PR (but I don't have a FreeBSD system handy to test it myself).


Updating "Hacking on Gogs"

Also, Gogs have 3 different pages with information about how to contribute:

https://github.com/gogits/gogs/blob/master/CONTRIBUTING.md
https://github.com/gogits/gogs/wiki/Contributing-Code
https://gogs.io/docs/advanced/hacking_on_gogs

And they aren't in sync! (Not all information is available on all pages). I think it would be better to consolidate these pages in a single one (making sure it is easily findable).
Just a suggestion.

Thanks!
CONTRIBUTING.md is a naming convention of GitHub, but the wiki one explains itself well.
Someone can find everything if he/she reads README.md.


ImportError: No module named run_calc_hungarian_alignment_score

When I ran xgb_train_cvBodyId.py, this error was happened.
Traceback (most recent call last):
  File "xgb_train_cvBodyId.py", line 16, in <module>
    from AlignmentFeatureGenerator import *
  File "/root/work/fnc-1/tree_model/AlignmentFeatureGenerator.py", line 11, in <                                                                                module>
    from run_calc_hungarian_alignment_score import *
ImportError: No module named run_calc_hungarian_alignment_score

Because there is no module "run_calc_hungarian_alignment_score".
Please check that we can run them completely if we follow the README.md.

Remove the line that imports AligenmentFeatureGenerator


UnsafeCell in Archived type incompatible with miri

I'm attempting to create a mutex type that can be used in its Archived form, to facilitate safe mutation of archived data mmap'd into multiple processes.
I have it implemented, and some basic tests working, but noticed that rkyv::archived_root results in an error in miri. rkyv::archived_root_mut works ok. Maybe there's a fundamental reason for that, and the answer is "just use rkyv::archived_root_mut", but I think it might be worth digging into a bit more.
I created a smaller test case that seems to show the problem is more fundamentally around UnsafeCell, which would be needed for interior mutability in any Archive type.
#![deny(unsafe_op_in_unsafe_fn)]

use std::cell::UnsafeCell;

#[repr(transparent)]
struct UnsafeU32 {
    val: UnsafeCell<u32>,
}

impl<S> rkyv::Serialize<S> for UnsafeU32
where
    S: rkyv::Fallible + ?Sized,
{
    fn serialize(&self, serializer: &mut S) -> Result<Self::Resolver, S::Error> {
        let val = unsafe { &*self.val.get() };
        val.serialize(serializer)
    }
}

impl rkyv::Archive for UnsafeU32 {
    type Archived = Self;
    type Resolver = <u32 as rkyv::Archive>::Resolver;

    unsafe fn resolve(&self, pos: usize, resolver: Self::Resolver, out: *mut Self::Archived) {
        unsafe { (&*self.val.get()).resolve(pos, resolver, out as *mut u32) };
    }
}

#[test]
fn test_rkyv_miri() {
    type T = UnsafeU32;
    let original: T = UnsafeU32 {
        val: UnsafeCell::new(10),
    };
    let bytes = rkyv::to_bytes::<_, 256>(&original).unwrap();

    // The archived mutex can be used to mutate the data in place.
    let archived = unsafe { rkyv::archived_root::<T>(&bytes[..]) };
    assert_eq!(unsafe { *archived.val.get() }, 10);
}

The test passes natively, but under miri fails:
$ MIRIFLAGS=-Zmiri-backtrace=full cargo +nightly miri test
Preparing a sysroot for Miri (target: x86_64-unknown-linux-gnu)... done
    Finished test [unoptimized + debuginfo] target(s) in 0.01s
     Running unittests src/lib.rs (target/miri/x86_64-unknown-linux-gnu/debug/deps/repro-982c58b2cf8c397a)

running 1 test
test test_rkyv_miri ... error: Undefined Behavior: trying to reborrow from <185489> for SharedReadWrite permission at alloc77718[0x0], but that tag only grants SharedReadOnly permission for this location
  --> /home/jnewsome/.cargo/registry/src/github.com-1ecc6299db9ec823/rkyv-0.7.39/src/util/mod.rs:67:5
   |
67 |     &*bytes.as_ptr().add(pos).cast()
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |     |
   |     trying to reborrow from <185489> for SharedReadWrite permission at alloc77718[0x0], but that tag only grants SharedReadOnly permission for this location
   |     this error occurs as part of a reborrow at alloc77718[0x0..0x4]
   |
   = help: this indicates a potential bug in the program: it performed an invalid operation, but the Stacked Borrows rules it violated are still experimental
   = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <185489> was created by a retag at offsets [0x0..0x4]
  --> src/lib.rs:38:29
   |
38 |     let archived = unsafe { rkyv::archived_root::<T>(&bytes[..]) };
   |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = note: backtrace:
   = note: inside `rkyv::archived_value::<UnsafeU32>` at /home/jnewsome/.cargo/registry/src/github.com-1ecc6299db9ec823/rkyv-0.7.39/src/util/mod.rs:67:5
   = note: inside `rkyv::archived_root::<UnsafeU32>` at /home/jnewsome/.cargo/registry/src/github.com-1ecc6299db9ec823/rkyv-0.7.39/src/util/mod.rs:148:5
note: inside `test_rkyv_miri` at src/lib.rs:38:29
  --> src/lib.rs:38:29
   |
38 |     let archived = unsafe { rkyv::archived_root::<T>(&bytes[..]) };
   |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside closure at src/lib.rs:30:1
  --> src/lib.rs:30:1
   |
29 |   #[test]
   |   ------- in this procedural macro expansion
30 | / fn test_rkyv_miri() {
31 | |     type T = UnsafeU32;
32 | |     let original: T = UnsafeU32 {
33 | |         val: UnsafeCell::new(10),
...  |
39 | |     assert_eq!(unsafe { *archived.val.get() }, 10);
40 | | }
   | |_^
   = note: inside `<[closure@src/lib.rs:30:1: 40:2] as std::ops::FnOnce<()>>::call_once - shim` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:248:5
   = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:248:5
   = note: inside `test::__rust_begin_short_backtrace::<fn()>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:572:5
   = note: inside closure at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:563:30
   = note: inside `<[closure@test::run_test::{closure#1}] as std::ops::FnOnce<()>>::call_once - shim(vtable)` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:248:5
   = note: inside `<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send> as std::ops::FnOnce<()>>::call_once` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/boxed.rs:1935:9
   = note: inside `<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>> as std::ops::FnOnce<()>>::call_once` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/panic/unwind_safe.rs:271:9
   = note: inside `std::panicking::r#try::do_call::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
   = note: inside `std::panicking::r#try::<(), std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
   = note: inside `std::panic::catch_unwind::<std::panic::AssertUnwindSafe<std::boxed::Box<dyn std::ops::FnOnce() + std::marker::Send>>, ()>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
   = note: inside `test::run_test_in_process` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:595:18
   = note: inside closure at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:489:39
   = note: inside `test::run_test::run_test_inner` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:527:13
   = note: inside `test::run_test` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:559:28
   = note: inside `test::run_tests::<[closure@test::run_tests_console::{closure#2}]>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:302:17
   = note: inside `test::run_tests_console` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/console.rs:293:5
   = note: inside `test::test_main` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:113:15
   = note: inside `test::test_main_static` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/test/src/lib.rs:132:5
   = note: inside `main`
   = note: inside `<fn() as std::ops::FnOnce<()>>::call_once - shim(fn())` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:248:5
   = note: inside `std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:122:18
   = note: inside closure at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:145:18
   = note: inside `std::ops::function::impls::<impl std::ops::FnOnce<()> for &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>::call_once` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ops/function.rs:280:13
   = note: inside `std::panicking::r#try::do_call::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
   = note: inside `std::panicking::r#try::<i32, &dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
   = note: inside `std::panic::catch_unwind::<&dyn std::ops::Fn() -> i32 + std::marker::Sync + std::panic::RefUnwindSafe, i32>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
   = note: inside closure at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:128:48
   = note: inside `std::panicking::r#try::do_call::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:492:40
   = note: inside `std::panicking::r#try::<isize, [closure@std::rt::lang_start_internal::{closure#2}]>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panicking.rs:456:19
   = note: inside `std::panic::catch_unwind::<[closure@std::rt::lang_start_internal::{closure#2}], isize>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/panic.rs:137:14
   = note: inside `std::rt::lang_start_internal` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:128:20
   = note: inside `std::rt::lang_start::<()>` at /home/jnewsome/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/rt.rs:144:17
   = note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)

I also tried wrapping the serialized buffer in an UnsafeCell, in case that's needed to signal that the raw buffer has interior mutability. It doesn't seem to make a difference, though.
#[test]
fn test_rkyv_miri_unsafecell_wrapper() {
    type T = UnsafeU32;
    let original: T = UnsafeU32 {
        val: UnsafeCell::new(10),
    };
    let bytes = UnsafeCell::new(rkyv::to_bytes::<_, 256>(&original).unwrap());

    // The archived mutex can be used to mutate the data in place.
    let archived = unsafe { rkyv::archived_root::<T>(&(&*bytes.get())[..]) };
    assert_eq!(unsafe { *archived.val.get() }, 10);
}

$ cargo +nightly miri test
Preparing a sysroot for Miri (target: x86_64-unknown-linux-gnu)... done
   Compiling repro v0.1.0 (/home/jnewsome/tmp/repro)
    Finished test [unoptimized + debuginfo] target(s) in 0.05s
     Running unittests src/lib.rs (target/miri/x86_64-unknown-linux-gnu/debug/deps/repro-982c58b2cf8c397a)

running 1 test
test test_rkyv_miri_unsafecell_wrapper ... error: Undefined Behavior: trying to reborrow from <185858> for SharedReadWrite permission at alloc77920[0x0], but that tag only grants SharedReadOnly permission for this location
  --> /home/jnewsome/.cargo/registry/src/github.com-1ecc6299db9ec823/rkyv-0.7.39/src/util/mod.rs:67:5
   |
67 |     &*bytes.as_ptr().add(pos).cast()
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |     |
   |     trying to reborrow from <185858> for SharedReadWrite permission at alloc77920[0x0], but that tag only grants SharedReadOnly permission for this location
   |     this error occurs as part of a reborrow at alloc77920[0x0..0x4]
   |
   = help: this indicates a potential bug in the program: it performed an invalid operation, but the Stacked Borrows rules it violated are still experimental
   = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
help: <185858> was created by a retag at offsets [0x0..0x4]
  --> src/lib.rs:53:29
   |
53 |     let archived = unsafe { rkyv::archived_root::<T>(&(&*bytes.get())[..]) };
   |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = note: backtrace:
   = note: inside `rkyv::archived_value::<UnsafeU32>` at /home/jnewsome/.cargo/registry/src/github.com-1ecc6299db9ec823/rkyv-0.7.39/src/util/mod.rs:67:5
   = note: inside `rkyv::archived_root::<UnsafeU32>` at /home/jnewsome/.cargo/registry/src/github.com-1ecc6299db9ec823/rkyv-0.7.39/src/util/mod.rs:148:5
note: inside `test_rkyv_miri_unsafecell_wrapper` at src/lib.rs:53:29
  --> src/lib.rs:53:29
   |
53 |     let archived = unsafe { rkyv::archived_root::<T>(&(&*bytes.get())[..]) };
   |                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: inside closure at src/lib.rs:45:1
  --> src/lib.rs:45:1
   |
44 |   #[test]
   |   ------- in this procedural macro expansion
45 | / fn test_rkyv_miri_unsafecell_wrapper() {
46 | |     type T = UnsafeU32;
47 | |     let original: T = UnsafeU32 {
48 | |         val: UnsafeCell::new(10),
...  |
54 | |     assert_eq!(unsafe { *archived.val.get() }, 10);
55 | | }
   | |_^
   = note: this error originates in the attribute macro `test` (in Nightly builds, run with -Z macro-backtrace for more info)

note: some details are omitted, run with `MIRIFLAGS=-Zmiri-backtrace=full` for a verbose backtrace

I have it implemented, and some basic tests working, but noticed that rkyv::archived_root results in an error in miri. rkyv::archived_root_mut works ok. Maybe there's a fundamental reason for that, and the answer is "just use rkyv::archived_root_mut", but I think it might be worth digging into a bit more.

Expanding on this a bit more, I'm hesitant to wrap the mmap'd buffer in a &mut [u8] instead of just a &[u8], because I think that breaks the "A mutable reference cannot be aliased" rule from https://doc.rust-lang.org/nomicon/references.html .
More concretely, I imagine that having the buffer behind a mutable reference signals to the compiler that nothing inside will be mutated behind its back, since it has the only reference. This assumption is false, of course, because the buffer will be mmap'd into other processes, with their own (mutable) references.

I actually think that using an &mut [u8] would resolve this issue since the mutable references have to be simultaneous to violate the aliasing rule. You can still have an &mut of some field reborrowed from an &mut of a larger structure. The compiler should pick up on that.

In my use case, multiple processes would be mmap'ing the same shared memory pages and deserializing it. If those processes each construct a &mut [u8] from the pointer to that shared memory so that they can call rkyv::archived_root_mut, then there would be simultaneous mutable references to the same memory.

Hm, that's true. It seems like this kind of external shared merry is not something Rust was meant to work with. I do still think that using a mutable buffer would work, but there is a problem with reads. We would need to treat everything as potentially volatile, which is something that rkyv is not able to do right now.

If I get rid of the intermediate slice and cast the pointer directly, miri seems to think it's ok:

Actually this is starting to make sense. In the current helpers, there's a point in time where the same underlying data has both a slice of &[u8] and a &T referring to it. In this case, since &T in this case contains UnsafeCell, miri knows that the underlying data has interior mutability, which is incompatible with the &[u8] alias.

I realized another issue when archiving types with interior mutability - rkyv::archived_root doesn't require pinning, unlike rkyv::archived_root_mut.
When T has interior mutability, it's possible to get a mutable reference and move some internals out, which may have contained relative pointers etc.
IDK if this is a bug per se, or covered by the safety requirements already put on the user via the Unsafe wrapper etc. Still, it'd be nice to make it easier to avoid this footgun.
I'm starting to sketch out a new helper for my own use that addresses both issues:
    /// Taking the AlignedVec here instead of a slice avoids a gray safety area
    /// that violates miri's current Stacked Borrows check.
    /// https://github.com/rkyv/rkyv/issues/303
    ///
    /// The returned value is Pin<T>, because T's that provide interior
    /// mutability could otherwise potentially be incorrectly moved.
    ///
    /// SAFETY:
    /// * `vec` must contain a value of type `T` at the end.
    unsafe fn archived_root<T>(vec: &rkyv::AlignedVec) -> pin::Pin<&T> {
        let pos = vec.len() - std::mem::size_of::<T>();
        // SAFETY: caller guarantees the cast is valid
        let reference = unsafe { &*vec.as_ptr().add(pos).cast() };
        // SAFETY: the resulting pin has an immutable borrow from `vec`.
        // Mutable methods of `vec` that might cause reallocation etc cannot
        // be called while this reference exists.
        unsafe { pin::Pin::new_unchecked(reference) }
    }

If a type has interior mutability and produces a mutable reference to some internal value, then it must ensure that the reference obeys the typical aliasing rules. For example, RefCell dynamically guarantees that a mutable and shared reference do not exist simultaneously. Cell gets around these rules by never producing a mutable reference, it only ever moves the value around. The core of the problem is not that we can't have mutable references to rkyv types (that's just a symptom), the real issue is that we can't allow rkyv types to be moved. Another way to think about this is that all rkyv types must be treated as pinned, which is not the way that standard library types with interior mutability treat their types by default.
I think the correct way to solve this issue would be to create an archived variant of UnsafeCell that only produces pinned mutable references to the interior type. Implementing Cell with that primitive would only be possible when T: Unpin, which is the bound we needed earlier (rkyv types are !Unpin by default). This seems persuasive to me.
Your idea for returning a pinned shared reference is on the right track, but not quite correct. It's not your fault though, pinning is notoriously difficult to get right. Pinning a shared reference doesn't actually do anything because you can just get the shared reference back out with get_ref. Rather than having some notion of a pinned value, rkyv is working moreso with a system of immovable types if that makes any sense.

Pinning a shared reference doesn't actually do anything because you can just get the shared reference back out with get_ref

Hah, whoops!

I think the correct way to solve this issue would be to create an archived variant of UnsafeCell that only produces pinned mutable references to the interior type. Implementing Cell with that primitive would only be possible when T: Unpin, which is the bound we needed earlier (rkyv types are !Unpin by default). This seems persuasive to me.

Hmm, yeah I think that makes sense.

I think I've addressed the pinning issues in my WIP mutex API. I have a PR out now being reviewed by a teammate; would be happy to have your review there too. Otherwise if you're interested I could separately take a stab at merging it into rkyv itself, and you can review there. https://github.com/shadow/shadow/pull/2386

I think the PR looks good in general. Because of the reliance on libc, I think this would be better suited for rkyv_contrib if you wanted to upstream it. I think it would be beneficial to add the pointer-only API for archived_root and friends. Issue filed here.

Thanks! Yeah, I could drop the nix dependency pretty easily, but dropping libc would be trickier. I'll have a look at adding to rkyv_contrib

Since it seems like this issue has been resolved, I'm going to close this issue. Feel free to reopen if I've missed something!

I'm still using my own replacement for archived_root to avoid the miri stacked borrows failure. Maybe this or something like it should be added to rkyv:
    /// Taking the AlignedVec here instead of a slice avoids a gray safety area
    /// that violates miri's current Stacked Borrows check.
    /// https://github.com/rkyv/rkyv/issues/303
    ///
    /// SAFETY:
    /// * `vec` must contain a value of type `T` at the end.
    unsafe fn archived_root<T>(vec: &rkyv::AlignedVec) -> &T::Archived
    where
        T: rkyv::Archive,
    {
        let pos = vec.len() - std::mem::size_of::<T::Archived>();
        // SAFETY:
        // * Caller guarantees that this memory contains a valid `T`
        // * The reference borrows from `vec`, ensuring that it can't
        //   be dropped or mutated.
        // * `AlignedVec::as_ptr` explicitly supports this use-case,
        //   including for `T` that has internal mutability via `UnsafeCell`.
        unsafe { &*vec.as_ptr().add(pos).cast() }
    }


getSearchTweetsWithQuery mishandles query, ie. "from:username"

A query of "from:username" will result in a search equivalent to "username", ie. any tweet with mention of that username, where it should instead return only tweets authored by username.
This seems to be caused by the ":" being ignored by the Twitter API due to STTwitter's encoding of the query string.
ie. removing  st_stringByAddingRFC3986PercentEscapesUsingEncoding on the query restores correct behaviour for the example query above
md[@"q"] = q; //[q st_stringByAddingRFC3986PercentEscapesUsingEncoding:NSUTF8StringEncoding];

ps. Otherwise, great library and much thanks! Used in this: http://tobyz.net/projects/screenrunner/

That was quick! Thanks.
ps. I am slightly surprised by this issue, as Twitter does ask the query be URL encoded as per https://dev.twitter.com/rest/public/search

I restored the previous code after someone mentioned the regression https://twittercommunity.com/t/conversations-with-search-api/31954
Maybe it was a temporary issue on Twitter' servers.

FYI, I think this issue is still unresolved: in my usage the reverted code has reverted to incorrect behaviour, all as per first post. Moreover, I'm the symptoms Garixi describes fit this issue: this issue doesn't stop results, it just returns a different subset of results relating to the username.


Add an authorization system using SvelteKit Auth

Done - Once you can successfully log in and stay logged in between sessions. Some routes should also only be accessible to authorized users.
Expected time needed: 8H

Hi @michalpij. Not sure I'm following what you wrote above.
Are you saying some routes are being exposed to unauthorized users?
Routes like /admin and /teachers check the role to make sure the user is authorized in their +page.server.ts files.


METRON-1925 Provide Verbose View of Profile Results in REPL

Motivation
When viewing profile measurements in the REPL using PROFILE_GET, you simply get back a list of values. It is not easy to determine from which time period the measurements were taken.
For example, are the following values all sequential?  Are there any gaps in the measurements taken over the past 30 minutes?  When was the first measurement taken?
[Stellar]>>> PROFILE_GET("hello-world","global",PROFILE_FIXED(30, "MINUTES"))
[2655, 1170, 1185, 1170, 1185, 1215, 1200, 1170]

The PROFILE_GET function was designed to return values that serve as input to other functions.  It was not designed to return values in a human-readable form that can be easily understood. We need another way to query for profile measurements in the REPL that provides a user with a better understanding of the profile measurements.
Solution
This PR provides a new function called PROFILE_DEBUG.  It is effectively a "verbose mode" for PROFILE_GET.
For lack of a better name, I just called it PROFILE_DEBUG.  I would be open to alternatives.  I did not want to add additional options to the already complex PROFILE_GET.

Description: Retrieves a series of measurements from a stored profile. Provides a more verbose view of each measurement than PROFILE_GET. Returns a map containing the profile name, entity, period id, period start, period end for each profile measurement.

Arguments:
profile - The name of the profile.
entity - The name of the entity.
periods - The list of profile periods to fetch. Use PROFILE_WINDOW or PROFILE_FIXED.
groups - Optional, The groups to retrieve. Must correspond to the 'groupBy' list used during profile creation. Defaults to an empty list, meaning no groups.

Returns: A map for each profile measurement containing the profile name, entity, period, and value.

Test Drive

Spin-up Full Dev and create a profile.  Follow the Profiler README.  Reduce the profile period if you are impatient.

Open up the REPL and retrieve the values using PROFILE_GET.  Notice that I have no idea when the first measurement was taken, if the values are sequential, if there are gaps in the values and how big.
[Stellar]>>> PROFILE_GET("hello-world","global",PROFILE_FIXED(30, "MINUTES"))
[1185, 1170, 1185, 1215, 1200, 1170, 5425, 1155, 1215, 1200]

Now use PROFILE_DEBUG to retrieve the same results.
[Stellar]>>> results := PROFILE_DEBUG("hello-world","global",PROFILE_FIXED(30, "MINUTES"))
[{period.start=1544119560000, period=12867663, profile=hello-world, period.end=1544119680000, groups=[], value=1185, entity=global}, {period.start=1544119680000, period=12867664, profile=hello-world, period.end=1544119800000, groups=[], value=1170, entity=global}, {period.start=1544119800000, period=12867665, profile=hello-world, period.end=1544119920000, groups=[], value=1185, entity=global}, {period.start=1544119920000, period=12867666, profile=hello-world, period.end=1544120040000, groups=[], value=1215, entity=global}, {period.start=1544120040000, period=12867667, profile=hello-world, period.end=1544120160000, groups=[], value=1200, entity=global}, {period.start=1544120160000, period=12867668, profile=hello-world, period.end=1544120280000, groups=[], value=1170, entity=global}, {period.start=1544120880000, period=12867674, profile=hello-world, period.end=1544121000000, groups=[], value=5425, entity=global}, {period.start=1544121000000, period=12867675, profile=hello-world, period.end=1544121120000, groups=[], value=1155, entity=global}, {period.start=1544121120000, period=12867676, profile=hello-world, period.end=1544121240000, groups=[], value=1215, entity=global}, {period.start=1544121240000, period=12867677, profile=hello-world, period.end=1544121360000, groups=[], value=1200, entity=global}]

For each measurement, I have a map containing the period, period start, period end, profile name, entity, groups, and value.  With this I can better answer some of the questions above.
{
  profile=hello-world,
  entity=global,
  period=12867663,
  period.start=1544119560000,
  period.end=1544119680000,
  groups=[],
  value=1185
}

When was the first measurement taken?
[Stellar]>>> GET(results, 0)
{period.start=1544119560000, period=12867663, profile=hello-world, period.end=1544119680000, groups=[], value=1185, entity=global}

I can see that the first period started at<PHONE_NUMBER>000 or Thu Dec 06 2018 18:06:00 UTC.

Are these measurements sequential?  Are there any gaps?
I can use MAP on the list of results to extract just the periods from each profile measurement. The period is a monotonically increasing value.
[Stellar]>>> MAP(results, m -> MAP_GET("period", m))
[12867663, 12867664, 12867665, 12867666, 12867667, 12867668, 12867674, 12867675, 12867676, 12867677]

From this I can tell that there is a gap in the measurements.

The first period here is 12867663 and the last is 12867677.
I can see that the first 6 measurements are sequential up to period 12867668.
Then there is a gap of 5 periods (12867669 - 12867673) before the measurements resume at period 12867674.

How big was that gap?
We know that we missed 5 periods, but how big is that gap?
Here is when each period starts in epoch milliseconds.
[Stellar]>>> MAP(results, m -> MAP_GET("period.start", m))
[1544119560000,<PHONE_NUMBER>000,<PHONE_NUMBER>000,<PHONE_NUMBER>000,<PHONE_NUMBER>000,<PHONE_NUMBER>000,<PHONE_NUMBER>000,<PHONE_NUMBER>000,<PHONE_NUMBER>000,<PHONE_NUMBER>000]

I can find the difference between the 5th and 6th period.  The different is about 720000 milliseconds, which is 12 minutes.
[Stellar]>>> MAP_GET("period.start", GET(results, 5))
1544120160000

[Stellar]>>> MAP_GET("period.start", GET(results, 6))
1544120880000

[Stellar]>>> MAP_GET("period.start", GET(results, 6)) - MAP_GET("period.start", GET(results, 5))
720000

Changes

Altered the ProfilerClient so that it returns the ProfileMeasurement values instead of just a list of the raw values stored in HBase.

Removed one of the methods of ProfileClient that was not being used to simplify things.

Updated PROFILE_GET to work with the altered ProfilerClient.

Added PROFILE_DEBUG.

Pull Request Checklist

[x] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[x] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[x] Has your PR been rebased against the latest commit within the target branch (typically master)?
[x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?
[x] Have you included steps or a guide to how the change may be verified and tested manually?
[x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
[x] Have you written or updated unit tests and or integration tests to verify your changes?
[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[x] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

Could the return be a full json document, that includes the query parameters? I can see  doing these things and writing to file, and wanting more than just the data, but having the meta data ( query ) as well.

@ottobackwards Maybe another optional array could be used, similar to the geoget functions, that would allow you to specify a list of desired fields to return.
https://github.com/apache/metron/blob/c08cd07f36cd9bf2608a586a209bf809130a069a/metron-platform/metron-enrichment/src/test/java/org/apache/metron/enrichment/stellar/GeoEnrichmentFunctionsTest.java#L140

Could the return be a full json document, that includes the query parameters?

what do you mean by query parameters?

Not sure where this is coming from:
Failed tests:
  RestFunctionsTest.restGetShouldTimeout:516 expected null, but was:<{get=success}>

This should get added to the README.

@nickwallen What I mean is that the returned value has the query parameters in it, so you have the data and the query you used for it.  Please excuse me if that is already the case

@ottobackwards: What I mean is that the returned value has the query parameters in it, so you have the data and the query you used for it

Gotcha. Yes, you have the query parameters returned in the results; the profile, entity, and period for each value.

@JonZeolla: PROFILE_VIEW or whatever it ends up as (PROFILE_DEBUG, PROFILE_VERBOSE? No preference for any) should get added to the README.

I added it to the README.

Ok, I like those suggestion. I can change it to PROFILE_DEBUG, if only for being fewer characters than PROFILE_VERBOSE.

oooh nice!   PROFILE_REPORT!!!!

I'm +1, pending everyone else who commented. I think @mmiklavc, @ottobackwards, @JonZeolla

Looks good, +1 from me as well.

Didn't review the PR but I'm good with that name


Attribute/Express validation should inherit from DB column type

In attributes we define the columns to index in search
    protected $searchIndexFieldDefinition = array(
        'type' => 'string',
        'options' => array('default' => null, 'notnull' => false),
    );

We know that the type of string is limited to 255 chars, and we should automatically add some basic validation because of that instead of silently truncating characters.

I see this as being relevant to : https://www.concrete5.org/developers/bugs/8-1-0/bug-express-e-mail-and-phone-valid-input-testing-incomplete/
Assuming it's intended to limit the form content length, I'm game for this! :+1:

This should be mostly resolved by https://github.com/concrete5/concrete5/pull/5198 but we probably want to consider some actual limits on certain attributes,
for example, and email or phone number should not be 4 million+ characters, its probably reasonable to limit that to something smaller.


Fix clitools entry point.

Addresses https://github.com/napalm-automation/napalm/issues/249
Related to https://github.com/napalm-automation/napalm/pull/250

Thanks! Will test and merge once you have solved the tiny issue in napalm-automation/napalm#250.

Fixes napalm-automation/napalm#249


feat: Support wildcard character or file list for io.copy

Describe the feature
io.copy support wildcard character or file list, for example:
io.copy("*.py","/")
io.copy(["a.py","b.py"],"/")
Why do you need this feature?
Copy multiple files of the same type, very redundant.
now:
io.copy("app.py", "/")
io.copy("args.py", "/")
io.copy("gen.py", "/")
io.copy("strings.py", "/")
io.copy("styles.py", "/")
io.copy("utils.py", "/")

expected:
io.copy("*.py", "/")

or
io.copy(["a.py","b.py"],"/")

Additional context
No response

I think it can copy the directory. But it doesn't support the ignore file.


Allow using onPanDrag while scrollEnabled=true

Does any other open PR do the same thing?
Previous work has been done in #2692 - where this was extracted from - and #2314.
The previous PR, while good, has been open since February and has seen no activity lately. Also, there are some concerns regarding the API it defines, where onZoom, onPinch etc could be a better alternative.
This PR is a much narrower scope than the previous one, and should be simpler to get merged. It also presents no conflict if the former one would get merged.
What issue is this PR fixing?
Allows onPanDrag to be used while scrollEnabled is true, which is currently not supported. This seems like the best entry point for determining when the user has interacted with the Map. There are multiple requests relating to this requirement; #2756 #1038 #1620 (last not explicitly the same - but seems like the same base issue).
Also, removes the [drag setMaximumNumberOfTouches:1]; call so that if one pans with more than one finger, the handler still fires. This is consistent with how it works on Android.
How did you test this PR?

Create a Map component with an onPanDrag handler
Pan the map
onPanDrag should fire

Verification
This is only enabled on iOS with Apple Maps. It's been tested both on the simulator and on device.

Any chance to get this reviewed, @christopherdro ?

Is the note on https://github.com/react-native-community/react-native-maps/blob/master/docs/mapview.md#events

Callback that is called when user presses and drags the map. NOTE: for iOS scrollEnabled should be set to false to trigger the event

then still valid?


Overhaul NCMC integrators

I think this should fix the NCMC issues we previously had with the symmetric protocol (which we had disabled before now). The integrator structure for both VVVR and GHMC have been overhauled, and protocol and shadow work are also separately accumulated now.

I've disabled the addUpdateContextState() since I think it is unnecessary and would be much more complex to incorporate this flawlessly.

This is ready to review and merge.

I should note that local testing of test_elimination.py show a test failure that does not show up on travis, but that I don't know how to fix:
======================================================================
ERROR: Failure: UnboundLocalError (local variable 'wt_chain' referenced before assignment)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/cbio/jclab/home/chodera/miniconda/lib/python2.7/site-packages/nose/loader.py", line 251, in generate
    for test in g():
  File "/cbio/jclab/home/chodera/github/choderalab/perses/perses.choderalab/perses/tests/test_elimination.py", line 160, in test_alchemical_elimination_mutation
    topology_proposal = proposal_engine.propose(system, topology)
  File "/cbio/jclab/home/chodera/github/choderalab/perses/perses.choderalab/perses/rjmc/topology_proposal.py", line 251, in propose
    old_chemical_state_key = self.compute_state_key(old_topology)
  File "/cbio/jclab/home/chodera/github/choderalab/perses/perses.choderalab/perses/rjmc/topology_proposal.py", line 937, in compute_state_key
    for wt_res, res in zip(wt_chain._residues, chain._residues):
UnboundLocalError: local variable 'wt_chain' referenced before assignment

I think it is important to document why, exactly, this change fixes some problem.

I think it is important to document why, exactly, this change fixes some problem.

There were two contributing issues:

OpenMM bugs in CustomIntegrator that have been fixed in the latest dev builds: https://github.com/pandegroup/openmm/pull/1630 https://github.com/pandegroup/openmm/pull/1603
The logical flow of the original NCMC integrators was flawed and has been restructured

In playing with various integrator structures with @gregoryross in openmm-constph, it looks like there may still be issues with manually unrolled integrator structures, but any problems that may appear there do not appear in the flow-controlled version we use here.


Fix CORS issues #38

Consider this a WIP as more can be done but this fixes issues mentioned in #38
Work done

Now use fetch api instead of xhr/$.JSON
Removes default values in the store for start and end chunks
Gets the initial values from the bin2file.json
No more synchronous requests
Added rudimentary loading indicator to App.js

Considerations
I'm adding a few considerations since I didn't have much time to get rid of all of that "hacky" feeling.

You will see a console error for "second chunk was earlier than first". I've left the error message but not sure this is actually an issue. If you confirm this is not an issue, I will remove the console output, or if it is, then i'll fix.
Should be a bit more robust but there's still a few code smells. e.g:

(my bad) I think the initial state for start/endChunkURL should come from one level higher. I think the call to bin2file.json should come from the component one level up in the tree. Either in its constructor or in getDerivedState (I don't know which).
presence of loadFirstJson and loadSecondJson could be made into one function (they essentially do the same thing). Can detect if is first or last from the app state

This is good work and it really sounds like you know what you're doing.  Jacob, Please merge in the latest from master. Hannah Sewell finished a PR (this is good news) that also affects App.js in a fairly straightforward way.  She moved the Link display to a second < Layer >.

"second chunk was earlier than first": not a timing issue except that processArray() can't proceed until both files have arrived.  This is due to the need to calculate cumulative X offset.
presence of loadFirstJson and loadSecondJson could be made into one function: I'm agnostic.  They're thin functions.
getDerivedState: I don't know enough about this design pattern to comment.  Perhaps it's best to hold off on a further refactor until we decide if a list of chunks may be necessary (I'm leaning towards yes) as I mentioned here:

For extra credit: We may eventually want to read in more than two chunks at a time.  If mobx could take a list of chunk filenames, that would allow us more spanning.  However I avoided coding this at first because it get complicated and mobx.arrays are a pain.  I recommend making the current startChunkURL and endChunkURL more robust first before attempting to replace them entirely.  That can be a separate issue following on #22.

For the first bullet point, I think that issue is fixed since I used promises. It's assured that the first file can't arrive after the second

Oh and forgot to mention about polyfills. I don't know what browsers you're targeting but fetch is fairly modern. As far as I'm aware, create react app polyfills it for you but I could be wrong

Thanks for updating with Hannah's content.  We'll get this reviewed today.

Thanks Jacob, I really like the style of code changes you made.  It'll be great to be rid of jQuery.  Just one more thing.  This appears to introduce a state bug in the leftOffset under some circumstances.
To reproduce:

Start with a fresh page
Mouse over a colored cell in a link column
Observed: Matrix jumps to the left but the arrows and link columns do not move.

Here's a video of the bug:
https://drive.google.com/open?id=1fs4NMysLUG_LYnxbA0RPl9PNLJKd2H_r

I've verified this bug exists in 05d5b52d45ff2ee629dbad1c2e7f45768cce0db3, so it doesn't have to do with CrookedY's changes being merged.

JavaScript is stylish!
I'll have a look at it later on today or tomorrow. Thanks for verifying it's my change that made the problem.

I really want to merge your changes, but we can't until the render bug is tracked down for the sake of following procedure.  It's also possible your change simply exposed an underlying race condition, which is why I was trying to get a history of React state "commits".  Something like rendering a component before leftOffset gets calculated would cause this problem.

It's a wide procedure. Unfortunately, I'm away from my computer at the moment.
Okay that's good to know and probably quite likely. I can't see anything obvious in my commits that would have caused this.
If it's urgent, feel free to push to my repo, I can give you rights.

Thank you for submitting this pull request.
After merging with the recent updates to master 9dd54119a57d9457f420909de03631301b03de4a
we merged in your changes
git checkout -b jacobwindsor-master master\ngit pull https://github.com/jacobwindsor/Schematize.git master\n
git branch

Testing was done with https://graph-genome.github.io/testing.
During step 10 we encountered an issue.  To reproduce this bug do the following.

Refresh the view
When mousing over the Links the screen does not refresh properly.

We will wait to merge this PR #46 until this can be fixed.
Issue does not exist in 9dd54119a57d9457f420909de03631301b03de4a

@jacobwindsor A fresh npm start and everything works fine. But hitting F5 and hovering over a link breaks the application as displayed above. The moment I click to show me the next part of the pangenome, this does not occur again. Maybe some states are not resolved properly?

Based on testing results from Christine, we've determined the weird render state isn't specific to this PR.  It's a state inconsistency that's possible to access from several different avenues. So Jacob didn't introduce this.  We'll merge this PR and open 2 new issues based on Christine's testing.


What is psigma ??!!!

In some places in the code we assume psigma = (E-E_0)/E_0 but on the manual we write psigma = (E-E_0)/(beta_0 P_0 c)  which is in fact  psigma = (E-E_0)/(beta_0^2 E_0). What's the truth?

psigma = (E-E_0)/(beta_0 P_0 c) = (E-E_0)/(beta_0^2 E_0)

psigma is not particularly useful, it originates from the conjugate momentum of sigma = s - beta0 T used in SixTrack. If there are bugs, it seems a good occasion to move to ptau= (E-E_0)/(P_0 c)= (gamma-gamma_0)/(beta_0 gamma_0 c)

Moved to tau (xpart v0.6.0, xtrack v0.11.0)


Use consistent capitalization for hash URIs

Current use:
/#catalog
/#Organization
/#dataset

I notice the Datatank is mixing them too (including trailing slash):
/#organization
#Government
#Mobility

@pietercolpaert, is there is a convention? I'd propose:
/#Catalog
/#Organization
/#Dataset

Pieter said its better without the slash
So I'll go with
#Catalog
#Organization
#Dataset

There is no convention at all, the choice is free

Confirmed fixed in v1.2


[backport -> release/3.2.x] fix(dns): Eliminate asynchronous timer in syncQuery() to prevent hang

backport https://github.com/Kong/kong/pull/11900 and https://github.com/Kong/kong/pull/11386 to 3.2.x

After we merged this https://github.com/Kong/kong/pull/12031, we can rebase this branch with the latest release 3.2.x.


[RESOLVED] High ICMP ping response time

Problem started at 02:09:14 on 2023.05.25
Problem name: High ICMP ping response time
Host: MF-FG
Severity: Warning
Operational data: Value: 179ms
Original problem ID: 22389
Event details in Zabbix: http://<IP_ADDRESS>/tr_events.php?triggerid=23132&eventid=22389

Problem has been resolved in 6m 0s at 02:15:14 on 2023.05.25
Problem name: High ICMP ping response time
Host: MF-FG
Severity: Warning
Original problem ID: 22389


python3Packages.gradio: init at 3.20.1

Description of changes
This PR adds gradio by @gradio-app, a web server framework for making AI (anything really) demos.
Original PR is #187032, which was closed due to a mass-ping resulting from a poorly thought out rebase onto staging.
Close #235184

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Pinging reviewers from prior pr:
@happysalada @Janik-Haag @VergeDX @SuperSandro2000

@ofborg build python3Packages.gradio

Have you seen usage of nix-update-script ?
It will generate automatic prs with updates of your packages. It will reduce the cost of maintenance quite a bit in my experience.
If you search for it in the repo there are plenty if exemples.
Sometimes it takes 1-2 days to generate an update but thats still better than having to do it by hand everytime.

@mweinelt the ci fails on aarch64-linux, the logs don't seem to say much. This could be due to a timeout.

My computer and network access this summer is spotty. I added you to my nixpkgs repo if you're interested in joint maintainership :)
@ofborg build python3Packages.gradio

This looks good to me.
Im happy to be pinged in case there are problems discovered during the staging PR.
@mweinelt might have more comments, so waiting a bit more.

@mweinelt ready for another round of review.
I've added two commits that will later be squashed. I didn't want to replace the commits so that the OP would loose attribution.
let me know if anything.

@SuperSandro2000 I think I've addressed all of your comments, but let me know if there is something that you think should be addressed differently.
I've also just tested to make sure this actually works.

I think this pr, with a few squashes, looks good! Thank you so much @happysalada for being such a champ in pushing this though.
Once some time when I'm back in civilization I will do a pr to bump gradio to handle the cyclical depdendency on gradio-client introduced in the latest version, and try to hoist the conftest network xfail code to a drop-in check hook.

I squashed your commits, adding you as co-author. This now builds on master, and I leave checking the 2k+ affected packages to the staging gods.

This looks good to me, waiting another 24h just in case someone else wants to have a look

@Janik-Haag there are 2 potential improvements (1 formatting and one that could be discussed potentially later). Would you mind if we merged and kept those further improvements for a subsequent PR. The current patch is still in a "let's try it and we will see if we like it" state, so I would say it would be a good candidate to revisit this in 6 months.
gradio is a dependency used by many machine learning tools, there is value in having it available earlier than later.
Feel free to let me know if you disagree, no problem at all.

I would prefer to do it right the first time since it's not a lot of work to do these changes and we currently can't merge it anyways since the builds are failing and it doesn't seem to be because of a timeout.

Fixed the formatting issue, and move the python code to a separate file. Now you get nice colors in your editor.
As (elaborate here)[https://github.com/NixOS/nixpkgs/pull/241558#discussion_r1285123038] i opt not to turn the python code into a patch.
Of course the build fails, it's staging and the closure is ginormous. When cherry-picked on master it builds fine.

I wanted to make sure the build passed.  Let me know if you disagree.
(in the process of testing I discovered several failures with upstream dependencies that I have reported. Note where related to the changes made here.)

Are you sure the package even works if there are so many test failures?

First step is merging i think. Trying to rebuild all staging everytime to test us just not very efficient. First we make sure nothing breaks, then we should be able toake changes that impact much less things. I have locally packaged fast chat, ill open a PR as soon as this hits master.

You can just do git switch master and then cherry pick the commits like this: git cherry-pick fdfcecedb72326c19772b4720a2e9c6df46347db 4324abf898a0ff5a14fe4b740f81f97ddc2e34a8 90500b167084b87a385c68f29de128418bc2da35 if you checked out this branch locally with gh pr checkout 241558. and then you can test locally without problem

I'm unable to reproduce your error. I also tried python 3.11, but that was blocked by tensorflow being marked broken on master. I'm disabling test_in_interface due to being flaky.

Are you sure the package even works if there are so many test failures?

The gradio devs tend to move fast and break things, and the functionality they provide is incredibly simple but pimped with lots of fancy webdev. The gradio test suite is mostly integration testing with all the major ML frameworks, hence the ginormous check closure.
Perhaps the future is doCheck = false, but as of now i consider this testing coverage a major success

You can just do git switch master and then cherry pick the commits like this: git cherry-pick fdfcecedb72326c19772b4720a2e9c6df46347db 4324abf898a0ff5a14fe4b740f81f97ddc2e34a8 90500b167084b87a385c68f29de128418bc2da35 if you checked out this branch locally with gh pr checkout 241558. and then you can test locally without problem

I use this one-liner, assuming this per is checked out in the worktree ../gradio-init:
git checkout master;  (cd ../gradio-init/; git log --oneline) | head -n3 | cut -d\  -f1 | tac | xe git cherry-pick

I'm surprised how nixpkgs-review does not have a cherry-pick or rebase strategy for staging PRs. (https://github.com/Mic92/nixpkgs-review/issues/287)

@Janik-Haag have your concerns been addressed ?
Regarding my test failure, ive had other test failures on staging that couldbt be reproduced. I have had one on thrift, that was tested on another machine and couldnt be reproduced. So the one i had could be just a fluke.

I currently am in a train so I have whonky internet, I'll try to check the package in the next ~8 hours once I have a stable connection again. But I really don't want to merge anything broken.

At this point, i dont think this paclage is broken, i think there could be flakiness in the tests or some potential problems with downstream dependents, but am not sure how we much more we could test.
Happy to wait one more day if you think we can potentially get more information.

I rebased onto the recent merge base between master and staging, and now it builds. So far I've waited one hour for the ofborg eval to be done. I'll marking the PR ready again once the eval is done, just for safety.

It now builds fine but when I try to run the binary it throws this error:
./result/bin/gradio
Traceback (most recent call last):
  File "/nix/store/3nap39bl8f7wbrbzfkl1dbj15lnab8aj-python3.10-gradio-3.20.1/bin/.gradio-wrapped", line 6, in <module>
    from gradio.reload import run_in_reload_mode
  File "/nix/store/3nap39bl8f7wbrbzfkl1dbj15lnab8aj-python3.10-gradio-3.20.1/lib/python3.10/site-packages/gradio/__init__.py", line 3, in <module>
    import gradio.components as components
  File "/nix/store/3nap39bl8f7wbrbzfkl1dbj15lnab8aj-python3.10-gradio-3.20.1/lib/python3.10/site-packages/gradio/components.py", line 33, in <module>
    from gradio.blocks import Block, BlockContext
  File "/nix/store/3nap39bl8f7wbrbzfkl1dbj15lnab8aj-python3.10-gradio-3.20.1/lib/python3.10/site-packages/gradio/blocks.py", line 24, in <module>
    from gradio import (
  File "/nix/store/3nap39bl8f7wbrbzfkl1dbj15lnab8aj-python3.10-gradio-3.20.1/lib/python3.10/site-packages/gradio/networking.py", line 17, in <module>
    from gradio.routes import App
  File "/nix/store/3nap39bl8f7wbrbzfkl1dbj15lnab8aj-python3.10-gradio-3.20.1/lib/python3.10/site-packages/gradio/routes.py", line 25, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

Great catch! I fixed it, and also added a postInstallCheck testing for this.
I'm surprised the pythonImportsCheck runs in the python stdenv.
I hosted a gradio service last year, so it didn't strike me to check the shell entrypoint as I kept bumping the version of the prior PR.

@Janik-Haag on the contrary, thank you for your thoroughness.
@pbsds thanks for sticking with this PR for so long!

Unfortunately, It doesn't look like I can build this on aarch64-darwin :/ — Anyone has an idea? I'm not super knowledgeable with the ins and outs of python.
error: builder for '/nix/store/j5m87whdmc1ryjy21l99r0rinf3rzmlz-python3.11-gradio-client-0.10.1.drv' failed with exit code 4;
       last 10 log lines:
       > /nix/store/9nasxys60wirmhqs07akfb68n83pip6x-python3.11-gradio-sans-client-4.20.1/lib/python3.11/site-packages/gradio/routes.py:81: in <module>
       >     from gradio.server_messages import (
       > /nix/store/9nasxys60wirmhqs07akfb68n83pip6x-python3.11-gradio-sans-client-4.20.1/lib/python3.11/site-packages/gradio/server_messages.py:64: in <module>
       >     class UnexpectedErrorMessage(BaseModel):
       > /nix/store/9nasxys60wirmhqs07akfb68n83pip6x-python3.11-gradio-sans-client-4.20.1/lib/python3.11/site-packages/gradio/server_messages.py:65: in UnexpectedErrorMessage
       >     msg: Literal[ServerMessage.unexpected_error] = ServerMessage.unexpected_error
       > /nix/store/c05vbvkjxarxkws9zkwrcwrzlsx9nd68-python3-3.11.8/lib/python3.11/enum.py:784: in __getattr__
       >     raise AttributeError(name) from None
       > E   AttributeError: unexpected_error
       > /nix/store/gamm276nsv0l0qm71qhywpz833niaxkx-stdenv-darwin/setup: line 1587: pop_var_context: head of shell_variables not a function context


Get rid of duplicate clamp functions

Perhaps this could be added as a mathutils module?

Whole math module would be usefull. We could add there Matrix struct that will simplify filters or convolve function that will be internally called by that functions.

Done by #247


Correct and usable normal distribution sampler

normal and standardNormal were severely broken based on sampling.
Not sure what algorithm was used, but this one is Box-Muller, which is very simple and proven.
The only disadvantage is that normal does not have bounded output, but this is acceptable for most uses of normally distributed variables, and has a more typical interface.

Thanks for the PR! This is kinda over my head but the code looks simple enough. I'll have time to investigate further over the weekend. It will also require a major version bump, so that implies a bit of scrutiny, planning, and coordination.

Here's some info on the Box-Muller transform.  It's pretty common way of generating numbers on the standard normal distribution from uniforms, and might actually be the most efficient in this case.  Other options usually require rejection sampling, i.e. repeated sampling of uniforms until they meet certain conditions, which seems tricky to do with Elm's generators.

Tell me about it...
Okay, so this looks good except the (1 - u) term. Why not just use u? Both expressions will be uniformly distributed on [0,1].

That was based on the assumption that the range of Random.float 0 1 doesn't actually include 1, which is the case for most RNG implementations, because of the asymmetry of 2's complement.  Thus, (1 - u) is meant to turn the [0,1) range into (0,1], avoiding ln(0) = NaN.
Looking at core, that assumption was correct, but it also looks like it could generate a number slightly below the minimum (about -1.16e10)!  If that's not intended behavior for Random.float, then my standardNormal would possibly need to be changed along with it.

Ah yes, in fact, that might cause problems with this implementation regardless, because if u is even slightly negative, then sqrt (logBase e (1 - u)) is not a real number.  Okay, maybe this should be postponed.

Oh no! Here's what I recommend:

Write the best normal generator you can with the current implementation of float core. If that means throwing a floor into this PR, do it.
Attempt to reproduce the issue using elm-random-pcg, of which I am also the author. If you can, open an issue or PR and we can fix it. If it doesn't repro, it's probably not worth fixing in core, since that library will be replacing core's implementation at some point.

Oh, that's good to know.  That indeed looks like a much better float implementation.  Substituting minInt and maxInt for the peeled values produces 0 and 0.9999999999999999, as expected.
I suppose I was a bit too hasty in opening an issue on core.  I'll amend this PR as well as that issue.

I wrote a program to find the bounds of what float 0 1 produces over ten million numbers, and core produces ( -1.1641532182693481e-10, 0.9999997616978362) and PCG produces (3.2658150628250837e-9, 0.9999999993300184). So I have both reproduced the issue in core and failed to reproduce it in PCG. That said, I don't think you did the wrong thing by opening the issue!
So, it seems that in the mean time the (1 - u) trick will help avert disaster if core produces a slightly negative number, since it will become a number slightly greater than one. So does that mean this is good to merge?

The above fixes were still necessary because there was an error with the parentheses and because a number slightly greater than one would produce a NaN from the square root.  But, now it should be good to go.

... I am so sorry.  There's still a missing ).

Got it, no worries.


FEATURE:NAVBAR NEEDED

Feature Summary
WE NEED A NAVBAR SO THAT WE CAN EASILY NIVAGATE TO THE OTHER PAGES WITH EASE OTHERVPAGES LIKE:- PRIVACY POLICIES,TESTIMONIALS ,DONATE AND MANY MORE.
Description
THIS PAGE IS HIGHLY NEEDED IN WEBSITE IT HELPS MAKE SINGLE PAGE WEBSITE INTO MULTI PAGE.
Proposed Solution
No response
Alternatives Considered
No response
Screenshots/Logs
No response
Additional Information

[X] I have searched for existing feature requests
[X] I am willing to help implement this feature
[X] I can provide more details or clarification if needed

@YadavAkhileshh THIS IS A HIGHLY PRIOR ISSUE BECAUSE EARLIER THIS FEATURE IS AVAILABLE IN THIS WEBSITE BUT NOW SOMEHOW THIS FEATURE IS UNABLE TO FIND


qemu: fix darwin build

Motivation for this change
ZHF: #122042
cc @NixOS/nixos-release-managers
Hoping to beat the branch-off, but I guess there's no real hurry in it either. We can always backport. 🙂
Fixes #121903. This is simply the workaround created by @mroi from this comment: https://github.com/NixOS/nixpkgs/issues/121903#issuecomment-836315774
I suggest we try move ahead with this and create a separate ticket for the aarch64-darwin issue.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[x] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Added a release notes entry if the change is major or breaking
[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 123872 at 44a41a2e run on x86_64-linux 1

6 packages marked as broken and skipped:

aqemu
linuxPackages-libre.virtualbox
linuxPackages_hardkernel_4_14.virtualbox
linuxPackages_hardkernel_latest.virtualbox
linuxPackages_latest-libre.virtualbox
multibootusb

35 packages skipped due to time constraints:

alpine-make-vm-image
cloud-init
cloud-utils
cot (python38Packages.cot)
libguestfs
linuxPackages.virtualbox (linuxPackages_5_10.virtualbox)
linuxPackages_4_14.virtualbox
linuxPackages_4_19.virtualbox
linuxPackages_4_4.virtualbox
linuxPackages_4_9.virtualbox
...

4 packages built successfully:

qemu_kvm
qemu_test
qemu_xen (qemu_xen_4_10)
qemu_xen-light (qemu_xen_4_10-light)

3 suggestions:

warning: missing-phase-hooks
installPhase should probably contain runHook preInstall and runHook postInstall.
Near pkgs/applications/virtualization/qemu/utils.nix:11:3:
   |
11 |   installPhase = ''
   |   ^

warning: build-tools-in-build-inputs
makeWrapper is a build tool so it likely goes to nativeBuildInputs, not buildInputs.
Near pkgs/applications/virtualization/qemu/default.nix:57:3:
   |
57 |   buildInputs =
   |   ^

warning: name-and-version
Did you mean to pass pname instead of name to mkDerivation?
Near pkgs/applications/virtualization/qemu/utils.nix:4:3:
  |
4 |   name = "qemu-utils-${version}";
  |   ^

Near pkgs/applications/virtualization/qemu/utils.nix:5:3:
  |
5 |   version = qemu.version;
  |   ^

Result of nixpkgs-review pr 123872 run on x86_64-darwin 1

11 packages marked as broken and skipped:

alpine-make-vm-image
cloud-init
cloud-utils
multibootusb
python38Packages.guestfs
python39Packages.guestfs
qemu_full
qemu_xen
qemu_xen-light
qemu_xen_4_10
qemu_xen_4_10-light

1 package failed to build:

out-of-tree

6 packages built:

cot (python38Packages.cot)
python39Packages.cot
qemu
qemu-utils
qemu_kvm
qemu_test

I had a failure on my darwin system, maybe the failure is not related

github.com/mattn/go-sqlite3
# github.com/mattn/go-sqlite3
In file included from _cgo_export.c:4:
callback.go:15:10: fatal error: 'sqlite3-binding.h' file not found

Looks like out-of-tree had a dependency on qemu, and fixing qemu unblocked that build. But yes, that failure is separate.


Remove gMSA references from Install-ADServiceAccount and Uninstall-ADServiceAccount

PR Summary

Removed text stating that Install-ADServiceAccount and Uninstall-ADServiceAccount interacts with gMSAs.
Added to notes of Install-ADServiceAccount and Uninstall-ADServiceAccount that the cmdlets are not necessary for gMSAs to run.
Fixes #2109

Rationale:
These cmdlets do not have anything to do with gMSAs, as stated today by Steve Syfuhs, working with Windows and Authentication at Microsoft:

It [Install-ADServiceAccount] does nothing. It’s a literal no-op. A gMSA is not bound to the machine like an MSA. There is no offline anything. It just doesn’t have a password until it can talk to a DC.

Source: https://twitter.com/SteveSyfuhs/status/1707271067293917606
If one or both of the cmdlets do in fact interact with gMSAs, then I suggest you clarify the interaction and add that information to #2109
PR Checklist

[x] Descriptive Title: This PR's title is a synopsis of the changes it proposes.
[x] Summary: This PR's summary describes the scope and intent of the change.
[x] Contributor's Guide: I have read the contributors guide.
[x] Style: This PR adheres to the style guide.

@microsoft-github-policy-service agree

Done @JohanFreelancer9


Early exit from cache save

Update deps
Early exit in cache save
Rebuild action

Test run: https://github.com/maciektr/cairo-test-run/actions/runs/10807604895

Test run: https://github.com/maciektr/cairo-test-run/actions/runs/10807921109/job/29979759413


Incoming Messages are encrypted

Hi Guys, just noticed that my incoming messages are now encrypted for some reason.It started doing this a few hours ago, WhatsApp probably changed something.I also noticed that not all my numbers are affected but only some of them.

@KapteinBabbalas Its normal, read other issues. The .NET API doesnt support E2E encryption, if you want to read encrypted messages use the php API
Regards

@KapteinBabbalas I have the solution for now, but i dont know for how long. you have to replace the respectives values on your WhatsAppApi.Settings.WhatsConstants class with this
public const string WhatsAppDigest = "xmpp/s.whatsapp.net";
    public const string WhatsAppHost = "c3.whatsapp.net";

    public const string WhatsAppRealm = "s.whatsapp.net";

    public const string WhatsAppServer = "s.whatsapp.net";

    public const string WhatsGroupChat = "g.us";

    public const string WhatsAppVer = "2.11.16";

    public const int WhatsPort = 443;

    public const string Device = "iPhone";

    public const string UserAgent = "WhatsApp/2.12.68 S40Version/14.26 Device/Nokia302";

    public static NumberStyles WhatsAppNumberStyle = NumberStyles.AllowLeadingSign | NumberStyles.AllowDecimalPoint;

    public static DateTime UnixEpoch = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc);

and then replace the respective value on WhatsAppApi.Register.WaToken with this
string token = "PdA2DJyKoUrwLw1Bg6EIhzh502dF9noR9uFCllGk1425519315543" + number;
remember to comment the old code, for me its working, but i dont know for how long. I think whatsapp will block iPhone too.

recompile the project and use the new whatsappapi.dll

@ricardoperovano  I got the same problem  yesterday ，and  iPhone will be block soon after

@michaelqu I think too, but now its working


AccessTokenCache questions / feedback

Differences between AccessToken.getToken and AccessToken.getTokenSync
In Mono<AccessToken> getToken it has a repeatWhenEmpty loop to keep resubscribing to the Mono until a token is acquired. AccessToken getTokenSync doesn't have a loop to keep trying while the return value is null. A quick review of the code shows that it should be possible for null to be returned from getTokenSync.
Logging level when a token is acquired
AccessTokenCache logs an INFO level log when a token is acquired. Should look into lowering the verbosity on this as it isn't really useful as information and is more of a verbose level log.
Tracing can cover observability gap there (records a span when actual request is made along with the status). We can also consider adding cache/token metrics  - there was a customer asking how to record them for .NET identity lib.

The log level piece is fixed by #41648.
@alzimmermsft Looking at the code for retrieveTokenSync/retrieveToken', I think the idea here is that in the sync case as we have taken a lock we know we are the only thread manipulating the cache. I don't actually see retrieveTokenSync` returning a null ever; it'll either refresh or use the cached token, and return that. If it fails it will throw.

Logging was resolved in #41648. After analysis I think the null concern is addressed.


scss/operator-no-unspaced rule fails on negative numbers in interpolated values

This is similar issue to #450
There is no fails
$fooBarFooBarFooBarFooBarFooBarFooBarFooBarFooBarFooBarFooBarFooBar: color.scale(
  #000000,
  $lightness: -50%,
  $alpha: -50%
);

In this case which similar to case above there is fails on $lightness and $alpha values
$fooBarFooBarFooBarFooBarFooBarFooBarFooBarFooBarFooBarFooBarFooBar: #{color.scale(
    #000000,
    $lightness: -50%,
    $alpha: -50%
  )};

In this case there is fail on $lightness value
$c: #{color.scale(#000000, $lightness: -50%, $alpha: -50%)};

It's reproduced with<EMAIL_ADDRESS>and<EMAIL_ADDRESS>

Thanks! Sounds like a bug.


Appstudio update test-component-pac-razl

Pipelines as Code configuration proposal
To start the PipelineRun, add a new comment with content /ok-to-test
For more detailed information about running a PipelineRun, please refer to Pipelines as Code documentation Running the PipelineRun
To customize the proposed PipelineRuns after merge, please refer to Build Pipeline customization

Pipelines as Code CI/test-component-pac-razl-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
8 seconds
init

✅ Succeeded
11 seconds
clone-repository

✅ Succeeded
46 seconds
build-container

✅ Succeeded
17 seconds
deprecated-base-image-check

✅ Succeeded
1 minute
inspect-image

✅ Succeeded
34 seconds
clamav-scan

✅ Succeeded
12 seconds
clair-scan

✅ Succeeded
12 seconds
sbom-json-check

✅ Succeeded
37 seconds
label-check

✅ Succeeded
7 seconds
show-summary

✅ Succeeded
7 seconds
show-sbom