BUG FIX: pearson3.stats returning moments as arrays rather than scalars.

Reference issue
Closes gh-11746
What does this implement/fix?
pearson3.moment was returning an array and now returns a scalar. Fixed by converting pearson._stats output variables to scalars.
Additional information
This is my first contribution so please review carefully. I followed the contribution guide fairly carefully. Changes pass testing and I added a regression test in _continous_distns.py.
There is a comment that I would like reviewed from the distribution:
    # The _argcheck function in rv_continuous only allows positive
    # arguments.  The skew argument for pearson3 can be zero (which I want
    # to handle inside pearson3._pdf) or negative.  So just return True
    # for all skew args.

I'm honestly not sure what the original intent here was as far as handling a skew of zero, so I just  defaulted to returning zeros in _stats, but please let me know if this is incorrect. I think that portion may be outside of my understanding of statistics / the standard approach for scipy devs in cases like this.

I started working on this but I got into a rabbit-hole, then got sidetracked without being able to document my thoughts. If I remember correctly, I uncovered a deeper issue - the rv_continuous.moment function is supposed to accept array arguments for loc, scale, etc., but it doesn't. Can you confirm?

I'm honestly not sure what the original intent here was as far as handling a skew of zero, so I just defaulted to returning zeros in _stats, but please let me know if this is incorrect

No, we can't just return zero unless that is the correct answer. I think the point was that for skews below a certain threshold, the calculation was to be performed by norm instead of the usual code.

Let me rework the norm section of that - I think that makes more sense now. I was assuming because it was returning an empty array it was supposed to be zero but that wouldn't make sense.
It looks like the following few examples work as expected:

expon.moment(n=np.array(1), loc=np.array(1), scale=np.array(1))
Out[16]: 2.0

pearson3.moment(n=np.array(1), skew=np.array(1), loc=np.array(1))
Out[13]: 1.0

I added _norm_stats as a method to pearson3 because, while methods like norm._pdf and norm._cdf are defined outside of the class, norm._stats is not; to replicate using those moments, this seemed like the easiest approach.
The rest of the pearson3 dist uses the methods defined outside of the normal class.

@WillTirone thanks for your patience, but it's going to take me a bit to get to this. But can you confirm that  moment is supposed to accept array arguments for loc and scale but doesn't? Accepting arrays is the root of the problem -because that's why moment might be to return an array, and if that's not working properly, we might need to hold off on this.

@mdhaber no worries at all! If you take a look at my comment above yours, I think that's what you're mentioning? Unless I'm misunderstanding. But it seems like it's accepting arrays just fine and returning a scalar. Do you have maybe a code snippet of something that's not working properly?

I meant arrays with more than one element.

I'm not positive if I'm using arrays in the args as intended, but from reading the docs https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.moment.html it looks like these arguments should accept array-like and are returning a ValueError. Let me know if these args aren't what you were thinking as far as them accepting arrays.
expon.moment(n=np.array([1,2]), loc=np.array([1,2]), scale=np.array([1,2]))
Traceback (most recent call last):

  File "<ipython-input-3-dc42edd1473f>", line 1, in <module>
    expon.moment(n=np.array([1,2]), loc=np.array([1,2]), scale=np.array([1,2]))

  File "/home/will/Desktop/code/scipy/scipy/stats/_distn_infrastructure.py", line 1164, in moment
    if not (self._argcheck(*args) and (scale > 0)):

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

pearson3.moment(n=np.array([1,2]), skew=np.array([1,2]))
Traceback (most recent call last):

  File "<ipython-input-5-e4467db2ee48>", line 1, in <module>
    pearson3.moment(n=np.array([1,2]), skew=np.array([1,2]))

  File "/home/will/Desktop/code/scipy/scipy/stats/_distn_infrastructure.py", line 1164, in moment
    if not (self._argcheck(*args) and (scale > 0)):

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Is this issue stalled due to another issue? If so, should I close it below and start on another issue?
Thanks :)

hi @mdhaber just wanted to check on this! Thanks

Thanks for the reminder. I'll take a look this week.

@WillTirone I dug into this today. When I marked gh-11746 as a good first issue, I hadn't really looked at pearson3 carefully. It turns out that there are a few other issues (gh-9168, gh-11186, gh-12192) that made this more challenging than it would have been otherwise. It was really not a good first issue! Can I take this one over and help you find a better first issue?

@mdhaber sure, that sounds good to me! Appreciate the help. Let me know if I need to close this one or what. I'll also keep an eye out on the new issues that come up.

Yes, let's close this. Would you take a look at gh-7168? That is a much better place to start.


Update GetIcons.js

HotFix 🚨 #823 removed #811 functionality, breaking a lot of files

Did it completely remove the code somehow or is the code still there but now not doing anything?

So, don't you have to remove the changes made with the PR #811 in this PR?
Or, there is an easy way to do that.
Just like unmerge  a PR?

Ok, just looking through the files. All the .toLowerCase code is still there in user profile component so I’m not sure why this has broken. Do we still need all the extra code in user profile component if the lower casing can be applied like this directly in getIcon?

Ok, just looking through the files. All the .toLowerCase code is still there in user profile component so I’m not sure why this has broken. Do we still need all the extra code in user profile component if the lower casing can be applied like this directly in getIcon?

The lower casing can be directly done in get icons component , we can remove the lower casing code from user profile without any issues.
Btw, what kind of broken files you were referring to?
I didn't saw anything broken on live site. 🙄

The lower casing can be directly done in get icons component , we can remove the lower casing code from user profile without any issues.

I’m not so sure as I think the icon name might also be used for styling, we’ll need to check it to make sure

The issue is here afaik https://github.com/EddieHubCommunity/LinkFree/pull/846#issuecomment-999789484

Can you add a telegram hover colour to the config/icons.json file also?

You can also correct arrow left icon it is not lower cased instead It is camel cased in switch case

We now have two identical pull requests for the same issue. As this one is a bit all over the place in the discussion (as it refers to other issues to start with) I'll close this one to make sure the wrong issues don't get closed when the PR is merged. Thanks for contributing :)


Build optimization from source code

A useful project.
But if it could use some package managers like govendor,glide to simplify the building from source.And add some dependencies version control.
It will be convenient to Integrate the glide into the Makefile.
Thanks.

I also ran into the #86.

With respect to #86, what dive version did you use? What docker version are you running?
Can you provide a stack trace?
With respect to vendoring tools, I've been trying to stick with vanilla modules (but there have been a few issues). What benefit would we get by vendoring with glide and govendor over vanilla modules?

1.The commit sha of master is 8b4c3021da69d946f8ce5d6b8698abc4eb0ed1c4.
2.The stack trace:
$ make all
rm -rf build
rm -rf vendor
go clean
go build -o build/dive
filetree/data.go:8:2: cannot find package "github.com/cespare/xxhash" in any of:
	/usr/local/go/src/github.com/cespare/xxhash (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/cespare/xxhash (from $GOPATH)
image/docker_image.go:7:2: cannot find package "github.com/docker/docker/client" in any of:
	/usr/local/go/src/github.com/docker/docker/client (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/docker/docker/client (from $GOPATH)
filetree/node.go:9:2: cannot find package "github.com/dustin/go-humanize" in any of:
	/usr/local/go/src/github.com/dustin/go-humanize (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/dustin/go-humanize (from $GOPATH)
cmd/analyze.go:6:2: cannot find package "github.com/fatih/color" in any of:
	/usr/local/go/src/github.com/fatih/color (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/fatih/color (from $GOPATH)
filetree/tree.go:5:2: cannot find package "github.com/google/uuid" in any of:
	/usr/local/go/src/github.com/google/uuid (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/google/uuid (from $GOPATH)
utils/exit.go:5:2: cannot find package "github.com/jroimartin/gocui" in any of:
	/usr/local/go/src/github.com/jroimartin/gocui (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/jroimartin/gocui (from $GOPATH)
cmd/root.go:10:2: cannot find package "github.com/k0kubun/go-ansi" in any of:
	/usr/local/go/src/github.com/k0kubun/go-ansi (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/k0kubun/go-ansi (from $GOPATH)
ui/detailsview.go:7:2: cannot find package "github.com/lunixbochs/vtclean" in any of:
	/usr/local/go/src/github.com/lunixbochs/vtclean (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/lunixbochs/vtclean (from $GOPATH)
cmd/root.go:11:2: cannot find package "github.com/mitchellh/go-homedir" in any of:
	/usr/local/go/src/github.com/mitchellh/go-homedir (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/mitchellh/go-homedir (from $GOPATH)
filetree/node.go:11:2: cannot find package "github.com/phayes/permbits" in any of:
	/usr/local/go/src/github.com/phayes/permbits (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/phayes/permbits (from $GOPATH)
cmd/build.go:4:2: cannot find package "github.com/sirupsen/logrus" in any of:
	/usr/local/go/src/github.com/sirupsen/logrus (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/sirupsen/logrus (from $GOPATH)
cmd/analyze.go:7:2: cannot find package "github.com/spf13/cobra" in any of:
	/usr/local/go/src/github.com/spf13/cobra (from $GOROOT)
	/Users/didi/Documents/go_workspace/src/github.com/spf13/cobra (from $GOPATH)
make: *** [build] Error 1

3.I found vanilla modules is fine.Just a suggestion and thanks for your help.


Deprecate high-level API

IMPORTANT: Please search among the Pull requests before creating one.
Title

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] New feature (non-breaking change which adds functionality)

Description

I've added DeprecationWarning for all high-level cellrank.tl functions + added notes to API in docs saying that:

high-level API will be removed in 2.0
cellrank.tl.kernels -> cellrank.kernels
cellrank.tl.estimators -> cellrank.estimators
cellrank.ul.models -> cellrank.models

How has this been tested?

Have release notes been modified?

Closes

closes #677
closes #678

Thanks Mike! We're moving quickly now!

I'll give this my review later today. We should also mention (in the docs) that cellrank.plremains unchanged, meaning that there will be 4 major modules from 2.0:

pl
kernel
estimators
models (we might need a more descriptive name here, like gene_models.

models (we might need a more descriptive name here, like gene_models.

For now, I'd stick to models (gene_models seems to long + I don't like the _). Merging this.


Feature/check for faststart enabled

I needed a way to only check if fast start had been applied on the media file, so I copy / pasta'd the first portion of the fastStartImpl method and added a check for a new moovAtomEncountered boolean at the end.
I also added some functional tests with an 8kb test video for both methods, and finally, I upgraded gradle to the latest version (4.8).

Coping/Pasting code instread of refactoring (e.g. introduce method) is never a good option.

Yes I agree, and I've refactored the locate logic into a method.
I did use some Java 8 Streams logic, a lambda, and some other current Java stuff which isn't necessary, but it is nice and clean code. If you want to keep the backward compatibility on this lib then please let me know and I'll remove it and also mark the source and target compatibility in the build.gradle file.
Some additional thoughts (not included in this PR):

This could really benefit from including a real logging library like slf4j or something similar.
The exceptions malformedFile & unsupportedFile could be thrown earlier inside the new locate method, but that would change existing behavior, so if you are open to another PR it would probably be wise to


Add references to other communication channels

We already have some communication channels that can be used to help our
users, but they are not very discoverable. Adding several links to the
README might help us to reach more Clangd users.

Nothing fancy here, just adding a couple links to already existing channels for more visibility. @sam-mccall let me know if you think it's a good idea and I will add those to LLVM's documentation page and to the GitHub pages.

Nothing fancy here, just adding a couple links to already existing channels for more visibility. @sam-mccall let me know if you think it's a good idea and I will add those to LLVM's documentation page and to the GitHub pages.

I totally agree that there's some value on adding that to clangd.github.io as well.


sp_migrate_user_to_contained - add warning that it's irreversible

It would be nice if there was a note on here mentioning that there is no way of reversing this command.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3affeb6a-cf0f-1b7a-abfd-7661435f863b
Version Independent ID: d4899b08-843d-2c5d-4bfa-d7a0ab130653
Content: sp_migrate_user_to_contained (Transact-SQL) - SQL Server
Content Source: docs/relational-databases/system-stored-procedures/sp-migrate-user-to-contained-transact-sql.md
Product: sql
Technology: system-objects
GitHub Login: @stevestein
Microsoft Alias: sstein

Hello @fowl2! I have added a note at the top of the page. It should go live soon. Thank you so much for your suggestions! Your feedback is appreciated! I'll go ahead and close this request now, but if you have any concerns, please feel free to let us know. Thanks again.
https://github.com/MicrosoftDocs/sql-docs-pr/pull/10606


Motion detail view: Amendments Display error

Describe the bug
Motions with amendments have display errors in the 'Show all changes' function. If amendments are not displayed in the motion, you can use this function to show the amendments. This works correctly. The error now occurs if you activate the amendments via 'Show all changes', then click on the motions menu and open the motion again, the amendments are visible even though the 'Show all changes' checkbox is not checked. The browser seems to remember the activated setting.
This also occurs if another menu instead of motions is opened.
Example - The amendment is visible, but the checkbox is not marked:

How to Reproduce
Steps to reproduce the behavior:

open a meeting
open motions and create a motion with an amendment
open the motion detail view and check 'show all changes'. The amendment should now be visible.
open motions menu
open motion -> Amendment is visibile and 'Show all changes' box is not marked

Expected behavior
When switching from a motion with activated 'Show All Changes' function to another menu and back to the motion, the browser should not have remembered the previously set checkmark. When reopening the motion, the amendments should not be visible if the check mark is not set.
Important: It is possible for amendments to be permanently displayed in the motion. This should continue to behave as usual and is excluded from this bug fix.

This error was introduced with 4.1.17 - maybe with the motion detail refactoring?


Fixes to SPOPropertyBag

Pull Request (PR) description
Fixes an issue where false positive drifts were being detected.
This Pull Request (PR) fixes the following issues
N/A

This change is

Codecov Report

Merging #430 into Dev will increase coverage by 0%.
The diff coverage is 12%.

@@          Coverage Diff          @@
##             Dev    #430   +/-   ##
=====================================
  Coverage     90%     90%
=====================================
  Files        104     105    +1
  Lines      11217   11283   +66
  Branches      10      10
=====================================
+ Hits       10197   10262   +65
- Misses      1010    1011    +1
  Partials      10      10


`ast_to_json' function - unability to free memory

There are 3 commenting tags "FIXME" in code where a try to free memory gets an application to be crashed for unknown reasons. Current assumtion is that not only heap memory is tried to be deallocated. But why?..

Fixed by allocating memory in `concat_array' for a single value.


[Bug]: No support for setting redirect URL when validating OAuth2 code.

Package
@lucia-auth/oauth
Describe the bug
When calling validateCallback on Discord OAuth2 provider, there's no option to set redirect URL, it will simply grab the redirect URL in the config.
This is a problem because I have different endpoints for a Discord OAuth2 callback that do different functions and Discord will throw an error when the redirect URL passed is not the same redirect URL used to generate the authorization URL.
I suspect this same issue would affect other OAuth2 providers too.
It would be great if we could pass in some optional config on validateCallback like:
await discordAuth.validateCallback(code, {
  redirectURL: "https://something.yeah/callback",
});

I would be happy to submit a pull request fixing this issue too.

Hm, might be better if you just create 2 Discord OAuth instances instead.

I didn't think of that, that's a good idea.  Thank you for the suggestion.  For now, I've just got a patch for the package in my repo.
I think because we have the option of specifying the redirect URL when creating the authorisation link we should have the option of specifying the redirect URL when validating.
If you don't like this change I think removing the ability to set the redirect URL when creating the authorisation link would make sense although I could see use cases for it.

I think because we have the option of specifying the redirect URL when creating the authorisation link we should have the option of specifying the redirect URL when validating.

Yeah, looks like some providers require the redirect uri when requesting the tokens , while others don't. Should be removed for Discord, or scrap it all together.

Removed the option to redirect uri to getAuthorizationUrl() with `<EMAIL_ADDRESS>


ScrapeEndpoint.params doc clarification, how to use params?

https://github.com/GoogleCloudPlatform/prometheus-engine/blob/v0.4.3-gke.0/doc/api.md#scrapeendpoint
Params says it is a map[string][]string, which is a map of string, Array of strings. How do I define this? Trying to pass in Authentication: Basic foobar, and I haven't been able to guess how I am supposed to do this.
latest version, added to our GCP k8s cluster today, non-autopilot

For anyone else who finds this, they are urlparams not headers, not sure what I was thinking.


Non resampling indices checks

Pull Request Checklist:

[ ] This PR addresses an already opened issue (for bug fixes / features)
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)
[ ] HISTORY.rst has been updated

After merging to master and before closing the branch:

[ ] bumpversion (minor / major / patch) has been called on master branch
[ ] Tags have been pushed

What kind of change does this PR introduce?

In the case where an indicator is not performing a resampling on the input data, checking for missing values per period didn't make sense. When pushing #312 , we had this discussion and decided to default the frequency to daily, which is the expected (and validated) input frequency anyway. However, the checks.missing_any performs a quite heavy check by computing the number of non-null days in each period. When the period is a day, then this is the exact same a da.isnull(), but with a lot more overhead.
This proposed solution assumes that the indicator doesn't perform resampling if freq is not given. Thus, the missing() check means all inputs must exist for an output.

Does this PR introduce a breaking change?

No.

Other information:

A caveat of the previous solution was that missing_any() was resampling to freq, meaning the output dataset would be rechunked daily. That's how I realized the error.
Also, running atmos.tg on a large datasets (not computing, simply creating the output) is 500x times faster with this change.

Pull Request Test Coverage Report for Build 1358

5 of 5 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 89.551%

Totals

Change from base Build 1356:
0.01%

Covered Lines:
3428

Relevant Lines:
3828

💛  - Coveralls


[BUG] Timestamp of frame event is behind host current monotonic time, impossibility

Describe the bug
I am currently exploring the use of FrameEvent message. It nice as it has low latency, as it is being sent out at the time that exposure stops and data starts streaming (do correct me if I am wrong)
However, the timestamp the FrameEvent message gives it strange. Sometimes the timestamp indicates a future time on the host monotonic clock. This is by a couple of millisecond
Minimal Reproducible Example
Reproducable on a OAK-D-LR (R0M0E0), with hardware sync from GPIO enabled
#!/usr/bin/env python3

# import cv2
import depthai as dai
import time
import datetime

enable_fsin = True
fps = 15
exposure = 8000

def configureColorAR0234(socket, streamName, frameEvent = False):

    # Define source and output
    cam = pipeline.create(dai.node.ColorCamera)
    cam.setBoardSocket(socket)
    xoutVideo = pipeline.create(dai.node.XLinkOut)

    xoutVideo.setStreamName(streamName)

    cam.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1200_P)
    # cam.setVideoSize(1920, 1200)
    # camA.initialControl.setAutoExposureEnable()
    cam.initialControl.setManualExposure(exposure, 200)

    cam.setFps(fps)

    xoutVideo.input.setBlocking(False)
    xoutVideo.input.setQueueSize(3)

    # Linking
    cam.isp.link(xoutVideo.input)

    if enable_fsin:
        # camA.initialControl.setFrameSyncMode(dai.CameraControl.FrameSyncMode.INPUT)
        cam.initialControl.setExternalTrigger(1,0)

    if frameEvent:
        xoutFrameEvent = pipeline.create(dai.node.XLinkOut)
        xoutFrameEvent.setStreamName("frame_event")
        xoutFrameEvent.input.setBlocking(False)
        xoutFrameEvent.input.setQueueSize(3)
        cam.frameEvent.link(xoutFrameEvent.input)

# Create pipeline
pipeline = dai.Pipeline()

configureColorAR0234(dai.CameraBoardSocket.CAM_A, "video_cama", True)
configureColorAR0234(dai.CameraBoardSocket.CAM_B, "video_camb")
configureColorAR0234(dai.CameraBoardSocket.CAM_C, "video_camc")

script = pipeline.create(dai.node.Script)
script.setProcessor(dai.ProcessorType.LEON_CSS)
script.setScript("""
    import GPIO
    import time

    MX_PIN = 41
    activeLevel = GPIO.HIGH
    timeActive = 0.001
    period = 0.1
    GPIO.setup(MX_PIN, GPIO.OUT, GPIO.PULL_DOWN)

    while True:
        time.sleep( period - timeActive )
        GPIO.write(MX_PIN, activeLevel)
        time.sleep( timeActive )
        GPIO.write(MX_PIN, not activeLevel)
""")

with dai.Device() as device:
    device.startPipeline(pipeline)

    # device.setTimesync(datetime.timedelta(seconds=5) , 20, False)

    videoA = device.getOutputQueue(name="video_cama", maxSize=4, blocking=False)
    videoB = device.getOutputQueue(name="video_camb", maxSize=4, blocking=False)
    videoC = device.getOutputQueue(name="video_camc", maxSize=4, blocking=False)

    frameEvent = device.getOutputQueue(name="frame_event", maxSize=4, blocking=False)

    while True:

        videoAIn = videoA.tryGet()
        videoBIn = videoB.tryGet()
        videoCIn = videoC.tryGet()

        frameEventIn = frameEvent.tryGet()

        if frameEventIn is not None:
            frameTs = frameEventIn.getTimestamp().total_seconds()

            hostNow = time.monotonic()

            if (frameTs >= hostNow):
                print(f"warning: host time is ahead of frame time {(frameTs - hostNow) * 1e3} ms, impossible (host = {hostNow}, frame ts = {frameTs})")

        # if videoAIn is not None:
        #     # cv2.imshow("camA", videoAIn.getCvFrame())
        #     print(f"camA {videoAIn.getTimestamp().total_seconds()}")

        # if videoBIn is not None:
        #     # cv2.imshow("camB", videoBIn.getCvFrame())
        #     print(f"camB {videoBIn.getTimestamp().total_seconds()}")

        # if videoCIn is not None:
        #     # cv2.imshow("camC", videoCIn.getCvFrame())
        #     print(f"camC {videoCIn.getTimestamp().total_seconds()}")

        # if cv2.waitKey(1) == ord('q'):
        #     break

Expected behavior
The timestamp should be older than the current host monotonic clock.
Screenshots
warning: host time is ahead of frame time 2.3374679999506043 ms, impossible (host = 2322.667376532, frame ts = 2322.669714)
warning: host time is ahead of frame time 2.6387759999124683 ms, impossible (host = 2322.771066224, frame ts = 2322.773705)
warning: host time is ahead of frame time 2.4276489998555917 ms, impossible (host = 2322.875276351, frame ts = 2322.877704)
warning: host time is ahead of frame time 2.6331309995839547 ms, impossible (host = 2322.979070869, frame ts = 2322.981704)
warning: host time is ahead of frame time 2.591991999906895 ms, impossible (host = 2323.083114008, frame ts = 2323.085706)
warning: host time is ahead of frame time 2.4143000000549364 ms, impossible (host = 2323.1872887, frame ts = 2323.189703)
warning: host time is ahead of frame time 2.5047849999282334 ms, impossible (host = 2323.291200215, frame ts = 2323.293705)
warning: host time is ahead of frame time 2.7853550000145333 ms, impossible (host = 2323.394918645, frame ts = 2323.397704)
warning: host time is ahead of frame time 2.896051999869087 ms, impossible (host = 2323.498969948, frame ts = 2323.501866)
warning: host time is ahead of frame time 2.7746120003939723 ms, impossible (host = 2323.602928388, frame ts = 2323.605703)
warning: host time is ahead of frame time 2.8310480001891847 ms, impossible (host = 2323.706874952, frame ts = 2323.709706)

@chengguizi
Using getTimestamp is currently vulnerable to such issue where time sync jumps in time with large offset into a wrong direction. This is current limitation of Timesync algorithm used. TBD to be properly resolved.
Also, time.monotonic is not the right clock to compare against. Use dai.Clock.now()

Thanks, glad to know the issue is known. Really hope it can be reolved eventually, timesync is quite important when using VPU data with external controllers :)
What is the difference of the dai.Clock.now() from the system's monotonic clock though?

@chengguizi if its a usb based system, then this issue is less glaring, but it's present more on PoE devices.
I am actually not 100% sure on the differences of the underlying impl. But in one if our other cases we've had there was a difference between a Pythons time.monotonic vs c++s steady_clock

Got it @chengguizi  Yes I am using C++ mainly. Python is only for debugging. Good to know you guys observe this undesirable discrepency of python monotonic vs C++ steady_clock. Will keep in mind.
I do use USB based system. but i still see quite a few millisecond of that slack in time estimation. Not sure the best way around, but is your current algorithm similar to this?
https://github.com/mavlink/mavros/blob/ros2/mavros/src/plugins/sys_time.cpp#L416

Yes, its similar to that - I'll see about giving that a try and have a branch with that timesync impl compared to the current one - perhaps it'll act better.
Maybe over the weekend if time will permit.

Thanks! This one estimate both the alpha and beta (time scale). Also it has sigmoid function to help convergence. Look forward!

@chengguizi
timesync_alt_impl - you'll have to build the wheels/lib locally.
Please give it a try & any kind of exploration using it - validation/comparisions/... as well as with PoE devices, would be greatly appreciated, to aid in faster mainlining if it turns out to be much better:)

The fix doesnt seem to work well. I actually seen time regression happens even on image timestamps (the next image in the sequence have earlier ts). This means the time sync correction might be making jumps.
Btw I am doing the following:

I tried both turning on and off IMU streaming
I did m_device->setTimesync(std::chrono::milliseconds(5000), 20, false); during setup. Does it affect anything?


A question about relative motion

In the code, use matrix = pose2motion(poses_SEs, skip) to calculate the relative motion.
But compare pose_curr , pose_next and calculate motion (motion = pose_curr.I*pose_next),  I find the result is not inconsistent.
What is the reason for this?
Thanks

I think the calculation should be a matrix multiplication rather than a numeric multiplication.


Possible issue with dispatching events

Consider the following component:
<template>
    <input
      @keyup.enter="$emit('enter')"
      @keyup.esc="$emit('esc')"
      @click="$emit('click')"
    >
</template>

And the following test:
import TestInput from '../../src/components/TestInput'

import { shallowMount } from '@vue/test-utils'

describe('TestInput', () => {
    it('should fire the enter key', () => {
        const wrapper = shallowMount(TestInput)

        wrapper.find('input').element.dispatchEvent(
            new KeyboardEvent('keyup', { keyCode: 13 })
        )

        console.log(wrapper.emitted())
        expect(wrapper.emitted().enter).toHaveLength(1)
    })

    it('should fire click', () => {
        const wrapper = shallowMount(TestInput)

        wrapper.find('input').element.dispatchEvent(
            new MouseEvent('click')
        )

        console.log(wrapper.emitted())
        expect(wrapper.emitted().click).toHaveLength(1)
    })
})

We would expect the test to pass (it did on Vue 2/VTU 1) but I've noticed two problems.

The event for the keyboard event never fires.
I believe this may be an underlying problem of how the keyup modifier is working on Vue 3 possibly. Im testing this with the 3.1 compact build.

The click event fires, BUT for both tests there's an additional emit that is happening.

The second test actually fails because the emitted() is adding a second event.
  console.log tests/unit/TestInput.spec.js:24
    {
      'hook:beforeMount': [ [] ],
      'hook:mounted': [ [] ],
      click: [ [], [ [MouseEvent] ] ]
    }

   Expected length: 1
    Received length: 2
    Received array:  [[], [{"isTrusted": false}]]

The first element in the array is the actual click emit that the input should fire, the second one is being injected by either vtu or vue.
On the first test, with the keyboard event we have a similar problem.
  console.log tests/unit/TestInput.spec.js:13
    {
      'hook:beforeMount': [ [] ],
      'hook:mounted': [ [] ],
      keyup: [ [ [KeyboardEvent] ] ]
    }

There is an emitted keyup event by the component, where there should not be since there is no keyup $emit anywhere.
Hope this was clear!
package.json for reference:
{
  "name": "vue31vtu",
  "version": "0.1.0",
  "private": true,
  "scripts": {
    "serve": "vue-cli-service serve",
    "build": "vue-cli-service build",
    "test:unit": "vue-cli-service test:unit",
    "lint": "vue-cli-service lint"
  },
  "dependencies": {
    "@vue/compat": "^3.1.0",
    "core-js": "^3.6.5",
    "vue": "^3.1.0"
  },
  "devDependencies": {
    "@vue/cli-plugin-babel": "~4.5.0",
    "@vue/cli-plugin-eslint": "~4.5.0",
    "@vue/cli-plugin-unit-jest": "~4.5.0",
    "@vue/cli-service": "~4.5.0",
    "@vue/compiler-sfc": "^3.1.0",
    "@vue/test-utils": "^2.0.0-rc.10",
    "babel-eslint": "^10.1.0",
    "eslint": "^6.7.2",
    "eslint-plugin-vue": "^7.0.0",
    "typescript": "~3.9.3",
    "vue-jest": "^5.0.0-0"
  },
  "eslintConfig": {
    "root": true,
    "env": {
      "node": true
    },
    "extends": [
      "plugin:vue/vue3-essential",
      "eslint:recommended"
    ],
    "parserOptions": {
      "parser": "babel-eslint"
    },
    "rules": {},
    "overrides": [
      {
        "files": [
          "**/__tests__/*.{j,t}s?(x)",
          "**/tests/unit/**/*.spec.{j,t}s?(x)"
        ],
        "env": {
          "jest": true
        }
      }
    ]
  },
  "browserslist": [
    "> 1%",
    "last 2 versions",
    "not dead"
  ]
}

Reopening because even with @xanf 's fix, the keyboard event is still not working with:
it('should fire the enter key', () => {
        const wrapper = shallowMount(TestInput)

        wrapper.find('input').element.dispatchEvent(
            new KeyboardEvent('keyup', { keyCode: 13 })
        )

        console.log(wrapper.emitted())
        expect(wrapper.emitted().enter).toHaveLength(1)
    })
    ```

We can just do like this: https://github.com/vuejs/test-utils/pull/1441/files.
Using dispatchEvent isn't documented, I'm not sure exactly on how we can make your test pass, but considering this isn't documented or officially supported, I'm going to go and and recommend we test like I've proposed in the PR.
If there's a case not covered by the example in the PR, we can re-open or make a new issue. For now I don't see any obvious actionable here, so I'll close this as inactive.


Modified download page to link to VS2015 runtimes

There's one issue I know this PR has that I think we should discuss, and that is how we link to the download. My concern is that users won't read and will download the 32 bit version by accident. Some possible solutions I can think of are:

Tell users to download the 64 bit runtime in the text (Done in this PR)

Directly link to the 64 bit version of the download (This link looks like it can break)

Host the runtime our selves and link to that

Any opinions?

This is... a little heavy handed. Perhaps do this temporarily? 6 months or so?


[feature] Include Neon-JS capabilities

By including this capabilities we gonna be able to easily test the network with other interfaces, instead of only the neo-python

Perhaps, as discussed with @igormcoelho, NEON gonna be embedded with Phatom and inside a container testing ecosystem.


Missing workloads to be analysed when the cluster has a considerable number of apps running

What steps did you take and what happened:
I have a lab cluster with 71 active replicasets and 38 daemonsets and the most I have managed to cover is 58 counting resources of both types. I think it is not powerful enough to cover such a big cluster. I would like to know if you know what are the limitations in terms of cluster size, because in my case, I am trying to test the tool on a lab cluster that does not have too much workloads compared to prod.
Here's an example of logs:
# 1. Get StatefulSet workload from cache
2023-12-14T07:08:12Z	DEBUG	reconciler.vulnerabilityreport	Getting workload from cache	{"kind": "StatefulSet", "name": {"name":"alertmanager-prom-op-istio-gateways-alertmanager","namespace":"istio-gateways"}}

# 2. Submit scan for the workload selected
2023-12-14T07:08:12Z	DEBUG	reconciler.vulnerabilityreport	Submitting a scan for the workload	{"kind": "StatefulSet", "name": {"name":"alertmanager-prom-op-istio-gateways-alertmanager","namespace":"istio-gateways"}, "podSpecHash": "6bc8c5bbcb"}

# 3. Checks if limits is OK. Yes in that case, only 1 workload scanning
2023-12-14T07:08:12Z	DEBUG	reconciler.vulnerabilityreport	Checking scan jobs limit	{"pre scan job processing for workload:": "alertmanager-prom-op-istio-gateways-alertmanager", "count": 1, "limit": 20}
"controllerKind": "StatefulSet", "controllerName": "alertmanager-prom-op-istio-gateways-alertmanager"}

# 4. Creates scan job
2023-12-14T07:08:12Z	DEBUG	reconciler.vulnerabilityreport	Creating scan job for the workload	{"kind": "StatefulSet", "name": "alertmanager-prom-op-istio-gateways-alertmanager", "namespace": "istio-gateways", "podSpecHash": "6bc8c5bbcb"}

# 5. Repeats step 1 (?)
2023-12-14T07:08:12Z	DEBUG	reconciler.vulnerabilityreport	Getting workload from cache	{"kind": "StatefulSet", "name": {"name":"alertmanager-prom-op-istio-gateways-alertmanager","namespace":"istio-gateways"}}

# 6. Get StatefulSet workload from cache
2023-12-14T07:08:12Z	DEBUG	reconciler.vulnerabilityreport	Scan job already exists	{"kind": "StatefulSet", "name": {"name":"alertmanager-prom-op-istio-gateways-alertmanager","namespace":"istio-gateways"}, "podSpecHash": "6bc8c5bbcb", "job": "trivy-system/scan-vulnerabilityreport-65c99bbcfd"}

# 7. Deletes job because of scan failed (why failed??)
2023-12-14T07:08:13Z	DEBUG	reconciler.scan job	Deleting failed scan job	{"job": "trivy-system/scan-vulnerabilityreport-65c99bbcfd"}

I have installed trivy-operator with the latest helm chart. Here is my values.yaml :
excludeNamespaces: "default" operator:   exposedSecretScannerEnabled: false   configAuditScannerEnabled: false   rbacAssessmentScannerEnabled: false   infraAssessmentScannerEnabled: false   metricsVulnIdEnabled: true   logDevMode: true   scanJobTTL: "24h"   scanJobsConcurrentLimit: 20 service:   metricsPort: 8080 serviceMonitor:   enabled: true   namespace: trivy-system   labels:      monitoredBy: monitoring-prometheus trivyOperator:   scanJobCompressLogs: false trivy:   command: filesystem   additionalVulnerabilityReportFields: "Target,Class,PackagePath,PackageType"   slow: false   resources:     requests:       cpu: 300m       memory: 300M     limits:       cpu: 700m       memory: 700M securityContext:   runAsUser: 0
What did you expect to happen:
I expect vulnerability reports for all workloads on the cluster but it isn't working as expected
Environment:

Trivy-Operator version: 0.19.1
Kubernetes version: v1.27.7-gke.1121000
OS: macOS 14.1.1

@borja-rivera do you have any logs for failed pod ?

@borja-rivera any update ?

@chen-keinan There are still workloads to be analysed. I disabled OPERATOR_SBOM_GENERATION_ENABLED but it still doesn't generate all the reports it should. I also excluded the kubernetes-specific kube-system namespace, but there are still apps missing.

@borja-rivera disabling OPERATOR_SBOM_GENERATION_ENABLED will not help in your case as you have errors , could you please share more log data with error to so I could track error context ?

Another recurring error is:
2023-12-27T08:47:33Z	ERROR	reconciler.scan job	Scan job container	{"job": "trivy-system/scan-vulnerabilityreport-684c445888", "container": "manager", "status.reason": "Error", "status.message": "2023-12-27T08:47:30.159Z\t\u001b[31mFATAL\u001b[0m\tinit error: cache error: unable to initialize the cache: unable to initialize fs cache: failed to create cache dir: mkdir /var/trivyoperator/trivy-db/fanal: permission denied\n"}
github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).processFailedScanJob
	/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:320
github.com/aquasecurity/trivy-operator/pkg/vulnerabilityreport/controller.(*ScanJobController).SetupWithManager.(*ScanJobController).reconcileJobs.func1
	/home/runner/work/trivy-operator/trivy-operator/pkg/vulnerabilityreport/controller/scanjob.go:81
sigs.k8s.io/controller-runtime/pkg/reconcile.Func.Reconcile
	/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/reconcile/reconcile.go:111
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
	/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:119
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:316
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/home/runner/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/home/runner/go/pkg/mod/[sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:227](http://sigs.k8s.io/controller-runtime@v0.16.3/pkg/internal/controller/controller.go:227)

2023-12-27T08:47:33Z	DEBUG	reconciler.scan job	Deleting failed scan job	{"job": "trivy-system/scan-vulnerabilityreport-684c445888"}

@borja-rivera I see a basic permission error for creating cache folder , are you running special config , can you share you config maps and operator deployment EnvVars ?
2023-12-27T08:47:33Z	ERROR	reconciler.scan job	Scan job container	{"job": "trivy-system/scan-vulnerabilityreport-684c445888", "container": "manager", "status.reason": "Error", "status.message": "2023-12-27T08:47:30.159Z\t\u001b[31mFATAL\u001b[0m\tinit error: cache error: unable to initialize the cache: unable to initialize fs cache: failed to create cache dir: mkdir /var/trivyoperator/trivy-db/fanal: permission denied\n"}

@chen-keinan Here's the trivy-operator-trivy-config config map:
apiVersion: v1 data:   trivy.additionalVulnerabilityReportFields: Target,Class,PackagePath,PackageType   trivy.command: filesystem   trivy.dbRepository: ghcr.io/aquasecurity/trivy-db   trivy.dbRepositoryInsecure: "false"   trivy.filesystemScanCacheDir: /var/trivyoperator/trivy-db   trivy.imagePullPolicy: IfNotPresent   trivy.imageScanCacheDir: /tmp/trivy/.cache   trivy.javaDbRepository: ghcr.io/aquasecurity/trivy-java-db   trivy.mode: Standalone   trivy.repository: ghcr.io/aquasecurity/trivy   trivy.resources.limits.cpu: 700m   trivy.resources.limits.memory: 700M   trivy.resources.requests.cpu: 300m   trivy.resources.requests.memory: 300M   trivy.severity: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL   trivy.skipJavaDBUpdate: "false"   trivy.slow: "false"   trivy.supportedConfigAuditKinds: Workload,Service,Role,ClusterRole,NetworkPolicy,Ingress,LimitRange,ResourceQuota   trivy.tag: 0.47.0   trivy.timeout: 5m0s   trivy.useBuiltinRegoPolicies: "true" kind: ConfigMap metadata:   annotations:     meta.helm.sh/release-name: trivy-operator     meta.helm.sh/release-namespace: trivy-system   creationTimestamp: "2023-12-27T08:46:22Z"   labels:     app.kubernetes.io/instance: trivy-operator     app.kubernetes.io/managed-by: Helm     app.kubernetes.io/name: trivy-operator     app.kubernetes.io/version: 0.17.1     helm.sh/chart: trivy-operator-0.19.1   name: trivy-operator-trivy-config   namespace: trivy-system   resourceVersion: "579337566"   uid: 0146087f-48f8-45ae-8993-1e4e311f17ca
And trivy-operator config map:
apiVersion: v1 data:   compliance.failEntriesLimit: "10"   node.collector.imageRef: ghcr.io/aquasecurity/node-collector:0.0.9   nodeCollector.volumeMounts: '[{"mountPath":"/var/lib/etcd","name":"var-lib-etcd","readOnly":true},{"mountPath":"/var/lib/kubelet","name":"var-lib-kubelet","readOnly":true},{"mountPath":"/var/lib/kube-scheduler","name":"var-lib-kube-scheduler","readOnly":true},{"mountPath":"/var/lib/kube-controller-manager","name":"var-lib-kube-controller-manager","readOnly":true},{"mountPath":"/etc/systemd","name":"etc-systemd","readOnly":true},{"mountPath":"/lib/systemd/","name":"lib-systemd","readOnly":true},{"mountPath":"/etc/kubernetes","name":"etc-kubernetes","readOnly":true},{"mountPath":"/etc/cni/net.d/","name":"etc-cni-netd","readOnly":true}]'   nodeCollector.volumes: '[{"hostPath":{"path":"/var/lib/etcd"},"name":"var-lib-etcd"},{"hostPath":{"path":"/var/lib/kubelet"},"name":"var-lib-kubelet"},{"hostPath":{"path":"/var/lib/kube-scheduler"},"name":"var-lib-kube-scheduler"},{"hostPath":{"path":"/var/lib/kube-controller-manager"},"name":"var-lib-kube-controller-manager"},{"hostPath":{"path":"/etc/systemd"},"name":"etc-systemd"},{"hostPath":{"path":"/lib/systemd"},"name":"lib-systemd"},{"hostPath":{"path":"/etc/kubernetes"},"name":"etc-kubernetes"},{"hostPath":{"path":"/etc/cni/net.d/"},"name":"etc-cni-netd"}]'   report.recordFailedChecksOnly: "true"   scanJob.podTemplateContainerSecurityContext: '{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"privileged":false,"readOnlyRootFilesystem":true}'   vulnerabilityReports.scanner: Trivy kind: ConfigMap metadata:   annotations:     meta.helm.sh/release-name: trivy-operator     meta.helm.sh/release-namespace: trivy-system   creationTimestamp: "2023-12-27T08:46:22Z"   labels:     app.kubernetes.io/instance: trivy-operator     app.kubernetes.io/managed-by: Helm     app.kubernetes.io/name: trivy-operator     app.kubernetes.io/version: 0.17.1     helm.sh/chart: trivy-operator-0.19.1   name: trivy-operator   namespace: trivy-system   resourceVersion: "579337565"   uid: 457a6483-6f94-43ad-97ed-19fa8140c03b

Env:
  - env:
    - name: OPERATOR_NAMESPACE
      value: trivy-system
    - name: OPERATOR_TARGET_NAMESPACES
    - name: OPERATOR_EXCLUDE_NAMESPACES
      value: default,kube-system
    - name: OPERATOR_TARGET_WORKLOADS
      value: pod,replicaset,replicationcontroller,statefulset,daemonset,cronjob,job
    - name: OPERATOR_SERVICE_ACCOUNT
      value: trivy-operator
    - name: OPERATOR_LOG_DEV_MODE
      value: "true"
    - name: OPERATOR_SCAN_JOB_TTL
      value: 24h
    - name: OPERATOR_SCAN_JOB_TIMEOUT
      value: 5m
    - name: OPERATOR_CONCURRENT_SCAN_JOBS_LIMIT
      value: "20"
    - name: OPERATOR_CONCURRENT_NODE_COLLECTOR_LIMIT
      value: "1"
    - name: OPERATOR_SCAN_JOB_RETRY_AFTER
      value: 30s
    - name: OPERATOR_BATCH_DELETE_LIMIT
      value: "10"
    - name: OPERATOR_BATCH_DELETE_DELAY
      value: 10s
    - name: OPERATOR_METRICS_BIND_ADDRESS
      value: :8080
    - name: OPERATOR_METRICS_FINDINGS_ENABLED
      value: "true"
    - name: OPERATOR_METRICS_VULN_ID_ENABLED
      value: "true"
    - name: OPERATOR_HEALTH_PROBE_BIND_ADDRESS
      value: :9090
    - name: OPERATOR_VULNERABILITY_SCANNER_ENABLED
      value: "true"
    - name: OPERATOR_SBOM_GENERATION_ENABLED
      value: "false"
    - name: OPERATOR_VULNERABILITY_SCANNER_SCAN_ONLY_CURRENT_REVISIONS
      value: "true"
    - name: OPERATOR_SCANNER_REPORT_TTL
      value: 24h
    - name: OPERATOR_CACHE_REPORT_TTL
      value: 120h
    - name: CONTROLLER_CACHE_SYNC_TIMEOUT
      value: 5m
    - name: OPERATOR_CONFIG_AUDIT_SCANNER_ENABLED
      value: "false"
    - name: OPERATOR_RBAC_ASSESSMENT_SCANNER_ENABLED
      value: "false"
    - name: OPERATOR_INFRA_ASSESSMENT_SCANNER_ENABLED
      value: "false"
    - name: OPERATOR_CONFIG_AUDIT_SCANNER_SCAN_ONLY_CURRENT_REVISIONS
      value: "true"
    - name: OPERATOR_EXPOSED_SECRET_SCANNER_ENABLED
      value: "false"
    - name: OPERATOR_METRICS_EXPOSED_SECRET_INFO_ENABLED
      value: "false"
    - name: OPERATOR_METRICS_CONFIG_AUDIT_INFO_ENABLED
      value: "false"
    - name: OPERATOR_METRICS_RBAC_ASSESSMENT_INFO_ENABLED
      value: "false"
    - name: OPERATOR_METRICS_INFRA_ASSESSMENT_INFO_ENABLED
      value: "false"
    - name: OPERATOR_METRICS_IMAGE_INFO_ENABLED
      value: "false"
    - name: OPERATOR_METRICS_CLUSTER_COMPLIANCE_INFO_ENABLED
      value: "false"
    - name: OPERATOR_WEBHOOK_BROADCAST_URL
    - name: OPERATOR_WEBHOOK_BROADCAST_TIMEOUT
      value: 30s
    - name: OPERATOR_SEND_DELETED_REPORTS
      value: "false"
    - name: OPERATOR_PRIVATE_REGISTRY_SCAN_SECRETS_NAMES
      value: '{}'
    - name: OPERATOR_ACCESS_GLOBAL_SECRETS_SERVICE_ACCOUNTS
      value: "true"
    - name: OPERATOR_BUILT_IN_TRIVY_SERVER
      value: "false"
    - name: TRIVY_SERVER_HEALTH_CHECK_CACHE_EXPIRATION
      value: 10h
    - name: OPERATOR_MERGE_RBAC_FINDING_WITH_CONFIG_AUDIT
      value: "false"
    - name: OPERATOR_CLUSTER_COMPLIANCE_ENABLED
      value: "true"

@borja-rivera since you are running in filesystem mode , you'll have to run scanJobPodTemplateContainerSecurityContext to runAsUser: 0
you need to uncomment this flag
see also in our docs First Option: Filesystem Scanning

@chen-keinan After a few days with this option activated, there are workloads that it now analyses that it did not do before. However, it still doesn't analyse all the workloads, as I have made a parallel script to analyse all the pods with the trivy tool, and it is analysing 101 different images, while trivy-operator is analysing only 23.

Excluding kube-system namespace, which reports a lot of vulns but it's not relevant for us at the moment

@borja-rivera try this workaround for this issue, disable this helm flag. and restart operator
Note: this bug which has been fixed in latest v0.18.0-rc

@chen-keinan I'm currently testing on it. Although there are some more reports than before, it's not scanning all of them. I'll let it scanning for few days more and then check any updates.

@borja-rivera  thanks , please do let me know if you get any errors in the logs

@borja-rivera issue should be fixed with latest trivy-operator v0.18.2 can you please check and confirm

i think mine is actually tied ot this one:
https://github.com/aquasecurity/trivy-operator/issues/757

Closing issue as it should be fixed with latests versions @borja-rivera feel free to open it if you thing that it has not been resolved.


allow adapters to handle their own state

Description
This PR allows for adapters (video and audio drivers) to handle their own state. This allows us to fix a few issues

Calling one of the Initialize functions resets the state of an adapter to StateClosed even if it's in use.
Disconnected drivers will return a stale state, possibly StateClosed or StateRunning.
Drivers used by other applications will start with a state of StateClosed.

Ideally, all drivers would return the actual state of the driver on queries. This PR returns real states for camera_linux.go fixing the above issues. All other drivers keep their current behavior.
Testing
This was manually tested on a Linux device and works as expected. It should not break any tests or change the current behavior for any drivers other than those on Linux.
Advice for Reviewers
The easiest way to review this code is to start with the changes in driver.go and wrapper.go. Afterwards, ensure none of the logic in any of the files besides camera_linux.go has changed. Lastly, review/test the changes in camera_linux.go and the new states in state.go. Simply running a background goroutine that continuously calls and prints the Status() of a Linux camera and observing what happens when the camera is disconnected and reconnected should be enough.

@at-wat could you take a second set of eyes on this?

@edaniels @at-wat Thanks for the reviews. After seeing all of the comments and speaking with @edaniels I updated this PR to make this functionality optional. It's far less changes now and will not break the API.


Nick: Watch digitalambesh.in

Nick requests the watch of the watch_keyword digitalambesh\.in. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

digitalambesh\.in has been seen in 0 true positives, 0 false positives, and 0 NAAs.

Approved by Jeff Schaller in Charcoal HQ


Write tests for the module

MANUAL TESTING IS A PAIN!!!!

Need to write tests using pytest

Wrote tests for Model, but need to write tests for AsyncModel

Having problems while writing tests for AsyncModel


Issue on Device Quantile DMatrix.

Hi team, it was mentioned during our offline chat that in some cases the Device Quantile DMatrix doesn't work.  It would be great if there's a description and a reproducible example for the issue so that we can try to fix it.

Hi @trivialfis, I took a look and fixed a couple of incompatibilities in #69. The error was on our end, the DeviceQuantileDMatrix itself seems to work fine.
As stated there I think it makes sense for us to investigate if we can add an interface that loads from files into GPU memory directly. However, this is unrelated to the original question and should be explored in a separate thread.


Upgrade quarkus to v0.19.1

Test are ok despite the build failed https://builds.apache.org/view/C/view/Apache Camel/job/camel-quarkus-pr/17/console


Remove unused Sphinx options and comments

Coverage remained the same at 85.263% when pulling b9f2439fd6c6729ee8b173e7328eca4c48b88d52 on dev into a7a7ea0320ed7f047d00fb5d1908aa7b58f986ce on master.


jps: Watch faroenewsfo.biz

jps requests the watch of the watch_keyword faroenewsfo\.biz. See the MS search here and the Stack Exchange search in text, in URLs, and in code.

Approved by Spevacus in Charcoal HQ


select array_difference([]) execute fail

Steps to reproduce the behavior (Required)

CREATE TABLE `array_agg_null` ( `c1` int, `c2` int, `c3` date, `c4` datetime, `c5` char(20), `c6` varchar(20), `c7` boolean, `c8` tinyint, `c9` smallint, `c10` int, `c11` bigint, `c12` largeint, `c13` float, `c14` double) DUPLICATE KEY(`c1`) DISTRIBUTED BY HASH(`c1`) buckets 4 PROPERTIES ("replication_num" = "1");

select array_difference([]);
select array_difference([]) from array_agg_null;
Expected behavior (Required)
mysql> select array_difference([]);
+----------------------+
| array_difference([]) |
+----------------------+
| []                   |
+----------------------+
1 row in set (0.01 sec)

Real behavior (Required)
ERROR 1064 (HY000): array_difference function only support numeric array types
StarRocks version (Required)
mysql> select current_version();
+------------------------+
| current_version()      |
+------------------------+
| MASTER_RELEASE 0a633d9 |
+------------------------+
1 row in set (0.01 sec)

is fixed by https://github.com/StarRocks/starrocks/pull/4520


zaproxy: Add version 2.9.0

Why did you take crossplatform binaries over windows specific?

@Ash258 I didn't know how to set windows specific binary as a source of package.

Are there any files, which should be persisted?

"plugin" folder should.
Could you add "persist" property at proper line?


FIX: SUMMARY.md section file name - Django with Mysql

Description
Modify the Connect Django With Mysql Database filename in docs --> user-guide --> use-cases to follow the other file names in that section as well as the SUMMARY.md file in the docs folder.
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

How Has This Been Tested?
Updated all locations where this file shows up i.e. in the use-cases folder and the SUMMARY.md file in the documentation.
Checklist:

[x] The title of the PR states what changed and the related issues number (used for the release note).
[x] Does this PR require documentation updates?
[x] I've updated documentation as required by this PR.
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have tested it for all user roles

Hi There,
We Thankyou for your valuable contributions. Please follow the steps below for the Giveaways Entry -

Once the PR is raised you can join the discord community for the upcoming updates.
After joining the community, send your PR link on the #contrib channel with the tag @Hacktoberfest.
We will evaluate the PR and once it is accepted , we will send you the swags.


No Support for aXAPI Version 3.0

On the docs it shows only support for aXAPI 2.1, when I run my playbook on aXAPI 3.0 A10 Appliance I get the following...
TASK [a10_server] *************************************************************************************************** task path: /etc/ansible/a10-disable-test.yml:3 Using module file /usr/lib/python2.7/dist-packages/ansible/modules/network/a10/a10_server.py <<IP_ADDRESS>> ESTABLISH LOCAL CONNECTION FOR USER: root <<IP_ADDRESS>> EXEC /bin/sh -c 'echo ~ && sleep 0' <<IP_ADDRESS>> EXEC /bin/sh -c '( umask 77 && mkdir -p " echo /root/.ansible/tmp/ansible-tmp-1508185896.69-71893879258939 " && echo ansible-tmp-1508185896.69-71893879258939=" echo /root/.ansible/tmp/ansible-tmp-1508185896.69-71893879258939 " ) && sleep 0' <<IP_ADDRESS>> PUT /tmp/tmpQfF6_Z TO /root/.ansible/tmp/ansible-tmp-1508185896.69-71893879258939/a10_server.py <<IP_ADDRESS>> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1508185896.69-71893879258939/ /root/.ansible/tmp/ansible-tmp-1508185896.69-71893879258939/a10_server.py && sleep 0' <<IP_ADDRESS>> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1508185896.69-71893879258939/a10_server.py; rm -rf "/root/.ansible/tmp/ansible-tmp-1508185896.69-71893879258939/" > /dev/null 2>&1 && sleep 0' fatal: [localhost]: FAILED! => {     "changed": false,     "failed": true,     "invocation": {         "module_args": {             "client_cert": null,             "client_key": null,             "force": false,             "force_basic_auth": false,             "host": "CN-A10-2",             "http_agent": "ansible-httpget",             "partition": "[]",             "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",             "server_ip": null,             "server_name": "test",             "server_ports": [],             "server_status": "disabled",             "state": "present",             "url": null,             "url_password": null,             "url_username": null,             "use_proxy": true,             "username": "ansible_a10",             "validate_certs": false,             "write_config": false         }     },     "msg": "failed to connect (status code 404), error was HTTP Error 404: Not Found"

Thanks for the response.

@aaronmlucas My sincerest apologies for failing to respond. I had closed existing issues as we're replacing our a10-ansible codebase wholesale with code automatically generated from our AXAPI v3.0 schema files.
This is currently available in the sdk_testing branch in this repository.
Again, my sincerest apologies for not responding when you first posted the issue. Thank you for bringing my attention to this.
@booneosborn - Tagging you as my failure in this regard should not go unnoticed.

Hi guys. I'm facing the same issue. Need it very badly. Will take a look at the sdk_testing  branch.

Issue seems to have been resolved. Closing due to age. Please re-open if issue persists.


How is AP(novel) > AP(base) on COCO?

Correct me if I am mistaken, but the reported AP values for base and novel might be swapped.
With the model only seeing 1/3 novel samples compared to the base sample, we expect the detection accuracy to also be lower accordingly. The reported accuracies for VOC seem to abide by this estimation. What is different in COCO for the results to be skewed the other way?

That's a good question!
I actually have a few guesses but not confirmed thoughts:

the number of ambiguous categories in COCO is larger than that in VOC, when the prediction branch being re-initialized and re-trained in the few-shot fine-tuning stage, the performance of base classes usually suffers a severe drop as the network tries to conduct a multi-class classification and predict class-specific box locations for each RoI. That performance drop is much obvious in COCO because our network conditions the output on the combination of query features and class features, the more ambiguous are the class features, the less precise are the outputs.

the class-agnostic box regression branch proposed in TFA seems to handle this issue very well by always using the same box regressor for all the classes, which I think could benefit most current few-shot object detection networks using a specific prediction bin for each class.

That's a good question!
I actually have a few guesses but not confirmed thoughts:

the number of ambiguous categories in COCO is larger than that in VOC, when the prediction branch being re-initialized and re-trained in the few-shot fine-tuning stage, the performance of base classes usually suffers a severe drop as the network tries to conduct a multi-class classification and predict class-specific box locations for each RoI. That performance drop is much obvious in COCO because our network conditions the output on the combination of query features and class features, the more ambiguous are the class features, the less precise are the outputs.

the class-agnostic box regression branch proposed in TFA seems to handle this issue very well by always using the same box regressor for all the classes, which I think could benefit most current few-shot object detection networks using a specific prediction bin for each class.


python3Packages.pyyaml: <IP_ADDRESS> -> 6.0

Motivation for this change
https://github.com/yaml/pyyaml/releases/tag/6.0
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

This should probably be done as part of a python-unstable jobset. This will likely cause alot of regressions

Yes, I tried this already due to home-assistant wanting it and it was bad.

I am not sure how we want to do this upgrade. Can python do overloading and we could create a temporary wrapper?

I am not sure how we want to do this upgrade.

In due time and carefully. We will not be able to not break stuff in the process unfortunately. Applications will receive overrides and that's it.

@ofborg eval

Would love this for packaging pyocd.

You can just use the older PyYAML, https://github.com/pyocd/pyOCD/commit/aa4ab29cb36140db6c0a9d164fd5d57e6083be6e makes me believe version 6 is not actually required.

Bisect says python3Packages.pyyaml: <IP_ADDRESS> -> 6.0 db01e386f4fe725bfd1f733c26e74dc838156233 broke awscli in staging:
$ nix build -f. awscli -L
...
awscli> installing
awscli> Executing pipInstallPhase
awscli> /build/awscli-1.22.14/dist /build/awscli-1.22.14
awscli> Processing ./awscli-1.22.14-py3-none-any.whl
awscli> Requirement already satisfied: colorama<0.4.4,>=0.2.5 in /nix/store/lsby77isz3iaivz4brh4h28rgpydxfg5-python3.9-colorama-0.4.3/lib/python3.9/site-packages (from awscli==1.22.14) (0.4.3)
awscli> Requirement already satisfied: rsa<5,>=3.1.2 in /nix/store/6avnhxkxy1h6h0viw8afpw70bp626rsv-python3.9-rsa-4.8/lib/python3.9/site-packages (from awscli==1.22.14) (4.8)
awscli> Requirement already satisfied: s3transfer<0.6.0,>=0.5.0 in /nix/store/6gqf8v5n55mdd7i3avijccn0xm36qlnl-python3.9-s3transfer-0.5.0/lib/python3.9/site-packages (from awscli==1.22.14) (0.5.0)
awscli> ERROR: Could not find a version that satisfies the requirement PyYAML<5.5,>=3.10 (from awscli) (from versions: none)
awscli> ERROR: No matching distribution found for PyYAML<5.5,>=3.10

Seems like we will need a pyyaml6 and pyyaml5?

@trofi Can you test whether awscli still works with
diff --git a/pkgs/tools/admin/awscli/default.nix b/pkgs/tools/admin/awscli/default.nix
index 0b9d6b47383..b04c554be84 100644
--- a/pkgs/tools/admin/awscli/default.nix
+++ b/pkgs/tools/admin/awscli/default.nix
@@ -30,7 +30,8 @@ with py.pkgs; buildPythonApplication rec {
   # https://github.com/aws/aws-cli/issues/4837
   postPatch = ''
     substituteInPlace setup.py \
-      --replace "docutils>=0.10,<0.16" "docutils>=0.10"
+      --replace "docutils>=0.10,<0.16" "docutils>=0.10" \
+      --replace "PyYAML>=3.10,<5.5" "PyYAML>=3.10"
   '';

   propagatedBuildInputs = [

Otherwise, we'll have to override PyYAML in packageOverrides.

@dotlambda your patch fixes build for me. The caveat is that I did not runtime-test it. I only noticed awscli build failure when checked unrelated changes locally.

According to the changelog, there was a breaking change with yaml.load(): https://github.com/yaml/pyyaml/blob/master/CHANGES
We should probably just add it to the packagesOverride

Mentioned here: https://github.com/NixOS/nixpkgs/pull/145807#issuecomment-969242447
Sorry, I said this should go via python-unstable and explicitly asked if someone was comfortable enough with this going via staging. I wasn't.

I was comfortable doing it through staging-next.
In general, there's a 2 week cadence for staging during non-ZHF periods, which is plenty of time to stabilize

the current staging-next will take another week at this pace. I already have a lot of python-unstable changes that I would like to do for the next staging-next. Which would push this back to 3 weeks before getting into the next python-unstable; or just merge to staging, and do stabilization then.

Bisected another failure down to this bump on staging: lirc:
$ nix build -f. -L lirc
...
lirc> PYTHONPATH=../python-pkg/lirc \
lirc> /nix/store/gn4gyzv93izh3lg5iqjb7l04k9r4hxv0-python3-3.9.9-env/bin/python ./data2hwdb ../configs ../configs \
lirc>     > lirc.hwdb
lirc> Traceback (most recent call last):
lirc>   File "/build/lirc-0.10.1/doc/./data2hwdb", line 112, in <module>
lirc>     main()
lirc>   File "/build/lirc-0.10.1/doc/./data2hwdb", line 100, in main
lirc>     db = database.Database(configdir, yamldir)
lirc>   File "/build/lirc-0.10.1/python-pkg/lirc/database.py", line 135, in __init__
lirc>     cf = yaml.load(f.read())
lirc> TypeError: load() missing 1 required positional argument: 'Loader'
lirc> make[2]: *** [Makefile:1263: lirc.hwdb] Error 1

Bisected another failure down to this bump on staging: lirc

We could apply https://sourceforge.net/p/lirc/git/merge-requests/39/

Bisected another failure down to this bump on staging: lirc

We could apply https://sourceforge.net/p/lirc/git/merge-requests/39/

Sounds good. Proposed possible fix as https://github.com/NixOS/nixpkgs/pull/150902

This PR breaks 3.x of mongodb because it uses a older version of scons that is not compatible with python 3+ to build
nix-build -A mongodb-3_6
error: PyYAML-6.0 not supported for interpreter python2.7
(use '--show-trace' to show detailed location information)


Search unresponsive after 0.53.1 update

Microsoft PowerToys version
0.53.1
Running as admin

[X] Yes

Area(s) with issue?
General
Steps to reproduce
Windows 10 Home 19043.1466 OR Windows 11
Powertoys Run Shortcut, begin typing
✔ Expected Behavior
Typing to be registered at near real time
❌ Actual Behavior
After a letter or two is typed, powertoys run stalls, for several seconds, and then catches up.
Other Software
No response

We currently working on a fix for this. We appreciate your patience.

/dup #15364

Please update to the latest version of PowerToys and see if the issue persists. 😄

It looks like it's gone! Thanks so much. I'll keep testing and reach back
out if needed. You guys rock!
On Wed, Jan 12, 2022 at 6:15 PM Franky Chen @.***>
wrote:

Please update to the latest version
https://github.com/microsoft/PowerToys/releases/ of PowerToys and see
if the issue persists. 😄
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/PowerToys/issues/15475#issuecomment-1011568561,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXIX6RDQEBM6JZ25LBQNAVTUVYKRZANCNFSM5LZIRN7Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>

Just had a similar freeze/crash. I used the report bug tool to generate the
attached.
It has been working pretty smoothly overall, but I was doing some quick
math and it seemed to get overwhelmed with very fast typing? No need to
respond, just wanted to share the report in the off chance it helps you out.
Thanks!
On Thu, Jan 13, 2022 at 9:35 AM Zach Lynde @.***> wrote:

It looks like it's gone! Thanks so much. I'll keep testing and reach back
out if needed. You guys rock!
On Wed, Jan 12, 2022 at 6:15 PM Franky Chen @.***>
wrote:

Please update to the latest version
https://github.com/microsoft/PowerToys/releases/ of PowerToys and see
if the issue persists. 😄
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/PowerToys/issues/15475#issuecomment-1011568561,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXIX6RDQEBM6JZ25LBQNAVTUVYKRZANCNFSM5LZIRN7Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>


support cyclecloud8 in cc_install.sh

Hello,
would it make sense to support cyclecloud8 installer python in cc_install.sh script?
https://github.com/Azure/azurehpc/blob/1adbceb6dac2151b5cffd76cc837482245fc42fa/scripts/cc_install.sh#L13
Maybe the goal of cc_install.sh script is different than I think of :-)
Thanks & Best

Please use https://github.com/Azure/azurehpc/blob/master/scripts/cyclecloud8_install.py instead.

https://github.com/Azure/azurehpc/blob/master/scripts/cyclecloud8_install.py#L506
Why is the installation commented?

This is designed to run on top of the CycleCloud marketplace image. Please look at the ./examples/cycleserver_msi example.


ability to show unique joycon patterns?

I'm not sure as to how this works and its capabillities, but is it possible to make the joycons/ pro controllers show up on the switch how smash joycons are (the smash ball pattern) or any of the other special edition controllers?

For the joycons you can use the hex codes from here:
https://switchbrew.org/wiki/Joy-Con#Colors

For the joycons you can use the hex codes from here: https://switchbrew.org/wiki/Joy-Con#Colors

i think what @Vincent-A-Rage means is how to get the smash logo (which is found on the smash joycons) to show up in the main window.
btw, i don't think that's possible.
hope that helps


Merge Test plan

I think test could have been more developed, there are a lot of stuff to test on a video game

I think it looks good


ostree: support for updates

Investigate building a ostree commit that has a parent commit and test updating an existing image.

This has also been done as part of #282. Support for testing this is added to ostree-osbuild-demo via #17.


Update to readiness for 2018 edition

r? @thejpster
(rust_highfive has picked a reviewer for you, use r? to override)

Ping from triage: @thejpster, could you review this? I believe this dyn construction was introduced more than two stables ago (in fact, over a year ago).

Needs a rebase after https://github.com/rust-embedded/embedded-hal/pull/167 landed.

Needs a rebase after #167 landed.

I close this pr because #167 landed - 2018 edition already applied. We don't need to update to becoming 2018 edition.


feat(components): adding sidesheet component

Description
Adding side sheet component to the web component library
What's included?

Added sidesheet component to the components library
Added storybook for side sheet component

Test Steps

[ ] npm run storybook
[ ] then log in an scroll to the side sheet story in the side bar
[ ] finally test the side sheet component

General Tests for Every PR

[ ] npm run start still works.
[ ] npm run lint passes.
[ ] npm run stylelint passes.
[ ] npm test passes and code coverage is not lower.
[ ] npm run build still works.

Screenshots or link to StackBlitz/Plunker

:tada: This PR is included in version 4.11.0 :tada:
The release is available on:

v4.11.0
GitHub release

Your semantic-release bot :package::rocket:


Http Status 204 - No Content

Hi,
can we somehow handle "No Content" response with the plugin? I wasn't able to see any example.
Many thnx

Nope, sorry this thing uses iFrames which don't really know anything about HTTP (they just see content). If you only need to support certain browsers this might be a better option for you https://github.com/eligrey/FileSaver.js/


constexpr doesn't work as expected when used on a function

Problem description
When constexpr is used on a function, the return value is expected to be available at compile-time.
But it doesn't seem to be working as expected.
Repro
The following code shows a simple function, "LoopCount", that returns a given input.
It has constexpr on the return type and the input type.
But it leads to an error when used for [ForceUnroll].
//TEST(compute):COMPARE_COMPUTE_EX:-slang -compute -shaderobj -output-using-type
//TEST(compute, vulkan):COMPARE_COMPUTE_EX:-vk -compute -shaderobj -output-using-type

//TEST_INPUT: set g_texture = Texture2D(size=8, content = one)
//TEST_INPUT: set g_sampler = Sampler
//TEST_INPUT:ubuffer(data=[0], stride=4):out,name=outputBuffer

Texture2D g_texture;
SamplerState g_sampler;
RWStructuredBuffer<float> outputBuffer;

float4 sample(float2 uv, constexpr int2 ofs )
{
    return g_texture.SampleLevel(g_sampler, uv, 0.0, ofs);
}

[ForceInline]
constexpr int LoopCount(constexpr int v) // <===== constexpr on a function
{
    return v;
}

[numthreads(1, 1, 1)]
void computeMain(int3 dispatchThreadID: SV_DispatchThreadID)
{
    float4 result = 0;
    [ForceUnroll] // <================== Works fine without this line
    for (int i = 0; i < LoopCount(3); i++)
        result += sample(float2(1.0), int2(i, i));
    outputBuffer[dispatchThreadID.x] = result.x;
}

The error message is also a little unclear what the real problem is.
It shows that the variable, "i", is not a compile-time constant, which is true but not a real reason for the error.
tests/language-feature/constants/constexpr-loop.slang(28): error 40006: expected a compile-time constant
    for (int i = 0; i < LoopCount(3); i++)
                                       ^~

Goal
We should make constexpr working for functions.
Or we should describe the limitation on our document.

Should be documented as not working.

I am leaving a note from our discussion.
The implementation of constexpr hasn't been never been completed.
We will make a document to keep track of features like this and follow up.
This task is WNF for now and revisit later when the priority arise.


Fixes #3392

This corrects the logic in isAtomCandForChiralH()
There's a bit of code modernization in here too.

Something odd about the OS X segfault, I’m going to investigate

@bp-kelley makes sense. I also restarted that failing job just in case it’s an intermittent thing

@greglandrum Looks good, some conflicts appeared though.

Yeah, I will fix those tomorrow morning. You ok if I merge after that’s done?

I will also take a look at that filter catalog test in valgrind


Theme service didn't work

send a MQTT: Publish to the S1E telneted,
topic: "homeassistant/select/0x0054ef4441dfa8/theme/settheme"
payload: "http://homeassistant.local:8123/local/theme_6_9.zip"
qos: 0
retain: false
the theme wont change, please help
大佬，你好！
S1E的主题可以直接进入ftp替换相关文件来更改，但是使用mqtt命令无法进行更改，不知道是为什么
已经将zip包放入/config/www/目录下，http://homeassistant.local:8123/local/theme_6_9.zip链接可以在浏览器直接下载
call service后没有任何变化

Topic: homeassistant/select/0x0054ef4441dfa8/theme/config
Payload
name: '妙控开关 S1E 1 '
unique_id: 0x0054ef4441dfa8_theme
icon: ''
options:

''
availability_topic: homeassistant/select/0x0054ef4441dfa8/theme/status
state_topic: homeassistant/select/0x0054ef4441dfa8/theme/state
command_topic: homeassistant/select/0x0054ef4441dfa8/theme/set
device:
name: 妙控开关 S1E 1
identifiers: A00173/SN1BRU04518
manufacturer: Aqara
sw_version: 2.0.7_0010
model: CJPKG01LM
platform: mqtt

no such "send_command_topic" anymore at the MQTT info now...

Can you try S1E2HA 1.0.4?

Same here, only command_topic, value like homeassistant/select/{device}/theme/set.

I have the same problem. The themes don't change. I've tried it:
homeassistant/select/{device}/theme/set
homeassistant/select/{device}/theme/settheme
"s1e2ha_version": "1.0.5"


Storage - Blobs Readme and Sample Issue

Section link1, link2, link3, link4:

Reason:
HttpResponseError: The specifed resource name contains invalid characters.
Suggestion1:
Update  to <connection_string>
Suggestion2:
Update  to mycontainer or <my_container>

Section link:

Reason:
RuntimeWarning: coroutine 'BlobServiceClient.get_service_properties' was never awaited
Suggestion:
Change code to
account_info = await blob_service_client.get_service_properties()
@rohit-ganguly, @lmazuel, @kyle-patterson, @kasobol-msft, @annatisch for notification.

Hi @ChenxiJiang333, thanks for catching this. I will add this and your other findings to a PR to resolve shortly.
Thanks!


chore(main): release 1.1.0

:robot: I have created a release beep boop
1.1.0 (2024-06-28)
Features

added mainbar/metabar button  border-left effect (416f934)

Bug Fixes

code refactoring (0c24459)
fixes #6 (2c757d0)
new release (306c1ff)
Update README (9a32422)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/iFadi/StudIP-ILIAS-Skin/releases/tag/v1.1.0 :sunflower:


Issue with Memory Enhancement Tool with Ollama models

I really like the concept of what you have built.
I'm reaching out to report an issue I've encountered while using Memory Enhancement Tool (MET). I've been trying to integrate MET with my local OLLAMA models, but unfortunately, it's not working as expected.
Specifically, I'm experiencing difficulties getting MET to function correctly with LLMs like llama3.1-8b. Despite reviewing the documentation and setup instructions for MET, I was unable to resolve the issue.
Could you please provide guidance on the following:

Are there any known compatibility issues between MET and local OLLAMA models?
Is there a specific model format or loading mechanism that MET requires?
Are there any LLM version limitations or restrictions that I should be aware of?

I would greatly appreciate any information or assistance you can provide to help me resolve this issue.
Thanks

Hi
Thanks for your feedback!
heading to your questions:

as I tested it with gpt-4o-2024 (the last released one) model, which is capable of function calling , it does good at handling complicated orders, like adding memory and detecting whether user wants to add or is it its own that decides to add those memories ; although still needs some enhancements;   AND local medium/small LLMs like llama3.2:latest doesn't handle this function (overall,complex function calling) properly.
So , today I will release two commits: 1)would be a simple one ONLY for llama models(simple memory adding) 2)an update for the gpt4o MET, which is an update where memories could be downloaded. I'm so sorry that time limitation doesn't let me to push upgrades everyday, I try to do so
no; The memories are saved in a folder called memories_jsons in the running folder (root folder of) open webui docker container (/app/backend i think); and i haven't tested this MET tool on installations like pip or etc.
As I said, the MET tool works best with models that are better capable of function calling; i will release a simple tool(actually the same as this, but much simpler for the LLM to work with) , that would be for tiny LLMs like llama3.2:latest and llama3.1 .

The problem is i just wanted to complete the functionality of the tool and I hadn't enough time to also complete the documentation. Sorry for that and I will do this very task today.
Thanks again for leaving a feedback!

Hey everyone,
Just sharing some thoughts on this approach.
I’ve been testing this with LLama3.2, but I’m not entirely sure what you mean by MET, @smcnaught1.
Are you referring to the usual Personalized Memory Manager?
I really appreciate @mhioi's concept of enhancing the memory functionality.
However, instead of writing files on the client side, I think it might be even more effective to explore the built-in Memories, MemoriesTable, and Files classes further to push them to the cache.
In my view, a good approach could be to send these to a pipeline server and bind the memories to a RAG (Retrieval-Augmented Generation) pipeline instead of using a tool. I do understand, though, that tools are easier to implement, while pipelines require an external server. Ideally, memories should be user-based and optional. If enabled, they should be added to the LLM context in a compressed form to minimize token usage or be integrated into the RAG pipeline.
In my early tests using Memories and MemoriesTable, I noticed that tools seem to be more of a model "recommendation" than a "mandatory option." The model decides when to read/write/update/delete from memory. However, I believe reading from memory should be mandatory. Also, due to limitations in the UI, such as confirming the deletion of a memory, it would be helpful if we could interact with memories directly within the chat window.
For LLama3.2 and LLava, it seems that the function prompts need to be more precise.
For example, when recalling from memory, the model often ignores it with the default prompt that @mhioi used.
In my case, the prompt needed to look more like this:

...
    async def recall_memories(
        self, __user__: dict, __event_emitter__: Callable[[dict], Any] = None
    ) -> str:
        """
        Retrieve all stored memories from the user's memory vault and provide them to the user.
        Be accurate and precise. Do not add any additional information. Always use the function to access memory or memories.
        If the user asks about what is currently stored, only return the exact details from the function. Do not invent or omit any information.

        :return: A numeric list of all memories. You MUST present the memories to the user as text. It is important that all memories are displayed without omissions. Please show each memory entry in full!
        """

        # get user id
        self.user_id = __user__.get("id")
...

This doesn’t fully prevent the model from hallucinating memories that were never written to the memory "vault" (whether that’s JSON files, the built-in memory class, etc.).
Also, retrieving memories is often hallucinated by the model. However, I observed that when the memories are pushed to the open-webui Memory, they are presented to the model, so it remembers, in a new chat window, what was previously stored. (I’m guessing they’re added to the context somewhere).
When using @mhioi's variation, you need to ensure memory reading is added to the system prompt or the initial chat message.
I’m still using an older version, @mhioi, so I haven’t been able to keep up with your rapid developments 😅.
Additionally, when a memory is already added to the personalized built-in memory, sometimes the memories are duplicated in this tool, and it becomes tricky to delete them properly. Even after deleting the JSON object, the personalized memory remains. It might be helpful to use different wording/naming to keep both "memories" separate when still writing to JSON files.
These are just some of my observations and conclusions.
Keep up the great work, and thanks for all the effort you’re putting into this!

Hey @dnl13

I’ve been testing this with LLama3.2, but I’m not entirely sure what you mean by MET, @smcnaught1. Are you referring to the usual Personalized Memory Manager?
I think what @smcnaught1  is talking about the gpt4 memory mimic tool;which is the reason I'm considering changing the name to MET or MEET (doesn't matter by now,we're focusing to more important things)...

I  really appreciate @mhioi's concept of enhancing the memory functionality. However, instead of writing files on the client side, I think it might be even more effective to explore the built-in Memories, MemoriesTable, and Files classes further to push them to the cache. In my view, a good approach could be to send these to a pipeline server and bind the memories to a RAG (Retrieval-Augmented Generation) pipeline instead of using a tool. I do understand, though, that tools are easier to implement, while pipelines require an external server. Ideally, memories should be user-based and optional. If enabled, they should be added to the LLM context in a compressed form to minimize token usage or be integrated into the RAG pipeline.

Thanks for your helps !
I really appreciate your observations!
Yes! There are much more enhancements for this tool !  as you mentioned, the crucial one is for merging it with built in memories. However I don't really know how to do that ( by now I mean).
However, I really wanna know where is the memories functionality in open webui? cause I have dev branch and still couldn't see any memories section in any place of the UI.Would be thankful if you guide me about that so we can implement this into MemoriesTable.
And for the pipelines concept, I never had a chance before to use pipelines. so actually don't know whether how to use it.

In my early tests using Memories and MemoriesTable, I noticed that tools seem to be more of a model "recommendation" than a "mandatory option." The model decides when to read/write/update/delete from memory. However, I believe reading from memory should be mandatory. Aleso, due to limitations in the UI, such as confirming the deletion of a memory, it would be helpful if we could interact with memories directly within the chat window.

yeah! that's why I used tools for that concept, cause if the LLM would be enough smart, it would decide if there is a need to add user's queryies to memories or not; if so, in how many indexes would it be? in what tags? etc...

For LLama3.2 and LLava, it seems that the function prompts need to be more precise.

😅yep; I was too hinged and excited to enriching the llm's tools , so haven't focused on prompts. sorry

For example, when recalling from memory, the model often ignores it with the default prompt that @mhioi used.
In my case, the prompt needed to look more like this:
...
    async def recall_memories(
        self, __user__: dict, __event_emitter__: Callable[[dict], Any] = None
    ) -> str:
        """
        Retrieve all stored memories from the user's memory vault and provide them to the user.
        Be accurate and precise. Do not add any additional information. Always use the function to access memory or memories.
        If the user asks about what is currently stored, only return the exact details from the function. Do not invent or omit any information.

        :return: A numeric list of all memories. You MUST present the memories to the user as text. It is important that all memories are displayed without omissions. Please show each memory entry in full!
        """

        # get user id
        self.user_id = __user__.get("id")
...

may I use your prompt and commit a change on prompts in the future?
If you mind ! or we can use another LLMs like chatgpt to enrich full prompts!(future versions actually...)

This doesn’t fully prevent the model from hallucinating memories that were never written to the memory "vault" (whether that’s JSON files, the built-in memory class, etc.). Also, retrieving memories is often hallucinated by the model. However, I observed that when the memories are pushed to the open-webui Memory, they are presented to the model, so it remembers, in a new chat window, what was previously stored. (I’m guessing they’re added to the context somewhere). When using @mhioi's variation, you need to ensure memory reading is added to the system prompt or the initial chat message.

I’m still using an older version, @mhioi, so I haven’t been able to keep up with your rapid developments 😅.

😂 just published a way to download any memory files user created ! hope you would give feedback to enhance or remove that feature! see, am toooo excited 😂

Additionally, when a memory is already added to the personalized built-in memory, sometimes the memories are duplicated in this tool, and it becomes tricky to delete them properly. Even after deleting the JSON object, the personalized memory remains. It might be helpful to use different wording/naming to keep both "memories" separate when still writing to JSON files.

yeah! if the LLM would be smart enough, it can merge duplicates !

These are just some of my observations and conclusions. Keep up the great work, and thanks for all the effort you’re putting into this!

Thanks ALOT! would be considering these in the future !
The most important thing is the MemoriesTable !
I would appreciate if you let me know how to use this PR as my open webui doesn't have that...

Hey @mhioi and @dnl13
I was refering MET as the GPT4 function that @mhioi created.
I recommend leveraging OpenWebUI API, if possible, to store and retrieve memories. Here's a high-level overview of how you can approach this:

Store Memories : When the chatbot recalls a memory, send a GET request to POST /memories/add with the user ID and key phrase from the conversation as JSON data. This will create or update a memory entry in OpenWebUI.
Retrieve Memories for Recalls : For each recall action (e.g., generating a response), use the GET /memories/ endpoint to fetch all memories associated with the current user. Then, filter and sort these memories based on relevance and importance.
Update Memory Entries : When updating or deleting memories, you can use the corresponding API endpoints (POST /memories/update and DELETE * /memories/delete/user, respectively).

`import requests
Initialize API endpoint URL
base_url = "https://api.openwebui.com"
def store_memory(user_id, key_phrase):
# Send a POST request to add a new memory entry
url = f"{base_url}/memories/add"
data = {
"user_id": user_id,
"key_phrase": key_phrase,
"content": ""
}
response = requests.post(url, json=data)
if response.status_code == 200:
    print(f"Memory added successfully: {response.json()['id']}")
else:
    print(f"Failed to add memory: {response.text}")

def retrieve_memories(user_id):
# Send a GET request to fetch memories for recalls
url = f"{base_url}/memories/"
params = {"user_id": user_id}
response = requests.get(url, params=params)
if response.status_code == 200:
    return response.json()
else:
    print(f"Failed to retrieve memories: {response.text}")`

I think there would be some issues with the constant API calling, and we would want to look at caching capabilities.
Just my thoughts at the moment, I have not had time to dig into this much further but I am looking to add automated memory into my OpenWebUI instance.
Thanks for the work!

yeah!
That sounds a good idea too!
the thing is can we implement memories both into openwbui itself(like the memoriestable @dnl13 said before) and into the API itself by using/developing one base code?
I think that's possible if we :

1st use memories class inside of the current code
2nd extend the results into API itself

@mhioi
fyi
The Memories class is already writing to the WebUI database, so perhaps using files might not be necessary anymore. However, we’ll need to be careful about how we can extend the root Memory class to support more diverse types of memories.
Additionally:
That’s why we should seriously consider generating it as a pipeline, so that it can later be integrated into an existing RAG model. An API endpoint for the memories could be easily implemented using the existing MemoryTables.
MHO


Fixes Ribbon Apache Client Configuration Effectively Ignoring PoolKeepAliveTime

Due to type mismatching (String vs Long) the PoolKeepAliveTime config was being ignored.
Unfortunately, the default and key definition still have a type mismatch (long vs Integer) outside this project.
This fix makes the assumption that the default will stay within integer boundaries.
Fixes gh-3038.

Codecov Report

Merging #3039 into 1.4.x will increase coverage by 0.02%.
The diff coverage is 100%.

@@             Coverage Diff             @@
##             1.4.x    #3039      +/-   ##
===========================================
+ Coverage     69.4%   69.43%   +0.02%
- Complexity    1927     1928       +1
===========================================
  Files          240      240
  Lines         8514     8515       +1
  Branches      1059     1058       -1
===========================================
+ Hits          5909     5912       +3
  Misses        2199     2199
+ Partials       406      404       -2

Impacted Files
Coverage Δ
Complexity Δ

...x/ribbon/apache/HttpClientRibbonConfiguration.java
95.65% <100%> (+3.81%)
3 <0> (ø)
:arrow_down:

...amework/cloud/netflix/zuul/web/ZuulController.java
87.5% <0%> (-12.5%)
2% <0%> (ø)

...oud/netflix/ribbon/okhttp/OkHttpRibbonRequest.java
87.23% <0%> (-1.9%)
10% <0%> (ø)

...strix/dashboard/HystrixDashboardConfiguration.java
25.24% <0%> (-0.25%)
5% <0%> (ø)

...etrics/RestTemplateUrlTemplateCapturingAspect.java
100% <0%> (ø)
2% <0%> (ø)
:arrow_down:

.../netflix/zuul/filters/post/SendResponseFilter.java
75.45% <0%> (+0.9%)
27% <0%> (ø)
:arrow_down:

...oud/netflix/zuul/filters/post/SendErrorFilter.java
82.35% <0%> (+5.88%)
13% <0%> (+1%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c181163...5b85711. Read the comment docs.

@ryanjbaxter I implemented it the way I think you suggested. Are you ok with propagating exceptions? I thought a typo/error in the config should stop initialization. What do you think?

@ryanjbaxter I noticed that you released 1.4.5 recently and it will probably be a while until the next release comes along that includes this. Is there a workaround you can think of that we can use until then?

Yes 1.4.5 is in the process of being released.  This will be merged forward into the 2.0.x stream and will be included in Finchley.SR1.  I would say we would probably have another Edgware release in the future just no sure when.


can't import

I get this:
FAILURE: Build failed with an exception.

What went wrong:
Execution failed for task ':buildSrc:plugins/edx-app-gradle-plugin:compileGroovy'.

java.lang.ExceptionInInitializerError (no error message)

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.6/userguide/command_line_interface.html#sec:command_line_warnings
CONFIGURE FAILED in 25s
Execution failed for task ':buildSrc:plugins/edx-app-gradle-plugin:compileGroovy'.

Please have a look at this issue's comment: https://github.com/edx/edx-app-android/issues/1062#issuecomment-366720223 for a detailed explanation on this issue and its possible fix.

So it's a known issue?
I don't understand the fix though. What to do in the project? Not update anything?

So it's a known issue?

Yes.

What to do in the project? Not update anything?

Yep! No need to update anything related to:

Build & SDK Tools
Gradle & Groovy versions

I am currently using Android Studio 3.1 with the default settings of the project i.e.
Android SDK Tools: 26.0.2
Android SDK Platform-Tools: 26.0.0
Gradle Version: 3.3
Android Plugin Repository: 2.3.0

As mentioned in this comment:
https://openedx.atlassian.net/browse/LEARNER-4224?focusedCommentId=307266&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-307266

Seems to work fine. Please fix it in the future..

Yep, will be fixing it soon.
For now, we have an OSPR (#1090) that might solve this issue too!

Now that I tried the sample app, I don't get what it is... :)
Can you please update the repo page, with explanation of what it is?

Which sample app are you talking about?

The only one you have here:
https://github.com/edx/edx-app-android

Our README file includes all the details that need to be setup for the app to work.

That's what I'm talking about. It doesn't tell what this is. It just says this:
"This is the source code for the edX mobile Android app. It is changing rapidly and its structure should not be relied upon. See http://code.edx.org for other parts of the edX code base.
"
What does it do?

Yep! Our own app's link is:
https://play.google.com/store/apps/details?id=org.edx.mobile&hl=en
We haven't added the link cuz we don't need to advertise our app through our open source project. This is not the purpose of our repo.
Whoever knows edX and Open edX initiatives, knows what our repos' purpose is. As any other open source projects we aim to help and collaborate with the Open edX communities through our repos.

I don't see any reason not to tell what the repository is about, and provide a link to the real app.
Even the opposite. It should always be this way: for anyone who goes to this repository, he should be able to see the output app of it.
And the opposite. The Play Store can also show that it's open sourced, and provide the link to here. This might be less important, because most users don't care about it.

Hmm, sounds legit!
We'll discuss it in our internal meetings and update accordingly, thanks for your feedback.

Many repositories out there show the app on the play store. It allows you to easily test the app, and might give you more reason to help there. It can also show how well the app was made, easily.


[feature]/support recursive type of type.

in codegen/models_build.go:69
why not support recursive type define?
we need support it .
input is work.
input R {
 id:Int
items:[R]
}

type dont work.
type R {
 id:Int
items:[R]
}

to add to this it looks like the code is generating the wrong pointer type when you have the situation above.
		case "OR":
			var err error
			var ptr1 filters.Catalog
			rawIf2 := v.([]interface{})
			ptr1 = make([]*filters.Catalog, len(rawIf2))
			for idx2 := range rawIf2 {
				var ptr3 filters.Catalog

				ptr3, err = UnmarshalCatalogFilter(rawIf2[idx2])
				ptr1[idx2] = &ptr3
			}
			it.OR = &ptr1
			if err != nil {
				return it, err
			}

var ptr1 should be a []*filters.Catalog

I need more info for this the code generated from those types look fine, perhaps a test case failing test case?
I think this might be a dupe of https://github.com/vektah/gqlgen/issues/44.
@Teddy-Schmitz I think you might have a different issue, that looks like arg unpacking (input) code, could you raise a bug with schema to reproduce it?