Update cache on delegate strategy

I am using the Distributed Cache store for my solution but now I am facing a new challenge, my strategy resolution is a delegate strategy which retrieves the tenantId from a claim of the authenticated user but I would like to retrieve the the tenant information from the database in the delegate and then save it in the distributed cache for tenant resolution. Do you have any advice of how I can get this?

Yes -- the last release contained an event sytem. See PR #359 for details. This will let you hook into when a tenant has been retrieved and if it is not in the cache you can add it there so that future requests will find it in the cache.
In this case you would have 2 stores. You'll want to register the distributed cache on first, then the slow database one. For a request the stores will be checked in the order registered so things in the cache are resolved fast, but it was from the database store then you can add into the cache at that point.
Let me know how it goes!

I followed your advice and it worked


Just wanted to try it out for fun

I'm stuck on this tho :D

Yes in the 1.19.3 it somehow loads for a long time I have to check. The inventory was for testing xd

could be good for minigames


Replaced Binance API with Kraken example

Fixes #1375  .
Changes proposed in this PR:

replaced Binance API with Kraken example.

The closing prices output after running the README steps:


Migrate onWheel event to be non-passive, which fixes #141

Hi @krnlde,
thanks for your help. I merged your work and released it with v3.5.0
I added you in contributors list


Add hotkeys!

Very useful to improve your games' UX!

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@52f3242). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master       #5   +/-   ##
=========================================
  Coverage          ?   80.62%
=========================================
  Files             ?       27
  Lines             ?      289
  Branches          ?       38
=========================================
  Hits              ?      233
  Misses            ?       56
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 52f3242...55e09d8. Read the comment docs.


Mixed content warnings

Load the avatar image securely
...

Add 'em as you find 'em

cf. #844

Hmm, I'm at an impasse on what to do about the mixed-content warnings.  Instagram doesn't let you add https without an "untrusted cert" popup (even worse than mixed-content IMO), and worse, in researching this I came across this post:
http://groups.google.com/group/instagram-api-developers/browse_thread/thread/96c3a7bff8187eff
Which sounds like we may not be able to display Instagram photos at all in our iframe soon.
Time to write up a story around proxying all this data through us?

n/m, looks like we already have proxying, working on that


Only let mut can use method

This is because nim allow var instance of Type invoke (self: var Type) procedure
But let has different semantic in elz. I don't think let is totally immutable, but nim does.
So we have to change whole let let mut to var only.
But avoid assign on let var

Fixed at v0.3.10


Fix memory bloat in EmsRefresh

reload saved record to clear caches and free memory
before a amazon refresh with public images would consume up to 4 gig of ram
after that it stays below 1 gig

@miq-bot add_labels bug
cc @Fryguy @jrafanie @blomquisg

/cc @kbrock

/cc @dmetzger57

(Stealing @jrafanie's point from gitter) .. so the trade off here is the classic speed for memory.  It's possible that we'll slow things down if any of the associations killed by reload are loaded later in the refresh process.
But, I think it's probably worth it since we wouldn't be carrying around both the hash of the provider's entire inventory plus many of the active record objects associated to basically every VM for the provider being refreshed.

Is it possible to not perform a db hit here (reload) here? We basically have introduced yet another an N+1
tangent: Do we even take advantage of :_object? I seem to remember us doing a ton of strange lookup logic around this.

Is it possible to not perform a db hit here (reload) here? We basically have introduced yet another an N+1

I could clear all caches, as reload does. Dont know if thats accesable via a public api though.

Do we even take advantage of :_object

Honestly, I dont understand what :_object does in that code 😢

Yeah I was thinking call clear_associations would get the same memory benefit without the DB hit

@durandom @Fryguy seems we're all on the same page.
While clearing the caches is not in the public api, we seem to be accessing @association_cache in a flew places.
Could we keep around the ems or other "parent" entries? I liked how miq_region.rb did:
@association_cache.except!(:tags, :taggings)

@association_cache.except!(:tags, :taggings)

I'm fine with that if it's public... Except it will still cause an N+1 if h[:_object] has any of the removed associations refetched.  If we don't even need [h:_object] or only some attributes from the object, I'd rather keep only what we need and not the whole object and it's related associations.

While clearing the caches is not in the public api,

yes it is... it's clear_association_cache

If we don't even need [h:_object] or only some attributes from the object, I'd rather keep only what we need and not the whole object and it's related associations.

that can probably be determined by checking the SQL statement made before and after the change and comparing...it should be pretty clear if the save_inventory suddenly starts making a ton more queries.

I'm good with using clear_association_cache instead of reloading the object and make sure that rails debug logging isn't showing N+1 as per:

that can probably be determined by checking the SQL statement made before and after the change and comparing...it should be pretty clear if the save_inventory suddenly starts making a ton more queries.

@durandom Any new stuff here?  I think it's pretty important performance wise, and I can see where the addition of h[:_object] can lead to enormous memory bumps.  Note that there are a couple other places where h[:_object] is set that you might want to investigate as well... git grep :_object finds them.

This code is basically the influence for save_inventory_multi/save_inventory_child and save_inventory_child pattern used throughout our code. It would be nice to make what ever changes we make here to save_store_ids_for_new_records. (and it would be nice if we could remove this code some day and just follow our basic pattern)
There was intent to move the h[:_object] logic from store_ids_for_new_records over to save_inventory_multi. We could remove a lot of hash processing and lookup if we just assigned that entry into the hash in the first place. If you notice, we call save_store_ids_for_new_records right after 54 of the 80 calls to save_inventory_multi.
But it looks like this bloats memory and a blind swap of that code may not be advisable.

unfortunately clear_association_cache has been moved to a private method: https://github.com/rails/rails/pull/16989/files#diff-a215bfc911ab0913d92695a0f20310efL141
I don't see any other quick fix than using reload here.
I'd say:

accept the N+1 problem here (it's only in the refresh worker, so it's not impacting noticeable response times) and merge it. This makes sure refresh at least works in darga
open an issue to address _object stuff
hopefully come up with a refresh worker that does not require to load all stuff into memory

I do not rescue on ActiveRecord::RecordNotFound because we rescue in the loop anyways and log the error

I did a quick check with an Amazon EMS that has 609 Images and 5 Instances:
ExtManagementSystem.find(7).vms_and_templates.count # => 613

poor mans measurement tool: grep SELECT log/* | wc
SELECT with reload: 3253
SELECT without reload: 2540
Diff: 613
So, at least for AWS CloudManager, it does not introduce more than N+1 overhead (and probably the code does not make use of the cached associations)
If you have any other scenarios to test against or better means of measurement, let me know :smile:

@durandom btw. you have to call .size to use the cached association, count always invokes the SQL query, but cache it, if you call it again


Error when npm install the project

I got this error when I ran npm install
157 error Windows_NT 10.0.10586
158 error argv "C:\Program Files\nodejs\node.exe" "C:\Program Files\nodejs\node_modules\npm\bin\npm-cli.js" "install"
159 error node v4.4.5
160 error npm  v2.15.5
161 error code EPEERINVALID
162 error peerinvalid The package<EMAIL_ADDRESS>does not satisfy its siblings' peerDependencies requirements!
162 error peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>162 error peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>162 error peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>162 error peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>162 error peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>162 error peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>162 error peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>162 error peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>162 error peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>163 verbose exit [ 1, true ]

check npm version:
$ npm --version
if <3 update to last:
npm install npm -g

Sorry, one of the dependencies was left in a old one. It should be fixed now.

thank you

@filipesilva
seems this issue happens again?
npm ERR! node v0.12.0
npm ERR! npm  v2.5.1
npm ERR! code EPEERINVALID
npm ERR! peerinvalid The package @angular/core does not satisfy its siblings' pe
erDependencies requirements!
npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>c.3
npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>-rc.3
npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>3
npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants @angular/co
<EMAIL_ADDRESS>npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants @an
<EMAIL_ADDRESS>npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>.0-rc.2
npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants @angular/c
<EMAIL_ADDRESS>npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>rc.3
npm ERR! peerinvalid Peer<EMAIL_ADDRESS>wants @angular/core@
2.0.0-rc.2

That node is reaaaaaaaaaally old.

indeed, it probably an old node
but I assume the error is because of the upgrade from 2.0.0-rc.2 to 2.0.0-rc.
3? No

@airhork I think it's actually because of node, I know that similar peerDeps issues are solved on newer versions. Can you try node 5 or 6?

Is NPM probably the problematic one, but I recommend Node 5 at least.

ok, thanks for the quick feedback. installing.

Thanks for the help.
I updated to the latest node, it need can skip this issue.
I saw the major version for node (official site) is still 4.4.6, which also have this issue.
reason seems the<EMAIL_ADDRESS>wants<EMAIL_ADDRESS>but core is already 3.
guess still better have a fix on it?

That is expected and will be fixed soon. Still with that, it should work.

I'm having the same issue when trying to deploy to AWS which uses node 4.4.3 (npm 2.15.1) , is there any way around this or am I stuck until a fix is in place?

I am not really sure if a solution for npm 2.x is going to come.
Can't it be updated to latest npm?
npm i -g npm

Hi there I'm on node v5.10.0 and after trying meteor npm install --save angular2-meteor I get the same error:
`The package<EMAIL_ADDRESS>does not satisfy its siblings' peerDependencies requirements!

@cooervo you'll have to ask at the angular2-meteor repo. We don't control the dependencies they use.

Guys, I had the same error. For me helps npm i -g npm

Update npm (npm install npm -g) solved for me!

Thanks you !!! (npm install npm -g) solved my issue also

I found something awesome in regards to node
anything you have a problem with just uninstall and reinstall it and then reinstall any dependencies they might require!


Cannot talk to t_daemon and m_daemon during measurement

When running a measurement file with do_device_sweep, the following error is shown in t_daemon and m_daemon:
error: uncaptured python exception, closing channel <utils.socket_subs.SockHandl
er connected <IP_ADDRESS>:54777 at 0x43792b0> (<class 'TypeError'>:a bytes-like obj
ect is required, not 'str' [C:\ProgramData\Anaconda3\lib\asyncore.py|write|91] [
C:\ProgramData\Anaconda3\lib\asyncore.py|handle_write_event|442] [C:\Users\admin
nus\Documents\Github\GrapheneLab-Measurement-Code\utils\socket_subs.py|handle_wr
ite|73] [C:\ProgramData\Anaconda3\lib\asyncore.py|send|361])
Listener disconnect!

This error is caused by changes in python 3. In python 2, strings are the same as bytes, while in python 3, strings are encoded in Unicode. In order to send over to channel, strings need to be encoded in bytes. And received bytes need to be decoded to string. Solved by commit 0ae28e511548a8620a3671354da7dd1786ccbdac


Bug: recvuntil

Tube::recvuntil optionally takes a timeout. If provided, it enters a loop like
while @timer.active?
  s = recv(1)
  return '' if s.empty?
  # process s
end

However, that return bails out of the whole function, not the @timer loop. Should be next if s.empty?

The original design for the timer / tube things is that we assume that after setting timeout_raw, recv_raw would either:

Return some data in at most timeout_raw seconds.
Return empty string if no data is available in timeout_raw seconds.
Raise EOFError.

So we somewhat assume that recv_raw is a blocking call.
We should definitely document this better (and fix the outdated document after #87), and maybe also provide a fallback implementation for non-blocking recv_raw.


Add new TOC toggle #38

Adds a more obvious way to open the Global TOC.

Codecov Report

Merging #47 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #47   +/-   ##
=======================================
  Coverage   73.86%   73.86%
=======================================
  Files           2        2
  Lines         199      199
=======================================
  Hits          147      147
  Misses         52       52

Flag
Coverage Δ

#pytests
73.86% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5b7226e...ebe8a5c. Read the comment docs.

@DrDrij @mmcky , updated the test_build_book.html file in tests folder. So that it has the latest HTML changes to compare with, in test regressions.

Thanks for setting this up @DrDrij
*Project Decision:  This feature won't be merged.


Load Query from a file

Is there a way to pipe in a set of queries stored in a .sql file?  In SQLCMD, the argument is -i.
I've attempted to pipe in a file into the -q argument, but it doesn't seem to work.  Any ideas how I could do that?

mssql -q ".run filename.sql"

@hasankhan This is great - would it be possible to expose a command line option to do this with an -i parameter, much like the sqlcmd utility directly on the shell?  main reason would be so that you you could use your history and tab completion, etc to quickly execute stuff like this.  our project has a sql folder where we keep all of our stored procedures. as i change those, i need to execute those against the database and it would nice if i had a command line way (without shelling into the server to run sqlcmd)

@hasankhan ill also say that i may be able to help add this functionality

Cool, please go ahead, make the change and send a pull request. Thanks.

@hasankhan just a heads up, i do plan to submit a PR for this, just got a bit backed up for a bit with other things first

Cool, looking forward to it. Please add unit test with your changes and target the pull request at dev branch.


[Refactor] add TableVersionRange to connectorMetadata interface

Why I'm doing:
Snapshot should not be recorded in the table, which is not semantic. It should be a query options.
What I'm doing:

remove snapshot field and getSnapshot() method in the IcebergTable
add TableVersionRange to LogicalScanOperator and PhysicalScanOperator for passing.
add getTableVersionRange interface to ConnectorMetadata
add TableVersionRange to ConnectorMetadata interface such as listPartitionNames/getTableStatistics/getRemoteFiles/getPrunedPartitions

TODO:
support timetravel in the next patch
Fixes #issue
https://github.com/StarRocks/starrocks/issues/47846
What type of PR is this:

[ ] BugFix
[ ] Feature
[ ] Enhancement
[x] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Does this PR entail a change in behavior?

[ ] Yes, this PR will result in a change in behavior.
[x] No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

[ ] Interface/UI changes: syntax, type conversion, expression evaluation, display information
[ ] Parameter changes: default values, similar parameters but with different default values
[ ] Policy changes: use new policy to replace old one, functionality automatically enabled
[ ] Feature removed
[ ] Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

[ ] This is a backport pr

Bugfix cherry-pick branch check:

[x] I have checked the version labels which the pr will be auto-backported to the target branch

[ ] 3.3
[ ] 3.2
[ ] 3.1
[ ] 3.0
[ ] 2.5

@megify backport branch-3.3


LOC: Japanese: The CC-BY 3.0 Unported link doesn't redirect to the Japanese page

Steps:

Launch the Brackets
Click Help->About Brackets
Click the link "CC-BY 3.0 Unported"

Result: The link would redirect to the English page
Expect: It should redirect to the Japanese page
Not reproduce on French

Took care of it.

Verified as fixed. It links to https://creativecommons.org/licenses/by-sa/2.5/deed.ja now


Correctness of AttestationValidatorImpl.validateAsync

Hi!
I do not fully grasp some of the intricacies of how the validateAsync function of AttestationValidatorImpl works with couroutines:
1 .What happens id any of lauched child coroutines take longer than the slowes the aysnc produces results? I'd assume that validateAsync returns successfully, not having completed all checks. Is that correct?
2. Any of the launched coroutines throwing an exception should cause the function as a whole to throw an exceptions, since launch is not used to create a root coroutine. Is that the reason for encapsulating the whole body in a coroutineScope?
Otherwise the code is easily comprehensible and a breeze to compare with Apple's documentation!

Hey there! Thanks for opening this issue to challenge the correctness of what this library does. It's highly appreciated.
As far as I can tell, you might not be familiar with Kotlin's approach to structured concurrency. I'd like to invite you to revise your questions with structured concurrency in mind as I assume it resolves both of your questions. If not, I'm looking forward to reading more of your thoughts!

Well, it seems like I was less familiar than i thought ;-) Thanks!


FAQ: Add an answer about the choice of HTTP.

This is an attempt as summarizing the answer @ara4n gave me to this question on Matrix HQ.
I'm sure some changes will be needed (I probably missed some points).

LGTM


Update detailed schedule

This was the schedule from March 2023:

08:50 - 09:00 Soft start and icebreaker question
09:00 - 09:15 Recap Git, any Hackmd questions to highlight

09:15 - 09:30 Concepts around collaboration

Explain terms. Pull, push, clone, fork. Focus on pull and not fetch.
Focus more on Clone and less on Generating from templates and importing

09:30 - 11:10 Centralized workflow + a break

9:30 - 9:45 : Explain concepts.
9:45 - 9:55 break
9:55 - 10:00 Inform clearly what is expected outcome.
10:00 - 10:30 Exercise
10:30 - 10:50 Instructors go through the exercise
10:50 - 11:00 - break

11:00 - 12:10 Distributed version control and forking workflow + one breake

11:00 - 11:15 - Concepts and what are exercise outcomes
11:15 - 11:45 - Exercise
11:45 - 12:00 - Instructors go through excercises
12:00 - 12:10 - break

12:10 - 12:30 - How to contribute changes to somebody else’s project and Q&A

New Schedule for September 2023

08:50 - 09:00 Soft start and icebreaker question
09:00 - 09:15 Recap Git, any Hackmd questions to highlight

09:15 - 09:30 Concepts around collaboration

Explain terms. Pull, push, clone, fork. Focus on pull and not fetch.
Focus more on Clone and less on Generating from templates and importing

09:30 - 11:00 Centralized workflow + a break

9:30 - 9:45 : Explain concepts.
9:45 - 9:55 break
9:55 - 10:00 Inform clearly what is expected outcome.
10:00 - 10:30 Exercise
10:30 - 11:00 Instructors go through the exercise

11:00 - 12:00 - Lunch break
12:00 - 13:10 Distributed version control and forking workflow + one breake

12:00 - 12:15 - Concepts and what are exercise outcomes
12:15 - 12:45 - Exercise
12:45 - 13:00 - Instructors go through excercises
13:00 - 13:10 - break

13:10 - 13:30 - How to contribute changes to somebody else’s project and Q&A

MR here https://github.com/coderefinery/git-collaborative/pull/255


Settings Placeholder text

📝 Provide a description of the new feature
Currently the application has various empty possibility's
Locations:

Fancy zones: Excluded Apps
Image Resizer: Filename format

Empty (mock):

Note: I'm not a graphical designer, so this can be changed
For both locations we need a placeholder text itself.

Currently
Empty:

Filled:

Placeholder text inspiration:

If you'd like to see this feature implemented, add a 👍 reaction to this post.

In general a good idea. But I think a placeholder text could be interpreted as exclusion.
I would prefer here an info icon with tooltip. Like we have for the file name format for ImageResizer.

In general a good idea. But I think a placeholder text could be interpreted as exclusion.
I would prefer here an info icon with tooltip. Like we have for the file name format for ImageResizer.

Placeholders can help with quick recognizing for the (type of) possible or intended input. even for helping to recognize it as being an input control in the first place.

I like the placeholder text idea.  Reason why is it clearly shows what is needed.  Could do "Example: outlook.exe"

I like the placeholder text idea.  Reason why is it clearly shows what is needed.  Could do "Example: outlook.exe"

When we do this we should write to apps to show that the user should write each app per line without delimiter sign:
ExampleApp1.exe
ExampleApp2.exe

@niels9001
Tag fix needed: "In progress"

Added in 0.25.  https://github.com/microsoft/PowerToys/releases/tag/v0.25.0


GRNs

I am very interested in joining this discussion

Description is too vague.... Are you going to talk about GRETA Thurnberg ?

This will be a test run for the new "clubs" format.
We will discuss the current status of GRETA, a benchmark tool that we are developing to evaluate multimodal GRN inference methods.
Requisits:

Read this review on multimodal GRN methods: https://doi.org/10.1038/s41576-023-00618-5

@PauBadiaM pls add more info: Trello card /board, also e.g. links to slidedecks that are not linked from the trello card

this has too little description - then it should not happen? how are people supposed to know topic and prepare for it?

Huh, it used to have the link to the trello board and the link to the review, I don't know what happened with it

Ah, found it, its a comment inside the chain of comments, the problem is that I do not have access to modify the first post so I can only comment below it


python: helloworld server is stuck if Exception is raised

What version of gRPC and what language are you using?
python, grpcio==1.16.0 (also tried with 1.16.1rc1)
What operating system (Linux, Windows, …) and version?
recent Arch Linux
What runtime / compiler are you using (e.g. python version or version of gcc)
python 3.7.1, packages installed from pypi with pip, so gcc is not relevant I guess. Not sure which gcc version was used to compile python.
What did you do?
I ran modified versions of the helloword greeter_server.py (based on https://grpc.io/docs/quickstart/python.html).
If I run it as it is (make no change to greeter_server.py) -> works -> OK.
If I run it without the loop that makes it sleep, the application exits immediately -> OK.
Code:
def serve():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
    server.add_insecure_port('[::]:50051')
    server.start()
    # try:
    #     while True:
    #         time.sleep(_ONE_DAY_IN_SECONDS)
    # except KeyboardInterrupt:
    #     server.stop(0)

If I add a raise Exception() after server.start() then the application is stuck, does not exits -> I don't think it's OK. (I needed to Ctrl + c it.)
Code:
def serve():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    helloworld_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
    server.add_insecure_port('[::]:50051')
    server.start()
    raise Exception()
    # try:
    #     while True:
    #         time.sleep(_ONE_DAY_IN_SECONDS)
    # except KeyboardInterrupt:
    #     server.stop(0)

What did you expect to see?
I would have expected that the traceback of the exception is printed then the application exists (with non zero status).
What did you see instead?
The traceback of the exception is printed, but the application does not exit!
 In Python 2 the program exit properly, I presume it might due to threading behavior difference between Python 2 and Python 3.

The gRPC Python will create 3 child threads for C-Core and one for server serving. In Python 3, the serving thread end, and main thread hung forever. There is a deadlock indeed.
Thanks to @ericgribkoff, I found the cause here is that a SHUTDOWN_TAG being added _server.py#L757 to completion queue, but never got propagated _server.py#L708 from C.
In Python 2
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff4686700 (LWP 189534)]
[New Thread 0x7ffff3e85700 (LWP 189535)]
[New Thread 0x7ffff3684700 (LWP 189536)]
[New Thread 0x7ffff2e83700 (LWP 189537)]
Traceback (most recent call last):
  File "greeter_server.py", line 47, in <module>
    serve()
  File "greeter_server.py", line 38, in serve
    raise Exception()
Exception
[Thread 0x7ffff3684700 (LWP 189536) exited]
[Thread 0x7ffff3e85700 (LWP 189535) exited]
[Thread 0x7ffff4686700 (LWP 189534) exited]
[Thread 0x7ffff2e83700 (LWP 189537) exited]

In Python 3
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff2505700 (LWP 190107)]
[New Thread 0x7ffff1d04700 (LWP 190108)]
[New Thread 0x7ffff1503700 (LWP 190109)]
[New Thread 0x7ffff0d02700 (LWP 190110)]
Traceback (most recent call last):
  File "greeter_server.py", line 47, in <module>
    serve()
  File "greeter_server.py", line 38, in serve
    raise Exception()
Exception
[Thread 0x7ffff0d02700 (LWP 190110) exited]

It might related to the object destroy behavior difference between Python 2 and Python 3. Still investigating...

Related Issue: Investigate server shutdown semantics in Cython code and possibly below #6597

Posting related info:

One of many thread behavior difference between 2/3 https://bugs.python.org/issue31516
PEP 442 -- Safe object finalization

@lidizheng Thanks for your investigations.
For a workaround I put my application code following the starting the grpc server to a try + except Exception, and do an os._exit() in the except body, e.g. os._exit(1) (I found that in https://stackoverflow.com/questions/1489669/how-to-exit-the-entire-application-from-a-python-thread).

Thanks for reporting this issue. The hang occurs because of the following interaction between the application's main thread and the daemon thread spawned by gRPC Python to poll the underlying C-Core server.
In the main thread:

When the exception is thrown, the Python interpreter begins to shutdown.
This triggers our grpc._Server.__del__() method to be called
This in turn calls sends _stop, which sends a shutdown request to the wrapped gRPC C-Core server, then waits for that operation to complete asynchronously.

This means that program exit is blocked until the core server shutdowns and the threading.Event is set. This is supposed to occur by means of the daemon thread started here. However, what actually happens is the following:

The daemon thread repeatedly polls a completion queue, which in normal operation would soon return an event from the core server indicating that shutdown is complete
The call from Cython into C-Core occurs here, with the GIL released.
The shutdown event pops out of the C-core completion queue as expected, leading to the exit of the with nogil block in our Cython code
Re-acquiring the GIL means invoking PyEval_RestoreThread
...and in particular, this conditional which, having noticed that the interpreter is shutting down, immediately exits the thread.

As a result, the main thread hangs forever, waiting for threading.Event.set() from a thread that, as soon as it tries to enter back into Python code, ceases to exist. This particular CPython behavior was introduced in this changeset, which explains why the example in this issue doesn't hang in Python2. However, given that the changeset fixed some serious sounding bugs with daemon threads and a shutting down interpreter, I'm not convinced that we actually work in this scenario even in Python2, as opposed to just getting lucky with this one particular repro case.
As with https://github.com/grpc/grpc/issues/17093, the root cause are our server deallocation methods,  blocking while waiting for a signal from threads that no longer exist. We will have a fix for this issue shortly.

@lidizheng @ericgribkoff thanks for your efforts.

The fix for this (#17444) is now merged and will be available in the next release.


Upgrade deps, fix some pow calls

This PR:

Upgrades deps, which includes an upgrade of a new thermodynamics version that fixes a performance bug here
Adds memory to one of the flame jobs (which sometimes seem to need it). cc @simonbyrne

Supersedes #748.

bors r+

bors r+

bors r+


Chaos Mesh compute-node-limit-bandwidth create table statement fails too slow?

Describe the bug
https://buildkite.com/risingwave-test/longevity-chaos-mesh/builds/367#018ca985-8aa6-42f4-bc2b-40843b071743
(Please omit the first execution/tab, it failed due to other environmental reasons)
This experiment is implemented by @xuefengze
experiments="[{\"action\": \"bandwidth\", \"cases\": [{\"mode\": [\"one\", \"all\"], \"label\": {\"key\": \"risingwave/component\", \"value\": \"compute\"}, \"duration\": \"10m\", \"direction\": \"to\", \"bandwidth\": {\"rate\": \"1mbps\", \"limit\": 20971520, \"buffer\": 10000}}]}]"

We have two experiments, one is limiting the bandwidth of only one compute node and the other one is limiting the bandwidth of all the compute nodes. The limitation lasts for 10 minutes.

In the first experiment, the limitation of bandwidth is applied around 12:34:02.
The create table t1 query was issued around 12:38:07 but only returned around 12:44:04.
There are two questions:

why did the select query before create table t1 succeed while the create table t1 failed?
why does the create table t1 statement require several minutes to return? Since we typically expect create type of statement to finish quickly, does it make sense to reduce the timeout and return a message soon, e.g. under 10 seconds. This is similar to the issue found in #14217.

Error message/log
No response
To Reproduce
No response
Expected behavior
No response
How did you deploy RisingWave?
No response
The version of RisingWave
No response
Additional context
No response

Might be the same as #14217?
Can we run an await tree dump if it reproduces?


Update dataflow client version to 2.4.1.RELEASE

Fixes #32 This version intended for Einstein release is compatible with SCDF 2.4.x .

Custom build includes app code, not generated. Apps based on boot 2.2.4 and spring-cloud-stream 3.x
ChannelInterceptor added to ensure compatibility with other Einstein release apps.
Manually tested on local 2.4.1 and 2.4.0 servers with http | tasklauncher-dataflow with and without OAuth2 configured.
README updated.

Merged in https://github.com/spring-cloud-stream-app-starters/tasklauncher-dataflow/commit/a6cc3c340551f48da86ae3179aa001a60a46fd28.


all items on library and all images on canvas have been removed

would anybody know why all my pasted images are suddenly greyed out, and all my items i had in the library have disappeared? i have not reset, changed anything on my laptop or whatnot.
the excalidraw elements like text and shapes still remain on the canvas, but everything else has been deleted overnight

The same thing happened to me. All the images have disappeared.


High Contrast theme does not look good under Linux Gnome Dark Style.

Description
The current High Contrast theme's colours are not good in my opinion. Probably a good idea to add a new "colours and fonts" preset for this theme.

UI elements' outlines/borders can be white. Currently, they are dark grey, almost invisible.

Null is black.

The SQL query is not bright enough. Should be a lighter green colour.

The value should be white text on black background.

This thing is white text on white background.

In the column editor the [ ]/ [v] becomes black when selected.

And so on... these are things I could find after using it around 10 minutes, and there should be more problems.
DBeaver Version
23.2.2
Operating System
Linux, Arch, Gnome, Wayland, Dark Style
Database and driver
Sample SQLite DB.
Steps to reproduce

Reset settings
Set "High Contrast" theme in settings.
Try to use the UI.

Additional context

GNOME Version:                               45.1
Windowing System:                            Wayland

Thanks!


Remove IfModule check

Since we're processing our custom configuration before reading system
config files, the MPM prefork module is not loaded at the time of the
evaluation. Since this image will always run with Prefork (threaded
workers don't work with Perl), we can always set the settings, not
needing to check for the module to be loaded.
Fixes RHBZ https://bugzilla.redhat.com/show_bug.cgi?id=1339514
@bparees PTAL

lgtm, but test issues.
[test]

I can reproduce it locally. I suspect a bug in tests, but not sure, investigating..

[test]

[test]

passed. merging.


Issue with GENERIC_CHART_AXES feature

**superset version 2.0.1
GENERIC_CHART_AXES enabled (yes)
Created dataset**
select
p0.day as day,
p0.host as host,
p0.referrer as referrer,
sum(p0.total) as page_views,
max(p2.t_page_views) as total_page_views,
(sum(p0.total)/max(p2.t_page_views)) as percent
from default.daily_page_views_dist  as p0
GLOBAL inner JOIN
(
select
day as p2day,
sum(total) as t_page_views
from default.daily_page_views_dist as p1
where
(
{%- if ','.join(filter_values('host')) != '' -%}
host in ({{ "'" + "','".join(filter_values('host')) + "'" }})
{%- endif -%}
{%- if ','.join(filter_values('host')) == '' -%}
true
{%- endif -%})
and
referrer not in ('', '-')
AND total > 25
AND (
{% if from_dttm is not none %}
day >= toDateTime('{{ from_dttm }}', 'America/Chicago') AND
{% endif %}
{% if to_dttm is not none %}
day <=  toDateTime('{{ to_dttm }}', 'America/Chicago') AND
{% endif %}
true
)
group by
day
) as p2
on  day = p2day
where
(
{%- if ','.join(filter_values('host')) != '' -%}
host in ({{ "'" + "','".join(filter_values('host')) + "'" }})
{%- endif -%}
{%- if ','.join(filter_values('host')) == '' -%}
true
{%- endif -%})
and
referrer not in ('', '-')
and
total > 25
AND (
{% if from_dttm is not none %}
day >= toDateTime('{{ from_dttm }}', 'America/Chicago') AND
{% endif %}
{% if to_dttm is not none %}
day <=  toDateTime('{{ to_dttm }}', 'America/Chicago') AND
{% endif %}
true
)
group by
referrer,
host,
day
order by day, host
==
Created chart

Chart works in chart builder as we can see via the above screen shots ^^
Next, add chart to dashboard

**See "select as day" code generated by dashboard as wrapper around the view query for the chart
See error msg caused by this generated code**

==
posted in superset slack, received input from members

posted this ticket
Happy to help anyway we can !!!    Please let me know how I can help get this resolved sooner !    We have report requests from both internal and external users requiring this feature !!
Joe Wilkerson
Data Engineer
True Anthem

This is likely fixed by now, and is pretty out of date if not. If people are still encountering this in current versions (3.x) please open a new Issue or a PR to address the problem.... or we can re-open this if it's still a problem. The feature flag is enabled by default now, ad will be gone in 4.0, so I hope it's all good now :)


Composite multifield: Underscore dependency missing

AEM version: 6.5.20
ACS AEM Commons Version: 6.3.2
Reproducible on Latest? yes
Expected Behavior
There is no JS error in the console.
Actual Behavior
Current error in the console:

The issue is caused by window._ is undefined. /apps/acs-commons/touchui-widgets/composite-multifield and more touch-ui widgets depend on lodash.underscore which is not found in AEM 6.5.20. Therefore, we always get this issue. A potential fix would be to depend on acs-commons.lodash instead (/apps/acs-commons/clientlibs/vendor/lodash) or depend on cq.integ.underscore (/libs/cq/personalization/clientlib/underscore)
Steps to Reproduce
Embed acs-commons.authoring.composite-multifield clientlib. If you open a dialog, the JS issue will appear.
Note: /apps/acs-commons/touchui-widgets/search-based-path-browser also depends on lodash.underscore.

FYI The same issue was fixed in https://github.com/Adobe-Consulting-Services/acs-aem-commons/issues/2821 by introducing lodash.underscore. But this category is not working anymore.

I am reopening the issue. I double-checked it and the issue is with the dependency of composite-multifield. On my local environment, I changed it manually to acs-commons.lodash and the issue was gone. Can you please double-check it on your side as well?

FTR: According to https://experienceleague.adobe.com/en/docs/experience-manager-65/content/release-notes/deprecated-removed-features

Deprecated Feature: Lodash/underscore client library. Adobe does not plan to further maintain and update the Lodash/underscore client library that is shipped as part of the distribution (Quickstart).

Replacement: Adobe recommends customers that still require Lodash/underscore for their code to add it into their project code base.


correspondence data structure, selection in separate layers to support interactive registration (and tracking)

🚀 Feature
I just had a user requesting help for registering two images using feature points so was thinking about how this could be done with napari.
A quick search revealed that interactive registration is one of the high-value plugin ideas in #939.
One of the prerequisites to make this happen is a mechanism to mark objects as corresponding across different layers (if necessary, time points or channels across nD arrays would have to be represented as layers).

For registration, this would be one-to-one correspondences, most likely between point objects in a Points layer.
For tracking one may also want to support one-to-many correspondences (e.g. dividing cells), these would most likely be between labels in labels layer.

One way to do this would be via an attribute to the individual object, e.g. with each data object (Point, label, ...) associate a unique ID that represents the same physical object across channels, timepoints. A plugin can then match the IDs for each object and can build/update a graph structure (I think that should live outside of napari). I see that there has already been some work on attributes for labels, e.g. to denote a class, merged with  #1281.
So what could be an easy way to create and mark two objects across different layers with the same attribute efficiently in the GUI, without constantly having to constantly switch between layers ?
I don't have a good idea, thinking about keyboard shortcuts but there should also be a different way.

One of the prerequisites to make this happen is a mechanism to mark objects as corresponding across different layers (if necessary, time points or channels across nD arrays would have to be represented as layers).

This is very intriguing. I think leveraging the properties dictionary might be the way to go. I think a particularly interesting example comes when you want to combine across two different layer types, for example labels and points. This happened for @constantinpape in https://github.com/hci-unihd/covid-if-annotations where he had a labels layer and then a points layer with one point for every centroid of the label.
I think ultimately he did that correspondence outside of napari, but it's interesting to think about ways we can make that and the other use cases like you mention.
This also makes me think about some of the linked data discussions that were taking place when we were thinking about how to best do 1D plots, where they might be functions of data in other layers.
I'm curious what @kevinyamauchi thinking of this. It could be worth picking one or two very concrete use cases and coding something up to do the correspondence "outside" of napari and then seeing what we could bring into napari to make that processes more simple.

Making this a bit more concrete for the two image feature points registration use case it's fairly easy now to get two image layers and two points layers, with the images side by side and the points overlaying using grid mode - see
import napari
from skimage import data
from skimage.transform import warp, AffineTransform
from skimage.color import rgb2gray

fixed = rgb2gray(data.astronaut())
tform = AffineTransform(scale=(0.9, 0.9), rotation=0.2, translation=(20, -10))
moving = warp(fixed, tform.inverse, output_shape=fixed.shape)
moving = rgb2gray(moving)

with napari.gui_qt():
    viewer = napari.Viewer()
    viewer.add_image(fixed)
    viewer.add_points(face_color='blue', name='fixed pts', size=20)
    viewer.add_image(moving)
    viewer.add_points(face_color='red', name='moving pts', size=20)
    viewer.grid_view(n_column=2, n_row=2, stride=2)

Which after some manual annotation looks as follows:

The big question then is how do I represent that the four blue points and four red points correspond to each other?
I guess the ideal might be something like using the properties dicts on both, so on each one we had a matches vector which could be used to store the indices of the matches with the other layer. Note that matches on each layer might be of different lengths, but they would be related by a simple indexing relationship.
Finally you'd need a custom mouse function that would let you click one point and then click the other corresponding point in a layer to make a match. I've not used done match of this matching points myself, but I think @berl has and he might have ideas on how he's want it to behave too.
Hope this helps making it a little more concrete!! Would be fun to get a minimal example of this working!!

Another approach would be to leverage the selected_data property of each layer and then use a keybinding to "match" whichever points are selected in both - something like
with napari.gui_qt():
    viewer = napari.Viewer()
    viewer.add_image(fixed)
    viewer.add_points(face_color='blue', name='fixed pts', size=20, properties={'matches':[np.nan]})
    viewer.add_image(moving)
    viewer.add_points(face_color='red', name='moving pts', size=20, properties={'matches':[np.nan]})
    viewer.grid_view(n_column=2, n_row=2, stride=2)

    @viewer.bind_key('m')
    def match_points(viewer):
        current_moving = viewer.layers['moving pts'].selected_data
        if len(current_moving) == 1:
            current_moving = list(current_moving)[0]
        else:
            current_moving = np.nan

        current_fixed = viewer.layers['fixed pts'].selected_data
        if len(current_fixed) == 1:
            current_fixed = list(current_fixed)[0]
        else:
            current_fixed = np.nan

        if not np.isnan(current_fixed):
            viewer.layers['fixed pts'].properties[current_fixed] = current_moving

        if not np.isnan(current_moving):
            viewer.layers['moving pts'].properties[current_moving] = current_fixed

        print(f'Match between moving {current_moving} and fixed {current_fixed}')

with the above example maybe. Unfortunately right now when you change between layers you loose which layer is currently being highlighted, but that could change.
We could also add a 5th shapes layer that tracked the edges between any matched points which would be pretty cool!
In general I find the keybindings pretty powerful ways to achieve a lot of functionality!!

Wow, that was quick. I just went outside to enjoy the sunshine after posting the issue and when I come back @sofroniewn is already on to it.
Regarding your suggested matches vector, I'm not sure that should go into napari or into a plugin.
As long as there is a convenient way to assign a unique attribute to a selection of objects (residing in different layers), the plugin can build up the representation of which objects are currently matching. Sometimes that might be a one-to-one, sometimes a directed graph (tracking) etc. The visualization of the correspondences could also be done from the plugin via hooks  (e.g. custom mouse function when hovering over an object), sometimes you may just want to highlight the corrsponding object in the other layer, in other casese you may want to draw a little arrow in a shapes layer.
As object get added or object attributes are modified in any way, a callback in the plugin gets notified and can update its correspondence representation and the correspondence visualization.

As I was typing the above, I wonder whether "layers" are the right tool to do the correspondences.
AFAIK your roadmap also has seperate camera views. With the separate camera views one could also have a second cursor functionality, i.e. as you move the cursor in one camera view you see a synchronized cursor in the other view (the transformations between views again handled by a plugin).

As I was typing the above, I wonder whether "layers" are the right tool to do the correspondences.
AFAIK your roadmap also has seperate camera views. With the separate camera views one could also have a second cursor functionality, i.e. as you move the cursor in one camera view you see a synchronized cursor in the other view (the transformations between views again handled by a plugin).

Indeed I think this particular example is also interesting for that reason, but we've still got some work to do before we get the multiview. I like the overall shape this discussion is taking though

I think a correspondence between layers would be better held inside napari so things like grid_view can act on those correspondences instead of having to modify stride all the time.  I was just about to make some comment about some kind of layer groups and .... now there's a new PR on the topic!?  amazing.
anyway, I do think some of this functionality belongs in napari because keeping track of correspondences between layers in a plugin is a hassle

@VolkerH I'm seeing now that this layer correspondence isn't exactly the issue here- you're more interested in addressing the correspondence one level down, like points 0,1,2 within a layer corresponding to points 4,33,1 within a different layer.
it does seem like there's some overlap, but maybe not as much as I initially thought.
carry on!

This is all super cool stuff. @sofroniewn , I like the idea of using properties to store correspondence between layer data for now. I definitely agree that coming up with an "official" model for that is going to take some thought.
I think I prefer a properties implementation for now because I think it can be more flexible (e.g., allow for many-to-one correspondence, as suggested by @VolkerH ). If you give each a layer a correspondence property, where any elements that have matching correspondence values are considered to be corresponding.
For example, if you have two Points layers, each with 3 points and none of them are corresponding, you have have the followings correspondence properties values (i.e., no points have matching values)
>>> points_layer_1.properties['correspondence']
[0, 1, 2]

>>> points_layer_2.properties['correspondence']
[3, 4, 5]

Now if the first point in each layer was corresponding and the others were not corresponding, you would have:
>>> points_layer_1.properties['correspondence']
[0, 1, 2]

>>> points_layer_2.properties['correspondence']
[0, 4, 5]

Finally if somehow all points in layer 2 corresponded to the first point in layer one, you would have
>>> points_layer_1.properties['correspondence']
[0, 1, 2]

>>> points_layer_2.properties['correspondence']
[0, 0, 0]

One could use a similar keybinding function as the one by @sofroniewn in  https://github.com/napari/napari/issues/1499#issuecomment-663342233 to modify the correspondence property values whenever the user wants to specify sets of points as matching. I think the tricky bit is coming up with an unused value for the correspondence, but perhaps one could store a value that just gets incremented each time a correspondence is assigned.
If you wanted to be really fancy, you could add a mouse_drag callback that recalculates the transformation based on the new point locations.
It's bedtime now, but if any of this sounds interesting, I am happy to try and code up and example!

@kevinyamauchi  what happens with multiple correspondence layers?  I think you would need N-1 arrays of np.nan for each of N layers.  If one happened to have many points, this could get painful.  maybe a sparse representation of these could be accomplished by making a list of  a layer ids and connected point indices  for each point.  if the list is empty, it's not connected to any other point in any layer.

cc'ing @adamltyson here- he's working on some tools for  brain image -> atlas registrations here with a napari gui and may have some related use cases or concerns

Thanks @berl. Very interested in this, and we will likely have multiple uses. I think the general concept discussed would suit our needs (picking corresponding points in N 2/3D images, then cropping to a bounding box, and aligning as an initial registration step).
Will keep an eye on this, as we will soon have someone dedicated to this project, who will hopefully be able to contribute upstream to napari.


Vulnerability roundup 63: libtasn1-4.13: 1 advisory

search, files

[ ] CVE-2018-1000654 (nixos-19.03)

Scanned versions: nixos-19.03: 5847485e3ec. May contain false positives.

Probably a no-op, see  #49785


Issue when configuring with Docker mysql instance.

I was trying to connect to a mysql instance run by docker engine:
docker run --name mysql_new -e MYSQL_ROOT_PASSWORD=admin -p 3306:3306 -d mysql
I am able to see the connection in mysql_workbench.
Now, I am trying to run
tables -i server/data/some.csv --db=“mysql://root:admin@localhost/db_name” --batch-size=20
But I get below error:
TABLES
▇▇▇▇▇▇
Model guessing
▀▀▀▀▀▀▀▀▀▀▀▀▀▀
• No models options provided, guessing data structure and types from first 1000 rows
• Streaming: server/data/some.csv
• Parser: csv
• [                                                  ] 100% | 0.0s elapsed | ~0.0s left
• Done guessing
Database
▀▀▀▀▀▀▀▀
• Connecting to DB
/usr/local/lib/node_modules/tables/lib/tables.js:541
if (this.dbConnectionConfg.protocol.toLowerCase().indexOf("mysql") === 0 ||
^
TypeError: Cannot read property 'toLowerCase' of null
at Tables.dbConnect (/usr/local/lib/node_modules/tables/lib/tables.js:541:39)
at Tables.wrapper [as dbConnect] (/usr/local/lib/node_modules/tables/node_modules/lodash/lodash.js:4941:19)
at Tables.start (/usr/local/lib/node_modules/tables/lib/tables.js:158:8)
at Tables.wrapper [as start] (/usr/local/lib/node_modules/tables/node_modules/lodash/lodash.js:4941:19)
at ReadStream.streamDone (/usr/local/lib/node_modules/tables/lib/tables.js:439:18)
at ReadStream.emit (events.js:165:20)
at endReadableNT (_stream_readable.js:1101:12)
at process._tickCallback (internal/process/next_tick.js:152:19)

Hmmmm, the error doesn't suggest anything that is particular to Docker or your db instance at all.  I'll see if I can reproduce locally.

I can't seem to recreate.  One thing I notice that might cause a problem, is that the command you sent over uses right and left double quotes, instead of just regular double quotes.  This would cause problems.
tables -i server/data/some.csv --db=“mysql://root:admin@localhost/db_name” --batch-size=20

Should be:
tables -i server/data/some.csv --db="mysql://root:admin@localhost/db_name" --batch-size=20

If it is working fine at your end, then probably I know the issue. Let me try that. I will let you know.

That was the issue with different instance of docker mysql installed in my machine.


[T5A3][W10-B1] Li Kai

Accepting this PR without a review because another of your PRs for this tutorial was reviewed in detail.


[HDInsight]Add New API Version 2023-08-15-Preview

…view
ARM (Control Plane) API Specification Update Pull Request
PR review workflow diagram
Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

[1] public repo review queue, private repo review queue
The PRs are processed by time opened, ascending. Your PR may show up on 2nd or later page.
If you addressed Step 1 from the diagram and your PR is not showing up in the queue, ensure the label ARMChangesRequested
is removed from your PR. This should cause the label WaitForARMFeedback to be added.
[2] https://aka.ms/azsdk/support/specreview-channel
[3] List of SDK breaking changes approvers in pinned Teams announcement
[4] public repo merge queue, private repo merge queue
If you need further help with anything, see Getting help section below.
Purpose of this PR
What's the purpose of this PR? Check all that apply. This is mandatory!

[ ] New API version. (Such PR should have been generated with OpenAPI Hub, per this wiki doc.)
[x] Update existing version for a new feature. (This is applicable only when you are revising a private preview API version.)
[ ] Update existing version to fix swagger quality issues in S360.
[ ] Other, please clarify:

edit this with your clarification

Due diligence checklist
To merge this PR, you must go through the following checklist and confirm you understood
and followed the instructions by checking all the boxes:

[x] I confirm this PR is modifying Azure Resource Manager (ARM) related specifications, and not data plane related specifications.
[ x] I have reviewed following Resource Provider guidelines, including
ARM resource provider contract and
REST guidelines (estimated time: 4 hours).
I understand this is required before I can proceed to Step 2, "ARM Review", for this PR.

Breaking changes review (Step 1)

If the automation determines you have breaking changes, i.e. Step 1 from the diagram applies to you,
you must follow the breaking changes process.
IMPORTANT This applies even if:

The tool fails while it shouldn't, e.g. due to runtime exception, or incorrect detection of breaking changes.
You believe there is no need for you to request breaking change approval, for any reason.
Such claims must be reviewed, and the process is the same.

ARM API changes review (Step 2)

If this PR is in purview of ARM review then automation will add the ARMReview label.
If you want to force ARM review, add the label yourself.
Proceed according to the diagram at the top of this comment.

Getting help

For general PR approval workflow, see the diagram at the top of this comment.
For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure
and https://aka.ms/ci-fix.
If the help provided by the previous points is not enough, post to https://aka.ms/azsdk/support/specreview-channel and link to this PR.

Hi @rkmanda. This is our preview API version and is not public yet. Could you please help us merge it?

Hi @rkmanda. This is our preview API version and is not public yet. Could you please help us merge it?

You need to follow the process described on https://github.com/Azure/azure-rest-api-specs/pull/25135#issuecomment-1663197460 to get an approval on the detection of the breaking changes

Is it OK if I fix the purpose of the PR to 'add a new api version' since its now a new api version?

Purpose of this PR
What's the purpose of this PR? Check all that apply. This is mandatory!

New API version. (Such PR should have been generated with OpenAPI Hub, per this wiki doc.)
Update existing version for a new feature. (This is applicable only when you are revising a private preview API version.)
Update existing version to fix swagger quality issues in S360.
Other, please clarify:
edit this with your clarification

I think all these avocado failures probably need to be fixed to get this PR properly linted.

The swagger JSON file is not referenced from the readme file.
readme: specification/hdinsight/resource-manager/readme.md
json: Microsoft.HDInsight/preview/2023-08-15-preview/applications.json

Which commits can I diff to see just the changes introduced in the new apiversion? If its hard to explain, it might even be worth redoing all your commits using openAPIHub and force pushing those commits over this branch, it should give us something easy to review. Or just adding commits that revert and then reapply the differences to the old api version.

Which commits can I diff to see just the changes introduced in the new apiversion? If its hard to explain, it might even be worth redoing all your commits using openAPIHub and force pushing those commits over this branch, it should give us something easy to review. Or just adding commits that revert and then reapply the differences to the old api version.

Which commits can I diff to see just the changes introduced in the new apiversion? If its hard to explain, it might even be worth redoing all your commits using openAPIHub and force pushing those commits over this branch, it should give us something easy to review. Or just adding commits that revert and then reapply the differences to the old api version.

I tried OpenAPIHub but failed with below error so have to do this manually. I combined all in the same commit and hopefully this works. Thanks.
URL: https://openapihub.azure-devex-tools.com/branch/create

ApiError: Server encountered an internal server error.
    During API SAVE_NEW_SWAGGER_VERSION({"baseBranch":"main","isNewService":false,"branchOptions":"UnprotectedBranch","isNewDataPlane":false,"baseBranchType":"Default","confidentiality":"public","serviceName":"hdinsight","serviceType":"resource-manager","hasNewDataPlane":false,"rpName":"Microsoft.HDInsight","releaseState":"preview","version":"2023-08-15-preview","copyExamples":true,"basedOnVersion":"preview/2023-04-15-preview"})
    at https://openapihub.azure-devex-tools.com/static/build.js:415:147161
    at Generator.next (<anonymous>)
    at a (https://openapihub.azure-devex-tools.com/static/build.js:415:146160)
The above error occurred in task SAVE_NEW_SWAGGER_VERSION
    created by CREATE_NEW/SAVE_NEW_SWAGGER_VERSION
    created by anonymous

I think all these avocado failures probably need to be fixed to get this PR properly linted.

The swagger JSON file is not referenced from the readme file.
readme: specification/hdinsight/resource-manager/readme.md
json: Microsoft.HDInsight/preview/2023-08-15-preview/applications.json

Done.

Hi @xinlifoobar, I'd like to double confirm that the only change introduced between 2023-04-15-preview and 2023-08-15-preview is "outboundDependenciesManagedType"?

Hi @xinlifoobar, I'd like to double confirm that the only change introduced between 2023-04-15-preview and 2023-08-15-preview is "outboundDependenciesManagedType"?

I have to do this because this saves effort than bypassing the breaking change process. Also, there are a lot more works behind this enum value so it is better to be in a new api version.

/pr RequestMerge


pytest fixture has database refusing connection

I am using python 3.7 and postgresql
@pytest.fixture(autouse=True, scope="session")
def postgres_factory():
    """
    Creates an initial fake database for use in unit tests.
    """
    postgres_factory = postgresql.PostgresqlFactory(cache_initialized_db=True, on_initialized=create_initial_data)
    return postgres_factory

@pytest.fixture(autouse=True, scope="class")
def postgres_instance(postgres_factory):
    fake_db = postgres_factory()
    return fake_db

I have these as pytest fixtures and when it tries to recreate them the first query throws a connection refused error.

Im getting the same problem.. Whenever I try to create a connection to the fake DB when imported as a fixture I get the error

This was perplexing for me too, as this works on and off on slight modifications, without good reason. Although I was using the Postgresql class not the factory, a fellow coworker pointed out to try to pass the port that the error refers to:
psycopg2.OperationalError: connection to server at "localhost" (<IP_ADDRESS>), port 5432 failed: Connection refused

And what solved the problem for me (although it worked at times):
pg = testing.postgresql.Postgresql(
    port=5432,
    postgres_args="-h <IP_ADDRESS> -F -c logging_collector=off -c max_locks_per_transaction=200",
)

No idea why.


Pixie doesn't work on Rancher Desktop

Describe the bug
Pixie doesn't work on Rancher Desktop.
To Reproduce
Steps to reproduce the behavior:

Install pixie on Rancher Desktop (k8s)
Select dockerd runtime
Install pixie
See error

Expected behavior
I expect pixie to work
Screenshots

Logs
Please attach the logs by running the following command:
./px collect-logs

pixie_logs_20220531221823.zip
App information (please complete the following information):

Pixie version: 0.7.9+Distribution.a47d77a.20220510221202.1
K8s cluster version Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.16+k3s1", GitCommit:"da16869555775cf17d4d97ffaf8a13b70bc738c2", GitTreeState:"clean", BuildDate:"2021-11-04T00:54:15Z", GoVersion:"go1.15.14", Compiler:"gc", Platform:"linux/arm64"}
Node Kernel version
Browser version

Additional context
Add any other context about the problem here.

Turns out that the VM started by rancher-desktop does not have Kernel config file, that’s a list of config entries originally used in building the kernel. Pixie uses it to patch the packaged headers to make them as close to the actual kernel as possible.
https://github.com/lima-vm/lima/issues/874
I have created this issue on lima-vm, which is the VM tool used by rancher-desktop.

The file is available in the boot image:
$ rdctl shell head /media/sr0/boot/config-virt
#
# Automatically generated file; DO NOT EDIT.
# Linux/x86_64 5.15.40 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="gcc (Alpine 10.3.1_git20211027) 10.3.1 20211027"
CONFIG_CC_IS_GCC=y
CONFIG_GCC_VERSION=100301
CONFIG_CLANG_VERSION=0
CONFIG_AS_IS_GNU=y
CONFIG_AS_VERSION=23700

Thanks @jandubois !
I've added /media/sr0/boot/config-virt into Pixie's config search path, and verified that Pixie now works fine on rancher-desktop, on Ubuntu Linux.
I still need to verify on mac. Since initial examination shows that /media/sr0/boot/config-virt does not exist on my m1 macbook air running with Lima VM.

@yzhao1012 Could you include instructions here on how you got Pixie installed on RD, especially once you have it working on macOS? Thank you!

@yzhao1012 Could you include instructions here on how you got Pixie installed on RD, especially once you have it working on macOS? Thank you!

Re:
Once rancher-desktop is installed and launched, follow https://docs.px.dev/installing-pixie/install-guides/community-cloud-for-pixie to install Pixie.
That instruction just uses local kubectl config to figure out the target kubernetes cluster and how to reach it. Then deploy Pixie with a predefined packages from public GCR repos.

We have added 2 kernel config paths used by rancher-desktop's CDRom-boot kernel image, under @jandubois' information.
See https://github.com/pixie-io/pixie/commit/57e1dfd2f335d59521ec35c2191ca84c4d12e89d
You should be able to test this after this week's Vizier release.

@yzhao1012 I got this exact error and I tried to build vizier and deploy it by following the development guide
But I keep getting the following output.

Output of `skaffold run -f skaffold/skaffold_vizier.yaml`
Generating tags...

gcr.io/pixie-oss/pixie-dev/vizier/pem_image -> gcr.io/pixie-oss/pixie-dev/vizier/pem_image:2022-06-12_04-50-10.549_UTC
gcr.io/pixie-oss/pixie-dev/vizier/kelvin_image -> gcr.io/pixie-oss/pixie-dev/vizier/kelvin_image:2022-06-12_04-50-10.549_UTC
gcr.io/pixie-oss/pixie-dev/vizier/metadata_server_image -> gcr.io/pixie-oss/pixie-dev/vizier/metadata_server_image:2022-06-12_04-50-10.549_UTC
gcr.io/pixie-oss/pixie-dev/vizier/query_broker_server_image -> gcr.io/pixie-oss/pixie-dev/vizier/query_broker_server_image:2022-06-12_04-50-10.549_UTC
gcr.io/pixie-oss/pixie-dev/vizier/cloud_connector_server_image -> gcr.io/pixie-oss/pixie-dev/vizier/cloud_connector_server_image:2022-06-12_04-50-10.549_UTC
gcr.io/pixie-oss/pixie-dev/vizier/cert_provisioner_image -> gcr.io/pixie-oss/pixie-dev/vizier/cert_provisioner_image:2022-06-12_04-50-10.549_UTC
Checking cache...
WARN[0002] Retrieving Bazel dependencies can take a long time the first time  subtask=-1 task=Build
gcr.io/pixie-oss/pixie-dev/vizier/pem_image: Error checking cache.
getting hash for artifact "gcr.io/pixie-oss/pixie-dev/vizier/pem_image": getting dependencies for "gcr.io/pixie-oss/pixie-dev/vizier/pem_image": getting bazel dependencies: running [bazel query kind('source file', deps('//src/vizier/services/agent/pem:pem_image.tar')) union buildfiles(deps('//src/vizier/services/agent/pem:pem_image.tar')) --noimplicit_deps --order_output=no --output=label]
stdout: ""
stderr: "Another command holds the client lock: \npid=74\nowner=client\ncwd=/pl/src/px.dev/pixie\n\nWaiting for it to complete...\nAnother command (pid=74) is running. Waiting for it to complete on the server (server_pid=84)...\nAnother command (pid=79) is running. Waiting for it to complete on the server (server_pid=84)...\nAnother command (pid=76) is running. Waiting for it to complete on the server (server_pid=84)...\nLoading: 0 packages loaded\nLoading: 20 packages loaded\n    currently loading: @com_llvm_lib//\nINFO: Repository base_image instantiated at:\n  /pl/src/px.dev/pixie/WORKSPACE:92:20: in \n  /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:68:28: in pl_container_images\n  /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:38:16: in _container_images_setup\n  /pl/src/px.dev/pixie/bazel/container_images.bzl:50:18: in base_images\n  /pl/src/px.dev/pixie/bazel/container_images.bzl:30:19: in _gcr_io_image\nRepository rule container_pull defined at:\n  /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl:294:33: in \nLoading: 34 packages loaded\nERROR: An error occurred during the fetch of repository 'base_image':\n   Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl", line 240, column 13, in _impl\n\t\tfail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image "gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee)\nERROR: /pl/src/px.dev/pixie/WORKSPACE:92:20: fetching container_pull rule //external:base_image: Traceback (most recent call last):\n\tFile \"/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl\", line 240, column 13, in _impl\n\t\tfail(\"Pull command failed: %s (%s)\" % (result.stderr, \" \".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image \"gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee)\nLoading: 34 packages loaded\nERROR: /pl/src/px.dev/pixie/BUILD.bazel:137:16: no such package '@base_image//image': Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image "gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee) and referenced by '//:pl_cc_base_image'\nINFO: Repository openjdk-base-musl instantiated at:\n  /pl/src/px.dev/pixie/WORKSPACE:92:20: in <toplevel>\n  /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:68:28: in pl_container_images\n  /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:38:16: in _container_images_setup\n  /pl/src/px.dev/pixie/bazel/container_images.bzl:68:18: in base_images\n  /pl/src/px.dev/pixie/bazel/container_images.bzl:30:19: in _gcr_io_image\nRepository rule container_pull defined at:\n  /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl:294:33: in <toplevel>\nERROR: An error occurred during the fetch of repository 'openjdk-base-musl':\n   Traceback (most recent call last):\n\tFile \"/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl\", line 240, column 13, in _impl\n\t\tfail(\"Pull command failed: %s (%s)\" % (result.stderr, \" \".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f)\nERROR: /pl/src/px.dev/pixie/WORKSPACE:92:20: fetching container_pull rule //external:openjdk-base-musl: Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl", line 240, column 13, in _impl\n\t\tfail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f)\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:73:16: no such package '@openjdk-base-musl//image': Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-musl'\nLoading: 50 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nLoading: 51 packages loaded\nINFO: Repository com_github_antlr_grammars_v4 instantiated at:\n  /pl/src/px.dev/pixie/WORKSPACE:10:8: in \n  /pl/src/px.dev/pixie/bazel/repositories.bzl:222:13: in _pl_deps\n  /pl/src/px.dev/pixie/bazel/repositories.bzl:158:16: in _cc_deps\n  /pl/src/px.dev/pixie/bazel/repositories.bzl:98:28: in _bazel_repo\n  /pl/src/px.dev/pixie/bazel/repositories.bzl:39:17: in _http_archive_repo_impl\nRepository rule http_archive defined at:\n  /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/http.bzl:353:31: in \nINFO: Repository 'com_github_antlr_grammars_v4' used the following cache hits instead of downloading the corresponding file.\n * Hash '9858e4a9944cac85830e6cf8edd9d567227af96d8b75f0b31accc525ec842c30' for https://github.com/antlr/grammars-v4/archive/e53d7a1228505bfc80d8637808ef60e7eea92cc2.tar.gz\nIf the definition of 'com_github_antlr_grammars_v4' was updated, verify that the hashes were also updated.\nERROR: An error occurred during the fetch of repository 'com_github_antlr_grammars_v4':\n   Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 10, in _http_archive_impl\n\t\tpatch(ctx, auth = auth)\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 167, column 22, in patch\n\t\tctx.patch(patchfile, strip)\nError in patch: Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists.\nERROR: /pl/src/px.dev/pixie/WORKSPACE:10:8: fetching http_archive rule //external:com_github_antlr_grammars_v4: Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/http.bzl", line 111, column 10, in _http_archive_impl\n\t\tpatch(ctx, auth = auth)\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/bazel_tools/tools/build_defs/repo/utils.bzl", line 167, column 22, in patch\n\t\tctx.patch(patchfile, strip)\nError in patch: Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists.\nERROR: /pl/src/px.dev/pixie/src/carnot/funcs/builtins/sql_parsing/BUILD.bazel:21:14: no such package '@com_github_antlr_grammars_v4//': Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists. and referenced by '//src/carnot/funcs/builtins/sql_parsing:cc_library'\nINFO: Repository openjdk-base-glibc instantiated at:\n  /pl/src/px.dev/pixie/WORKSPACE:92:20: in \n  /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:68:28: in pl_container_images\n  /pl/src/px.dev/pixie/bazel/pl_workspace.bzl:38:16: in _container_images_setup\n  /pl/src/px.dev/pixie/bazel/container_images.bzl:62:18: in base_images\n  /pl/src/px.dev/pixie/bazel/container_images.bzl:30:19: in _gcr_io_image\nRepository rule container_pull defined at:\n  /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl:294:33: in \nERROR: An error occurred during the fetch of repository 'openjdk-base-glibc':\n   Traceback (most recent call last):\n\tFile "/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl", line 240, column 13, in _impl\n\t\tfail("Pull command failed: %s (%s)" % (result.stderr, " ".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d)\nERROR: /pl/src/px.dev/pixie/WORKSPACE:92:20: fetching container_pull rule //external:openjdk-base-glibc: Traceback (most recent call last):\n\tFile \"/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/io_bazel_rules_docker/container/pull.bzl\", line 240, column 13, in _impl\n\t\tfail(\"Pull command failed: %s (%s)\" % (result.stderr, \" \".join([str(a) for a in args])))\nError in fail: Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d)\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:66:16: no such package '@openjdk-base-glibc//image': Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-glibc'\nERROR: /pl/src/px.dev/pixie/BUILD.bazel:137:16: no such package '@base_image//image': Pull command failed: 2022/06/12 04:55:09 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:09 Image pull was unsuccessful: reading image \"gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/base_image/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/distroless/base@sha256:34e96e21516698913035a62ef1ce484d91184de8a44209a33c7e134547e20dee) and referenced by '//:pl_cc_base_image'\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:66:16: no such package '@openjdk-base-glibc//image': Pull command failed: 2022/06/12 04:57:08 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:57:08 Image pull was unsuccessful: reading image "gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d": error getting credentials - err: exec: "docker-credential-osxkeychain": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-glibc/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:d7048f5a32ca7598f583c492c960496848cc9017fdb55942370f02603c83561d) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-glibc'\nERROR: /pl/src/px.dev/pixie/src/stirling/source_connectors/perf_profiler/java/agent/BUILD.bazel:73:16: no such package '@openjdk-base-musl//image': Pull command failed: 2022/06/12 04:55:22 Running the Image Puller to pull images from a Docker Registry...\n2022/06/12 04:55:22 Image pull was unsuccessful: reading image \"gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f\": error getting credentials - err: exec: \"docker-credential-osxkeychain\": executable file not found in $PATH, out: \n (/home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/go_puller_linux_amd64/file/downloaded -directory /home/tharindu/.cache/bazel/_bazel_tharindu/4c31fb537ca0f31ab15bbd6a8445d3b6/external/openjdk-base-musl/image -os linux -os-version  -os-features  -architecture amd64 -variant  -features  -name gcr.io/pixie-oss/pixie-dev-public/docker-deps/library/openjdk@sha256:25b910311bfe15547ecab6895d5eb3f4ec718d6d53cced7eec78e4b889962e1f) and referenced by '//src/stirling/source_connectors/perf_profiler/java/agent:image-musl'\nERROR: /pl/src/px.dev/pixie/src/carnot/funcs/builtins/sql_parsing/BUILD.bazel:21:14: no such package '@com_github_antlr_grammars_v4//': Error applying patch /pl/src/px.dev/pixie/bazel/external/antlr_grammars.patch: Cannot rename file (near line 6), new file name (sql/mysql/Positive-Technologies/MySQLLexer.g4) already exists. and referenced by '//src/carnot/funcs/builtins/sql_parsing:cc_library'\nERROR: Evaluation of subquery "deps(//src/vizier/services/agent/pem:pem_image.tar)" failed (did you want to use --keep_going?): errors were encountered while computing transitive closure\nLoading: 101 packages loaded\nLoading: 101 packages loaded\n"
cause: exit status 7

I am just trying to build and deploy vizier with a macbook. Using minikube to build and deploy was also not successful since minikube uses a separate docker runtime and mounting host file system to minikube vm was problematic.

Related issue: #450

You should be able to test this after this week's Vizier release.

@yzhao1012 - I just tested this on my Intel Macbook and it now works for me (I was previously able to reproduce the issue). Given that you've confirmed that it also works on M1 Macbook and Linux, should we add Rancher Desktop as a supported environment to Pixie's Requirements page?

Related issue: #450

with rancher desktop and dockerd runtime

Dockerd runtime wont work.
Actually, you have to use the VM driver, like what Hannah and I tested with rancher-desktop.
As docker (or any container-based kubernetes desktop environment) is not compatible with Pixie (PEM specifically).
The reason is that PEM needs to access the host kernel's system filesystem to compile BPF code.
Container-based kubernetes desktop environment hides the host filesystem from PEM and breaks it.


feat: Add message field to Barcodes struct

Type of Change

[x] New feature

Description
This pull request introduces a new feature to the Barcodes struct in the ExPass repository. The following changes have been made:

New message Field: A new required field called message has been added to the Barcodes struct. This field represents the payload or message that will be displayed as a barcode. It must be a non-empty string, and its presence is now enforced when creating a Barcodes struct.

Validation: The message field is validated using a new rule that ensures the field is present and contains a valid string. It is now required for struct creation.

Updates to Tests: The test suite has been updated to cover cases related to the new message field. Tests now ensure that the field is enforced, that valid values work correctly, and that invalid or missing values result in appropriate error handling.

Testing

Unit Tests:

Tests were added to ensure that the message field is required and must be a non-empty string.
Tests were updated to validate that the Barcodes struct cannot be created without this field.
Additional tests were included to check valid message values, as well as cases where the message field is missing, an invalid type, or an empty string.

Impact

Codebase Impact:

The Barcodes struct now requires a valid message field, which must be a non-empty string.
Existing code that constructs Barcodes instances without a message field will now raise errors.

System Behavior:

The new required message field strengthens data integrity by ensuring that the content of the barcode is explicitly provided.

Additional Information

Breaking Changes: The addition of a required message field represents a breaking change to the Barcodes struct. Any code that previously created a Barcodes struct without this field will need to be updated to include it.

Error Handling: Clear error messages are returned when the message field is missing, an invalid type, or an empty string.

Checklist

[x] My code adheres to the coding and style guidelines of the project.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have made corresponding changes to the documentation.
[x] My changes generate no new warnings.

closes: #58


Add/Plus button need to show to chat with registered users

Add/Plus button need to show to chat with registered users
Expected Results
Add/Plus button need to show on click of it in next page it has to display list of registered users so that
user can chat with registered users.
Actual Results

Steps to Reproduce
Launch applozic chat screen. No Icon is displayed to view registered user list.
Code Sample
ALChatManager file we tried by changing values.But no icon is displayed.
Applozic SDK version: ? 3.6
Xcode version: ? 8.2.1
iOS version: ? 10
Dependency manager + version: ?

You can try our sample app. We have a contacts button on top.

Thanks for the reply.
Yeah i tested sample add button is displayed.
And i have updated to applozic cocoapods to 3.8 version and but still add but is not displayed in our app.
Code Snippet :
Jaswanth Kumar<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>[ALApplozicSettings setColorForNavigation:k_Nav_BackGround_Color];
[ALApplozicSettings setColorForNavigationItem:[UIColor whiteColor]];
// [ALApplozicSettings setColorForNavigationItem:[MEUtilities infoColourForKey:@"k_Nav_Tint_Color"]];
[ALApplozicSettings hideRefreshButton:NO];
[ALUserDefaultsHandler setNavigationRightButtonHidden:NO];
[ALUserDefaultsHandler setBottomTabBarHidden:YES];
[ALApplozicSettings setTitleForConversationScreen:@"Chats"];
[ALApplozicSettings setCustomNavRightButtonMsgVC:NO];
Please let us know  if we missed anything.


Support directory diffs

The side-by-side diff view of staged & unstaged changes is completely fantastic. I want to use it for all my git diff commands. Frequently I'm diffing against something other than HEAD, though. For example, I'd like to see the equivalent of git diff master in vscode.
I realize that you can set up vscode as a git difftool, but this forces you to view the files one-by-one, rather than en masse.
A way around this would be to support directory diffs. Then I'd be able to run git difftool --dir-diff to see all the diffs at once with vscode.
My .gitconfig looks like this:
[difftool "vscode"]
    cmd = code --wait --diff $LOCAL $REMOTE

I'm using vscode 1.1.1.
For comparison, the webdiff project does something like this to support running git webdiff instead of git diff at the command line. It works great.

Code is probably not going to implement this. I suggest implementing an extension which supports it or using a Git tool for that purpose.


forget root

Be sure and add yourself to AUTHORS along with your changes.

Codecov Report

Merging #977 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #977      +/-   ##
==========================================
- Coverage    77.3%   77.28%   -0.02%
==========================================
  Files          52       52
  Lines        5574     5570       -4
==========================================
- Hits         4309     4305       -4
  Misses       1265     1265

Impacted Files
Coverage Δ

vent/core/rq_worker/watch.py
46.15% <100%> (-0.06%)
:arrow_down:

vent/api/plugins.py
80.39% <0%> (-0.72%)
:arrow_down:

vent/menus/main.py
77.89% <0%> (+0.52%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 311f66f...c036e64. Read the comment docs.


nd Conv1x1

Hey,
I have modified Conv1x1 to work with 1d, 2d and 3d data. Would you be interested in pull requests for this repository?
Cheers

Hi,
That's great! Yes, feel free to send pull requests :-)

Merged now, thanks!


Build Fails on windows

$ go build
github.com/karalabe/hid
In file included from ....\pkg\mod\github.com\karalabe\hid@v1.0.0\hid_enabled.go:40:
<EMAIL_ADDRESS>In function 'hid_enumerate':
....\pkg\mod\github.com\karalabe\hid@v1.0.0/hidapi/windows/hid.c:431:5: warning: 'strncpy' specified bound depends on the length of the source argument [-Wstringop-overflow=]
strncpy(cur_dev->path, str, len+1);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
....\pkg\mod\github.com\karalabe\hid@v1.0.0/hidapi/windows/hid.c:429:11: note: length computed here
len = strlen(str);
^~~~~~~~~~~
This looks to be an underlying bug in hidapi.
built on go1.20.3

Thanks @dbboyd17 for reporting this!

Immediate status
Unfortunately, I no longer have a Windows PC to recreate this issue and investigate.
And since I am not experiencing the issue on Linux or macOS  (both using go1.20.3), I tentatively concur that hidapi is the culprit.
I will have Windows access again next week. so I'll report back (in this thread) once I've had a chance to debug on there.
Who should be responsible
Given that this bug is like 2 degrees separated from ~Kev...  ~ this repository:
<EMAIL_ADDRESS>  (Kevin Bacon)
└──<EMAIL_ADDRESS> (Go module)
    ├── hidapi                      (C library)
    └── libusb                      (C library)

So I think it would be more appropriate to report the bug to the other maintainers; specifically to @karalabe, since they have constructed their project source repo monolithically, disconnecting these libraries from upstream development.
Workaround
Since this bug is not appearing in Linux, I would recommend trying to build from a WSL instance on Windows (until the bug is fixed):
# From a WSL shell, you'll need to install one of the Go release
# packages for Linux. Do not try to use the Windows build you
# currently have installed.
#
# Once you've verified the Linux toolchain builds without error,
# you can then use that to cross-compile back to Windows.
#
# For example:
% GOOS=windows GOARCH=amd64 go build ./src


Add support for SlickGrid maintained fork

SlickGrid is no longer maintained (as of 2 years ago). The maintained fork is now at https://github.com/6pac/SlickGrid
This repo has being actively maintained.
Moving qgrid to the active fork should be considered.

Thanks for this suggestion.  This is fixed because I'm now using my own fork of the 6pac fork in the new qgrid 1.0 beta.  That fork is here: https://github.com/quantopian/SlickGrid in case you're curious.

It would be helpful to know why there's a quantopian fork of 6pac's.


[Bug] Neko crashes when scrolling in the chapter tab

Device information

Tachiyomi version: 2.4.3
Android phone & version: Redmi Note 6 Pro, Android 11

Steps to reproduce

open any manga (tested with 2 mangas)
Scroll through the chapters

Expected behavior
Scroll
Actual behavior
App scrolls a little and crashes
Other details
Crash log: https://del.dog/pracanulet
Idk if I read it right but it seems like it's an issue with the API implementation/the session.

Logout then relogin


Configuration of log settings has no effect

In my config.yaml file, the settings I entered for the log section have no effect (they are not taken into consideration)
For example, if I set:
log:
  fancy: false

Whenever I run the application, I always get the fancy output which does not render well on containerized (dockerized) environments
I am using the latest Rapidoid release (5.4.6)

Thanks for reporting, I am working on this now...

thanks for rapid fix @nmihajlovski !
😄


Cannot detect using SSD-mobilenet

I have concerns on my term project. I have 800+ data on this one and trained around 5 epochs (initial) but can’t seem to detect using the camera. Any idea why?

Hi @qasmendoza, can you check your PyTorch pth model (not the onnx) with this script:   https://github.com/dusty-nv/pytorch-ssd/blob/8ed842a408f8c4a8812f430cf8063e0b93a56803/run_ssd_example.py
If PyTorch model doesn't work, then it needs trained for longer.  I normally train detection models for at least 30 epochs.
  Then TensorRT will generate the .engine again (otherwise it will be running off of old model)

@dusty-nv you are correct! The main issue is that the old .engine file is the one being loaded on the new workflow. Once I deleted it and re-export onnx after re-training the model for 50 epochs it worked! THANK YOU!


Examples Update Node-RED 1.x

I'm submitting a ...  (check one with "x")
Problem

( ) bug report => please search github for a similar issue or PR and test Modbus problems with the modbus-serial package before submitting
(X) unexpected behavior => please visit the Wiki before

Current behavior

The example flows are very strange in their behavior if they are designed with early v0.x of Node-RED.
Expected behavior

We need new example flows for Node-RED v1.x with the 5.+ based settings.
Minimal reproduction of the problem with instructions

Import Example inside Node-RED
Deploy
unexpected errors, but package is working well with new designed flows

Your Environment
Please tell us about your environment:

( x) I am using just the Node-RED package and got modbus-serial package installed with it
( ) I have installed modbus-serial package global from source (using git clone)
( ) I have installed modbus-serial as a global package (using npm i -g)

Yes, modbus-serial is the name but it does all, TCP and Serial.
The node-modbus package is just for the Modbus Server node.

Device :

(x) macOS: v
(x) Linux: v
( ) Windows: v
(x) Raspbian: v
( ) Other:

Name:
Version:

node-red-contrib-modbus version: v5.6.0

Node: (min. 10.x)
node --version = v10.19.0

v.5.9.0