Alternative scaling for tree graph layout

Overview: What does this pull request change?

Fix errors when trying to use the tree layout with a single-vertex
graph (was caused by a division by 0)
scale can now be None (or a tuple containing None). A
None value keeps the spacings computed by SageMath's tree layout
implementation in the corresponding dimension (useful for keeping
trees compact)
Adds a keyword argument scaling_type (that can be used with the
layout_config argument).

Its default value is "relative", which keeps the original meaning of the scale parameter (but includes the previously described changes).

If set to "absolute", the scale parameter can either be a scalar
or a tuple. It now scales SageMath's layout by the given factor in
each dimension, allowing for a finer control on compact tree
layouts.

This improvement is especially useful for rendering large trees (deep
and/or wide) when used with the auto_zoom method of MovingCamera.

Motivation and Explanation: Why and how do your changes improve the library?

When using manim to make videos of tree search algorithms, I encountered some issues when rendering large trees :

Setting the scale parameter of the layout to a set value in order to fit the scene dimensions led to vertices (which include some custom graphics to display game positions) overlapping for wide trees, and edges disappearing behind vertices for deep trees.
Trying to dynamically compute values for the scale parameter in order to make compact layouts led to re-computing the whole tree layout (as already done in tree_layout(...)

I've used the scaling_type parameter (from this PR) in order to keep the tree layout as small as possible (using the scale parameter to account for vertex size). Together with the auto_zoom method of MovingCamera, I've been very happy with the results.

Reviewer Checklist

[ ] The PR title is descriptive enough for the changelog, and the PR is labeled correctly
[ ] If applicable: newly added non-private functions and classes have a docstring including a short summary and a PARAMETERS section
[ ] If applicable: newly added functions and classes are tested

This looks very interesting! Your have marked your proposed changes as a draft, but as soon as you want anyone to look over them, please mark this as ready for review and/or request a review from me! :-)

Oh, I've only marked it as a draft because I was thinking it needed more input before oficially submitting it for review. But I can mark it ready for review now.
I tried to make sure the previous behaviour was kept when using previously valid values for the scale parameter.
I also planned to include videos demonstrating the change, but did not have the time to render them...

Apologies for things moving rather slow here! In general the changes look good to me; do you, by any chance, have some code fro an example scene where you do this?

I've used the scaling_type parameter (from this PR) in order to keep the tree layout as small as possible (using the scale parameter to account for vertex size). Together with the auto_zoom method of MovingCamera, I've been very happy with the results.

I'm curious to see how exactly you have pieced things together.
Other than that, to get this merged all that remains is adapting the documentation of the parameters. The current situation is slightly inconvenient (in the sense that only the Graph constructor itself has all the documentation for the parameters). Maybe _tree_layout should be unprivated and renamed to tree_layout so that the corresponding docstring also gets rendered. But that might be a story for another day; if you could adapt the description of layout_scale and layout_config in the docstring of Graph, and add a minimal docstring like
"""Determine a suitable tree layout.

Parameters
----------

for the tree layout function, that would be much appreciated.

do you, by any chance, have some code fro an example scene where you do this?

I do have a piece of code I used to produce a video of the negamax algorithm being applied to tic tac toe. It is however quite messy and has some pretty heavy dependencies. I will try to make a minimal working example that only requires this branch to work.

I'm curious to see how exactly you have pieced things together.

The current implementation of the tree layout uses fixed distances between nodes, then uses the scale parameter as a target size for the whole tree. What I did is provide a way to use the scale parameter to contract or dilate the fixed-distance layout by a constant factor (that does not depend on the number of nodes). I then use the bounding box of the resulting tree as the target field of fiew for the auto_zoom method of MovingCamera. The fundamental difference is that adding nodes to the tree and refreshing the whole thing causes the tree to expand and the camera to zoom out in sync, while the current implementation only allows for fixed-size trees.
The motivation behind this is that nodes started to overlap when the number of nodes grew too large. This variation allows for larger trees to be displayed without nodes overlapping, at the cost of the nodes becoming smaller and smaller on screen as you add more nodes to the tree.
I will also look into updating the docstring, and post an update here with the minimal code example along with thesponding rendered video when it is done

Hey @behackl ! I've provided with a code example (see commit https://github.com/ManimCommunity/manim/pull/2478/commits/c86c11affa8c70fd0b645944de3b798eb6d33b0c ) that shows the differences between the absolute and relative scaling types. You can see the results without rendering them yourself at https://www.youtube.com/watch?v=uu0DFlO8tWA&list=PLOefHdC8B_hxok0rrylz4xVnixBYJZYoa
I've also updated the docsting for Graph (https://github.com/ManimCommunity/manim/pull/2478/commits/223684adfd4fd9d27239994036a4f16f375516b0). Is this good for merging ?
PS : There are some issues in pre-commit.ci (has to do with mypy) but I think it comes from upstream.

The reworked docstring looks good, thank you @pcouy! I'm not sure whether the code snippets should be included in example_scenes, I think chances for someone finding them there are rather low. It might be better to move the sample scenes to an examples-section of the module docstring of graph.py so that the scenes also appear in our documentation; what do you think?
I can help move the code there if you would like me to.
I'm not entirely sure I understand why the pre-commit action fails. Seems weird.

@behackl

The reworked docstring looks good, thank you @pcouy! I'm not sure whether the code snippets should be included in example_scenes, I think chances for someone finding them there are rather low. It might be better to move the sample scenes to an examples-section of the module docstring of graph.py so that the scenes (or at least their code) also appear in our documentation; what do you think?

I agree with you. In my opinion, the examples are the best way to show what this new argument does. Documentation seems to be the right place to display them. Is there a way to add the resulting in the doc as well ?

I can help move the code there if you would like me to.

I'd love that :)

I'm not entirely sure I understand why the pre-commit action fails. Seems weird.

Yeah. I don't know wht it is about. I did not even touch the manim/cli/render/commands.py file, and I don't know what the global_options modifier does.

I quickly wanted to take care of this earlier today, but then unfortunately ran into a not so easy to resolve issue: your code doesn't run on the latest main branch, basically because recent changes have made it impossible to run both graph.animate.add_vertices and graph.animate.add_edges in the same play call.
A more or less clean workaround I can think of is improving the add_edges method so that keyword arguments like position, vertex type, ... can be passed to add_edges as well so that these arguments are then properly respected when creating new vertices within that method.
I'll try to implement that in a separate PR. As for this PR, we could either merge this without the additional example code and take care of that in a follow-up PR after the improved add_edges is available, or we wait with merging this including the example code until this is the case. What do you think @pcouy?

The example took 8 seconds to render, which is too long to comfortably add to the docs. I have downgraded the example to a code-only one.

Wow ! Thank you @behackl for sorting this out and merging it ! :)


Drop referer check

I was going to close #2929 as "won't fix", but then I realized that we probably don't need that Referer check, because we never send the CSRF token over insecure connections (this PR modifies a test to make sure of that).
Conflicts with #3022, I'll rebase after it's merged.

Rebased on master.


flutter i18n make sligt changes to make it easier to debug

Should move the i18n makefile to boilerplate i think.
SO its ready for Alex to move all boilerplates into Boostrap, where the example code can use it !!


Adding Percona 5.7 docker image

We use percona 5.7 at our shop (instead of MySQL or Maria 5.7) so I wanted push this back upstream so that vitess had support for percona version 5.7 as well.

I signed it!

LGTM thanks!


Rewrite convert to minimize dize when minimized

Way too many duplicated calculations. Calculate once, use again and again.

Also check if its correct that we should be operating on val. Seems to me we should be using t - or flip around for the formatting is after the.

Closed as part of https://github.com/AlaSQL/alasql/pull/1719


Majestic (MJ12) Bot

Added Majestic (MJ12) bot it's quite common seo bot on the internet.
As per request: https://github.com/WhichBrowser/Parser-PHP/issues/100#issuecomment-624314133

Coverage remained the same at 100.0% when pulling bc9d37e4ede1687e4b072d70cdd54f5d6068b766 on ayumi-cloud:Majestic into 4899110cd2f87b01e04ced62dbb9dec541031dee on WhichBrowser:master.

Going to close this pr as it's not passing tests and all the other pr's I have created all pass fine - so I don't want to mix them up.


restore body of requests before calling handlers

Decoding the request message from the http.Request was consuming the request's body.
Creating a new ReadCloser makes this transparent to users' code.
Fixes: https://github.com/GoogleCloudPlatform/go-endpoints/issues/71

LGTM


redb: Add parallel insertion test

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 94.90%. Comparing base (c6be3c5) to head (2e353bc).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #453      +/-   ##
==========================================
+ Coverage   94.72%   94.90%   +0.17%
==========================================
  Files          47       47
  Lines        2446     2492      +46
==========================================
+ Hits         2317     2365      +48
+ Misses        129      127       -2

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[CI] Add cpp file format checker

This patch adds a Github Action workflow to check cpp file format

this file imported from deviceMLOps.MLAgent
this file use cpp_linter marketplace actions

Self evaluation:

Build test:	 [X]Passed [ ]Failed [ ]Skipped
Run test:	 [X]Passed [ ]Failed [ ]Skipped

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2425. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/.

:octocat: cibot: @DonghakPark, The last line of a text file must have a newline character. Please append a new line at the end of the line in .github/workflows/cpp_linter.yml.

quick question. current TAOS/pr-prebuild-clang specify the exact line to fix. would this work the same?

This action based on marketplace gitaction : https://github.com/marketplace/actions/c-c-linter
And according to explain they give line by line feedback !

As far as I know, the official extension is yaml. I understand that yml was created because the extension is limited to 3 characters in Windows. How about changing it to yaml?

I totally agree with you . The three-character limit is no longer a meaningful rule.
However, since the default value for adding a new action in GitHub is defined as YML, all the GitAction files in NNTrafter have been unified to YML. I believe it is for wider compatibility and GitHub uses YML as the default value for GitAction.
What do you think about conforming to the default settings on Github or is it better to change official express ?


Aurora Constantly uploading ~2000 B/s of data

What are you experiencing an issue with?

[ x] Aurora

What kind of an issue is it?

[x ] Performance problems     ?

Expected behavior:
Should not be using the network, or only occasionally.
Actual behavior:
Aurora is constantly uploading ~2000 B/s. What is it uploading and why is it doing that/should it be doing that.
How often does this problem occur?
Always
Please describe the steps to reproduce this issue:
The issue can be reproduced by doing following steps:

Be connected to the internet
Open Aurora

What is the version of Aurora you are experiencing this issue with?
0.6.0b
Does the previous version of Aurora have this issue?
have not used the previous version
Please include any logs, screenshots, or files that are related to this issue:
https://pastebin.com/Y2iV7mgX
Other information
The minor background update option is turned off

[INFO] 4/03/2018 5:13:58 PM: Device, AtmoOrb, was initialized
Using AtmoOrb? If yes thats your Upload, cause data must be send. Having an AtmoOrb myself in use and getting an constant upload too.

See #985


/pr webeditor start

hello the "/pr webeditor start" command isent working could you help?

Make sure that you have installed the web-editor add-on.
Download the add-on at: https://addons.powerranks.nl/
Installation instructions are available at the top of that website

Make sure that you have installed the web-editor add-on.
Download the add-on at: https://addons.powerranks.nl/
Installation instructions are available at the top of that website


[WIP] Bintray

          1 Error

:no_entry_sign:
src/Caches/Bintray/Uploading.hs#L7 - Found Parse error

1  uploadBinary ::
2>
3

Error description

1

          2 Warnings

:warning:
PR is classed as Work in Progress

:warning:
src/Lib.hs#L474 - Found Reduce duplication

 1frameworkNameWithFrameworkExtension = appendFrameworkExtensionTo f
 2platformBuildDirectory = carthageBuildDirectoryForPlatform platform
 3frameworkDirectory
 4  = platformBuildDirectory </> frameworkNameWithFrameworkExtension
 5dSYMNameWithDSYMExtension
 6  = frameworkNameWithFrameworkExtension <> ".dSYM"
 7dSYMdirectory
 8  = platformBuildDirectory </> dSYMNameWithDSYMExtension
 9bcSybolMapPath d = platformBuildDirectory </> bcsymbolmapNameFrom d
10

Why Not

1Combine with src/Lib.hs:536:5

  Generated by :no_entry_sign: Danger

          1 Error

:no_entry_sign:
src/Caches/Bintray/Uploading.hs#L7 - Found Parse error

1  uploadBinary ::
2>
3

Error description

1

          2 Warnings

:warning:
PR is classed as Work in Progress

:warning:
src/Lib.hs#L474 - Found Reduce duplication

 1frameworkNameWithFrameworkExtension = appendFrameworkExtensionTo f
 2platformBuildDirectory = carthageBuildDirectoryForPlatform platform
 3frameworkDirectory
 4  = platformBuildDirectory </> frameworkNameWithFrameworkExtension
 5dSYMNameWithDSYMExtension
 6  = frameworkNameWithFrameworkExtension <> ".dSYM"
 7dSYMdirectory
 8  = platformBuildDirectory </> dSYMNameWithDSYMExtension
 9bcSybolMapPath d = platformBuildDirectory </> bcsymbolmapNameFrom d
10

Why Not

1Combine with src/Lib.hs:536:5

  Generated by :no_entry_sign: Danger


RuntimeWarning: overflow encountered in power   return 1.0/(1.0 + power(e, -x_value))

I am getting this error while running " python train.py 0.1 100 " , even I tried with 60 instead of 100 . same error is repeating.
autonomous-rc-car-master/sigmoid.py:9: RuntimeWarning: overflow encountered in power
return 1.0/(1.0 + power(e, -x_value))

Try to change the dimensions

I changed image dimension to 50,50 from 75,75 in configuration.py .
IMAGE_DIMENSIONS = (50, 50)
but still the same warning occuring , can you suggest me dimension value ?
Note : I am using raspberry pi 3 (raspbian Os) with pi cam v2

Is there any workaround for this issue? Even I'm getting the same warning message.


Decrease Output Shown On Terminal

While refactoring the drivers to add support for both timeseries and climo input, we should take this time to also cleanup the text printed on the screen. This has been a complaint from many users.
It would be a good idea to actually implement #135 if possible.

Done in the climo branch. Other issues, such as #135, #144, and #148 will further cleanup the output.


Execute getLock in unordered way

The HZ's partition-aware operations will be blocked if thread of
the same parition is blocked. Run the getLock in an ordered way
will result that a first partition-thread-blocked operation blocks
following operation even the operation is on another partition.
Signed-off-by: Richard MENG<EMAIL_ADDRESS>

This is for 3.5 branch. The master branch seems already good.


Remove the Renderable trait

Description
As briefly discussed in #1512.
This PR also updates the documentation to no longer use the Renderable trait for rendering children. Since version 0.17 that's no longer possible anyway but the documentation hasn't been updated until now.
Checklist:

[x] I have run ./ci/run_stable_checks.sh
[x] I have reviewed my own code

I don't think it makes sense to fix the aspell issue in this PR so I'm gonna go ahead and merge this.


memory leak

My English is not very good, please understand.As the concurrency of my project increases, the memory also increases. After investigation, it was found that the reason was due to the publisher.startNotifyBlockedHandler() method's failure to exit and release memory in a timely manner. I am not sure what happened that caused the failure to exit. My expected outcome is publisher Close() can close normally.
After commenting out this line of code, I found it to be effective, but it is not elegant.

Close your publishers and consumers when you're done with them and do not attempt to reuse them. Only close the connection itself once you've closed all associated publishers and consumers.
The issue has been resolved through conn.close.But it's not possible to repeat newConn.I did not reuse the publisher, but I reused Conn. When the publisher closed, startNotifyBlockedHandler was still blocking, so I had to reuse the publisher

It seems to me that the most likely issue is that you have a lot of Publish() calls getting blocked. This function: startNotifyBlockedHandler waits for any in-flight publishes to finish and then stops new ones from happening. So if they are hanging indefinately for some reason that could be the root cause. I'd argue something at the infrastructure level is probably the issue.
You could add a context timeout, and that should stop them from hanging forever.

Hello, this seems still to be an issue.
I could reproduce this goroutine leak by using the example of the readme and printing the stacktrace before and after closing the publisher. The goroutine started at startup will not terminate with closing the publisher.
The problem is the goroutine is started and passes a notify chan (calling go channels chan to prevent confusion with rabbitmqs Channels) to the Connection. This chan will only be closed if the connection to rabbitmq is closed here.
But a long lived service will stay connected to the rabbitmq therefore hold the Connection alive. And only creating and closing publishers as needed. This results in the connection holding all channels in the blocks array created on startNotifyBlockedHandler, not closing it when a publisher shutdowns and keeping the goroutine at the for loop alive.
Contrary to the startNotifyFlowHandler which is also started on the creation of a publisher its chan is managed by a Channel. This channel is closed if the publisher Close function is called. Which terminates this routine started by the publisher.
I have two ideas for a fix:

your above mentioned context which is passed down into the function and acts as a lifetime, if the publisher is closed this lifetime is exceeded and the routine finishes. This is NOT allowed to close the channel or it will panic if the Connection is terminated and trying to close the channel aswell
tag the channel passed, notify the connection to remove the channel on publisher close. This would require changes in the wrapped library as well

I could implement the first. The second option would need discussion and implementation in amqp091-go library first.
Take care!

Hello, this seems still to be an issue.
I could reproduce this goroutine leak by using the example of the readme and printing the stacktrace before and after closing the publisher. The goroutine started at startup will not terminate with closing the publisher.
The problem is the goroutine is started and passes a notify chan (calling go channels chan to prevent confusion with rabbitmqs Channels) to the Connection. This chan will only be closed if the connection to rabbitmq is closed here.
But a long lived service will stay connected to the rabbitmq therefore hold the Connection alive. And only creating and closing publishers as needed. This results in the connection holding all channels in the blocks array created on startNotifyBlockedHandler, not closing it when a publisher shutdowns and keeping the goroutine at the for loop alive.
Contrary to the startNotifyFlowHandler which is also started on the creation of a publisher its chan is managed by a Channel. This channel is closed if the publisher Close function is called. Which terminates this routine started by the publisher.
I have two ideas for a fix:

your above mentioned context which is passed down into the function and acts as a lifetime, if the publisher is closed this lifetime is exceeded and the routine finishes. This is NOT allowed to close the channel or it will panic if the Connection is terminated and trying to close the channel aswell
tag the channel passed, notify the connection to remove the channel on publisher close. This would require changes in the wrapped library as well

I could implement the first. The second option would need discussion and implementation in amqp091-go library first.
Take care!

once.Do rabbitmq.NewPublisher；Reusing this connection has not caused any problems after large-scale high-concurrency testing

For me this is not a fix of the problem. This does not change the fact that even initializing one publisher and closing it will not stop all started goroutines from this publisher thus resulting in a leaking goroutine.
Furthermore someone can have multiple publisher which are written to simultaneously with different publisher options configured. So i'm confused how it should be possible to reuse a publisher created with different settings which does not allow to change them. Can you elaborate this more?

rabbitmq.NewPublishe

var once sync.Once
var oncePublish sync.Once
var conn *rabbitmq.Conn
var publisher *rabbitmq.Publisher

func GetMqConn() *rabbitmq.Conn {
	once.Do(func() {
		var err error
		conn, err = rabbitmq.NewConn(GetMqUrl())
		if err != nil {
			logging.Logger.Error("rabbitmq conn err", zap.Error(err))
		}
	})
	return conn
}

func GetPublisher() (*rabbitmq.Publisher, error) {
	var err error
	oncePublish.Do(func() {
		publisher, err = rabbitmq.NewPublisher(
			GetMqConn(),
			rabbitmq.WithPublisherOptionsLogging,
		)
	})
	return publisher, err
}

Our business is relatively simple, with only one configuration. We can use GetPublisher globally, which has withstood the test of high concurrency. If you have multiple configurations, my suggestion is to put different configurations of Publishers into the Publisher pool to reuse the same configuration of Publishers

For me this is not a fix of the problem. This does not change the fact that even initializing one publisher and closing it will not stop all started goroutines from this publisher thus resulting in a leaking goroutine. Furthermore someone can have multiple publisher which are written to simultaneously with different publisher options configured. So i'm confused how it should be possible to reuse a publisher created with different settings which does not allow to change them. Can you elaborate this more?

var once sync.Once
var oncePublish sync.Once
var conn *rabbitmq.Conn
var publisher *rabbitmq.Publisher

func GetMqConn() *rabbitmq.Conn {
	once.Do(func() {
		var err error
		conn, err = rabbitmq.NewConn(GetMqUrl())
		if err != nil {
			logging.Logger.Error("rabbitmq conn err", zap.Error(err))
		}
	})
	return conn
}

func GetPublisher() (*rabbitmq.Publisher, error) {
	var err error
	oncePublish.Do(func() {
		publisher, err = rabbitmq.NewPublisher(
			GetMqConn(),
			rabbitmq.WithPublisherOptionsLogging,
		)
	})
	return publisher, err
}

Our business is relatively simple, with only one configuration. We can use GetPublisher globally, which has withstood the test of high concurrency. If you have multiple configurations, my suggestion is to put different configurations of Publishers into the Publisher pool to reuse the same configuration of Publishers

Ah thank you for your example @lmb1113 . We have a different use case, we open a lot of publishers because each client gets its own queue and exchange for incoming messages

Hello,
dont want to necro bump but @wagslane should i open a new issue for this? Or can this issue be reopened?

Sorry I've been crazy busy at work. I want to look at this when I have some time just haven't had a chance yet

Hello, I noticed that each publisher has created the startNotifyBlockedHandler function. Can we adjust it to use the same function for each conn? My understanding is that the function of startNotifyBlockedHandler is to prevent publishing if the connection is not available
@mjurczik @wagslane
Reproduce memory leak process, when globally connected once, multiple publishers

publisher.startNotifyBlockedHandler()->conn.startNotifyBlockedHandler()?

We also met this issue. We don't reuse publishers or consumers, but when getting a lot of messages to publish we receive OOMs with ~2-3gb memory usage when regular memory usage is about 100-200mb. Hope you can fix it.

publisher.startNotifyBlockedHandler()->conn.startNotifyBlockedHandler()?

We also met this issue. We don't reuse publishers or consumers, but when getting a lot of messages to publish we receive OOMs with ~2-3gb memory usage when regular memory usage is about 100-200mb. Hope you can fix it.

Suggest reuse publishers or consumers

@lmb1113 we tried that approach but getting a lot of errors like here:

https://github.com/wagslane/go-rabbitmq/issues/162
https://github.com/wagslane/go-rabbitmq/issues/165

@lmb1113 we tried that approach but getting a lot of errors like here:

reconnection bug #162
reconnection err while rabbitmq server handshake_timeout #165

Reuse connections and reuse publishers or consumers?

For me this is not a fix of the problem. This does not change the fact that even initializing one publisher and closing it will not stop all started goroutines from this publisher thus resulting in a leaking goroutine. Furthermore someone can have multiple publisher which are written to simultaneously with different publisher options configured. So i'm confused how it should be possible to reuse a publisher created with different settings which does not allow to change them. Can you elaborate this more?

var once sync.Once
var oncePublish sync.Once
var conn *rabbitmq.Conn
var publisher *rabbitmq.Publisher

func GetMqConn() *rabbitmq.Conn {
	once.Do(func() {
		var err error
		conn, err = rabbitmq.NewConn(GetMqUrl())
		if err != nil {
			logging.Logger.Error("rabbitmq conn err", zap.Error(err))
		}
	})
	return conn
}

func GetPublisher() (*rabbitmq.Publisher, error) {
	var err error
	oncePublish.Do(func() {
		publisher, err = rabbitmq.NewPublisher(
			GetMqConn(),
			rabbitmq.WithPublisherOptionsLogging,
		)
	})
	return publisher, err
}

Our business is relatively simple, with only one configuration. We can use GetPublisher globally, which has withstood the test of high concurrency. If you have multiple configurations, my suggestion is to put different configurations of Publishers into the Publisher pool to reuse the same configuration of Publishers

@hotrush Reuse connections and reuse publishers or consumers?

@lmb1113 connection reused always. Publisher reused - connection/reconnection issues, publisher not reused - memory leaks. Smth like that

@mjurczik So all resources are not necessarily expected to be cleaned up when you close the publisher, you would need to also close the underlying connection itself. When you close both are you still seeing a "leak"?
(And if that's not a suitable use case you might want to just drop down to the amqp lib)

Hello,

So all resources are not necessarily expected to be cleaned up when you close the publisher, you would need to also close the underlying connection itself.

Imo this should not be expected. A connection should be reusable, removing a publisher and have them free their memory along side their started routines should not force you to close the connection aswell.
In the readme itself it says: Close your publishers and consumers when you're done with them and do not attempt to reuse them. Only close the connection itself once you've closed all associated publishers and consumers.

When you close both are you still seeing a "leak"?

Sure if i shutdown my programme and gracefully close all publisher and afterwards close the connection the routines will be stopped gracefully. That happens because on conn.Close() function shutdown closes the channels  blocking the startNotifyFlowHandler.
Closing the connection seems not to be a suitable fix:
Lets say we have 3 Publishers, and we want to close one of them because it is not needed anymore.
We can't simply close it AND the connection. This will eventually break the connection required by the other still in use publishers as well. Closing only the publisher will keep the startNotifyFlowHandler routine alive, which doesn't do anything anymore, because the publisher it is assigned to and the notify will be reported to is already closed and discarded.
I thought about my "fix" with the lifetime, the problem is this would finish the routine and allow the publisher + routine to be garbage collected but the chan whould still be hold in the Connection blocks array in amqp lib. I guess it would be better to implement something similar in the ConnectionManager it holds the block channels, but on NotifyBlockedSafe does not pass them down, instead the ConnectionManager passes one channel down to amqp and on signal iterates through the selfmanaged blocks channels. Additionally a function to remove a channel from the connection manager would be needed allowing to remove it.
If you like i can implement this, create a PR and you can still decide if this does make sense for you :)

Yeah this makes sense to me @mjurczik ! If we have 3 publishers on a single connection, we should be able to close 1 publisher, keep the connection alive for the other 2, and release any publisher specific resources.
If you'd like to propose a PR I will take a look when I have time

Hey @wagslane,
sorry for the hold up. Just pushed my fixed, hope this helps.


structure for write-up on lessons learned

So far just placeholders but once we converge on the text I will paste in in there and we are ready to go.
Will be published on /blog/2020/04/09/first-online-workshop/ any comments on URL structure? That is something that should not be changed later but can easily be changed now.

i think the URL structure is fine, and typical for blogs right?
one aspect is that i think newsletter articles should also be published as blog posts (and perhaps only linked from the actual tinyletter email sent out), but that doesn't change anything for the URL structure

I was also thinking yesterday that newsletters would probably get more visibility for those not the email list if they are also published as blogs.

Ready for review.

It's the same as hackmd with minute edits.


Only 366 Items are returned in JSON and CSV Outputs

Only 366 Items are returned in JSON and CSV Outputs
Great Tool! - Thanks for sharing!
Use Case:
Need to extract unique Hosts from 18 Months of Apache Logs with total count of sightings for each IP Address (~18 Gigabytes of raw log data).
Issue
On the basis of documentation reviews I selected goaccess as my first choice.  Up and running against test data in 15 minutes.  Config file optimized in another 15 minutes.
Good Job on Tool and Docs!
Unfortunately, it appears I misinterpreted the Documentation regarding Item Limits in Output files.
This may just be Semantics.  Documentation states:

NOTES
For now, each active window has a total of 366 items. Eventually this will be customizable. All these
items are available by default in the CSV and JSON exports, and as an expandable panel in the
HTML report (upper-right corner).

Is Intent?:

NOTES
For now, each active window has a total of 366 items. Eventually this will be customizable.
These 366 items are all available by default in the CSV and JSON exports, and
as an expandable panel in the HTML report (upper-right corner).

Question:
Is there a way to extend this Item Limit for Hosts (for data extraction only)?

Thanks for clarifying those notes. I've updated the website. Regarding the ability to extend this limit, there's no option right now, it's on the to-do list. However, you could modify line 1542 (haven't really tested it, but it should do it):
h->holder_size = size > MAX_CHOICES ? MAX_CHOICES : size;

to
h->holder_size = !conf.output_html && size > MAX_CHOICES ? MAX_CHOICES : size;

Note that if you are outputting HTML, browsers might not be able to handle a large amount of data in the report.

Closing this. Feel free to reopen it if needed.


 I’m fairly new to dart but I think the fat arrow explicitly returns. When I tried to use the return statement following the example it was showing an error

Yep, good call, that's definitely a typo. Thanks for the report -- I'll fix it up!


Insecure cipher suites

The default array of cipher suites includes a number of cipher with known vulnerabilities and a few other ciphers which are now computationally cheap to crack (given modern computing power).
OpenVPN's hardening wiki: https://community.openvpn.net/openvpn/wiki/Hardening
Background (conversation continued from: https://github.com/Nyr/openvpn-install/pull/90)

No, no, no and no. I'm not going to force my users to upgrade/break their system, this scripts installs OpenVPN and that's it. I'm definitively not forcing an upgrade on anyone because that's not the expected outcome of an OpenVPN installer.

You also install iptables, openssl, wget and another 3rd party tool via wget (the latter of which you've only just secured thanks to our discussions). You also do an apt-get update prior to installing the aforementioned, which means any dependant software will also need to be updated regardless of running an $(which apt-get yum) upgrade. So you are updating and modifying the users' server in all sorts of ways which you're not documenting anywhere.
Since your script isn't just an OpenVPN installer but more an "OpenVPN for dummies" (so to speak) I think it makes a lot of sense to also handle the suggestions myself and others have made. But if you feel that's a step too far then that's fine. I can relate to that. However you then still need to be clear that this is doing more than just installing OpenVPN and that you make no effort to secure the install. eg you make no mention of anywhere that you're installing and editing iptables rules and the scope for breaking things there is much greater than a simple yum update. And you make no mention that you're supporting insecure ciphers. The README would lead many to assume that this is a secure out-of-the-box solution which it seems is not the case judging from the issues you've closed and your reasons behind those decisions.
I'm genuinely not trying to be rude. I can see this is a popular project so I'm only offering you my experience so this project can improve and nobody gets any unexpected, nasty, surprises.

You've only listed one "vulnerability": a one-liner which is completely optional. Open a new issue if you find any reasonable vulnerability within openvpn-install.sh and I'll make sure to address it. Until then, don't waste more of my time.

I listed quite a few vulnerabilities actually. But I don't really want to get into an argument. I'm just offering up friendly suggestions for hardening your installer. :+1:

I'll quote myself again because you seem to have big problems understanding English:

Unfortunately, update in yum = upgrade in apt-get

See? I mention yum update, not yum upgrade. And my statement is right.

You also install iptables as well as other 3rd party tools via wget

Because both iptables (which is likely already present) and easy-rsa (which is not a package, just a third party script) are NEEDED for the script to do its work. I obviously install REQUIRED dependencies because else the script couldn't work.

You're using the default array of cipher suites which include a number of cipher with known vulnerabilities and a few other ciphers which are now computationally cheap to crack (given modern computing power).

You are right that OpenSSL supports a lot of the "less secure" ciphers and they could theoretically be used for the control channel, but downgrading during negotiation is nearly impossible and by default weaker cyphers are the latest in order of preference so stronger ones are used.
I'll also quote what's the position from the OpenVPN guys in this matter:

--tls-cipher really is an expert feature. [...] unlike browsers, OpenVPN does not perform a 'downgrade dance', making it a lot saner to rely on the SSL libary's own version/cipher downgrade protection.

Anyway, setting specific TLS cyphers is a bad idea which could cause more trouble than benefit. It would be way better instead to forbid weak ciphers instead. Please also remember that this is a default/barebones installer and those are the defaults suggested by the developers. I try to leave personal preference (from both me and you) away from the script as much as possible.


URL normalization

URL normalization
Description:
Can't find an URL normalization option (https://en.wikipedia.org/wiki/URL_normalization).
Now we have an option prefix rewrite for vhosts. For example we can replace /service-hello with / and proxy to root address of service. But in case of /service-hello -> / replacement if we request to this address: http://envoy:8888/service-hello/_healthz then we will get a request to http://helloservice//_healthz
It would be nice if we had this feature of URL normalization if we don't have one.

But in case of /service-hello -> / replacement if we request to this address: http://envoy:8888/service-hello/_healthz then we will get a request to http://helloservice//_healthz

@Lookyan could you elaborate more on this? Thank you!

@dio at least I want to have a feature of multiple slash replacement with one slash during proxying to upstream.
So, we can have URL like this: http://smth//a/b///c
And we should make a request to this URL in this case: http://smth/a/b/c

@Lookyan got it. Sorry I didn't get that additional slashes.


TypeError in main.jsbundle:1072

Error in Pride in London 2018 app
TypeError in main.jsbundle:1072
TypeError: undefined is not an object (evaluating 'e.sys')
This error is located at:
in t
in Connect(t)
in RCTView
in Unknown
in t
in RCTView
in n
in t
in n
in RCTView
in RCTView
in t
in RCTView
in e
in r
in Unknown
in n
in n
in t
in RCTView
in RCTView
in t
in RCTView
in RCTView
in n
in t
in RCTView
in t
in r
in withCachedChildNavigation(r)
in Unknown
in n
in n
in t
in RCTView
in RCTView
in RCTView
in n
in t
in n
in RCTView
in RCTView
in t
in RCTView
in e
in r
in Unknown
in n
in n
in RCTView
in n
in e
in i
in Unknown
in n
in t
in RCTView
in RCTView
in t
View on Bugsnag
Stacktrace
main.jsbundle:1072 -
main.jsbundle:1072 - shouldComponentUpdate

View full stacktrace
Created automatically via Bugsnag

I have a bit of a plan for how to guard against this kind of error. Essentially we need to have a better flow handling of these kinds of properties and we would have been forced to handle this. Once I am done on past event filtering.

An error linked to this issue has been marked as fixed in Bugsnag
TypeError in main.jsbundle:1072

This should be fixed with the decoding etc that we do now. If we see it pop up again we can investigate further.


FR: Even darker UI - less element containers with solid grey background.

Is your feature request related to a problem? Please describe.
It would be nice to be able to remove/configure element container - either configure the color/border/background (and possibly set it to black to make it go away) or have and option to disable it.
I'm not particularly fond of MaterialYou design (de gustibus…) and I find previous UI (see screenshots https://f-droid.org/packages/com.best.deskclock/) much cleaner.
Considering that on AMOLED true black background is desired it would be nice to be able to remove as much of the grey-ish backgrounds as much as possible.

As you can imagine, it's very complicated to keep everyone happy with the design of an application... 😬

I'm not particularly fond of MaterialYou design ... and I find previous UI ... much cleaner.

I personally prefer the new interface, which clearly separates each alarm, for example.

Even if I prefer the new interface, I'll try to implement your request. That's why I've left this topic open.

@woj-tek: Can you test the app version attached? I think I've met your expectations.
In the settings, I've added 2 options to show/hide backgrounds and show/hide background borders.
I'll wait for your feedback before merging my changes.
Clock_2.1-debug.zip

Tiny comment - settings page doesn't follow it

Indeed, I forgot to mention it. 😬
I didn't manage to do it, but it can be done later.
In any case, I'm very happy to have met your expectations. 😁


add support for rocky linux and alma linux; add support for RHEL9 and…

Chat Release 3.0.3, brings

support for Rocky Linux and Alma Linux;
support for RHEL9 and its derivatives

LGTM


Make it possible to reattach NEGs automatically

The current behaviour of Autoneg introduces race conditions for us, the scenario goes:

A service is deployed with existing BackendService and NEGs
Autoneg connects the service to existing NEGs and updates its annotations
User removes BackendService resource
Autoneg does nothing because the NEGs status stays unchanged but the service is actually detached from the NEGs
User recreates a new BackendService
Autoneg will do nothing because the NEGs status stays unchanged

The proposed solution here without us doing big architectural change is to make Autoneg reconcile periodically.
Since this is a big behaviour change, I added a flag to enable it but defaults to false. Along with another flag to configure the reconciliation period.

Don't get me wrong, I support this, but couldn't the user just delete the status annotation to re-reconcile?

@rosmo Yes, that was a workaround, but it adds toils. And we think this kind of mishaps can happen without users notice and cause incident.

LGTM

Hello, Do you know when this will be released? :) Thank You!

@joshw123 I'll roll out a new pre-release after issue in #113 is solved.


registerHandlerClass function is not there?

I have been upgrading to swift 3 for the passed hours i have put everything to work, except for DeepLinkKit. I am not able to find the function registerHandlerClass anymore. The weird thing is it is in the DPLDeepLinkRouter.h but when i command click on DeepLinkKit within my own code i get the following:

I am not sure that this is a problem I have or this is a problem in the pod. I have deleted the pods, deleted Podfile.lock, deleted derived data, cleaned my project, everything, but it still doesn't show up. Does anyone know what to do?

@borek2 I'm assuming that the header is in swift syntax because it was run through a Swift conversion? It looks like that converter made an error on interpreting the registerHandlerClass: method... although I'm not entirely sure why.
How is DLK being imported? With an umbrella header import you should be able to leave this as Objective C syntax, no?

Thanks for the quick reply @chrismaddern !
I have added DeepLinkKit as a static library now, and it still gives problems. The actual function is there now, but i am not able to get a reference to it. With some trial and error (I am not very skillful in Objective-C) i noticed the following:
This is the line that gives problems:
- (void)registerHandlerClass:(Class <DPLRouteHandler>)handlerClass forRoute:(NSString *)route;
When i use another class in instead of DPLRouteHandler it works fine (as, i can reference to it). But here is when it will cause problems. If you use a protocol (i created a new one to test this), it will fail to make a reference.
Sorry if it sounds a little vague, i only program in Swift so Objective C is sometimes a little vague to me. Might this problem have something to do with xCode 8?

Again it might be my lack of knowledge of Objective C, though in my understanding within the code DPLRouteHandler is both a protocol and a class? It is defined as a protocol in the file DPLDeepLinkRouter.h. But it has an implementation in DPLRouteHandler.m. Is it a naming problem maybe?

+1, In the same seat as @borek2.

@doohoo @borek2 It looks likely that there may have to be an interface change and bump to 2.0.0 for support for Xcode 8 / Swift 3.
We'll take a look in the early week and try to get something out asap!
In the meantime, if either of you find a sensible interface change that works, feel free to fork / share.

@chrismaddern Will look into it! Thanks for the help!

I'm in the same situation. Hope you're able to sort it out asap 👍

Hi @borek2 , thanks for reporting the issue.
This branch should fix the problem.
You can try it like this:
pod 'DeepLinkKit', :git => 'git@github.com:button/DeepLinkKit.git', :branch => 'pavel/111_registerHandlerClass_swift3_interface'

Awesome! Thanks so much! I will be testing this tomorrow @pantuspavel

It seems to work 👍  Thanks @pantuspavel

Anyone got this working using Pods?

@doohoo Hi!
PR in question was merged into master.
Please try:
pod 'DeepLinkKit', :git => 'git@github.com:button/DeepLinkKit.git', :branch => 'master'

We restored fix branch, so you can try:
pod 'DeepLinkKit', :git => 'git@github.com:button/DeepLinkKit.git', :branch => 'pavel/111_registerHandlerClass_swift3_interface'

@pantuspavel it didn't go to well.
Basically get:
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

@doohoo , seems like server rejects your connection.

Hehe, something is messup. Will check it out 👀 . Thanks.

@pantuspavel just went for the https way instead :)

No problem!
Glad it worked out for you.

Sorry this took so long. The change is available now in 1.2.2


pixi add is really slow in some scenarios

Checks

[X] I have checked that this issue has not already been reported.

[X] I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example
[project]
name = "glum"
channels = ["conda-forge"]
platforms = ["linux-64", "osx-arm64", "win-64"]

[build-dependencies]
c-compiler = "*"
cxx-compiler = "*"
cython = "*"
setuptools-scm = ">=8.1"

[host-dependencies]
python = ">=3.9"
pip = "*"

[dependencies]
numexpr = "*"
pandas = ">=1.1"
tabmat = ">=4.0.0"
scikit-learn = ">=0.23"
scipy = "*"
tqdm = "*"
formulaic = "*"
pyarrow = "*"

[feature.no-tabmat.dependencies]
numexpr = "*"
pandas = ">=1.1"
scikit-learn = ">=0.23"
scipy = "*"
tqdm = "*"
formulaic = "*"
pyarrow = "*"

[feature.test.dependencies]
statsmodels = "*"
pytest = "*"
pytest-xdist = "*"
git_root = "*"
click = "*"
attrs = "*"
psutil = "*"
mypy = "*"

[feature.dev.dependencies]
ipdb = "*"
ipython = "*"
line_profiler = "*"
memory_profiler = "*"
ipykernel = "*"

[feature.lint.dependencies]
pre-commit = "*"
cython-lint = "*"
ruff = "*"

[feature.docs.dependencies]
jinja2 = "*"
jupyterlab = "*"
jupytext = "*"
make = "*"
matplotlib-base = "*"
nbclassic = ">=0.2.8"
nbsphinx = ">=0.8.3"
sphinx = ">=3.5.3"
sphinx_rtd_theme = "*"
sphinxcontrib-apidoc = "*"
sphinxext-altair = "*"

[feature.tutorial.dependencies]
jupyterlab = "*"
altair = "*"
openml = "*"
dask-ml = ">=2022.5.27"
shapely = "*"
libpysal = "*"

[feature.benchmark.dependencies]
h2o-py = "*"
openjdk = "*"
[feature.benchmark.target.win-64.dependencies]
blas = "=*=*mkl"
[feature.benchmark.target.linux-64.dependencies]
blas = "=*=*mkl"

[feature.build-tabmat.dependencies]
c-compiler = "*"
cxx-compiler = "*"
cython = "!=3.0.4"
setuptools-scm = "*"
make = "*"
mako = "*"
xsimd = "<11|>12.1"
pip = "*"
[feature.build-tabmat.target.unix.dependencies]
jemalloc-local = "*"

[feature.py39.dependencies]
python = "3.9.*"
[feature.py310.dependencies]
python = "3.10.*"
[feature.py311.dependencies]
python = "3.11.*"
[feature.py312.dependencies]
python = "3.12.*"

[feature.oldies.dependencies]
python = "3.9.*"
pandas = "1.2.*"
scikit-learn = "0.24.*"
tabmat = "4.0.*"
scipy = "1.7.*"
formulaic = "0.6.*"

[environments]
default = ["dev", "test"]
docs = ["docs", "tutorial"]
benchmark = ["benchmark"]
py39 = ["py39", "test"]
py310 = ["py310", "test"]
py311 = ["py311", "test"]
py312 = ["py312", "test"]
oldies = ["oldies", "test"]
lint = { features = ["lint"], no-default-feature = true}
nightly = ["py312", "build-tabmat", "test"]
glum-tabmat = { features = ["no-tabmat", "build-tabmat", "dev", "test"], no-default-feature = true}

pixi.lock.txt
$ pixi add python
<takes 90s until it prints anything>
updating lockfile ...
-> total 140s

Issue description
We want to adopt pixi in https://github.com/Quantco/glum.
When playing around with it, we noticed, that pixi add polars takes a long time to finish.
What I noticed is that most of this time is spent while pixi doesn't print anything and pixi also doesn't do any parallel work.
Only once the updating lock-file was printed, pixi used all cores on my machine.
Here a recording of what's happening:
https://github.com/prefix-dev/pixi/assets/29506042/0702e828-3b1e-4bf9-bce6-2205de39761f
https://github.com/prefix-dev/pixi/assets/29506042/2f8291d6-e56b-4fff-a840-c47a75a6c764
Do you know what's happening in the first part and why it is so slow?

Pixi add doesnt currently follow the same code path as other “solve” stages. We plan on fixing that soon. That will do parallel work as well as provide much better feedback and some other nice features.
Ill see if I can maybe whip something up tomorrow.

This is what it looks like with this PR
https://github.com/prefix-dev/pixi/assets/4995967/a78e13c7-9ecf-4c69-94f5-e0f9e35eec3b


Downgrade?

Any plans to downgrade to 1.20.1? I wanna use this mod but a lot of my mods are not up to date with 1.20.4, and fishing is the only mod that is on 1.20.4, at least for my list of mods(apart from optimizations)

i'd really like this mod to be downgraded to 1.20.1, i love it and it fixes one of the main issues minecraft has... fishing... i also would love to see the team that made pescatore work on a farming mod which is also just boring and oversimplified in a way that's a actual boring thing to do (like vanilla fishing)

1.20.1 please!!!!

Ty <3


BLM Timeline's Procs row shows Sharpcast being consumed by Paradox under Umbral Ice

The "Procs" row of the timeline for BLM erroneously shows UI Paradox consuming Sharpcast, although the "Sharpcast" row tracks it correctly if you open it.
It's evident in this log: https://xivanalysis.com/fflogs/8ayPA7dvMmz4pXBR/3/7 at roughly 3:15.

Adding a pic for clarity's sake:

This is fixed now as of PR #1537


Product Request: implementation of 'only-live-offers' and 'days' params and tests

Discussing this commit: https://github.com/akaszynski/keepa/pull/96/commits/30d3f354b991fc462d6f789790c263c89fbd9675
Product Request:
✔ 'only-live-offers' is implemented, tested, and passes test.
❌ 'days' is implemented, tested, but FAILS tests.
It would seem specifying 'days' as a parameter to the product request has no incidence of the API response... ?
I don't know where the issue is. Could it possibly be from Keepa? I find it unlikely.
Keepa Request Products documentation
I hope we can discuss this in this pull request,
Thank you,

A few months ago, I modified version 1.1.0 to include these options.
With the following call, the 'days' option worked fine, limiting the amount of data to ~90 days:
api.query(asinList, stats=30, domain=mkt, history=False, offers=100, update=24, days=90, only_live_offers=True)
Recently, I have been using the follow which does NOT work (returning years of data) since I only need Buy Box history:
api.query(asinList, stats=30, domain=mkt, history=False, update=24, buybox=True, days=90)
It seems that if I request the 'offers' option the 'days' option works. (The 'offers' option includes the Buy Box history so I do not need to specify buybox=True)
Given the above, it may be a Keepa issue.
Hope this helps.

A few months ago, I modified version 1.1.0 to include these options.
With the following call, the 'days' option worked fine, limiting the amount of data to ~90 days:
api.query(asinList, stats=30, domain=mkt, history=False, offers=100, update=24, days=90, only_live_offers=True)
Recently, I have been using the follow which does NOT work (returning years of data) since I only need Buy Box history:
api.query(asinList, stats=30, domain=mkt, history=False, update=24, buybox=True, days=90)
It seems that if I request the 'offers' option the 'days' option works. (The 'offers' option includes the Buy Box history so I do not need to specify buybox=True)
Given the above, it may be a Keepa issue.
Hope this helps.

Thank you for this, it is helpful to know we both have issues with this.
I opened a bug report on the Keepa forums.

According to Keepa support, the bug should be fixed. The days param should now limit historical data for the following "csv , buyBoxSellerIdHistory , salesRanks , offers and offers.offerCSV".
❌ However, my test still fails... ?

Issue with the tests failing is due to the private keys I use to run the tests.  Since it might be possible to scrape the key from a malicious PR, GitHub disables foreign PRs.  I'll have to figure out a workaround for this.
I'm getting a test failure at:
================================== FAILURES ===================================
___________________________ test_productquery_days ____________________________

api = <keepa.interface.Keepa object at 0x7f16455f10d0>

    def test_productquery_days(api):
        """Tests that 'days' param limits historical data to X days.
        This includes the csv, buyBoxSellerIdHistory, salesRanks, offers and offers.offerCSV fields."""

        def any_date_out_of_range(date_objects) -> bool:
            """Returns True if any date in date_objects if older than max_days, relative to now()."""
            today = datetime.datetime.now()
            return any((today - hist_date).days > max_days for hist_date in date_objects)

        max_days = 30
        request = api.query(PRODUCT_ASIN, offers=20, days=max_days)
        product = request[0]

        # Check if any Dateframes are out of date range.
        dataframes_dates = [set(df.index.tolist()) for key, df in product['data'].items() if key.startswith('df_')]
        dataframes_dates = chain.from_iterable(dataframes_dates)
        breakpoint()
>       assert not any_date_out_of_range(dataframes_dates)
E       assert not True
E        +  where True = <function test_productquery_days.<locals>.any_date_out_of_range at 0x7f1644202670>(<itertools.chain object at 0x7f163e907b20>)

tests/test_interface.py:225: AssertionError

Many of the dates are over 30 days:
2021-03-03 00:02:00
2021-02-17 16:54:00
2021-03-04 00:00:00
2021-02-25 11:02:00
2021-02-22 11:02:00
2021-03-04 11:00:00
2021-02-27 11:02:00
2021-02-26 00:02:00
2020-04-20 01:28:00
2020-04-20 01:28:00
2020-04-12 20:38:00
2020-12-25 17:00:00
2019-10-29 01:24:00
2021-03-01 19:14:00
2019-11-23 00:12:00
2021-03-01 19:14:00

Yes, the test fails although I thought I implemented the 'days' param properly.
I don't know why the Keepa API response is not limiting historical data.

Did some more digging:
I have confirmed the Keepa API reponse does filter out historical data for 'sales_ranks', and 'offers'.
However, as of now (2021-03-26), contrary to the Keepa documentation, it does not filter historical data for 'buy_box_seller_id_history' and 'offers_csv'.
Reported the bug to Keepa:
https://keepa.com/#!discuss/t/api-product-search-days-param-only-works-along-with-offers-param/12567/2
Once bug is fixed will submit new PR with fully passing tests.


HTML-encoded symbols are being parsed incorrectly

Good morning team,
Recently I've found that HTML-encoded symbols are being parsed incorrectly.
Example:
> Quote

> Quote

First case scenario will be parsed correctly. But the second scenario will be parsed incorrectly despite of being exactly same but HTML-encoded symbol.
This is a common situation because some of the text editors/services converts some symbols into HTML-encoded ones.
I've managed to reproduce it only with quotes. But it can be the case with other rules as well.

Are you expect that second example will be transformed to visually quoted text.
This beheaviour is not expectable for MD.
Look like this two lines works in gh markdown editor:

Quote

> Quote


 with no options matching the bound value should not select the first item

Description of the bug:
When a bound <select> value is set to something that doesn't match any options, it is ignored, and first <option> is selected instead.
Related: http://stackoverflow.com/questions/36409229/using-ractive-js-how-do-you-unselect-a-selected-select-box
Versions affected:
both 0.7.3 (<select> and model go out of sync) and edge (model value replaced by first <option>)
Reproduction:
http://jsfiddle.net/7bx6pc8h/1/
Code:
{{selected}}<br>

<select value="{{selected}}">
  <option>1</option>
  <option>2</option>
</select>

var ractive = new Ractive({
  el: 'body',
  template: '#template',
  data: { selected: 'x' }
});

setTimeout(() => {
  ractive.set('selected', 'y');
  console.log(ractive.get('selected')); // logs 1
}, 3000);

Note:
The current behavior is correct in a situation where there had been a selected <option>, but it was removed from the DOM. In that case, we should select the first option. However, I think this should not be the case when ractive.set() is called explicitly.

I think this is more or less forced by browsers now. Firefox as of mid-last-year (at least that's the first encounter I had with it) and Chrome as of 49 don't seem to let you have a select with options and no selected value. There's a strong chance I've missed something though :smile:

Try select.selectedIndex = -1

@evs-chris I posted this as a solution on SO and it works in all browsers.


FAB-17214 Update Azure yml files

This patch removes the python installation steps from azure yml files
and updates the script section.

@suryalnvs please review..

@rameshthoomu I am good with the changes. But smoke test with operator is failed at BeforeSuite while pulling the images from nexus.

Should this still point to 1.4.4-stable images, they are starting to age out of nexus since 1.4.5 is out


ntr: osmosensory signaling pathway

negative regulation of osmolarity signalling pathway

@ukemi
I am making the generic 'regulation of osmolarity signalling pathway' in order to appropriately define 'negative regulation of osmolarity signalling pathway', which is the the term Val requested.
I was wondering if I need to directly assert a subclass with the 'regulates' relationship. I see terms with and without the asserted subclass, so it's not completely clear to me what the appropriate pattern to follow should be.
'regulation of Kit signaling pathway':
Equivalent To: biological_process and (regulates some 'Kit signaling pathway')
Asserted SubClass Of: regulates some 'Kit signaling pathway'
'regulation of Fas signaling pathway'
Equivalent To: biological_process and (regulates some 'Fas signaling pathway')

Currently, I haven't put those in, and the stanzas look like this presently:
[Term]
+id: GO:0120027
+name: regulation of osmosensory signaling pathway
+namespace: biological_process
+def: "Any process that modulates the frequency, rate or extent of osmosensory signaling pathway." [GOC:vw]
+intersection_of: GO:0008150 ! biological_process
+intersection_of: regulates GO:0007231 ! osmosensory signaling pathway
+created_by: kchris
+creation_date: 2017-03-31T14:45:01Z
+
+[Term]
+id: GO:0120028
+name: negative regulation of osmosensory signaling pathway
+namespace: biological_process
+def: "Any process that stops, prevents or reduces the frequency, rate or extent of osmosensory signaling pathway." [GOC:vw]
+intersection_of: GO:0008150 ! biological_process
+intersection_of: negatively_regulates GO:0007231 ! osmosensory signaling pathway
+created_by: kchris
+creation_date: 2017-03-31T15:00:28Z

We need to talk about this on the call today. It appears that it is inconsistent. I think they should be asserted.

@cmungall advised that it should not be necessary to manually assert SubClass Of relationships of the form:
regulates some 'Kit signaling pathway'
The equivalence axiom of the form:
Equivalent To: biological_process and (regulates some 'Kit signaling pathway')
is sufficient to generate the appropriate regulation relationship in the fully reasoned file (though note that we won't see it in Protege or in the obo stanza of the working file).
Thus for these terms, I will make the equivalence axiom but not the asserted subclass.

Ticket to computationally remove unnessecary asserted regulation relationships: https://github.com/geneontology/go-ontology/issues/13275


Missing libssl.so.x.x symlink in final docker image

This only affects newly built images; docker pull lycheeverse/lychee as per the README fetches a working image ("Created": "2024-03-22T23:29:28.775271855Z").
Due to the floating versions of the base images used in the Dockerfile, i.e. rust:latest & debian:bullseye-slim, there is a issue between the libssl-dev version of the builder stage & that of the final base image. This causes the following error when running a freshly built container:
lychee: error while loading shared libraries: libssl.so.3: cannot open shared object file: No such file or directory

As of time of writing, building the Dockerfile fetches:
> docker pull rust:latest
latest: Pulling from library/rust
...
Digest: sha256:00e330d2e2cdada2b75e9517c8359df208b3c880c5e34cb802c120083d50af35
...

> docker pull debian:bullseye-slim
bullseye-slim: Pulling from library/debian
...
Digest: sha256:a165446a88794db4fec31e35e9441433f9552ae048fb1ed26df352d2b537cb96
...

The contents of /usr/lib/x86_64-linux-gnu/ for each of these images is: libssl.so, libssl.so.3 for rust:latest, & libssl.so.1.1 for debian:bullseye-slim. Hence, the lychee from builder is expected libssl.so.3 where the final image contains only libssl.so.1.1.
Solutions could be to:

add a step to the Dockerfile to create a symlink for the expected libssl.so.x.x file, assuming its compatible across base-images.
fix the base images to revision tags that are known to be compatible, e.g. FROM rust@sha256:...
use the same base image for both builder & final stages

Thanks.
From the options you presented, we'd ideally use the same image. Maybe we could change it to rust:slim-buster, which would ensure that the image stays small?

The current lychee docker image (available via docker pull lycheeverse/lychee) weighs 107MB & uses rust:latest & debian:bullseye-slim. That rust:latest aliases to rust:bookworm (docker) which is Debian 12 (releases). However, debian:bullseye-slim is Debian 11 - I believe that's why there's different libssl.so.x.x files present.
I think the best solution is to, instead, ensure the two base images use the same Debian version. I tested the Dockerfile with both the same base images, separately, as rust:slim-bullseye & rust:slim-buster (despite being Debian 10) as you suggested. In short, rust images are  b i g (~1GB).
I'd suggest using rust:bookworm & debian:bookworm-slim, which produces a 375MB image & appears to work as expected. I cannot account for the difference in lychee executable size between this new image & the original (~292MB vs. 84MB)

I cannot account for the difference in lychee executable size between this new image & the original (~292MB vs. 84MB)

Yes, I can. The executable isn't stripped in Dockerfile as it is in the GitHub Releases here. I have realised that the Dockerfile isn't used for the public image release.

As far as I can see, this can be closed because of the excellent work by @MattTimms.
Thanks for looking into this and fixing it so cleanly and thoroughly. Props!
The libssl issues should be resolved now; please reopen in case that's incorrect.)


Added entry to config so that the line ending is configurable

I've also made a commit that fixes a bug with the BST


ssh and mysql preflight

[ ] could connection be established
[ ] does configured filesystem and database exist
[ ] collect all database tables to include it in the mysqldump command

closed in favor of #73


feat(Storage): use VirtualTable

https://github.com/ydb-platform/ydb-embedded-ui/issues/613
You need to turn on the experiment "Use table with data load on scroll for Nodes and Storage cluster tabs" in settings to enable this table

I noticed that when I switch filter to All in Storage tab (no matter what type of storage type is selected), api request is duplicated.

I noticed that when I switch filter to All in Storage tab (no matter what type of storage type is selected), api request is duplicated.

I know this issue, I didn't figured out how to fix it yet. I created an issue: https://github.com/ydb-platform/ydb-embedded-ui/issues/631
I'd tackle it separately, from user perspective table works as supposed, though not totally effective


HELP cant install truffle

npm ERR! code EACCES
npm ERR! syscall mkdir
npm ERR! path /usr/local/lib/node_modules/truffle
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, mkdir '/usr/local/lib/node_modules/truffle'
npm ERR!  [Error: EACCES: permission denied, mkdir '/usr/local/lib/node_modules/truffle'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/usr/local/lib/node_modules/truffle'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.
npm ERR!      /Users/sam/.npm/_logs/2022-04-24T13_42_37_145Z-debug-0.log
sam@SAMs-MacBook-Pro nft-erc721-collection-2.3.0 % root/Administrator
zsh: no such file or directory: root/Administrator
sam@SAMs-MacBook-Pro nft-erc721-collection-2.3.0 %

I'm sorry @samdupnamgyal but this is an invalid issue.
Please read the instructions carefully and use issues for bug reports only. If yo need support, please use the Discussions section.

@samdupnamgyal
We also have a dedicated "Video Tutorials" section in the README file with instructions on how to use the tools. The first videos are about dependencies installation on any operating system: https://github.com/hashlips-lab/nft-erc721-collection

@samdupnamgyal hello, I had the same issue. I don't know if you were able to fix it but i just ran "sudo npm i -g truffle" and it worked perfectly for me. hope that helps. :)


Investigate what to do with missing starter files

What
Users can delete starter files either accidentally or deliberately - this can cause different issues, for example:

deleting .gitignore means node_modules gets committed, which can cause issues deploying or running on other machines
deleting package.json would cause similar issues
deleting files like routes.js, application.js or application.scss means our documentation that refers to these files can be confusing

Investigate how we might best deal with file deletions
Why

This has caused support issues in the past (especially .gitignore)

Who needs to work on this

Developers
Interaction Designer
Content Designer

Who needs to review this

Developers
Interaction Designer
Content Designer

Done when

[ ] We know the effects of deleting different starter files and folders
[ ] Ideas of what to do about it
[ ] Decision made of what to do next

We're just going to focus on .gitignore for now as its the only commonly reported issue

This may have been fixed by the new processes in v13. .gitignore files going missing used to happen roughly 4 times a year, let's revisit if it comes up again, if it hasn't occurred again by August 2023, we can consider closing this.


IOS app: Poll self-hosted server setting

To resolve the slow response time of up to 20minutes for notifications to come through via iOS app
Would it not be possible for the iOS app to simple do a GET request to the self-hosted server to fetch any new notifications?
I know next to nothing about the iOS app limitations, but a background GET request every minute doesn't seem like the biggest resource draw
The reason i focus on self hosting is to avoid bombarding https://nfty.sh/
but would offer developers the advantage of boosting their own service without having to rely to heavily on the 20 minute delay

Well that isn't great at all :-/
I guess the only way is to create your own firebase application ?
https://ntfy.sh/docs/config/#ios-instant-notifications
I tried adding the upstream-base-url: "https://ntfy.sh" but i get no notifications at all :-/
sucks having to pay 99$ a year just for notifications

Try adding the upstream URL and then also enabling trace mode via the log-level setting. Then publish a test message to a test topic and post it here.
The base URL you define in server.yml and the one you enter in the iOS app have to match exactly.


lax.dynamic_slice inside jit

Should this work?
import jax
import jax.numpy as np

@jax.jit
def sum_first_k(a, k):
  return np.sum(lax.dynamic_slice(a, (0,), (k,)))

sum_first_k(np.arange(3.0), 2)

Here's the traceback I get:
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-167-645715d2be42> in <module>()
----> 1 sum_first_k(np.arange(3.0), 2)

13 frames
/usr/local/lib/python3.6/dist-packages/jax/api.py in f_jitted(*args, **kwargs)
    121     _check_args(args_flat)
    122     flat_fun, out_tree = flatten_fun_leafout(f, in_tree)
--> 123     out = xla.xla_call(flat_fun, *args_flat, device_values=device_values)
    124     return out if out_tree() is leaf else tree_unflatten(out_tree(), out)
    125

/usr/local/lib/python3.6/dist-packages/jax/core.py in call_bind(primitive, f, *args, **params)
    661   if top_trace is None:
    662     with new_sublevel():
--> 663       ans = primitive.impl(f, *args, **params)
    664   else:
    665     tracers = map(top_trace.full_raise, args)

/usr/local/lib/python3.6/dist-packages/jax/interpreters/xla.py in xla_call_impl(fun, *args, **params)
    604 def xla_call_impl(fun, *args, **params):
    605   device_values = FLAGS.jax_device_values and params.pop('device_values')
--> 606   compiled_fun = xla_callable(fun, device_values, *map(abstractify, args))
    607   try:
    608     return compiled_fun(*args)

/usr/local/lib/python3.6/dist-packages/jax/linear_util.py in memoized_fun(f, *args)
    206       if len(cache) > max_size:
    207         cache.popitem(last=False)
--> 208       ans = call(f, *args)
    209       cache[key] = (ans, f)
    210     return ans

/usr/local/lib/python3.6/dist-packages/jax/interpreters/xla.py in xla_callable(fun, device_values, *abstract_args)
    617   pvals = [pe.PartialVal((aval, core.unit)) for aval in abstract_args]
    618   with core.new_master(pe.JaxprTrace, True) as master:
--> 619     jaxpr, (pval, consts, env) = pe.trace_to_subjaxpr(fun, master, False).call_wrapped(pvals)
    620     assert not env  # no subtraces here (though cond might eventually need them)
    621     compiled, result_shape = compile_jaxpr(jaxpr, consts, *abstract_args)

/usr/local/lib/python3.6/dist-packages/jax/linear_util.py in call_wrapped(self, *args, **kwargs)
    145
    146     del gen
--> 147     ans = self.f(*args, **dict(self.params, **kwargs))
    148     del args
    149     while stack:

<ipython-input-165-9a17ef1ee2d8> in sum_first_k(a, k)
      1 @jax.jit
      2 def sum_first_k(a, k):
----> 3   return np.sum(lax.dynamic_slice(a, (0,), (k,)))

/usr/local/lib/python3.6/dist-packages/jax/lax/lax.py in dynamic_slice(operand, start_indices, slice_sizes)
    607   return dynamic_slice_p.bind(
    608       operand, start_indices, slice_sizes=tuple(slice_sizes),
--> 609       operand_shape=operand.shape)
    610
    611 def dynamic_update_slice(operand, update, start_indices):

/usr/local/lib/python3.6/dist-packages/jax/core.py in bind(self, *args, **kwargs)
    145
    146     tracers = map(top_trace.full_raise, args)
--> 147     out_tracer = top_trace.process_primitive(self, tracers, kwargs)
    148     return full_lower(out_tracer)
    149

/usr/local/lib/python3.6/dist-packages/jax/interpreters/partial_eval.py in process_primitive(self, primitive, tracers, params)
    100       tracers = map(self.instantiate_const, tracers)
    101       avals = [t.aval for t in tracers]
--> 102       out_aval = primitive.abstract_eval(*avals, **params)
    103       eqn = JaxprEqn(tracers, None, primitive, (), False, False, params)
    104       return JaxprTracer(self, PartialVal((out_aval, unit)), eqn)

/usr/local/lib/python3.6/dist-packages/jax/lax/lax.py in standard_abstract_eval(shape_rule, dtype_rule, *args, **kwargs)
   1405     return ShapedArray(shape_rule(*args, **kwargs), dtype_rule(*args, **kwargs))
   1406   elif least_specialized is ShapedArray:
-> 1407     return ShapedArray(shape_rule(*args, **kwargs), dtype_rule(*args, **kwargs))
   1408   elif least_specialized is UnshapedArray:
   1409     return UnshapedArray(dtype_rule(*args, **kwargs))

/usr/local/lib/python3.6/dist-packages/jax/lax/lax.py in _dynamic_slice_shape_rule(operand, start_indices, slice_sizes, operand_shape)
   2608            "start_indices, got start_inidices length {} and slice_sizes {}.")
   2609     raise TypeError(msg.format(len(start_indices), slice_sizes))
-> 2610   if not onp.all(onp.less_equal(slice_sizes, operand.shape)):
   2611     msg = ("slice slice_sizes must be less than or equal to operand shape, "
   2612            "got slice_sizes {} for operand shape {}.")

/usr/local/lib/python3.6/dist-packages/jax/core.py in __bool__(self)
    340   def __getitem__(self, idx): return self.aval._getitem(self, idx)
    341   def __nonzero__(self): return self.aval._nonzero(self)
--> 342   def __bool__(self): return self.aval._bool(self)
    343   def __float__(self): return self.aval._float(self)
    344   def __int__(self): return self.aval._int(self)

/usr/local/lib/python3.6/dist-packages/jax/abstract_arrays.py in error(self, *args)
     36 def concretization_function_error(fun):
     37   def error(self, *args):
---> 38     raise TypeError(concretization_err_msg(fun))
     39   return error
     40

TypeError: Abstract value passed to `bool`, which requires a concrete value. The function to be transformed can't be traced at the required level of abstraction. If using `jit`, try using `static_argnums` or applying `jit` to smaller subfunctions instead.

I know XLA can't have variable sized outputs, but here I'm summing the outputs, so in principle that shouldn't be an issue.

No, it shouldn't work: actually it's not just that XLA (and JAX's jit, which is what's actually raising the error here for tracing reasons) require fixed output shapes, but all the shapes of the intermediates need to be fixed too. So summing the output doesn't help; that lax.dynamic_slice alone is a problem.
Here are a two alternatives, both of which you probably know about:
from __future__ import print_function
from functools import partial

import jax
import jax.numpy as np

@partial(jax.jit, static_argnums=(1,))
def sum_first_k(a, k):
  return np.sum(jax.lax.dynamic_slice(a, (0,), (k,)))

print(sum_first_k(np.arange(3.0), 2))

@jax.jit
def sum_first_k(a, k):
  n = len(a)
  return np.sum(np.where(np.arange(n) < k, a, 0))

print(sum_first_k(np.arange(3.0), 2))

The first is a way of solving the problem with recompilation. The second is a way to solve it with masking, for which XLA can still generate very efficient code by fusing the selection into the reduction rather than round-tripping several arrays to memory. A third strategy is to use a loop construct.
WDYT?

It would be nice if the error message said something like this, rather than
sending me down a rabbit hole. What actually happened is that I first tried
using indexing like a[:k], which generated an error encouraging me to try
lax.dyanmic_slice.
On Tue, Jul 9, 2019 at 9:49 PM Matthew Johnson<EMAIL_ADDRESS>wrote:

No, it shouldn't work: actually it's not just that XLA (and JAX's jit,
which is what's actually raising the error here for tracing reasons)
require fixed output shapes, but all the shapes of the intermediates need
to be fixed too. So summing the output doesn't help; that
lax.dynamic_slice alone is a problem.
Here are a two alternatives, both of which you probably know about:
from future import print_functionfrom functools import partial
import jaximport jax.numpy as np
@partial(jax.jit, static_argnums=(1,))def sum_first_k(a, k):
return np.sum(jax.lax.dynamic_slice(a, (0,), (k,)))
print(sum_first_k(np.arange(3.0), 2))
@jax.jitdef sum_first_k(a, k):
n = len(a)
return np.sum(np.where(np.arange(n) < k, a, 0))
print(sum_first_k(np.arange(3.0), 2))
The first is a way of solving the problem with recompilation. The second
is a way to solve it with masking, for which XLA can still generate very
efficient code by fusing the selection into the reduction rather than
round-tripping several arrays to memory. A third strategy is to use a loop
construct.
WDYT?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/google/jax/issues/1007?email_source=notifications&email_token=AAJJFVXUEEPZYGHSO7K6WE3P6VS7HA5CNFSM4H7I6LE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZSI4PI#issuecomment-509906493,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAJJFVWEJNLZMCW3JTWJYJTP6VS7HANCNFSM4H7I6LEQ
.

Is there a way to make the construct with static_argnums work with sum_first_k inside vmap?
Calling sum_first_k as a standalone function works fine:
print(sum_first_k(np.arange(3.0), 2))
But it doesn't work when calling it from vmap
vmap_sum_first_k = jax.vmap(sum_first_k,(None,0))
print(vmap_sum_first_k(np.arange(10.0), np.arange(4)))
TypeError: Abstract value passed to bool, which requires a concrete value. The function to be transformed can't be traced at the required level of abstraction. If using jit, try using static_argnums or applying jit to smaller subfunctions instead.

It's a little awkward, but you can make something like this work using explicit masking:
@jax.jit
def sum_first_k(a, k):
  return np.sum(a * (np.arange(a.size) < k))

Ok, thanks, that works.  But it is slow in my use case: a  dimension is roughly 3e6 x 100 while each k is a slice, not a number, with different slice lengths where median slice length is about 50e3. So explicit masking results in huge number of multiplications by 0. Which is too bad - other parts of my program are unbelievably fast using jax with gpu, much faster than anything else I tried.

I think it's worth adding that slice_sizes needs to be static to the dynamic_slice() docstring. I ran into the same issue as shoyer@ above, where I want dynamic slice_sizes() and first tried indexing a[:k] to get this error message, before ending up here.
I can send in a PR if that sounds good, WDYT?

I think it's worth adding that slice_sizes needs to be static to the dynamic_slice() docstring. I can send in a PR if that sounds good, WDYT?

This is a good suggestion! I added clarifications to both the dynamic_slice documentation and this error message in: https://github.com/google/jax/pull/3795
I'm close this issue, since hopefully users will see the more descriptive errors in the future and won't be misled.

@shoyer Is there a workaround for situations where masking doesn't apply? My end goal here it to define a train/test split function that is jit-able and which I could apply vmap over.

It's a little awkward, but you can make something like this work using explicit masking:
@jax.jit
def sum_first_k(a, k):
return np.sum(a * (np.arange(a.size) < k))

@shoyer Is there a workaround for situations where masking doesn't apply? My end goal here it to define a train/test split function that is jit-able and which I could apply vmap over.

There are probably work-arounds in other specific cases, but this is a pretty fundamental limitation in XLA.


[Namespace] Using namespaces in chat engine

Is this your first time submitting a feature request?

[X] I have searched the existing issues, and I could not find an existing issue for this feature
[X] I am requesting a straightforward extension of existing functionality

Describe the feature
from canopy.knowledge_base import KnowledgeBase
kb = KnowledgeBase(index_name="{{TestIndex}}",namespace="{{TestNamespace}}")
kb.connect()
Similar to chain using namespace in vector stores, namespaces should be supported in Chat Engines
Describe alternatives you've considered
from canopy.knowledge_base import KnowledgeBase
from canopy.chat_engine import ChatEngine
kb = KnowledgeBase(index_name="demo")
kb.connect()
ContextEngine(kb,global_metadata_filter={'key': {'$eq': 'value'}})
chat_engine=ChatEngine(context_engine)
Who will this benefit?
Balancing costs of using a new index for each set instead of multiple namespaces in an index. Metadata filtering works but is not backwards compatible with existing namespaces code.
Are you interested in contributing this feature?
No
Anything else?
No response

Hi @shubhampatilappy !
Sorry for the late response 🙏
We are working on adding this very soon, right now aiming for api level setting of namespace
Meaning the base api would be something like:
host:port/{namespace}/chat/completions

Heck yeah!


Enhancement of split op with indices option

Description
Support split by arbitrary ranges through addition of indices option.
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[ ] The PR title starts with [MXNET-$JIRA_ID], where $JIRA_ID refers to the relevant JIRA issue created (except PRs with tiny changes)
[ ] Changes are complete (i.e. I finished coding on this PR)
[ ] All changes have test coverage:
Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
[ ] Code is well-documented:
For user-facing API changes, API doc string has been updated.
For new C++ functions in header files, their functionalities and arguments are documented.
For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
Check the API doc at http://mxnet-ci-doc.s3-accelerate.dualstack.amazonaws.com/PR-$PR_ID/$BUILD_ID/index.html
[ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

[x] Indices option for split op
[x] Unit tests

Comments

I don't see the docs that you linked. Could you relink?

@larroy the string in the describe call on the next line is the doc for the argument (after you click into the provided link), and there's also an extra example added in the .cc file for this operator. Have you read the tutorial on how to add an operator? I feel like you're asking for more documentations but you haven't read the existing ones, please checkout Parameter Registration&Operator Registration section of this doc: https://mxnet.incubator.apache.org/faq/add_op_in_backend.html, thanks

Thanks, to be specific about my question, you added an "indices" parameter but you didn't add that to the example code which you modified. Would it make sense to use this added parameter in the examples?

@larroy There was a typo in the example, the indices argument was named sections before, so indeed there was a related example added.

@HyperZealot Thanks for the contribution. Can you look into failing builds so that we can move forward with this PR?

@HyperZealot Update [pr-awaiting-response]


New version: Hibbiki.Chromium version 124.0.6367.79

Created by 🥟 Dumplings in workflow run #961.
Logs
Updated: 124.0.6367.61 → 124.0.6367.79
Cannot validate argument on parameter 'ChatID'. The argument is null, empty, or consists of only white-space characters. Provide an argument that contains non white-space characters, and then try the command again.
Submitting WinGet manifests
Cannot validate argument on parameter 'ChatID'. The argument is null, empty, or consists of only white-space characters. Provide an argument that contains non white-space characters, and then try the command again.

For moderators
This account is automated by Github Actions and the source code was created by CoolPlayLin. If you have any questions about any pull request, don't hesitate to ping Exorcism0666.

[!warning]
With the new version of Komac, numerous Pull Requests may contain bugs or unwanted elements. If such issues arise, please tag russellbanks so that these bugs can be addressed in future updates.

[!important]
Please carefully review these Pull Request before merging. If it is a Pull Request for removing incorrect content and the URLs are issue, free upon manual checking, please close this Pull Request directly. (It is best to inform Exorcism0666 of the closure.)

良い一日をお過ごしください！


Try to increase open files limit and warn if it is still too low afterwards

Fixes #1368

Note: the warning is the most important thing, trying to set it (which may only work on MacOS) is more of a convenience


Disappearance of alerts/notifications bars

These bars should not disappear automatically.

@dawid-g fixed

verified as fixed.


[networking_mapping] Add DNS field to the networking definition.

As a pull request owner and reviewers, I checked that:

[x] Appropriate testing is done and actually running
[x] Appropriate documentation exists and/or is up-to-date:

[x] README in the role
[x] Content of the docs/source is reflecting the changes

recheck

@pablintino can you please rebase this pr ?

/approve

/lgtm

recheck


Update requirements.txt

For some reason it only worked for me when I explicitly added the argon2-cffi dependency to the backend requirements
Pull Request Checklist

[X ] Description: Briefly describe the changes in this pull request.
[ X] Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
[ ] Documentation: Have you updated relevant documentation?
[ ] Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?

Description
I kept getting the following error when trying to bring up the docker compose.
open-webui  |   File "/app/backend/main.py", line 18, in <module>
open-webui  |     from litellm.proxy.proxy_server import ProxyConfig, initialize
open-webui  |   File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 12, in <module>
open-webui  |     from argon2 import PasswordHasher
open-webui  | ModuleNotFoundError: No module named 'argon2'

I fixed it by adding argon2-cffi to the requirements.txt in the backend

Changelog Entry
Added

Added argon2-cffi to the requirements

Fixed
open-webui  |   File "/app/backend/main.py", line 18, in <module>
open-webui  |     from litellm.proxy.proxy_server import ProxyConfig, initialize
open-webui  |   File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 12, in <module>
open-webui  |     from argon2 import PasswordHasher
open-webui  | ModuleNotFoundError: No module named 'argon2'

Changed

Added argon2-cffi to the requirements

Thanks for the PR! This is has been addressed with the latest release!


Integrating a ci pipeline into the project

What about integrating a ci pipeline into the project?
In this way, we could automate the process of checking that the new changes are breaking the codebase or not.
I think that a Github Actions pipeline would fit well here.

@b0iq Are you running the example site?
It's quite annoying currently, for me it was 3 terminals, one running the liveserver, one watching to build the package, and one that was running the tests when the package changed 😤

@ekwoka
Yeah, me too, I'm running three terminals just like you and I did install playwright but it keeps failing 🤬

@b0iq Which of the tests are failing for you?

Test cloud be easily automated. Even running the example site (even tough I think leerobs pr to add vercel should have done it?) Anyways:  👍


Datasource getting hit x times with x styles

Is it normal for a datasource to get x queries when x styles are defined to a layer?
I am using this SQL Server plugin: https://github.com/gaspardle/mapnik-mssql and it seems that it's sending two queries to sql server when I have two styles defined for one layer. Performance is getting hit hard because of it.
I am asking here to see if its a normal behavior in mapnik, or a bug in nodejs mapnik, or a bug in the plugin (or the plugin needs some sort of cache on when to reuse resultsets).

After some research seems I want cache-features according to https://github.com/mapnik/mapnik/wiki/XMLConfigReference
Will wait for a reply to confirm, and then close.

Is it normal for a datasource to get x queries when x styles are defined to a layer?

Yes, it is intended behavior now. The reason why cache-features is not used by default is that it can take a lot of memory, see https://github.com/mapnik/mapnik/issues/657.
I think that can make sense to reevaluate this decision as I'm using cache-features on all layers with more than one style without any memory issue so far.


fix: releasing devnet docker img

increase arm build timeout to 60
use ubuntu-latest for docker manifest stuff

Benchmark results
Metrics with a significant change:

avm_simulation_time_ms (Token:transfer_public): 21.6 (-34%)

Detailed results
All benchmarks are run on txs on the Benchmarking contract on the repository. Each tx consists of a batch call  to create_note and increment_balance, which guarantees that each tx has a private call, a nested private call, a public call, and a nested public call, as well as an emitted private note, an unencrypted log, and public storage read and write.
This benchmark source data is available in JSON format on S3 here.
Proof generation
Each column represents the number of threads used in proof generation.

Metric
1 threads
4 threads
16 threads
32 threads
64 threads

proof_construction_time_sha256_ms
5,752
1,561
711
763 (+2%)
770

proof_construction_time_sha256_30_ms
11,504
3,113 (+1%)
1,396
1,442
1,466 (+1%)

proof_construction_time_sha256_100_ms
43,965
11,761 (-1%)
5,714 (+5%)
5,677 (+4%)
5,793

proof_construction_time_poseidon_hash_ms
78.0
34.0
34.0
58.0 (+2%)
88.0 (+1%)

proof_construction_time_poseidon_hash_30_ms
1,525
421
202
231 (+3%)
270

proof_construction_time_poseidon_hash_100_ms
5,637
1,509
688 (+2%)
732 (-1%)
743 (-1%)

L2 block published to L1
Each column represents the number of txs on an L2 block published to L1.

Metric
4 txs
8 txs
16 txs

l1_rollup_calldata_size_in_bytes
4,356
7,876
14,884

l1_rollup_calldata_gas
50,208
93,032
178,096

l1_rollup_execution_gas
1,396,459
2,130,179
3,915,299

l2_block_processing_time_in_ms
253
462 (+4%)
817 (-1%)

l2_block_building_time_in_ms
11,337
22,486 (+1%)
44,344 (-1%)

l2_block_rollup_simulation_time_in_ms
11,336
22,486 (+1%)
44,343 (-1%)

l2_block_public_tx_process_time_in_ms
9,620
20,606 (+1%)
42,502 (-1%)

L2 chain processing
Each column represents the number of blocks on the L2 chain where each block has 8 txs.

Metric
3 blocks
5 blocks

node_history_sync_time_in_ms
2,961 (+1%)
4,096 (+6%)

node_database_size_in_bytes
12,640,336
16,719,952

pxe_database_size_in_bytes
16,254
26,813

Circuits stats
Stats on running time and I/O sizes collected for every kernel circuit run across all benchmarks.

Circuit
simulation_time_in_ms
witness_generation_time_in_ms
input_size_in_bytes
output_size_in_bytes
proving_time_in_ms

private-kernel-init
84.6 (-5%)
400 (+1%)
21,755
44,860
N/A

private-kernel-inner
163 (-2%)
711 (+1%)
72,566
45,007
N/A

private-kernel-reset-tiny
304 (-1%)
719 (-1%)
65,590
44,846
N/A

private-kernel-tail
162 (-1%)
137 (+1%)
50,643
52,257
N/A

base-parity
5.65 (+1%)
N/A
160
96.0
N/A

root-parity
36.0 (+2%)
N/A
73,948
96.0
N/A

base-rollup
3,475 (+1%)
N/A
189,136
664
N/A

block-root-rollup
41.8
N/A
58,205
2,448
N/A

public-kernel-setup
84.3 (+1%)
N/A
105,085
71,222
N/A

public-kernel-app-logic
96.0 (+1%)
N/A
104,911
71,222
N/A

public-kernel-tail
876
N/A
390,582
16,414
N/A

private-kernel-reset-small
306 (-1%)
N/A
66,341
45,629
N/A

private-kernel-tail-to-public
1,384
617 (+1%)
455,020
1,825
N/A

public-kernel-teardown
82.6
N/A
105,349
71,222
N/A

merge-rollup
21.0
N/A
38,174
664
N/A

undefined
N/A
N/A
N/A
N/A
80,671 (+2%)

Stats on running time collected for app circuits

Function
input_size_in_bytes
output_size_in_bytes
witness_generation_time_in_ms

ContractClassRegisterer:register
1,344
11,731
344

ContractInstanceDeployer:deploy
1,408
11,731
18.3

MultiCallEntrypoint:entrypoint
1,920
11,731
406

FeeJuice:deploy
1,376
11,731
393 (+1%)

SchnorrAccount:constructor
1,312
11,731
74.1 (-1%)

SchnorrAccount:entrypoint
2,304
11,731
398

Token:privately_mint_private_note
1,280
11,731
106 (+2%)

FPC:fee_entrypoint_public
1,344
11,731
26.3 (-10%)

Token:transfer
1,312
11,731
229 (-3%)

Benchmarking:create_note
1,344
11,731
88.6 (+1%)

SchnorrAccount:verify_private_authwit
1,280
11,731
27.5

Token:unshield
1,376
11,731
527

FPC:fee_entrypoint_private
1,376
11,731
803 (+13%)

AVM Simulation
Time to simulate various public functions in the AVM.

Function
time_ms
bytecode_size_in_bytes

FeeJuice:_increase_public_balance
58.0 (+6%)
7,739

FeeJuice:set_portal
11.0 (-9%)
2,354

Token:constructor
80.2 (-2%)
26,051

FPC:constructor
54.7 (+4%)
18,001

FeeJuice:mint_public
40.7 (+2%)
5,877

Token:mint_public
71.3 (+3%)
10,917

Token:assert_minter_and_mint
41.6 (+1%)
7,512

AuthRegistry:set_authorized
38.8 (+2%)
4,391

FPC:prepare_fee
232
7,043

Token:transfer_public
:warning: 21.6 (-34%)
39,426

FPC:pay_refund
53.3
10,234

Benchmarking:increment_balance
1,215 (+1%)
6,563

Token:_increase_public_balance
40.5 (-1%)
8,433

FPC:pay_refund_with_shielded_rebate
64.9 (+2%)
10,783

Public DB Access
Time to access various public DBs.

Function
time_ms

get-nullifier-index
0.159 (+1%)

Tree insertion stats
The duration to insert a fixed batch of leaves into each tree type.

Metric
1 leaves
16 leaves
64 leaves
128 leaves
256 leaves
512 leaves
1024 leaves

batch_insert_into_append_only_tree_16_depth_ms
2.22 (+2%)
3.96 (+3%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_count
16.8
31.7
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_16_depth_hash_ms
0.116 (+2%)
0.112 (+3%)
N/A
N/A
N/A
N/A
N/A

batch_insert_into_append_only_tree_32_depth_ms
N/A
N/A
11.4
17.8 (+2%)
30.4
60.4 (+3%)
112 (-1%)

batch_insert_into_append_only_tree_32_depth_hash_count
N/A
N/A
95.9
159
287
543
1,055

batch_insert_into_append_only_tree_32_depth_hash_ms
N/A
N/A
0.109
0.103 (+2%)
0.0986
0.104 (+2%)
0.100

batch_insert_into_indexed_tree_20_depth_ms
N/A
N/A
14.3 (-1%)
26.8 (+4%)
43.0 (-1%)
86.5 (+5%)
160 (-1%)

batch_insert_into_indexed_tree_20_depth_hash_count
N/A
N/A
109
207
355
691
1,363

batch_insert_into_indexed_tree_20_depth_hash_ms
N/A
N/A
0.109 (-1%)
0.107 (+5%)
0.104 (-1%)
0.107 (+6%)
0.0993 (-2%)

batch_insert_into_indexed_tree_40_depth_ms
N/A
N/A
16.9 (+2%)
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_count
N/A
N/A
132
N/A
N/A
N/A
N/A

batch_insert_into_indexed_tree_40_depth_hash_ms
N/A
N/A
0.108 (+2%)
N/A
N/A
N/A
N/A

Miscellaneous
Transaction sizes based on how many contract classes are registered in the tx.

Metric
0 registered classes
1 registered classes

tx_size_in_bytes
64,779
668,997

Transaction size based on fee payment method
| Metric |  |
| - |  |


Update the sample-runs script to run any new examples and miniapps

Script: https://github.com/mfem/mfem/blob/master/config/sample-runs.sh
Also, update the sample-runs script in the internal tests.

Currently missing:
Miniapps

[ ] adjoint
[ ] gslib
[ ] navier
[ ] nurbs
[ ] shifted
[ ] solvers
[ ] tools
[ ] toys

Examples

[ ] amgx
[ ] caliper
[ ] ginkgo
[ ] hiop
[ ] pumi
[ ] superlu

Let's use this issue to keep track of some problems with sample runs uncovered in #2224: https://github.com/mfem/mfem/pull/2224#issuecomment-840012326
We should re-open this issue when it is auto-closed by merging #2224.