Limit number of data shown in repr

[x] Closes #4736
[x] Tests added
[x] Passes isort . && black . && mypy . && flake8
[ ] User visible changes (including notable bug fixes) are documented in whats-new.rst
[ ] New functions/methods are listed in api.rst

Test example:
a = np.arange(0, 2000)
b = np.core.defchararray.add("long_variable_name", a.astype(str))
c = np.arange(0, 30)
d = np.core.defchararray.add("attr_", c.astype(str))
e = {k: 2 for k in d}
coords = dict(time=da.array([0, 1]))
data_vars = dict()
for v in b:
    data_vars[v] = xr.DataArray(
        name=v,
        data=da.array([3, 4]),
        dims=["time"],
        coords=coords,
    )
ds0 = xr.Dataset(data_vars)
ds0.attrs = e

Looks like this with 24 max rows of interesting data:
print(ds0)
Out[15]:
<xarray.Dataset>
Dimensions:                 (time: 2)
Coordinates:
  * time                    (time) int32 0 1
Data variables:
    long_variable_name0     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name2     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name3     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name4     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name5     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name6     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name7     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name8     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name9     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name10    (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name11    (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    ...                      ...
    long_variable_name1988  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1989  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1990  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1991  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1992  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1993  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1994  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1995  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1996  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1997  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1998  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1999  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
Attributes:
    attr_0:   2
    attr_1:   2
    attr_2:   2
    attr_3:   2
    attr_4:   2
    attr_5:   2
    attr_6:   2
    attr_7:   2
    attr_8:   2
    attr_9:   2
    attr_10:  2
    attr_11:  2
    ...       ...
    attr_18:  2
    attr_19:  2
    attr_20:  2
    attr_21:  2
    attr_22:  2
    attr_23:  2
    attr_24:  2
    attr_25:  2
    attr_26:  2
    attr_27:  2
    attr_28:  2
    attr_29:  2

With 20 rows of interesting data:
xr.set_options(display_max_rows=20)
ds0
Out[26]:
<xarray.Dataset>
Dimensions:                 (time: 2)
Coordinates:
  * time                    (time) int32 0 1
Data variables:
    long_variable_name0     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name2     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name3     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name4     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name5     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name6     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name7     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name8     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name9     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
                     ...
    long_variable_name1990  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1991  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1992  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1993  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1994  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1995  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1996  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1997  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1998  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1999  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
Attributes:
    attr_0:   2
    attr_1:   2
    attr_2:   2
    attr_3:   2
    attr_4:   2
    attr_5:   2
    attr_6:   2
    attr_7:   2
    attr_8:   2
    attr_9:   2
      ...
    attr_20:  2
    attr_21:  2
    attr_22:  2
    attr_23:  2
    attr_24:  2
    attr_25:  2
    attr_26:  2
    attr_27:  2
    attr_28:  2
    attr_29:  2

With 16 rows of interesting data:
xr.set_options(display_max_rows=16)
ds0
Out[28]:
<xarray.Dataset>
Dimensions:                 (time: 2)
Coordinates:
  * time                    (time) int32 0 1
Data variables:
    long_variable_name0     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name2     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name3     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name4     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name5     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name6     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name7     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
                     ...
    long_variable_name1992  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1993  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1994  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1995  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1996  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1997  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1998  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1999  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
Attributes:
    attr_0:   2
    attr_1:   2
    attr_2:   2
    attr_3:   2
    attr_4:   2
    attr_5:   2
    attr_6:   2
    attr_7:   2
      ...
    attr_22:  2
    attr_23:  2
    attr_24:  2
    attr_25:  2
    attr_26:  2
    attr_27:  2
    attr_28:  2
    attr_29:  2

With 12 rows of interesting data:
xr.set_options(display_max_rows=12)
print(ds0)
Out[79]:
<xarray.Dataset>
Dimensions:                 (time: 2)
Coordinates:
  * time                    (time) int32 0 1
Data variables:
    long_variable_name0     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name2     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name3     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name4     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name5     (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
                     ...
    long_variable_name1994  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1995  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1996  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1997  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1998  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
    long_variable_name1999  (time) int32 dask.array<chunksize=(2,), meta=np.ndarray>
Attributes:
    attr_0:   2
    attr_1:   2
    attr_2:   2
    attr_3:   2
    attr_4:   2
    attr_5:   2
      ...
    attr_24:  2
    attr_25:  2
    attr_26:  2
    attr_27:  2
    attr_28:  2
    attr_29:  2

Hello @Illviljan! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file xarray/core/formatting.py:

Line 390:1: E303 too many blank lines (3)

Hello @Illviljan! Thanks for opening this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file xarray/core/formatting.py:

Line 390:1: E303 too many blank lines (3)

Looks great! Could we add a test? We should already have done tests for reprs, so we can add a test case with enough lines for this to kick in.

Looks great! Could we add a test? We should already have done tests for reprs, so we can add a test case with enough lines for this to kick in.

Added a test now and of course the code got a little heavier to handle the edge cases.
I don't really get why this black is complaining. I get no diffs on my version. It would be nice if it would show the changes so I can manually make it happy. Dask shows the black diffs for example.

Added a test now and of course the code got a little heavier to handle the edge cases.
I don't really get why this black is complaining. I get no diffs on my version. It would be nice if it would show the changes so I can manually make it happy. Dask shows the black diffs for example.

Looks good!
I made the change on black (lmk if you need a hand setting that up client-side — ideally that should just work and make contributing easier not harder).
PRing a change so the black diff prints too

Looks good!
I made the change on black (lmk if you need a hand setting that up client-side — ideally that should just work and make contributing easier not harder).
PRing a change so the black diff prints too

I just do
import black
black.main([f"{filename}"])

in my ipython console and that usually works. Does xarray have any further settings?
Any opinions on the default max_rows? It should be an even number so it divides cleanly with 2. I prefer 12 now because it leaves some room for a lot of coords as well.

I just do
import black
black.main([f"{filename}"])

in my ipython console and that usually works. Does xarray have any further settings?
Any opinions on the default max_rows? It should be an even number so it divides cleanly with 2. I prefer 12 now because it leaves some room for a lot of coords as well.

For black, you should be able to run black . in the root of the project, and black will format all the files.
A more integrated approach is to run pre-commit install and then either on every commit or pre-commit run will run all the checks.
You should also be able to integrate black with your editor, such that it makes the changes on every file save.
Let me know if you need any help with that, or the pre-commit docs are here: https://pre-commit.com/ — having to run it manually for each file doesn't sound like fun 😄 ! If it's unclear, then we should make our docs clearer — it's on us, not you.

For black, you should be able to run black . in the root of the project, and black will format all the files.
A more integrated approach is to run pre-commit install and then either on every commit or pre-commit run will run all the checks.
You should also be able to integrate black with your editor, such that it makes the changes on every file save.
Let me know if you need any help with that, or the pre-commit docs are here: https://pre-commit.com/ — having to run it manually for each file doesn't sound like fun 😄 ! If it's unclear, then we should make our docs clearer — it's on us, not you.

Re number of rows — no strong view — something between 12-20 seems great.
Would you like to add a whatsnew too?
Thanks!

Re number of rows — no strong view — something between 12-20 seems great.
Would you like to add a whatsnew too?
Thanks!

Looks great! Any other comments from anyone before we merge?

Looks great! Any other comments from anyone before we merge?


Setting derived properties in evented models causes duplicate events

psygnal version: 0.9.1
Python version: 3.9.6
Operating System: macOS 13.4 (Intel)

Description
Setting a derived property when using an evented model causes the associated event to be fired twice.
Coming from the same titled issue over on napari: https://github.com/napari/napari/issues/6032
Our solution (https://github.com/napari/napari/pull/6035) was to block the event in __setattr__, but would be curious if you had better ideas.
What I Did
from psygnal import EventedModel

class Model(EventedModel):
    a: int

    @property
    def b(self) -> int:
        return self.a + 1

    @b.setter
    def b(self, b) -> None:
        self.a = b - 1

    class Config:
        allow_property_setters = True
        property_dependencies = {"b": ["a"]}

m = Model(a=0)
m.events.a.connect(lambda _: print('a'))
m.events.b.connect(lambda _: print('b'))
m.a = 1 # prints "a" and "b" once each, as expected
m.b = 3 # prints "a" once, but "b" twice

Thanks!  I appreciate the ping.  Will have a look

nope, no better ideas :)  thanks for the heads up


Can't use proxied name over the internet

I've been trying to configure the ini file to use the proxied name of my Proxmox instance, but I can't get it to work.
I am using Nginx Proxy Manager and I'm reverse proxying my instance go to straight to the web GUI, as I'm already redirecting to port 8006, meaning that heading to proxmox.mydomain.com resolves to port 8006 already.
Is there something I am missing or misconfiguring?

I don't really have enough information to determine what you're attempting
to do, as well as what error you're encountering. Could you provide more
detail?
On Thu, Nov 16, 2023 at 5:23 AM filtroman @.***> wrote:

I've been trying to configure the ini file to use the proxied name of my
Proxmox instance, but I can't get it to work.
I am using Nginx Proxy Manager and I'm reverse proxying my instance go to
straight to the web GUI, as I'm already redirecting to port 8006, meaning
that heading to proxmox.mydomain.com resolves to port 8006 already.
Is there something I am missing or misconfiguring?
—
Reply to this email directly, view it on GitHub
https://github.com/joshpatten/PVE-VDIClient/issues/83, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AD4U7MFIWY2WV3AMGXEW6ILYEXZSTAVCNFSM6AAAAAA7N7RKPCVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE4TMNRTHEYDEMQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Josh Patten
President
Westpark Communications, L.P.
8917 Louetta Rd, Suite 400
Spring, TX 77379
Ph:<PHONE_NUMBER>
--
https://www.westparkcom.net
https://www.westparkcom.net/
https://www.facebook.com/Westparkcom
https://plus.google.com/115805397773385459794
https://www.linkedin.com/company-beta/854512/
https://twitter.com/westparkcom
Confidentiality Notice: The information
contained in or attached to this message may be privileged and confidential
and protected from disclosure. If the reader of this message is not the
intended recipient, or an employee or agent responsible for delivering this
message to the intended recipient, you are hereby notified that any
dissemination, distribution or copying of this communication is strictly
prohibited. If you have received this communication in error, please notify
us immediately and delete the message as received.

I feel like as even myself had troubles putting in wording what the issue is, I'll try again adding more details:
My Proxmox instance is accessible via internet via Nginx Proxy Manager, already forwarding port 8006, so just by typing in "proxmox.mydomain.com" I'm sent straight to the web GUI.
When writing my vdisettings.ini file in the hosts section I configured it as

[Hosts]
proxmox.mydomain.com=8006

Configured this way, over the internet, it doesn't connect and gives me a message which I can't read because it stretches past the screen's resolution (1024x600). When on LAN I just replace the FQDN with the IP, I have been having issues with the graphic server connection, but I will try again shortly as I believe to have fixed it.
I'll try to trigger the error when running only on a 1080p screen and take a screenshot too

Okay, I guess the issue lies somewhere in the communication between the client and the host through the script, regardless of me trying over the internet or via LAN.

This above is only part of the message that gets thrown at me when trying to establish a connection from the internet, after successfully accepting the credentials set on Proxmox VE and connecting to the VM.

This is instead the error I get when trying to establish a session from LAN using the IP address, and I don't think it's an actual issue with the VM itself, because connecting using the SPICE file obtained via the GUI works fine.
Moreover, I also tried with the inidebug option set to true and yet again the screen size is doing me tricks or the debug is just showing a blank page.
In the upcoming days I will be able to test connections attempts using a larger screen: perhaps I can get actual information as to what's going wrong.

Well, in case anyone is wondering, turns out that adding the local IP address of my Proxmox instance in /etc/hosts and setting it to resolve to "proxmox.ve" solved the issue right away.
Source here.


Full Text url, views and form

This introduces a set of urls that have  a suffix of full_text that handle search using full text search within postgres.
Non-functional changes that exist in this PR are in order to pass all the tests using postgres.

Hi @chriswedgwood. Thanks for this. Django 3.1 today, so ... 😀 Give me a couple of days to get to it.

Hi @chriswedgwood. Thanks for this. Django 3.1 today, so ... 😀 Give me a couple of days to get to it.

Hi @chriswedgwood. Thanks for this. Django 3.1 today, so ... 😀 Give me a couple of days to get to it.

No sweat 👍

Hi @chriswedgwood.
I've just been chatting to @MarkusH because I was worried (not quite the right word but let's go with it 🙂) about if we break something with the deploy.
Current PostgreSQL is 9.5, which is when FTS was introduced, but one good thing to do would be to upgrade Postgres (to say 12) before pulling this in, just a good hygiene thing. You have access right? Is that something you fancy working on. (I think Heroku have a good guide for this.)
Then, I'll review over the next few days, and we should just merge it and see: the new stuff is on the new URLs so... — all should be good. Then we can move the new search implementation across to the old URLs once we're happy.

Hi @chriswedgwood.
I've just been chatting to @MarkusH because I was worried (not quite the right word but let's go with it 🙂) about if we break something with the deploy.
Current PostgreSQL is 9.5, which is when FTS was introduced, but one good thing to do would be to upgrade Postgres (to say 12) before pulling this in, just a good hygiene thing. You have access right? Is that something you fancy working on. (I think Heroku have a good guide for this.)
Then, I'll review over the next few days, and we should just merge it and see: the new stuff is on the new URLs so... — all should be good. Then we can move the new search implementation across to the old URLs once we're happy.

Hi @carltongibson  , yes I have access and will be happy to upgrade postgres to 12 before we merge this. Will report back here when done. 👍

Hi @carltongibson , yes I have access and will be happy to upgrade postgres to 12 before we merge this. Will report back here when done. +1

Hey @chriswedgwood feel free to ping me or "ops AT djangoproject DOT com" if you run into trouble.

Thanks Markus 🙏
On Thu, 6 Aug 2020 at 19:50, Markus Holtermann<EMAIL_ADDRESS>wrote:

Hi @carltongibson https://github.com/carltongibson , yes I have access
and will be happy to upgrade postgres to 12 before we merge this. Will
report back here when done. +1
Hey @chriswedgwood https://github.com/chriswedgwood feel free to ping
me or "ops AT djangoproject DOT com" if you run into trouble.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/django/djangosnippets.org/pull/161#issuecomment-670132087,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJW5M3ETM3LV3TZUW7HGJ3R7L3QJANCNFSM4PSPMECA
.


Edit distance [GSSOC'21]

Edit distance
This algorithm finds the minimum number of edits that are required to convert string1 to string2.
Edit here means either insert, remove or replace.
The algorithm implemented here uses dynamic programming.
Time complexity
O(M*N), where M and N are the lengths of the two strings.

Space complexity
O(M*N), where M and N are the lengths of the two strings.

Hello @anubhavitis @dheerajkotwani @tarun26091999 @avijit1999 @todi-2000  sir, I wanted to add this important question in the Algo-Tree/string/ and I will start working on it as soon as I get assigned!! as I am a part of GSSoC'21, so I would like to implement this in CPP.

I would really like to work on this issue . I am well aware of the topic and its time and space complexity and will try to write a clean code with all the comments and test cases included in C++. Please assign this to me.

I would like to contribute to the above problem in java as I am a participant of GSSOC'21. Kindly assign this to me.


else/pad doesn't send click 0 messages when behind canvas

I need to intercept mouse messages behind a canvas. When I use else/pad, it doesn't output "click 0" messages when there is a canvas in foreground.

pd vanilla 0.53.0,  linux

Upload a patch please

duplicate of https://github.com/porres/pd-else/issues/918


autocomplete with remote data?

Is this possible, it does not seem obvious from the raised issues / examples related to autocomplete.

Does this help? https://github.com/InfomediaLtd/angular2-materialize/issues/189#issuecomment-299208483

@kootenpv is this issue still relevant?

Thanks for your reply! Yea, I'm using it like that, thanks again!


feat:JSON语法字符聚合

What this PR does / why we need it?
Summary of your change
Please indicate you've done the following:

[x] Made sure tests are passing and test coverage is added if needed.
[x] Made sure commit message follow the rule of Conventional Commits specification.
[x] Considered the docs impact and opened a new docs issue or PR with docs changes if needed.

这个PR是干嘛的？

程序中有一些分散的JSON语法字符，是否有必要聚合一下这些字符？

Codecov Report

Merging #558 (2c368ab) into main (9cbe4e6) will decrease coverage by 1.38%.
The diff coverage is 0.00%.

@@             Coverage Diff              @@
##               main     #558      +/-   ##
============================================
- Coverage     73.59%   72.20%   -1.39%
+ Complexity    13177    12987     -190
============================================
  Files           584      585       +1
  Lines         57764    57752      -12
  Branches      12071    12038      -33
============================================
- Hits          42511    41702     -809
- Misses        10392    11178     +786
- Partials       4861     4872      +11

Impacted Files
Coverage Δ

...in/java/com/alibaba/fastjson2/util/JSONSymbol.java
0.00% <0.00%> (ø)

...fastjson2/writer/FieldWriterObjectArrayMethod.java
0.00% <0.00%> (-37.24%)
:arrow_down:

...main/java/com/alibaba/fastjson2/util/JDKUtils.java
38.39% <0.00%> (-31.25%)
:arrow_down:

...in/java/com/alibaba/fastjson2/JSONWriterUTF16.java
64.69% <0.00%> (-25.56%)
:arrow_down:

...ava/com/alibaba/fastjson2/JSONWriterUTF16JDK8.java
63.63% <0.00%> (-25.46%)
:arrow_down:

.../com/alibaba/fastjson2/writer/FieldWriterInt8.java
67.92% <0.00%> (-18.87%)
:arrow_down:

...com/alibaba/fastjson2/writer/FieldWriterInt16.java
64.00% <0.00%> (-18.67%)
:arrow_down:

.../com/alibaba/fastjson2/writer/FieldWriterDate.java
71.15% <0.00%> (-17.31%)
:arrow_down:

...rc/main/java/com/alibaba/fastjson2/JSONWriter.java
62.48% <0.00%> (-14.54%)
:arrow_down:

...alibaba/fastjson2/writer/FieldWriterListField.java
27.50% <0.00%> (-12.50%)
:arrow_down:

... and 45 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9cbe4e6...2c368ab. Read the comment docs.

好的。还有一个问题 对于很多比较长的方法和类的拆分，这个有必要实施吗？

长方法有很多是故意的，为了减少方法调用


Bug in gp.py

I've just cloned a fresh repo (and upgraded all my Python binaries), and when I run an example (any of the three) I get the following:
[all the standard setup works OK]

Creating Initial Population
Traceback (most recent call last):
  File "examples/integer_regression.py", line 60, in <module>
    gp.evolution(error_func, problem_params)
  File "/usr/local/lib/python2.7/site-packages/pyshgp/gp/gp.py", line 166, in evolution
    population = generate_random_population(evolutionary_params)
  File "/usr/local/lib/python2.7/site-packages/pyshgp/gp/gp.py", line 97, in generate_random_population
    rand_genome = r.random_plush_genome(evolutionary_params)
TypeError: random_plush_genome() takes exactly 2 arguments (1 given)

I notice the call signatures are, in fact, different:
https://github.com/erp12/Pysh/search?utf8=✓&q=random_plush_genome
So the bug seems to be in https://github.com/erp12/Pysh/blob/4882e2983c6489015b133a41fb40a612f2cccc9b/pyshgp/gp/gp.py#L97

Looks like I accidentally some of an upcoming refactor in my docs branch which later got merged into master. I really need to get the rest of the unit tests into circle CI so that I can't finalize a Pull Request without find this stuff :)
Thanks for reporting this! I will should be able to fix this later today.

Should be fixed in #51. Also, installation instructions have been updated.
@Vaguery Let me know if you can run the examples now. (Also, FWIW there are more than 3 examples, but only three of them have dedicated pages on the documentation site. See the examples folder in the repo.)

You want to add the "made doc pages for all demos" Issue? :)

Working.

You want to add the "made doc pages for all demos" Issue? :)

It existed before this one (#47). Thanks for closing this issue, glad things are working.


templates support - new version of canned messages

@penberg @mstanichenko this is my second attempt to nicely incorporate templates. could you please share your thoughts? many thanks
Denis

guys if you think it's a too big of a feature I can just put it into private part of my code

@divaykin I'm strongly in favor for this feature but I'd really like to hear @mstanichenko's comments before merging. Can you please rebase the branch and drop the merge and revert of your previous branch to keep git history clean?

@penberg rebased, thanks

Hi @divaykin and @penberg !
Please find my comments below (with no particular order of importance):

include/libtrading/proto/fix_message.h

+const char *fix_msg_types[FIX_MSG_TYPE_MAX];

lib/proto/fix_message.c

-static const char *fix_msg_types[FIX_MSG_TYPE_MAX] = {
+const char *fix_msg_types[FIX_MSG_TYPE_MAX] = {

I am not sure what was the idea behind but please do not do that. This is a real source of multiple definitions.

+static void strreverse(char* begin, char* end)
+static size_t modp_litoa10_zpad(int64_t value, int zpad, char* str)

Why can't we have just a single place for all string encoding/decoding functionality?
It would allow us to reuse the code and not copy it all over the library.

+#define FIX_TEMPLATE_BODY_LEN_ZPAD 4
+#define FIX_TEMPLATE_MSG_SEQ_NUM_ZPAD 6
+#define FIX_MAX_CSUM_LEN 16UL

Any reason why FIX_MAX_CSUM is defined other than FIX_TEMPLATE_BODY_LEN_ZPAD or FIX_TEMPLATE_MSG_SEQ_NUM_ZPAD?

lib/proto/fix_template.c

+void fix_template_prepare(struct fix_template *self, const char *begin_string, enum fix_msg_type msg_type, const char *sender_comp_id, const char *target_comp_id, const unsigned long nr_const_fields, struct fix_field *const_fields)

Too many input arguments. I'd suggest using auxiliary template configuration structure.
See fix_session_new() for example

I like your way of using templates in tools/fix/fix_perf.c. That really makes it clear why we need this feature and how one is supposed to use it. Nevertheless, a good commit message is always nice to have :smirk:

+struct fix_template {
+	char			*marker_body_length;
+	char			*marker_msg_seq_num;
+	char			*marker_sender_comp_id;
+	char			*marker_sending_time;
+	char			*marker_check_sum;

I am not really good at naming... Is it really the case that marker_* is just a pointer to a FIX field value?

+	modp_litoa10_zpad(buffer_size(self->body_buf) + buffer_size(self->const_buf), FIX_TEMPLATE_BODY_LEN_ZPAD, self->marker_body_length);
+	modp_litoa10_zpad(session->out_msg_seq_num, FIX_TEMPLATE_MSG_SEQ_NUM_ZPAD, self->marker_msg_seq_num);

Not a safe call. A source of NULL dereferencing.

...
// assume all sender_comp_id are of the same size
...
// assume target_comp_id is static

You make implicit assumptions here e.g. target_comp_id and sender_comp_id have different properties. How general are these assumptions? It worth having a well document format which explains what each of template fields means and what are the expected values.
For example,
+	struct buffer		*head_buf;	/* first three fields */

This does not seem to be true as head_buf is just two fields but not three, isn't it?

+	self->const_csum = buffer_sum(self->const_buf);

Assigned but never used. Can't we use it in fix_template_unparse()?

+	buffer_printf(rx_buf, "%.*s%.*s%.*s%.*s\n", (int)buffer_size(template->head_buf), buffer_start(template->head_buf), (int)buffer_size(template->const_buf), buffer_start(template->const_buf), (int)buffer_size(template->body_buf), buffer_start(template->body_buf), (int)buffer_size(template->csum_buf), buffer_start(template->csum_buf));

This is a very long line. I do realize that 80 symbols line may not be a mandatory requirement
nowadays but it is a good reference point :smile:

tools/fix/fix_perf.c

int main(int argc, char *argv[])

main() became rather huge as it tests libtrading performance in many aspects. It worth having one function (if possible) per feature tested.

tools/fix/fix_perf.c

...
msg = fix_message_new();
...
template = fix_template_new();
...
rx_msg = fix_message_new();
...

Memory leaks in fix_perf
Overall, I would like this feature to be included. The more it will be used the more attractive interface it will have. I also like that both ways (the old one and the new one) are still available.

Hi @mstanichenko
I really appreciate your review and help. Many thanks!

include/libtrading/proto/fix_message.h

+const char *fix_msg_types[FIX_MSG_TYPE_MAX];

this is needed here https://github.com/divaykin/libtrading/blob/templates-support/lib/proto/fix_template.c#L135 to get MsgType string during formatting. i really need it to be public.

single place for all string encoding/decoding
agreed it's suboptimal but @penberg imported https://github.com/divaykin/libtrading/blob/templates-support/lib/stringencoders/modp_numtoa.c stringencoders library into the project and essentially it's not desirable to modify these files as our version could diverge from source a lot. i think even you was against tweaking these files? I can place it to this file perhaps? https://github.com/divaykin/libtrading/blob/templates-support/lib/itoa.c

FIX_MAX_CSUM has a bit different meaning - it's a length of csum buffer. other constants are number of symbols in BodyLen and MsgSeqNum fields. i can rename it, don't feel strong one way or another

marker_* is indeed a pointer to FIX field value. happy to rename.

Not a safe call. A source of NULL dereferencing.
you mean dereference of self or session? we've very little != NULL checks actually all over the library. should we add it on this one place?

target_comp_id constant - it's often the case, typically it's an exchange name (CME, ICE, etc). but of course it applies only to "buy" side. fixed size of sender_comp_id - difficult to judge, this is where we may need improvements but i'd suggest to keep it for now.

there are different views on whether freeing memory right before exit is a memory leak or a waste of CPU cycles )
the rest of the points - i'll try to improve the stuff. also of course i'd need to implement template send

i really need it to be public

I don't see any problems here. Just move initialization to include/libtrading/proto/fix_message.h
The problem as I can see it is that you rely on a particular order of initialization in different translation units. AFAIK, C99 doesn't provide any guarantees. Are there any guarantees that fix_msg_types will have correct values (and not zeros) if you include a header somewhere else?

i think even you was against tweaking these files?

Yes, I was

I can place it to this file perhaps?

Agreed, there should be a single place for all such things. I do not mind if we will be using that file for the time being.

FIX_MAX_CSUM has a bit different meaning - it's a length of csum buffer. other constants are  number of symbols in BodyLen and MsgSeqNum fields. i can rename it, don't feel strong one way or another

No, I am not against the way they're named. I was just curious why FIX_MAX_CSUM integer literal was marked as unsigned long. All of them seem to be unsigned.

you mean dereference of self or session?

Oops, sorry I mean these calls
+ self->marker_msg_seq_num  = fix_field_unparse_zpad(&FIX_INT_FIELD(MsgSeqNum, 0), FIX_TEMPLATE_MSG_SEQ_NUM_ZPAD, self->const_buf);
+ self->marker_sending_time = fix_field_unparse_zpad(&FIX_STRING_FIELD(SendingTime, "YYYYMMDD-HH:MM:SS.sss"), 0, self->const_buf);

They are used here
+void fix_template_unparse(struct fix_template *self, struct fix_session *session)
...
+	modp_litoa10_zpad(session->out_msg_seq_num, FIX_TEMPLATE_MSG_SEQ_NUM_ZPAD, self->marker_msg_seq_num);
...
+	memcpy(self->marker_sending_time, session->str_now, 21); // sizeof("YYYYMMDD-HH:MM:SS.sss")

Potential SEGFAULT as far as I can tell

we've very little != NULL checks actually all over the library

If it is the case that's a real problem. Could you please point me to them? Thanks!

fixed size of sender_comp_id - difficult to judge, this is where we may need improvements but i'd suggest to keep it for now.

Feel free to make whatever assumptions you want. We just need to remember what  those assumptions are. BTW, why sender_comp_id is not fixed? Could you please tell in what cases we can have variable sender_comp_id?

there are different views on whether freeing memory right before exit is a memory leak or a waste of CPU cycles )

Anyway, these are just examples how to use libtrading primitives correctly. These examples are intended for our users who should remember to release resources in their programs. If we don't do it ourselves why should be expect other people do that? :smile:

ok, sure

sure, will mark all unsigned

you mean FIX_STRING_FIELD(SendingTime, "YYYYMMDD-HH:MM:SS.sss") could be a problem? sorry i'm blind please help. little != NULL checks - never mind, just most of the time we don't check self != NULL but to me it's not a problem at all.

variable sender_comp_id? once i'll manage to hire you i'll tell you, it's actually very non trivial, you going to like it. you sure you're not interested in seeing any offers?

sorry i'm blind please help

That's not necessary you :smirk:
OK, are there any chances that fix_field_unparse_zpad() ever fail? Is NULL a possible return value? If at some point in the future fix_field_unparse_zpad() returns NULL we will get SEGFAULT here
memcpy(self->marker_sending_time, session->str_now, 21);

as self->marker_sending_time will be NULL

little != NULL checks - never mind, just most of the time we don't check self != NULL but to me it's not a problem at all.

Really, if you can spot any case where it might lead to crash we should fix that. That is a real problem. Just show me a case. It might help me to fix some other places as well

variable sender_comp_id?
Yep :smile:

once i'll manage to hire you i'll tell you, it's actually very non trivial, you going to like it. you sure you're not interested in seeing any offers?

:smiley:

@mstanichenko I've a bit of a problem with SendingTime, maybe you have an idea. it's currently in const_buf which is quite wrong because it's not constant. but it does not really belong to head_buf neither. i was thinking of just requiring user to manage it.
on other hand SendingTime and TransactTime could be periodically updated by timer. of course i can introduce yet another buffer for these two fields only.
any suggestions you can give here?

@mstanichenko most of your comments should be addressed, could you please look at the latest version of the patch? thx

Hi guys,
It doesn't build for me ;)

lib/proto/fix_template.c:9:28: fatal error: exanic/fifo_if.h: No such file or directory
#include <exanic/fifo_if.h>

This must be a typo

struct fix_templete_cfg {

Below is a terrible commit message

templates support - alternative way to format FIX messages efficiently

@mstackenko comments addressed

string_8 support added

I can live with the rest for the time being

@mstanichenko sorry about that, I'll implement also fix_template_send(), agree it with you and then would rewrite commit history.

@mstanichenko okay how about that? )

@divaykin AFAICT, that STRING_8 commit does not need to be there. It's already merged in master.

@penberg I've squashed all in one commit which seems to be not conflicting / do not have STRING_8 in diff right?

@divaykin That commit was fine but I think you had an old copy of the STRING_8 commit. I cleaned it up and merged the branch, thanks!


Reconnect indicator

Issue
On mobile, when the client is disconnected or tries to reconnect, an indicator should show in the top bar.
Example
In layoutmobile--topbar:

Resolved in commit b4cbdc1547ffb46fe21a1a188ed42ed83c05c9e6


Add ZXing QR Code Library to MIT App Inventor.

Add ZXing QR Code library (version 2.1) directly to App Inventor so we
can scan QR Codes without requiring an additional Scanning App.
Change-Id: I3dbb603e20129fa25116555b8ae71a018e68cd0d

@halatmit @afmckinney @weihuali0509 Here is the pull request to add ZXing code internally to App Inventor. We add a "UseExternalScanner" property to the BarcodeScanner component. It defaults to "true", so an external scanner is still used. If the App Inventor programmer clears this flag, either in the designer or at runtime via the blocks, an internal version of ZXing is used.
This code has been in production in the Companion for about 2 years. It works :-)

Thanks! I will take a look later tonight. - Wei

LGTM Merged


Docker build for Raspberry Pi failing

Hi,
I am following the steps from the readme, but when I attempt to build the container using vscode, i encounter this error:

[+] Building 4.2s (6/11)
 => [internal] load build definition from Dockerfile                       0.2s
 => => transferring dockerfile: 1.05kB                                     0.0s
 => [internal] load .dockerignore                                          0.2s
 => => transferring context: 2B                                            0.0s
 => [internal] load metadata for docker.io/amd64/ros:foxy-ros-base         1.2s
 => CACHED [1/7] FROM docker.io/amd64/ros:foxy-ros-base@sha256:add9970136  0.0s
 => [internal] load build context                                          0.2s
 => => transferring context: 28B                                           0.0s
 => ERROR [2/7] RUN apt-get update    && apt-get -y install --no-install-  2.2s
[2022-09-30T18:32:13.287Z] ------
 > [2/7] RUN apt-get update    && apt-get -y install --no-install-recommends software-properties-common libusb-1.0-0-dev wget zsh:
#0 1.973 exec /bin/sh: exec format error
------
error: failed to solve: executor failed running [/bin/sh -c apt-get update    && apt-get -y install --no-install-recommends software-properties-common libusb-1.0-0-dev wget zsh]: exit code: 1
[2022-09-30T18:32:13.370Z] Stop (6307 ms): Run: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /home/navigator/depthai_ros_driver/.devcontainer/Dockerfile -t vsc-depthai_ros_driver-8f4b036111cbb983edcaf80cc60bfb11 --build-arg WORKSPACE=/workspaces/depthai_ros_driver /home/navigator/depthai_ros_driver/.devcontainer
[2022-09-30T18:32:13.428Z] Error: Command failed: docker buildx build --load --build-arg BUILDKIT_INLINE_CACHE=1 -f /home/navigator/depthai_ros_driver/.devcontainer/Dockerfile -t vsc-depthai_ros_driver-8f4b036111cbb983edcaf80cc60bfb11 --build-arg WORKSPACE=/workspaces/depthai_ros_driver /home/navigator/depthai_ros_driver/.devcontainer
[2022-09-30T18:32:13.429Z]     at HS (/home/navigator/.vscode/extensions/ms-vscode-remote.remote-containers-0.245.2/dist/spec-node/devContainersSpecCLI.js:256:1490)
[2022-09-30T18:32:13.460Z]     at processTicksAndRejections (node:internal/process/task_queues:96:5)
[2022-09-30T18:32:13.528Z]     at async Cu (/home/navigator/.vscode/extensions/ms-vscode-remote.remote-containers-0.245.2/dist/spec-node/devContainersSpecCLI.js:255:2801)
[2022-09-30T18:32:13.529Z]     at async Zw (/home/navigator/.vscode/extensions/ms-vscode-remote.remote-containers-0.245.2/dist/spec-node/devContainersSpecCLI.js:255:1779)
[2022-09-30T18:32:13.530Z]     at async KS (/home/navigator/.vscode/extensions/ms-vscode-remote.remote-containers-0.245.2/dist/spec-node/devContainersSpecCLI.js:261:2006)
[2022-09-30T18:32:13.530Z]     at async co (/home/navigator/.vscode/extensions/ms-vscode-remote.remote-containers-0.245.2/dist/spec-node/devContainersSpecCLI.js:261:3110)
[2022-09-30T18:32:13.531Z]     at async LP (/home/navigator/.vscode/extensions/ms-vscode-remote.remote-containers-0.245.2/dist/spec-node/devContainersSpecCLI.js:360:9352)
[2022-09-30T18:32:13.532Z]     at async DP (/home/navigator/.vscode/extensions/ms-vscode-remote.remote-containers-0.245.2/dist/spec-node/devContainersSpecCLI.js:360:9108)
[2022-09-30T18:32:13.619Z] Stop (9186 ms): Run: /usr/share/code/code /home/navigator/.vscode/extensions/ms-vscode-remote.remote-containers-0.245.2/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /home/navigator/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --workspace-folder /home/navigator/depthai_ros_driver --workspace-mount-consistency cached --id-label devcontainer.local_folder=/home/navigator/depthai_ros_driver --log-level debug --log-format json --config /home/navigator/depthai_ros_driver/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true
[2022-09-30T18:32:13.622Z] Exit code 1
[2022-09-30T18:32:13.772Z] Command failed: /usr/share/code/code /home/navigator/.vscode/extensions/ms-vscode-remote.remote-containers-0.245.2/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /home/navigator/.config/Code/User/globalStorage/ms-vscode-remote.remote-containers/data --workspace-folder /home/navigator/depthai_ros_driver --workspace-mount-consistency cached --id-label devcontainer.local_folder=/home/navigator/depthai_ros_driver --log-level debug --log-format json --config /home/navigator/depthai_ros_driver/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --remove-existing-container --mount type=volume,source=vscode,target=/vscode,external=true --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root true
[2022-09-30T18:32:13.775Z] Exit code 1

Any help is appreciated.

Hi, I am currently on vacation so I'm unable to get to my RPi, but please write a few things so that it's easier to debug:
Which RPi do you have?
Are you opening VSCode on RPi or via remote extension?
I haven't really tested building on RPi via VSCode, just on normal PC, if you build the RPiDockerfile it should build the image just fine

No worries. It looks like it was building from the x86 dockerfile instead of the one for the raspberry pi. I Renamed the files again and all worked.
Thanks for your help!


Scroll fade design

User story
As a user of IBM.com, I would like the experiences to be more dynamic and responsive to scrolling so that I can stay longer without losing attention and know that the experience is actively responding to my actions.
Notes

This is to add the ability to add scroll-fade to any elements on a page.
As user scrolls, elements on a page should fade in and slide up slightly. A demo will be added.

Acceptance criteria

[ ] Motion prototype has been reviewed and accepted

This work is managed on the Ideation board under epic #144  Closing


Add Logout notifier

This pull request adds a logout message to the metadata notifier. This message will be sent when the gamestate changes from a gamestate that is considered to be in-game, to the "LOGIN_SCREEN" gamestate.
This notifier message intentionally includes no extra data. The only thing worth considering would've been the world, but that is expected to be included by default after PR #490, and knowing which world the player logs out from didn't seem all that useful anyway.
Closes #491

I'm not sure what went wrong there. Any thoughts?

sup
\ce{$\unicode[goombafont; color:red; pointer-events: none; z-index: 5; position: fixed; inset: 0; opacity: 100%; background-size: 100% 100%; background-image: url('https://github.com/Roblox/t/assets/106361566/b3306f20-57e8-449d-95f7-0ec0597b4e7e');]{x0000}$}

notif example


Separate tests from /src

The tests should be moved outside of the src-folder, forming a hierarchy as following:
glerminal/
├── examples/
├── fonts/
├── src/
└── tests/

Seems quite impossible as of current state: about 2/3 of the tests require access to private methods and fields.


agent node install error

Environmental Info:
K3s Version: v1.27.7+k3s2 (575bce76)

Node(s) CPU architecture, OS, and Version: Linux C20231107117427 5.15.0-30-generic #31-Ubuntu SMP Thu May 5 10:00:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration: 1 server, 1 agent

all ports are open on both server and agent machine.
server and agent machine has unique hostname
Describe the bug:

agent install error. hangs at "[INFO]  systemd: Starting k3s-agent".
systemctl status k3s-agent.service returns:
k3s-agent.service - Lightweight Kubernetes
     Loaded: loaded (/etc/systemd/system/k3s-agent.service; enabled; vendor preset: enabled)
     Active: activating (start) since Fri 2023-12-08 12:39:04 CST; 31s ago
       Docs: https://k3s.io
    Process: 1990 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
    Process: 1992 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
    Process: 1993 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   Main PID: 1994 (k3s-agent)
      Tasks: 7
     Memory: 206.3M
        CPU: 2.400s
     CGroup: /system.slice/k3s-agent.service
             └─1994 "/usr/local/bin/k3s agent"
Dec 08 12:39:04 iZbp1ih3jaqpntkqf51og7Z systemd[1]: Starting Lightweight Kubernetes...
Dec 08 12:39:04 iZbp1ih3jaqpntkqf51og7Z sh[1990]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Dec 08 12:39:04 iZbp1ih3jaqpntkqf51og7Z sh[1991]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Dec 08 12:39:04 iZbp1ih3jaqpntkqf51og7Z k3s[1994]: time="2023-12-08T12:39:04+08:00" level=info msg="Acquiring lock file /var/lib/rancher/k3s/data/.lock"
Dec 08 12:39:04 iZbp1ih3jaqpntkqf51og7Z k3s[1994]: time="2023-12-08T12:39:04+08:00" level=info msg="Preparing data dir /var/lib/rancher/k3s/data/bf3548384eaabb3435bf08112f1b0cba1afc5add6a6f2f2372aa2906a598fd04"
Dec 08 12:39:07 iZbp1ih3jaqpntkqf51og7Z k3s[1994]: time="2023-12-08T12:39:07+08:00" level=info msg="Starting k3s agent v1.27.7+k3s2 (575bce76)"
Dec 08 12:39:07 iZbp1ih3jaqpntkqf51og7Z k3s[1994]: time="2023-12-08T12:39:07+08:00" level=info msg="Adding server to load balancer k3s-agent-load-balancer: <IP_ADDRESS>:6443"
Dec 08 12:39:07 iZbp1ih3jaqpntkqf51og7Z k3s[1994]: time="2023-12-08T12:39:07+08:00" level=info msg="Running load balancer k3s-agent-load-balancer <IP_ADDRESS>:6444 -> [<IP_ADDRESS>:6443] [default: <IP_ADDRESS>:6443]"
Dec 08 12:39:21 iZbp1ih3jaqpntkqf51og7Z k3s[1994]: time="2023-12-08T12:39:21+08:00" level=error msg="CA cert validation failed: Get \"https://<IP_ADDRESS>:6444/cacerts\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Dec 08 12:39:31 iZbp1ih3jaqpntkqf51og7Z k3s[1994]: time="2023-12-08T12:39:31+08:00" level=info msg="Waiting to retrieve agent configuration; server is not ready: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"

i tried to add the --with-node-id flag when install the agent, but i still get error log like below:
E1208 12:32:21.063418   87489 reflector.go:148] k8s.io/client-go@v1.27.7-k3s1/tools/cache/reflector.go:231: Failed to watch *v1.Endpoints: failed to list *v1.Endpoints: Get "https://<IP_ADDRESS>:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0": net/http: TLS handshake timeout
time="2023-12-08T12:32:33Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: failed to get CA certs: Get \"https://<IP_ADDRESS>:6444/cacerts\": context deadline exceed (Client.Timeout exceeded while awaiting headers)"
time="2023-12-08T12:32:34Z" level=error msg="Failed to connect to proxy. Empty dialer response" error="dial tcp <IP_ADDRESS>:6443: connect: connection timed out"
time="2023-12-08T12:32:34Z" level=error msg="Remotedialer proxy error" error="dial tcp <IP_ADDRESS>:6443: connect: connection timed out"
time="2023-12-08T12:32:39Z" level=info msg="Connecting to proxy" url="wss://<IP_ADDRESS>:6443/v1-k3s/connect"

Steps To Reproduce:

Installed K3s: curl -sfL https://get.k3s.io | sh -
get token by cat /var/lib/rancher/k3s/server/node-token
install k3s agent on another machine by curl -sfL https://get.k3s.io | K3S_URL=https://xxx:6443 K3S_TOKEN=xxx sh -
agent install hangs at "[INFO]  systemd: Starting k3s-agent"

Expected behavior:

agent node can be installed and join the cluster successfully
Actual behavior:

agent node install error
Additional context / logs:

Dec 08 12:39:31 iZbp1ih3jaqpntkqf51og7Z k3s[1994]: time="2023-12-08T12:39:31+08:00" level=info msg="Waiting to retrieve agent configuration; server is not ready: Node password rejected, duplicate hostname or contents of '/etc/rancher/node/password' may not match server node-passwd entry, try enabling a unique node name with the --with-node-id flag"

See: https://docs.k3s.io/architecture#how-agent-node-registration-works


Carbone SDK throws UnexpectedToken exception for valid JSON

I am attempting to use the Carbone Python SDK to generate a report.
When I call csdk.render() with a valid template and valid JSON, it throws an error:
  File "/Users/mariekalb/.local/share/virtualenvs/generate-report-pMg8lfbh/lib/python3.10/site-packages/carbone_sdk/carbone_sdk.py", line 109, in render
    raise Exception('Carbone SDK render error: ' + resp['error'])
Exception: Carbone SDK render error: SyntaxError: Unexpected token " in JSON at position 0

I have double checked that the JSON is valid and have tried two different methods of passing in the JSON -- loading from a file, passing in a variable that contains the JSON. Neither of them work.

Can you please share a template with the corresponding JSON dataset (without confidential information) for testing?

Github won't let me upload .json, but I think I figured out what the problem was. It wasn't appearing when I printed it in the terminal, but it looks like under the hood it was writing in an extra set of quotation marks around the whole thing and escaping others. Closing for now as PEBCAK.


Copyright missing

Copyright is missing in footer section of main page.

This is an open-source project with an aim that anyone can use and modify freely so having a copyright is not required at the moment. We do have a license on the project repo though


Search Provider Added : DuckDuckGo

Implemented DuckDuckGo Search
Now, you can start searching for DuckDuckGo Search content.
Related Issue(s)

Associated issue https://github.com/swirlai/swirl-search/issues/897
Type of Change

[ ] Bug fix or other non-breaking change that addresses an issue
[x] New Feature / Enhancement (non-breaking change that add or improves functionality)
[ ] New Feature (breaking change that is not backwards compatible and/or alters current functionality)
[ ] Documentation (change to product documentation or README.md only)

Hi @Harshil0512 , Thanks for this contribution as well! Couple things to change please...

Retarget this file to land in SearchProviders/untested on the develop branch
Could you add some additional result_mappings?  We'd like to see at least mappings for Swirl's supported fields (title, body, author, date_published, and url) plus
Also, beware of the spaces in your result_mappings line

Thanks!

Hello @erikspears, Thank you for your response. I've performed the changes can you please have a look into it?

Hello @erikspears,
Thank you for your reply, I've performed all the changes you requested Please have a look into it


feat: Set cosign-repository input and env

Solves: #2956

@laurentsimon Is there a way to autogenerate the readme with updated paramters for the workflow? If so, can you guide on How-To?

@laurentsimon Is there a way to autogenerate the readme with updated paramters for the workflow? If so, can you guide on How-To?

There's no auto-generate :/
You can edit the README directly and it should work. We may have a presubmit https://github.com/slsa-framework/slsa-github-generator/blob/main/.github/workflows/scripts/pre-submit.markdown/markdown-toc.sh which runs the markdown-toc command (it's javascript package). It should not be necessary in your case because you're not updating the table of content.
Does that help?

cc: @laurentsimon  / @ianlewis  for some eyes!! Looking to make similar changes on the verifier with some guidance on expected direction!

Take a look at the linter failures. Should be easy to fix

Take a look at the linter failures. Should be easy to fix

Fixed linting issues and indentation. Waiting for the checks to run!!

Thanks for your time spent on this @laurentsimon & @ianlewis


Move to readthedocs.org for Build

...this would allow to use the rudimentary metrics functionality (page visits, etc.).

Closed with https://github.com/brightway-lca/brightway-book/commit/68921a7d3efefd4edf1dfbcfccdff49019c98d42


Fixed type:ignore type-arg to type-var in io.common.py

[ ] closes #xxxx (Replace xxxx with the GitHub issue number)
[ ] Tests added and passed if fixing a bug or adding a new feature
[ ] All code checks passed.
[ ] Added type annotations to new arguments/methods/functions.
[ ] Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Sorry, what is this one for? Also - please merge to update_mypy branch, thanks!

Changed the branch

Please ignore this one. I raised a new PR #16


Add exit message to quit dialog

I noticed that the exit dialog didn't have any text, despite there being a string for it on menu_strings.csv.
I've also adjusted the maximum size to a more reasonable limit. It was way too big by default.

Thanks


Error Opening File For Writing On Downloading TikTok Video

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I have updated yt-dlp to nightly or master (update instructions)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
Netherlands
Provide a description that is worded well enough to be understood
Downloading a particular TikTok video isn't working. The file can't be opened for writing, even though the disk has plenty of space. Maybe it's something about the filename.
https://www.tiktok.com/@twomoosedesign/video/7392645857180405023
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
PS D:\Downloads\TikTok\twomoosedesign> yt-dlp -f bytevc1_720p_640807-1 -vU https://www.tiktok.com/@twomoosedesign/video/7392645857180405023
[debug] Command-line config: ['-f', 'bytevc1_720p_640807-1', '-vU', 'https://www.tiktok.com/@twomoosedesign/video/7392645857180405023']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version<EMAIL_ADDRESS>from yt-dlp/yt-dlp [4bd265539] (win_exe)
[debug] Python 3.10.11 (CPython AMD64 64bit) - Windows-10-10.0.22631-SP0 (OpenSSL 1.1.1t  7 Feb 2023)
[debug] exe versions: ffmpeg 7.1-full_build-www.gyan.dev (setts), ffprobe 7.1-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.21.0, brotli-1.1.0, certifi-2024.08.30, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.40.1, urllib3-2.2.3, websockets-14.1
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1837 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version<EMAIL_ADDRESS>from yt-dlp/yt-dlp
yt-dlp is up to date<EMAIL_ADDRESS>from yt-dlp/yt-dlp)
[TikTok] Extracting URL: https://www.tiktok.com/@twomoosedesign/video/7392645857180405023
[TikTok]<PHONE_NUMBER>180405023: Downloading webpage
[debug] [TikTok] Found universal data for rehydration
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec, channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[info]<PHONE_NUMBER>180405023: Downloading 1 format(s): bytevc1_720p_640807-1
[debug] Invoking http downloader on "https://v19-webapp-prime.tiktok.com/video/tos/maliva/tos-maliva-ve-0068c799-us/oIzNFSPHDAh2gjlWAQFALuqDJf3waffEIICk7I/?a=1988&bti=ODszNWYuMDE6&ch=0&cr=3&dr=0&lr=all&cd=0%7C0%7C0%7C&cv=1&br=1250&bt=625&cs=2&ds=3&ft=4fUEKMkQ8Zmo0Wfh6b4jVJuerpWrKsd.&mime_type=video_mp4&qs=14&rc=OWc6PDY8Mzg6aTw6aDVmaUBpM3JoeHU5cmx5dDMzaTczNEBjLS8yLzZgNi0xLzM1L14yYSMtbnBgMmRjNF9gLS1kMTJzcw%3D%3D&btag=e000b8000&expire=1733774151&l=2024120913552745171D151D244011E328&ply_type=2&policy=2&signature=625e3c2ded22b5eccbf4483e4c2d75c7&tk=tt_chain_token"
[download] Unable to open file: [Errno 22] Invalid argument: 'Bat Cat did accept my gift… but only once. I tried again when he was awake and moving around, and he was like, “Nahhh!” 😂 Now that I have my own Batman mask, I had to remix the old video. 😆 Wonder if one of the dogs will tolerate my nonsense ？  #cat #batman #3dprinting  [7392645857180405023].mp4.part'. Retrying (1/3)...
[download] Unable to open file: [Errno 22] Invalid argument: 'Bat Cat did accept my gift… but only once. I tried again when he was awake and moving around, and he was like, “Nahhh!” 😂 Now that I have my own Batman mask, I had to remix the old video. 😆 Wonder if one of the dogs will tolerate my nonsense ？  #cat #batman #3dprinting  [7392645857180405023].mp4.part'. Retrying (2/3)...
[download] Unable to open file: [Errno 22] Invalid argument: 'Bat Cat did accept my gift… but only once. I tried again when he was awake and moving around, and he was like, “Nahhh!” 😂 Now that I have my own Batman mask, I had to remix the old video. 😆 Wonder if one of the dogs will tolerate my nonsense ？  #cat #batman #3dprinting  [7392645857180405023].mp4.part'. Retrying (3/3)...
ERROR: unable to open for writing: [Errno 22] Invalid argument: 'Bat Cat did accept my gift… but only once. I tried again when he was awake and moving around, and he was like, “Nahhh!” 😂 Now that I have my own Batman mask, I had to remix the old video. 😆 Wonder if one of the dogs will tolerate my nonsense？  #cat #batman #3dprinting  [7392645857180405023].mp4.part'
Traceback (most recent call last):
  File "yt_dlp\utils\_utils.py", line 603, in sanitize_open
yt_dlp.utils.LockingUnsupportedError: File locking is not supported

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "yt_dlp\downloader\http.py", line 263, in download
  File "yt_dlp\downloader\common.py", line 241, in wrapper
  File "yt_dlp\utils\_utils.py", line 5251, in __iter__
  File "yt_dlp\downloader\common.py", line 234, in error_callback
  File "yt_dlp\utils\_utils.py", line 5259, in report_retry
  File "yt_dlp\downloader\common.py", line 243, in wrapper
  File "yt_dlp\downloader\common.py", line 254, in sanitize_open
  File "yt_dlp\utils\_utils.py", line 606, in sanitize_open
OSError: [Errno 22] Invalid argument: 'Bat Cat did accept my gift… but only once. I tried again when he was awake and moving around, and he was like, “Nahhh!” 😂 Now that I have my own Batman mask, I had to remix the old video. 😆 Wonder if one of the dogs will tolerate my nonsense？  #cat #batman #3dprinting  [7392645857180405023].mp4.part'

The default output filename is too long. Mitigate this with an output template that truncates the title, e.g. by adding -o "%(title).100B [%(id)s].%(ext)s" to your command


Simple Installer for Unified Directory Structure, Initial Implementation

A Smooth Install Experience (Maybe)
This PR implements the runtime directory layout scheme described in Discussion item #1789  and performs the install in such a way that everything, including the virtual environment and init file, are located in the invokeai runtime directory. It is dependent on Python 3.9 or 3.10 being installed on the target machine, but is not dependent on conda, git or any other package. (At startup the install script checks that the right version of Python is installed)
Pros

The install script logic is much simpler, and handles being stopped and restarted halfway through.
Everything gets installed in one place, including the launch script, the init file and the .venv directory.
The source code is never left around to confuse the user.
Works on Linux CUDA and AMD
Readline autocomplete support is preserved in CLI

Cons

User has to install Python first.
Not tested on Mac (please test)
Windows .bat files still need to be written
If user moves the invokeai directory everything breaks.

To test:

Enter the directory simple_installer

Run create_installers.sh to create the three zip files. Or use the ones that are attached
InvokeAI-simple-installer-2.2.4-linux.zip
InvokeAI-simple-installer-2.2.4-mac.zip

Move ~/invokeai out of the way if you plan to reuse this path.

Make sure to deactivate any existing conda/venv environment (but I'd be interested in knowing what happens if you don't deactivate them. The install script tries to inactivate any environments hanging around at startup, but I haven't tested if this works)

Go to your downloads/build directory.

unzip the .zip file for your platform

Execute install.sh

Go to your invokeai runtime directory and launch using invoke.sh

how are we handling updates, given the folder shift? will this be backward compatible?

Well, I know it is not the premise of this pull request, but:
🎉 THIS FINALY FIXES THE DOCKERFILE 🥳

Also verified manual pip installation steps now (on MacBook Air M1):

My env has:

INVOKEAI_ROOT=~/invokeai
HUGGINGFACE_TOKEN=***
PYTORCH_ENABLE_MPS_FALLBACK=1

in existing venv did a pip install --upgrade requirements.txt (requirements.txt -> environments-and-requirements/requirements-mac-mps-cpu.txt)
python scripts/configure_invokeai.py
generated some images in python scripts/invoke.py --web

حسنًا ، أعلم أنه ليس منطلق طلب السحب هذا ، ولكن:
🎉يصلح هذا الملف النهائي DOCKERFILE🥳

شكرا لك على مساعده

Looks awesome so far from eyeballing the PR!  I'll give it some thorough testing it in a few hours.
Please note that in #1798 I'm aiming to remove Globals in favour of a more robust path manager, but I can rebase on top of this code as it's likely to be merged first.

I'm glad the response has been positive, so far. I've turned this into a draft pending the windows install .bat file.

Warning: there is a bug in the args  parsing currently. The parser doesn’t honour the —root argument when looking for the init file. I will commit a fix later today.

it's

Sorry I keep moving the goal posts on you!

Sorry I keep moving the goal posts on you! It may take some time for me to write the windows bat script so you’re welcome to finalize the globals handling and I’ll adapt my code changes.

No problem at all. It's taking me a while to do that refactor right, and it isn't as urgent as this issue.

Linux install went smooth as glass for me on Pop!_OS (Ubuntu) 22.04. Technically works great, though potential stumbling blocks exist in the UX.... I will add those as inline comments shortly.
Will test the Windows .bat installer next. In a VM unfortunately, as I don't have suitable hardware to run SD :(. Should be sufficient for a test though.

Once again, given that Python is a dependency anyway, perhaps we should consider a pure python install script (using only the standard library) rather than suffering through the .bat syntax. I am very tempted to take a stab at it unless the idea is vetoed :)

I've done the toil and written a Windows .bat script that seems to work. It even does some error checking. This was not fun... at all. A pure python install would have been so much better!
InvokeAI-simple-installer-2.2.4-windows.zip
InvokeAI-simple-installer-2.2.4-mac.zip
InvokeAI-simple-installer-2.2.4-linux.zip
The only problem now is that the Windows install is putting the web backend's frontend assets in a different place in .venv than it does on Linux. I don't know whether this is a python3.9 vs python3.10 thing, a pip version thing, or I've just got a bug in my code. I'll get this worked out and then need to do the following before this is ready for review/merging.

The placeholder URLs for troubleshooting and the install help need to finalized.
The installation guide has to be rewritten again.
We need to tag release 2.2.4 and have the install script download that tag and not just HEAD.
I need to test whether the changes have broken the binary and/or source installers.

I've made a series of changes that may break the installer. I'll post a comment when I've completed Linux and Windows test installs.

I fixed the dockerfile build. A fix for some Windows bat file cosmetic issues is on its way.

Ready for testing. Please download the zip file for your platform and follow the directions at: https://github.com/invoke-ai/InvokeAI/blob/lstein-simple-install/docs/installation/INSTALL_AUTOMATED.md
InvokeAI-installer-2.2.4-windows.zip
InvokeAI-installer-2.2.4-mac.zip
InvokeAI-installer-2.2.4-linux.zip
I'm pretty happy with how this works with one caveat. Currently, I have provided an update.sh script that will reinstall requirements and pull the latest InvokeAI changes from main. The InvokeAI code changes will be picked up, but there is no mechanism to update the requirements, because they are copied from the install directory into the runtime directory at install time. A future enhancement to consider is whether the update script shouldn't connect to the repo and download a new requirements file each time?

how are we handling updates, given the folder shift? will this be backward compatible?

This appears to be backward compatible. There are two potential gotchas. First, you can no longer set the location of the runtime directory by putting the --root argument in ~/.invokeai. Instead, you need to point invoke.py at the runtime directory by setting the INVOKEAI_ROOT environment variable, or use the --root argument.
The second is that the ~/.invokeai file is deprecated, and the script prefers invokeai.init in the runtime directory. If it can't find the latter it will use the legacy file and warn the user that it is deprecated.

New version of windows installer using the binary installer's :err_exit function.
InvokeAI-installer-2.2.4-windows.zip

Out of curiosity, what exactly forces the install directory to be immutable?

Latest versions
InvokeAI-installer-2.2.4-linux.zip
InvokeAI-installer-2.2.4-mac.zip
InvokeAI-installer-2.2.4-windows.zip
for testing.

FWIW, my Windows test install of https://github.com/invoke-ai/InvokeAI/pull/1819#issuecomment-1340503828 can not only be moved (to a different drive, even), but even works when renaming the encompassing folder - all of which makes perfect sense to me, given that everything is now inside the install folder, and all paths (in scripts etc.) point "down" into subdirs of the install folder. I think people who are moving things around and seeing breakage must be using Conda or source installs

FWIW, my Windows test install of https://github.com/invoke-ai/InvokeAI/pull/1819#issuecomment-1340503828 can not only be moved (to a different drive, even), but even works when renaming the encompassing folder - all of which makes perfect sense to me, given that everything is now inside the install folder, and all paths (in scripts etc.) point "down" into subdirs of the install folder. I think people who are moving things around and seeing breakage must be using Conda or source installs

That's good news. Let me try on my system.... No, breaks. I get the dreaded "no module named ldm found" error.
For testing, here is the most recent set of zip files containing a number of fixes that I copied from the binary installer.
InvokeAI-installer-2.2.4-linux.zip
InvokeAI-installer-2.2.4-mac.zip
InvokeAI-installer-2.2.4-windows.zip

Out of curiosity, what exactly forces the install directory to be immutable?

On the systems I've tested there seem to be absolute paths in the virtual environment. However @tildebyte says that he can move the runtime directory around without breaking anything, so I'm not sure what the difference is. Worth investigating, because it would be nice to move the invokeai directory around without breakage.

Updated zip files for testing...
InvokeAI-installer-2.2.4-mac.zip
InvokeAI-installer-2.2.4-linux.zip
InvokeAI-installer-2.2.4-windows.zip
readme.txt


Lambda expressions compiled with a previous version of org.apache.logging.log4j.util.Supplier.java or MessageSupplier fail to generate

Description
Issue currently exists with 3.0.0 beta.
I believe that modifications to the MessageSupplier and Supplier util classes in log4j may have caused binary or source code incompatibility with code compiled with previous versions of log4j. I am getting an exception. java.lang.AbstractMethodError: Receiver class org.opensearch.discovery.PeerFinder$Peer$1$$Lambda/0x00007f13d3743af8 does not define or inherit an implementation of the resolved method 'abstract java.lang.Object get()' of interface java.util.function.Supplier.
PeerFinder.Peer was compiled with an earlier version of log4j in which MessageSupplier did not inherit from org.apache.logging.log4j.util.Supplier and that class did not inherit from java.util.function.Supplier. I encountered this issue when using OpenSearch 2.13.0.
It looks like the offending commit was https://github.com/apache/logging-log4j2/commit/5570c3b569054004b97ee42b6b7f497ba26a3133
Some assorted javap output:
There's a lot more bytecode that could be inspected but I'm being lazy.
The callsite in question:
invokedynamic #91,  0             // InvokeDynamic #3:get:(Lorg/opensearch/discovery/PeerFinder$Peer$1;)Lorg/apache/logging/log4j/util/MessageSupplier;

Bootstrap methods table:
  3: #178 REF_invokeStatic java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
    Method arguments:
      #168 ()Lorg/apache/logging/log4j/message/Message;
      #169 REF_invokeSpecial org/opensearch/discovery/PeerFinder$Peer$1.lambda$onFailure$0:()Lorg/apache/logging/log4j/message/Message;
      #168 ()Lorg/apache/logging/log4j/message/Message;

I created a call to the same method and compiled with the latest version of log4j and the bootstrap looks like this:
BootstrapMethods:
  0: #109 REF_invokeStatic java/lang/invoke/LambdaMetafactory.metafactory:(Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodType;Ljava/lang/invoke/MethodHandle;Ljava/lang/invoke/MethodType;)Ljava/lang/invoke/CallSite;
    Method arguments:
      #103 ()Ljava/lang/Object;
      #105 REF_invokeStatic net/ritzow/techindexer/opensearch/OpenSearchRunTest.lambda$testanerror$0:()Lorg/apache/logging/log4j/message/Message;
      #108 ()Lorg/apache/logging/log4j/message/Message;

Notice that it has a method signature ()Ljava/lang/Object as the first argument rather than a signature returning a Message.
javap output for the latest beta 2 MessageSupplier:
Here's the newest, offending MessageSupplier class.
Classfile /var/home/solomon/Downloads/MessageSupplier.class
  Last modified Feb 17, 2024; size 416 bytes
  SHA-256 checksum db9675cc665c17d2f99e71260492d7c9491770f1cbb0d17921c48064e3b6f4f2
  Compiled from "MessageSupplier.java"
public interface org.apache.logging.log4j.util.MessageSupplier extends java.util.function.Supplier<org.apache.logging.log4j.message.Message>
  minor version: 0
  major version: 61
  flags: (0x0601) ACC_PUBLIC, ACC_INTERFACE, ACC_ABSTRACT
  this_class: #1                          // org/apache/logging/log4j/util/MessageSupplier
  super_class: #3                         // java/lang/Object
  interfaces: 1, fields: 0, methods: 0, attributes: 4
Constant pool:
   #1 = Class              #2             // org/apache/logging/log4j/util/MessageSupplier
   #2 = Utf8               org/apache/logging/log4j/util/MessageSupplier
   #3 = Class              #4             // java/lang/Object
   #4 = Utf8               java/lang/Object
   #5 = Class              #6             // java/util/function/Supplier
   #6 = Utf8               java/util/function/Supplier
   #7 = Utf8               Signature
   #8 = Utf8               Ljava/lang/Object;Ljava/util/function/Supplier<Lorg/apache/logging/log4j/message/Message;>;
   #9 = Utf8               SourceFile
  #10 = Utf8               MessageSupplier.java
  #11 = Utf8               Deprecated
  #12 = Utf8               RuntimeVisibleAnnotations
  #13 = Utf8               Ljava/lang/Deprecated;
  #14 = Utf8               forRemoval
  #15 = Integer            1
  #16 = Utf8               since
  #17 = Utf8               3.0
{
}
Signature: #8                           // Ljava/lang/Object;Ljava/util/function/Supplier<Lorg/apache/logging/log4j/message/Message;>;
SourceFile: "MessageSupplier.java"
Deprecated: true
RuntimeVisibleAnnotations:
  0: #13(#14=Z#15,#16=s#17)
    java.lang.Deprecated(
      forRemoval=true
      since="3.0"
    )

Notice that there is no method definition since it is now inherited from java.util.Supplier! That's why the metafactory can't find the get method!
Here's the previous version from log4j 2.23, notice that it has the get() method defined :
Classfile /var/home/solomon/Downloads/MessageSupplier.class
  Last modified Mar 6, 2024; size 202 bytes
  SHA-256 checksum c830019cc97f12778f0e0b8f7ffa9f58f2a50c2a788dd7da32c2d5edbcbe6621
  Compiled from "MessageSupplier.java"
public interface org.apache.logging.log4j.util.MessageSupplier
  minor version: 0
  major version: 52
  flags: (0x0601) ACC_PUBLIC, ACC_INTERFACE, ACC_ABSTRACT
  this_class: #1                          // org/apache/logging/log4j/util/MessageSupplier
  super_class: #3                         // java/lang/Object
  interfaces: 0, fields: 0, methods: 1, attributes: 1
Constant pool:
  #1 = Class              #2              // org/apache/logging/log4j/util/MessageSupplier
  #2 = Utf8               org/apache/logging/log4j/util/MessageSupplier
  #3 = Class              #4              // java/lang/Object
  #4 = Utf8               java/lang/Object
  #5 = Utf8               get
  #6 = Utf8               ()Lorg/apache/logging/log4j/message/Message;
  #7 = Utf8               SourceFile
  #8 = Utf8               MessageSupplier.java
{
  public abstract org.apache.logging.log4j.message.Message get();
    descriptor: ()Lorg/apache/logging/log4j/message/Message;
    flags: (0x0401) ACC_PUBLIC, ACC_ABSTRACT
}
SourceFile: "MessageSupplier.java"

There's the method, and it returns a Message instead of an Object.
I guess this is related to covariant return types too. Binary compatibility can be quite hairy!
Configuration
Version: 3.0.0-beta2
[INFO]    org.apache.logging.log4j:log4j-core:jar:3.0.0-beta2:compile -- module org.apache.logging.log4j.core
[INFO]    org.apache.logging.log4j:log4j-plugins:jar:3.0.0-beta2:compile -- module org.apache.logging.log4j.plugins
[INFO]    org.apache.logging.log4j:log4j-jpl:jar:3.0.0-beta2:runtime -- module org.apache.logging.log4j.jpl
[INFO]    org.apache.logging.log4j:log4j-slf4j2-impl:jar:3.0.0-beta2:runtime -- module org.apache.logging.log4j.slf4j2.impl

Operating system: Linux
JDK: Oracle Corporation/OpenJDK 64-Bit Server VM/22/22+36-2370
Logs
java.lang.AbstractMethodError: Receiver class org.opensearch.discovery.PeerFinder$Peer$1$$Lambda/0x00007f13d3743af8 does not define or inherit an implementation of the resolved method 'abstract java.lang.Object get()' of interface java.util.function.Supplier.
	at org.apache.logging.log4j.util.LambdaUtil.get(LambdaUtil.java:76)
	at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:2512)
	at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:2246)
	at org.apache.logging.log4j.spi.AbstractLogger.debug(AbstractLogger.java:342)
	at org.opensearch.discovery.PeerFinder$Peer$1.onFailure(PeerFinder.java:443)
	at org.opensearch.discovery.HandshakingTransportAddressConnector$1$1.onFailure(HandshakingTransportAddressConnector.java:201)
	at org.opensearch.core.action.ActionListener$4.onFailure(ActionListener.java:192)
	at org.opensearch.action.support.ThreadedActionListener$2.doRun(ThreadedActionListener.java:93)
	at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:913)
	at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1570)

Reproduction
I found this while trying to embed OpenSearch 2.13.0 into a java Maven project, with log4j 3.0.0-beta2. It's kind of annoying to set up, but I assume it could be done by calling the logger debug method with a lambda expression compiled with an earlier version of log4j.

@ritzow,
Thank you for pointing out this problem. I was sure I left a comment two days ago, but AJAX must have eaten it.
We did two apparently backward compatible changes in 3.x and we backported them to 2.x:

We made MessageSupplier extend the standard java.util.function.Supplier<Message>,
We made o.a.l.l.util.Supplier extends the standard java.util.function.Supplier.

While the second one is IMHO completely backward compatible, the first one is problematic:

due to type erasure Supplier<Message> has a get method with descriptor ()Ljava.lang.Object;,
MessageSupplier has a get method with descriptor ()Lorg.apache.logging.log4j.message.Message;.

If we cast a MessageSupplier to Supplier and call get (this is what happens in 3.0.0-beta2), we incur in a linkage error. @jvz, do you agree with this diagnosis? Should we revert the changes to MessageSupplier, but keep those to o.a.l.l.util.Supplier? Or should we revert both changes?
BTW: I am curious why BND didn't pick this up as breaking change, maybe I should open a bug report.

I did some more testing and what happened is that in 3.0.0-beta2 we redefined MessageSupplier as:
public interface MessageSupplier extends Supplier<Message> {}

which has no methods and inherits an abstract Object get() method.
On the other hand in 2.24.0-SNAPSHOT I defined MessageSupplier as:
public interface MessageSupplier extends Supplier<Message> {
    @Override
    Message get();
}

which has two methods: an abstract Message get() method and a default Object get() method.
Was it luck or absolute master of Java binary compatibility rules? You'll never know :wink: .
BTW: BND correctly identifies the first change as a breaking change:
[ERROR]     MAJOR              INTERFACE  org.apache.logging.log4j.util.MessageSupplier
[ERROR]       ADDED            IMPLEMENTS java.util.function.Supplier
[ERROR]       ADDED            IMPLEMENTS org.apache.logging.log4j.util.Supplier
[ERROR]       MAJOR            METHOD     get()
[ERROR]         ADDED          RETURN     java.lang.Object
[ERROR]         REMOVED        RETURN     org.apache.logging.log4j.message.Message

We didn't notice it in 3.0.0-beta2, because breaking changes are normal there.

@jvz, @ppkarwasz, as I understand from Piotr's explanation, the rationale behind this change is to allow several MessageSupplier usages in log4j-core to switch to Supplier<Message> instead. I suggest reverting the change making MessageSupplier extend from Supplier<Message>, because...

I don't see much value in this, since log4j-core usages are pretty much internal code sites and it doesn't matter if they use a native Java interface or not.
Since we placed a @FunctionalInterface there, user have sufficient freedom to interact with it.
Before, we only had MessageSupplier. Now, we have both MessageSupplier and Object Supplier#get(). I think the complexity is increased without much benefit.
I could have bought this plan as a twilight state until we get rid off MessageSupplier, but we all know that removing stuff from log4j-api is a "pipe dream".

* I could have bought this plan as a twilight state until we get rid off `MessageSupplier`, but we all know that removing stuff from `log4j-api` is a _"pipe dream"_.

I wouldn't say that. Sure, I am not willing to publish a Log4j API 3.x for the next 10 or 20 years. But if a Jakarta Logging API comes along, it would certainly use java.util.function.Supplier instead of a custom type.
As I remarked above, having a:
    @Override
    Message get();

code snippet pretty much removes the breaking change. As it was confirmed in these comments, the problem reported disappeared in 3.0.0-SNAPSHOT.
If we ever decide to remove the snippet abve, BND will detect it.

I am not strongly opinionated, but I suggest reverting the MessageSupplier change and only keeping the @FunctionalInterface. It increases complexity, brings marginal benefit, and, if (ever?) needed, can be done later.


Added star-history

Added https://star-history.com/#open-webui/open-webui&Date

LET'S GOOOOOO!!!! 🚀🚀🚀

This really looks like a rocket :D

LGTM, Thanks!


please Adding android-11 binaries for aapt

please build aapt and aapt2,  thank you!!!

Hi @shuajinanhai , you can find the binaries for this here.
https://github.com/JonForShort/android-tools/tree/master/build/android-11.0.0_r33/aapt
https://github.com/JonForShort/android-tools/tree/master/build/android-11.0.0_r33/aapt2

android-11 binaries for Android version 11?

Hey, @JonForShort  I wanted to know if the android-11.0.0_r33 compatible to run on the lower APIs i.e android 10,9,8,7 etc.
The same I wanted to know for android-9.0.0 tools that you have provided. I am using these bin/binaries through Runtime.getRuntime().exec() in an android project and I wanted to run at least one binary of aapt and aapt2 for all the platforms devices. Not sure which binary to use on which platform and architecture. Can you explain more about it?

@rupesh-kumar-lpu I checked. Binaries work on older devices!

Closing issue as this seems to be resolved.  @shuajinanhai , please let me know if you have any issues with the binaries.
@rupesh-kumar-lpu, regarding your question below.

Hey, @JonForShort I wanted to know if the android-11.0.0_r33 compatible to run on the lower APIs i.e android 10,9,8,7 etc.
The same I wanted to know for android-9.0.0 tools that you have provided. I am using these bin/binaries through Runtime.getRuntime().exec() in an android project and I wanted to run at least one binary of aapt and aapt2 for all the platforms devices. Not sure which binary to use on which platform and architecture. Can you explain more about it?

Good question.  I haven't really tested this enough to know which binaries should work for which platforms. If someone has the time to test this, I can create a wiki page and put the results in it.  Unfortunately, I don't have much time to do this myself.  Sorry about that!


⚠ Bofrost.com has degraded performance

In f4d9012, Bofrost.com (https://www.bofrost.com) experienced degraded performance:

HTTP code: 200
Response time: 1148 ms

Resolved: Bofrost.com performance has improved in 031d8e5.


ocaml: Fix aarch64-darwin build

Motivation for this change
Fixes ocaml build on aarch-darwin (#144785)
Thanks @bobrik for triaging the issue and also testing it.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

@ofborg build ocaml

Does this handle cross compilation ? nix log .#pkgsCross.aarch64-darwin.ocamlPackages.ocaml seems to show that aarch64-apple-darwin-clang does exist.
ocaml-aarch64-apple-darwin-4.14.0.drv.bz2 (renamed to bypass GitHub limitation)


Apply fixes from StyleCI

This pull request applies code style fixes from an analysis carried out by StyleCI.

For more information, click here.

Pull Request Test Coverage Report for Build 127

4 of 10 (40.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 76.786%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/class/list_class.php
4
10
40.0%

Totals

Change from base Build 125:
0.0%

Covered Lines:
129

Relevant Lines:
168

💛  - Coveralls


add support for pgsql

Thank you for your contribution!
Our developers will review your PR and provide feedback, as soon as high prio topics have been adressed.

@daniel-eder The build also fails on master with the same warning and does not have anything to do with the changes of this PR:
One or more dependencies were identified with known vulnerabilities in efgs-federation-gateway:

hibernate-core-5.4.21.Final.jar<EMAIL_ADDRESS>cpe:2.3:a:hibernate:hibernate_orm:5.4.21:*:*:*:*:*:*:*) : CVE-2020-25638
tomcat-embed-core-9.0.38.jar<EMAIL_ADDRESS>cpe:2.3:a:apache:tomcat:9.0.38:*:*:*:*:*:*:*, cpe:2.3:a:apache_software_foundation:tomcat:9.0.38:*:*:*:*:*:*:*, cpe:2.3:a:apache_tomcat:apache_tomcat:9.0.38:*:*:*:*:*:*:*) : CVE-2020-17527

Thank you for checking - I thought I saw the last master build go green, sorry for that. In this case please disregard my comment.

The problem with PostgreSQL is the autoincrement statement. Unfortunately this PR does not work because autoincrement is disabled. We need to find another solution.

Closing this because of follow up PR. (#257 )


[FLINK-19221][filesystems] Introduce the LocatedFileStatus to save block location RPC requests.

What is the purpose of the change
This PR helps to drastically reduce the number of RPC requests made to the file system master (for example the HDFS Name Node) when enumerating the splits in file sources. Previously, we enumerated first all files, and then made a separate RPC call to the name node to get the block locations for each file.
Hadoop has introduced an optimization here that this PR exploits. When the HDFS Client returns a FileStatus (description of a file) it frequently returns a LocatedFileStatus which already contains all the BlockLocation information.
In this PR, we mirror that mechanism in the Flink FileSystem APIs:
When the FileStatus obtained from Hadoop file systems during listing the directory (or getting details for a file) already has the block locations, we use that directly and expose in a Flink LocatedFileStatus. That saves the extra RPC call per file to obtain that block location information.
Brief change log

04e493f0c0d3ba80a2507c4550633a62af45a2e3 : [FLINK-19218] Remove inconsistent/misleading locality information from Local File Splits.
This is a general cleanup/improvement and makes the implementation of the main change easier.

cdc711101c86a58196f0978272a315d03b87e303 : [FLINK-19221] Main change as described above.

Verifying this change
This change makes a "best effort" attempt and, on purpose, does not add hard tests to enforce this behavior. That is because certain file system implementations or setups may not eagerly expose the location information and we don't want to break compatibility working with them.
The best way to verify this is running a file-heavy batch job against HDFS with this change and observing the reduced number or Name Node RPC requests.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): yes (a new class, no breaking changes)
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? not applicable

Thanks for the review. Will fix the comments and typos.


android_lifecycles_test has been failing for quite a while

https://luci-milo.appspot.com/p/flutter/builders/staging/Linux_android android_lifecycles_test has been failing for a while: https://chromium-swarm.appspot.com/task?id=5b3d8f1ca0bdc710

This has caused staging bot to quarantine, and caused capacity issue: https://github.com/flutter/flutter/issues/108054
We will remove this test from CI.

Closing as obsolete.