offlinetest fails on test_httplib_validation_errors

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting a bug unrelated to a specific site
[X] I've verified that I'm running yt-dlp version 2023.07.06 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood
After 227bf1a33be7b89cd7d44ad046844c4ccba104f4 was committed, performing make offlinetest PYTHON=python3, shows the following error:
FAILED test/test_networking.py::TestUrllibRequestHandler::test_httplib_validation_errors[Urllib] - AssertionError: Regex pattern did not match.
This above error occurs with both python 3.7 and 3.9.

E                Regex: "method can't contain control characters"
E                Input: 'HTTP Error 400: Bad Request'

test/test_networking.py:793: AssertionError
======================================================================= short test summary info =======================================================================
SKIPPED [1] test/test_jsinterp.py:111: Not implemented
SKIPPED [1] test/test_jsinterp.py:373: Not implemented
SKIPPED [1] test/test_jsinterp.py:335: Not implemented
FAILED test/test_networking.py::TestUrllibRequestHandler::test_httplib_validation_errors[Urllib] - AssertionError: Regex pattern did not match.
================================================ 1 failed, 460 passed, 3 skipped, 5548 deselected in 99.00s (0:01:38) =================================================

What are the exact versions of python you are testing on?
Have a suspicion that the expected behaviour might not be present in older minor versions (as it is a result of patches for a number of security vulnerabilities)

I tried Python 3.7.5 (from ubuntu) and self-compiled Python 3.9.17.  Both versions of python produce the same issue.

The one shown in the log isn't patched in 3.7.5, so will need to make an exemption for that: https://github.com/advisories/GHSA-w7gf-rpqw-gx4f

Python 3.x before 3.5.10, 3.6.x before 3.6.12, 3.7.x before 3.7.9, and 3.8.x before 3.8.5

I'm guessing another one of the checks in that test is failing in 3.9.17.

I'm guessing another one of the checks in that test is failing in 3.9.17.
Maybe to keep it simple we skip the test if an HTTPError is raised 🤔

Skipping the test if an error is raised sounds reasonable.

Actually, checking for HTTPError won't suffice, so I'm going to have to whitelist Python versions.
Can you show the output for 3.9.17? All of these should be working for that version :confused:

I just added https://github.com/yt-dlp/yt-dlp/pull/7677 and did a build with 3.9.17:
test/test_networking.py::TestHTTPRequestHandler::test_headers[Urllib] PASSED                                                                                [ 33%]
test/test_networking.py::TestHTTPRequestHandler::test_timeout[Urllib] PASSED                                                                                [ 34%]
test/test_networking.py::TestHTTPRequestHandler::test_source_address[Urllib] Traceback (most recent call last):
  File "/usr/lib/python3.9/socketserver.py", line 683, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.9/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.9/socketserver.py", line 747, in __init__
    self.handle()
  File "/usr/lib/python3.9/http/server.py", line 433, in handle
    self.handle_one_request()
PASSED                                                                             [ 34%]  File "/usr/lib/python3.9/http/server.py", line 422, in handle_one_request
    self.wfile.flush() #actually send the response if not already done.
  'GET\n' (found at least '\n')

yt_dlp/networking/_urllib.py:458: RequestError

During handling of the above exception, another exception occurred:

self = <test.test_networking.TestUrllibRequestHandler object at 0x7f57c8d40c10>
handler = functools.partial(<class 'yt_dlp.networking._urllib.UrllibRH'>, logger=<class 'yt_dlp.utils._YDLLogger'>)

    @pytest.mark.parametrize('handler', ['Urllib'], indirect=True)
    def test_httplib_validation_errors(self, handler):
        with handler() as rh:

            # https://github.com/python/cpython/blob/987b712b4aeeece336eed24fcc87a950a756c3e2/Lib/http/client.py#L1256
            with pytest.raises(RequestError, match='method can\'t contain control characters') as exc_info:
                validate_and_send(rh, Request('http://<IP_ADDRESS>', method='GET\n'))
            assert not isinstance(exc_info.value, TransportError)

            # https://github.com/python/cpython/blob/987b712b4aeeece336eed24fcc87a950a756c3e2/Lib/http/client.py#L1265
            with pytest.raises(RequestError, match='URL can\'t contain control characters') as exc_info:
>               validate_and_send(rh, Request('http://<IP_ADDRESS>', method='GET\n'))
E               AssertionError: Regex pattern did not match.
E                Regex: "URL can't contain control characters"
E                Input: "method can't contain control characters. 'GET\\n' (found at least '\\n')"

test/test_networking.py:798: AssertionError
======================================================================= short test summary info =======================================================================
SKIPPED [1] test/test_jsinterp.py:111: Not implemented
SKIPPED [1] test/test_jsinterp.py:373: Not implemented
SKIPPED [1] test/test_jsinterp.py:335: Not implemented
FAILED test/test_networking.py::TestUrllibRequestHandler::test_httplib_validation_errors[Urllib] - AssertionError: Regex pattern did not match.
================================================ 1 failed, 465 passed, 3 skipped, 5548 deselected in 88.62s (0:01:28) =================================================

I am also using pytest 7.4.0.  I wonder if that has any affect on this issue.

It looks like there is a typo on this line (extra space in the address):
https://github.com/yt-dlp/yt-dlp/blob/86aea0d3a213da3be1da638b9b828e6f0ee1d59f/test/test_networking.py#L798

Sorry, that extra space was already fixed by your latest PR.

I changed this line, but it did not fix the issue.
--- a/test/test_networking.py
+++ b/test/test_networking.py
@@ -796,7 +796,7 @@ def test_verify_cert_error_text(self, handler):
         # https://github.com/python/cpython/blob/987b712b4aeeece336eed24fcc87a950a756c3e2/Lib/http/client.py#L1265
         # bpo-38576: Check implemented in 3.7.8+, 3.8.3+
         (
-            Request('http://127.0.0. 1', method='GET'),
+            Request('http://<IP_ADDRESS>', method='GET'),
             'URL can\'t contain control characters',
             lambda v: v < (3, 7, 8) or (3, 8, 0) <= v < (3, 8, 3)
         ),

It looks like the real error here is this:
ValueError: I/O operation on closed file.

Since the same error is occuring in both 3.7.5 and 3.9.17, this could also be related to websockets 11.0.3, since that is where the trace starts.

The extra space is intentional. I think it was the control character I had left in the method for that one.
The ValueError is by the test server and is expected, can ignore that.

OK. https://github.com/yt-dlp/yt-dlp/pull/7677 skips the test_source_address[Urllib] as expected for python 3.7.5.  Also, the test is not failing anymore with python 3.9.17 when this change is applied. There is still an exception occurring on test_source_address[Urllib] but overall it passes now with python 3.9.17.
test/test_networking.py::TestHTTPRequestHandler::test_source_address[Urllib] ----------------------------------------
Exception occurred during processing of request from ('<IP_ADDRESS>', 37019)
Traceback (most recent call last):
  File "/usr/lib/python3.9/socketserver.py", line 683, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.9/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
PASSED                                                                             [ 34%]  File "/usr/lib/python3.9/socketserver.py", line 747, in __init__
    self.handle()
  File "/usr/lib/python3.9/http/server.py", line 433, in handle
    self.handle_one_request()
ValueError: I/O operation on closed file.
----------------------------------------


patchelf tool

Please update patchelf for Alpine 3.16 due issue for armv7 cpus.
https://github.com/NixOS/patchelf/issues/474
https://github.com/brenoguim/patchelf/tree/breno.474

@magicse Is this in any way related to Gitlab? If not, I'd recommend you open an issue on https://gitlab.alpinelinux.org/alpine/aports/-/issues/


Declare supported languages for apps

This will allow us to filter the app store to show only applications that match the user's language.
We could use dc:language for that.

Done https://github.com/activitypods/activitypods/commit/ea66d21aa0b463a465e33d484ac5b649cbc6581e


Improve selection handling

This is going to take a rewiring of selection states within the editor.
Things that need to be supported:

[x] Selection of items (including Ctrl/Cmd + a)
[x] Deselection of items
[x] Drag movement of all items in the workspace
[x] Storing of item location after move
[x] Deletion of items
[x] Ctrl/Cmd+x to cut
[x] Ctrl/Cmd+c to copy
[x] Ctrl/Cmd+v to paste

[x] Graphs
[x] URLs

[x] Reset graph location
[x] Drag and Drop from Components panel
[x] Auto-selection of pasted & dropped items
[x] Handle disambiguation of edges
[x] Bring back comments

[x] Check comments work in terms of dragging etc

[x] Clean up old events and logic

[x] Editor
[x] Graph Renderer
[x] Graph Node
[x] Graph
[x] Events

Declaring victory.


Select scripting language for game engine development

We've talked briefly about possibly shelving BeanShell as language of choice, mainly because BeanShell doesn't have great security (it can instantiate anything on the system class loader) and is super old.
In the interests of getting ModLoader feature-complete (again) after removing BeanShell, we should select a new implementation language.
I would suggest Lua or Javascript mainly because they are the most common (and easiest to access because they have implementations that will run on the JVM) but am open to other suggestions.
Ultimately I feel like this one should be Hawk's final choice. I'll happily implement whatever language, but a choice does need to be made.

I think we should use Common Lisp. Or Erlang. ;)
I am not all that familiar with Lua or JS, but I feel like Lua is probably the better language, at least syntactically. Although choosing JS would permit Pathway devs to use any of the large set of languages that compile to JS, which fits in with my (probably impractical) desire to permit multiple scripting languages.

I believe it has been decided that we will definitely be going with Java 8's Nashorn JavaScript implementation for security reasons?

Not especially for security reasons, but the fact that it makes security easy-ish will be nice. If Hawk agrees, then we can go with this.

Nashorn seems like a good choice to me; it's backed by Oracle, so it should be well-maintained and supported, and JS is a common language that a lot of people know.

Here are some assorted additional thoughts I've had that we might want to consider:

What version of ExcemaScript does the Nashorn interpreter support? If it's ES6 compliant, we might want to design our script-facing API with the new features in ES6 in mind.
What methods are there for optimising the performance of JS running in Nashorn? Can commonly used scripts or hot code paths be precompiled to Java bytecode? Will this be done automagically or do we have to ask Nashorn to do it for us? Does JS running in Nashorn benefit from JVM JIT compilation?
Are there ways that we can provide Nashorn with more convenient access to specific Java classes and objects? It would be cool to provide less gross syntax for accessing Pathway model components.
The Web Resource Optimizer for Java (wro4j) is a Java library for managing web assets pipelines. I believe it contains a Java implementation of the CoffeeScript compiler. If we wanted to, I think we could use this to compile CoffeeScript scripts to JS before passing them to Nashorn to interpret. I don't know much about CoffeeScript, but I think @ArcticLight likes it...?

I think we could use this to compile Coffeescript [...]

I don't really think it's our job to compile Coffeescript for our mod authors, since they can do this themselves (and for any other compiled-to-JS language), but if Hawk likes this strongly I suppose we could do that.

What version of ExcemaScript does the Nashorn [...]

I don't know, we'd have to find out.

What methods are there [...]

I think (may be mistaken) that performance is Pretty Good(TM) according to Oracle, and there were advancements in using the J8 new dispatch things in bytecode that helped. I don't think we have to ask the platform to optimize for us, but I'm also not 100% certain

we can provide Nashorn with more convenient access to specific [...]

If nothing else, we can still use the prelude / script context style insert-the-boilerplate scripts to provide this functionality for us. Since we handle the entire boot process, we can mangle the script environment however we like before it is passed to any mods.

I don't really think it's our job to compile Coffeescript for our mod author

Fair, I was sort of thinking that, too. Just figured it was worth putting out there.

[...] performance is Pretty Good(TM) [...]

I'm sure it is; I just feel like Pathway is kind of spanning a lot of layers of indirection and I feel like we want to optimise those edges as aggressively as is reasonable.

[...] we can still use the prelude / script context style insert-the-boilerplate scripts [...]

Yeah, I figured we'd do that, but we ought to see if there's an easier way to do it from the Java end. Storing types as vars seems to help though.

I think this issue is closable? It seems like we've definitely agreed on Nashorn JS and are just discussing the specifics of how we're going to do that. I opened #57 to continue this dialogue.

We might want to re-open this if we're still considering writing our own DSL using Truffle/Graal. @ArcticLight, is that something that's actually on the table?

Not at the moment, I don't think.

That's what I thought.


Add translatesfx to gitlab pipeline

Sign the translatesfx exe and darwin binaries
Include the translatesfx binaries as assets for the github release

Should we expect to see translatesfx in the deb and rpm CI artifacts?

I don't really think that that's necessary, any reason to have deb/rpm for this?

I mean in the collector artifacts, not as individual libraries:
rpm -qlp splunk-otel-collector-0.36.0~post-1.x86_64.rpm  | grep translate
/usr/lib/splunk-otel-collector/agent-bundle/lib/python3.8/site-packages/oslo_i18n/_translate.py
/usr/lib/splunk-otel-collector/agent-bundle/lib/python3.8/site-packages/oslo_i18n/tests/test_translate.py
...
dpkg -c ./splunk-otel-collector_0.36.0-post_amd64.deb |grep translate
-rwxr-xr-x splunk-otel-collector/splunk-otel-collector     1642 2021-09-29 18:13 ./usr/lib/splunk-otel-collector/agent-bundle/lib/python3.8/site-packages/oslo_i18n/tests/test_translate.py
-rwxr-xr-x splunk-otel-collector/splunk-otel-collector     2844 2021-09-29 18:13 ./usr/lib/splunk-otel-collector/agent-bundle/lib/python3.8/site-packages/oslo_i18n/_translate.py


Move initial guess to a new structure

With this PR, you can now create a linear initialization of states (X) and controls (U) values. Before, you gave a list with your unique initial value.
The example here is based on the other example 'align_marker_on_segment'.

@Kilperic13
I will accept this PR since I think this makes sense to keep what was done so far and it seems ready, please open a new one with the linear (the one line commented). And providing a test with you example


Automatically create PR to homebrew-core after the new version has been released

For now I see the following solution:

Create GH action that do:

git clone git@github.com:kvendingoldo/homebrew-core.git
update fork
update fork's formula based on this repo
open MR to homebrew-core

@Nmishin wdyt?

related to https://github.com/tofuutils/tenv/issues/120

https://github.com/Homebrew/homebrew-core/pull/170564

no need to automate the release, homebrew-core has it part of autobump flow.

@chenrui333 how homebrew-core will take changes from this repository?

we query new version, and then run the source build.

all the workflow runs captured in here, https://github.com/Homebrew/homebrew-core/actions/workflows/autobump.yml


[FEATURE] MS SQL Support

Hello All,
Is there any plan to add support for more DB's like mssql server, or any workaround to make it possible.
My goal is to provide some data analysis through my training the agent on my structured data.
Best

yes! we have that on one of the features to develop

let the force be with you then :-)

sending you force to make you, achieve this one ;-)
let the force be with you

#761


Post-course 2020 tutorial processing

TODO list to track which days have been updated and who is working on what

[ ] W0D1
[ ] W0D2
[ ] W1D1 (MW)
[x] W1D2
[ ] W1D3
[ ] W1D4
[ ] W1D5
[ ] W2D1
[ ] W2D2
[ ] W2D3
[ ] W2D4
[ ] W2D5
[ ] W3D1
[ ] W3D2
[x] W3D3
[x] W3D4
[ ] W3D5
[ ] W4D1
[ ] W4D2
[ ] W4D3
[ ] W4D4
[ ] W4D5
[ ] W5D1
[ ] W5D2
[ ] W5D3
[ ] W5D4
[ ] W5D5

Assign yourself by editing the issue; check the box when you've finished the work


Flash status synchronisation tb fix

TB has been updated to check for flash_status bits correctness.
SPI-monitor now triggers SV events on each byte sampled on the SPI bus as well as when the CSB is deasserted.
The intercept VSEQ has been updated so reads/writes to flash_status won't occur just sequentially before/after a SPI-txn but also concurrently, as well as sending two flash_status writes in a row (less likely) where the first  writes the upper bits of the flash_status register, and the second would clear the busy/wel bits.
The Scoreboard has been updated to be in sync with the latest  flash_status behaviour changes in the RTL:

SPI reads to FLASH_STATUS (READ_STATUS#1/2/3 commands ) no longer wrap around the register if the CSB was left "active" after the first byte is returned to the host by the RTL
Writes to flash_status make their way towards the CDC SPI-side at each of the bytes sent to/from the host.
TL-UL view of flash_status updates on CSB being inactive.

This PR takes care of  issue #21111

Was a significant subset of tests run on this? It looks like regression results have taken a big dive between before this change (https://reports.opentitan.org/hw/ip/spi_device_2p/dv/2024.03.29_04.21.20/report.html) and after (https://reports.opentitan.org/hw/ip/spi_device_2p/dv/2024.04.01_04.52.26/report.html).
Are those real failures?

Was a significant subset of tests run on this? It looks like regression results have taken a big dive between before this change (https://reports.opentitan.org/hw/ip/spi_device_2p/dv/2024.03.29_04.21.20/report.html) and after (https://reports.opentitan.org/hw/ip/spi_device_2p/dv/2024.04.01_04.52.26/report.html).
Are those real failures?

 I had some spurious failures of tests timing out which passed once I rerun them.
I'll investigate!


User details page

As a MET system user, I want to be able to see other users' detail pages so that I can see their roles and the project they worked on.
System user is on the user management page (listing) (https://app.zenhub.com/workspaces/met-ops-6254279dcfa60b0010ea5a0d/issues/bcgov/met-public/714) and clicks on a user name.
https://xd.adobe.com/view/61e16dd6-919e-4374-a074-d09fd2184766-34b2/screen/24e95953-0a12-4052-8818-9977fd6c0b79/specs/
The "Assign Superuser Role" toggle is only enabled if the user is a Superuser/Instance Admin."
Add to Engagement" button is disabled if user is a viewer
**NOTE: card blah blah handles the The "Deactivate User" toggle is only enabled if the user is Instance Admin.

Hey team! Please add your planning poker estimate with Zenhub @tiagograf-AOT @jadmsaadaot @VineetBala-AOT


Update README.md

Summary
Fixed wording and typos in readme only.

@jirihofman Thank you so much for your contribution.
But could you please follow the semantic PRs?

But could you please follow the semantic PRs?
Done.

@jirihofman  i think you need to change the commit not only the title

@KarimMokhtar Done. I thought so too, but this line from the actions confused me a bit

No release type found in pull request title "Update README.md". Add a prefix to indicate what kind of release this pull request corresponds to


machine-friendly output format

Having some common machine-readable format(s) would be helpful for analyzing results. There should be a format that is easy for plotting packages like gnuplot and matplotlib to consume - I expect CSV (which is a subset of the gnuplot text data format) is the right starting point for this.
There might be other formats of interest, maybe JSON that looks something like:
{
    "name": "Benchmark name",
    "optimize": "ReleaseFast",
    "target": "x86-64-linux-gnu",
    "summary": {
        "mean_wall_time": {
            "value": 123438432,
            "stddev": 1234,
            "min": 12314566,
            "max":<PHONE_NUMBER>,
        },
        "other_summary_statistic": {
            "value": 12345,
            "stddev": 123432
            "min": 23432,
            "max": 12382132
        }
    },
    "samples": [
        { "wall_time": 123455677, "peak_rss": 1234, "cpu_cycles": 123, "instructions": 1232, "cache_references": 12321, "cache_misses": 234232, "branch_misses": 128234 },
        { "wall_time": 123455677, "peak_rss": 1234, "cpu_cycles": 123, "instructions": 1232, "cache_references": 12321, "cache_misses": 234232, "branch_misses": 128234 },
    ]
}

JSON is pretty verbose for using as a storage format, but it would compress well.

Machine-friendly output could also lead to potential plotting options as well.


HTML entities are being decoded

A template containing > is decoded to >.

This issue appears to have been resolved.


Update documentation episode

The coderefinery episode on documentation was updated, as well as the instructor notes. It no longer includes our shortened schedule "A lesson with a focus on in-code-documentation and readme files".
This schedule can still be retrieved from the github history:
https://github.com/coderefinery/documentation/blob/077906ba8634d1b1840f804b04c895c08feff042/content/guide.md

Add to the instructor notes that sphynx is a demo.

@svenvanderburg is this adressed in #40? If so, can be closed.

Yes, this is addressed with #40, closing the issue.


Fix operator monitoring

ServiceMonitor only generated once
Metrics accessible via http without rbac-proxy

jira: OSD-10788

/lgtm


[C++] [Acero] Incorrect results in inner join

Describe the bug, including details regarding any error messages, version, and platform.

Before explaining, if recreating previously submitted issues like this is considered a mistake, I apologize.

#37729

Overview
In the previous issue, the user is discussing the occurrence of incorrect results when performing an inner join.
Although it was well-explained in the previous issue, to reiterate, the user has created this issue because they are getting incorrect results when performing an inner join between two tables, namely table_1 and table_2.
Each of these tables (table_1 and table_2) has columns col_1, col_2, col_3, and table_2 has an additional col_4. Upon my investigation, it appears that table_2.parquet has 7 more records, but for col_1, col_2, and col_3, it contains the same values as table_1.
The number of records in each table is 6282 and 6289, respectively.
So, when performing an inner join using col_1, col_2, and col_3 as the join keys, the result should be 6282, regardless of the order of the tables.
Reason
To start with the cause, there is an issue with the BloomFilter logic.
When testing in C++, if you set the BloomFilter option (disable_bloom_filter) to true, the join operation is performed without any issues.
// with bloom_filter (default)
HashJoinNodeOptions join_opts{
    JoinType::INNER,
    {"col_1", "col_2", "col_3"},
    {"col_1", "col_2", "col_3"},
    literal(true),
    "_l",
    "_r"
};

// without bloom_filter
HashJoinNodeOptions join_opts{
    JoinType::INNER,
    {"col_1", "col_2", "col_3"},
    {"col_1", "col_2", "col_3"},
    literal(true),
    "_l",
    "_r",
    true // disable_bloom_filter
};

Additionally, the findings from further investigation are as follows.

The number of matching records between the left and right tables must exceed 1024.
When the number of matching records is close to the number of records in the right table, errors occur.
The number of matching records does not appear to have any correlation with the number of records in the left table.

Up to this point, this is what I have gathered about the issue, and I am working hard to fix the bug.
However, I am a novice about Arrow, which is causing it to take longer than expected. Nonetheless, I will continue to make efforts to resolve the bug.
Any advice or insights from those who are more experienced would be greatly appreciated, and it would also be great if someone with expertise could tackle this issue first. So, I'm sharing this here.
Component(s)
C++

@llama90 thank you for investigating this and working to solve it.
There was a recent conversation on the Arrow developer mailing list that might be relevant (but I'm not sure):
https://lists.apache.org/thread/pq10zd4sr494hd09t93o3hb6zswn5wll
When batch sizes are too large, silent overflows can occur, producing incorrect hash join results.
Could this explain the incorrect results you are seeing?

Hello, I am creating the schema and generating data as follows for testing.
It seems like a different issue from a Swiss join. Even in the smallest case, when both table_1 and table_2 have 1025 identical records each, the result still shows 1024 join results. When you change the number of right records to 1537, you will still get 1025 values.
Based on the information you mentioned, I became interested and conducted a test by increasing the number of records to 100,000.

Left: 100,000
R: 100,000
Match: 100,000

without BloomFilter: 100,000
with BloomFilter: 84,334

In this case, I was able to confirm that the results were coming out in batches of 32,768 each. Maybe it is a morsel unit.

Left: 32,768
R: 32,768
Match: 32,768

without BloomFilter: 32,768
with BloomFilter: 17,010

In my opinion, it seems clear that there might be an issue with the Bloom Filter. Cases where accurate results were obtained, whether using BloomFilter or not, were as follows:

When the number of match records is 1024 or less, regardless of the number of records in the Left and Right tables.
When the number of match records exceeds 1024, but the number of records in the Right table is greater than the number of match records.

For example, R - 5000, M - 5000, L - 10000 (In this case, reducing it to 9000 records results in 4904 as the output).

Below is an example of the tables.
table_1 (left)

schema ({timestamp(TimeUnit::MICRO, "UTC"), large_utf8(), large_utf8()})

col_1: All values are 2023-09-14 21:43:18.678917
col_2: 1, 2, 3, 4, 5 ... N, -1, -1, -1, -1, -1, ... -1
col_3: All values are foo

N is the number of matching records between table 1 and table 2

table_2 (right)

schema ({timestamp(TimeUnit::MICRO, "UTC"), large_utf8(), large_utf8(), large_utf8()})

col_1: All values are 2023-09-14 21:43:18.678917
col_2: 1, 2, 3, 4, 5 ... M
col_3: All values are foo
col_4: All values are bar

M is the number of records for table 2

Example

The number of records for table 1: 5
N: 3
M: 4

table 1

col_1
col_2
col_3

2023-09-14 21:43:18.678917
1
foo

2023-09-14 21:43:18.678917
2
foo

2023-09-14 21:43:18.678917
3
foo

2023-09-14 21:43:18.678917
-1
foo

2023-09-14 21:43:18.678917
-1
foo

table 2

col_1
col_2
col_3
col_4

2023-09-14 21:43:18.678917
1
foo
bar

2023-09-14 21:43:18.678917
2
foo
bar

2023-09-14 21:43:18.678917
3
foo
bar

2023-09-14 21:43:18.678917
4
foo
bar

I haven't managed to find the exact issue yet (I'm not too familiar with this code in particular), but this section is fairly suspicious: https://github.com/apache/arrow/blob/3697bcdd514fc5a0472dabd3928f6b4731554cda/cpp/src/arrow/acero/hash_join_node.cc#L1131-L1146
Note that this function is exclusive to the bloom filter path and the the value of kMiniBatchLength happens to be 1024...

It seems correct.
I've noticed that when modifying the kMiniBatchLength value arbitrarily to be larger than the number of match records, the results come out fine.
I suspect there might be an issue with handling the remainder when it exceeds the mini-batch size, so I'm looking into that part.
Thank you for checking!

It seems that the issue has been fixed.
I will clean up the code, write unit tests, and aim to submit a PR as soon as possible. Thanks to your review, I was able to reproduce the symptoms on a smaller scale for testing. I appreciate your review once again. @benibus


Update wav2vec2 baseline

Issue
Description

Fixed csv path
change save path from client_19999 to server (easier to look after)
add pre-trained huggingface path so we only need to download once
remove unnecessary elements in sb config

Related issues/PRs

Proposal
Explanation

Checklist

[ ] Implement proposed change
[ ] Write tests
[ ] Update documentation
[ ] Update changelog
[ ] Make CI checks pass
[ ] Ping maintainers on Slack (channel #contributions)

Any other comments?

@tuanct1997, thanks for creating the PR. If I run the code i'm getting the error below shortly after starting the experiment:
INFO flwr 2023-10-15 20:32:56,240 | server.py:280 | Received initial parameters from one random client
[2023-10-15 20:32:56,240][flwr][INFO] - Received initial parameters from one random client
INFO flwr 2023-10-15 20:32:56,240 | server.py:91 | Evaluating initial parameters
[2023-10-15 20:32:56,240][flwr][INFO] - Evaluating initial parameters
params["data_folder"] = 'data/server'
speechbrain.core - Beginning experiment!
speechbrain.core - Experiment folder: /home/javier/projects/tuanwav2vec2/baselines/fedwav2vec2/outputs/2023-10-15/20-32-13/client_19999server
speechbrain.dataio.encoder - Load called, but CTCTextEncoder is not empty. Loaded data will overwrite everything. This is normal if there is e.g. an unk label defined at init.
speechbrain.core - Info: device arg overridden by command line input to: cuda
speechbrain.core - 316.5M trainable parameters in ASR
DEBUG flwr 2023-10-15 20:33:00,246 | client.py:46 | Starting client 19999
speechbrain.utils.checkpoints - Would load a checkpoint here, but none found yet.
ERROR flwr 2023-10-15 20:33:00,725 | app.py:294 | Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/home/javier/.cache/pypoetry/virtualenvs/fedwav2vec2-n5e6cC0Z-py3.10/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 302, in _worker_loop
    data = fetcher.fetch(index)
  File "/home/javier/.cache/pypoetry/virtualenvs/fedwav2vec2-n5e6cC0Z-py3.10/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/javier/.cache/pypoetry/virtualenvs/fedwav2vec2-n5e6cC0Z-py3.10/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 58, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/home/javier/.cache/pypoetry/virtualenvs/fedwav2vec2-n5e6cC0Z-py3.10/lib/python3.10/site-packages/speechbrain/dataio/dataset.py", line 165, in __getitem__
    return self.pipeline.compute_outputs({"id": data_id, **data_point})
  File "/home/javier/.cache/pypoetry/virtualenvs/fedwav2vec2-n5e6cC0Z-py3.10/lib/python3.10/site-packages/speechbrain/utils/data_pipeline.py", line 464, in compute_outputs
    return self._compute(data, self._exec_order, self.output_mapping)
  File "/home/javier/.cache/pypoetry/virtualenvs/fedwav2vec2-n5e6cC0Z-py3.10/lib/python3.10/site-packages/speechbrain/utils/data_pipeline.py", line 496, in _compute
    values = item(*args)  # Call the DynamicItem to produce output
  File "/home/javier/.cache/pypoetry/virtualenvs/fedwav2vec2-n5e6cC0Z-py3.10/lib/python3.10/site-packages/speechbrain/utils/data_pipeline.py", line 72, in __call__
    return self.func(*args)
  File "/home/javier/projects/tuanwav2vec2/baselines/fedwav2vec2/fedwav2vec2/dataset.py", line 86, in audio_pipeline
    info = torchaudio.info(wav)
  File "/home/javier/.cache/pypoetry/virtualenvs/fedwav2vec2-n5e6cC0Z-py3.10/lib/python3.10/site-packages/torchaudio/backend/sox_io_backend.py", line 106, in info
    return _fallback_info(filepath, format)
  File "/home/javier/.cache/pypoetry/virtualenvs/fedwav2vec2-n5e6cC0Z-py3.10/lib/python3.10/site-packages/torchaudio/backend/sox_io_backend.py", line 14, in _fail_info
    raise RuntimeError("Failed to fetch metadata from {}".format(filepath))
RuntimeError: Failed to fetch metadata from data/audio/TEDLIUM_release-3/legacy/test/sph/JaneMcGonigal_2010.sph

Hi @jafermarq ,
I saw this error due to the missing of sph audio file from audio folder. Could you please first verify if data/audio/TEDLIUM_release-3/legacy/test/sph this path is exist? If yes, There is no audio named "JaneMcGonigal_2010.sph" inside which could due to the download link. Which TED-LIUM3 url you used?
Thank you in advance.

I used the code you are merging here (so i cloned your repp+branch) and run it. It seems that file is there but there is another TEDLIUM_release-3 path in between and JaneMcGonigal_2010.sph does exist (i'm not sure by your comment if it should or should not exist)
(fedwav2vec2-py3.10) ➜  fedwav2vec2 git:(fedw2v2_fix) ✗ ls data/audio/TEDLIUM_release-3/TEDLIUM_release-3/legacy/test/sph
AimeeMullins_2009P.sph  BillGates_2010.sph  DanBarber_2010.sph  DanielKahneman_2010.sph  EricMead_2009P.sph  GaryFlake_2010.sph  JamesCameron_2010.sph  JaneMcGonigal_2010.sph  MichaelSpecter_2010.sph  RobertGupta_2010U.sph  TomWujec_2010U.sph

Yes. I think it's due to the configs file. I edited the extract_subdirectory from data to data/TED-TEDLIUM_release-3 since when you extract it, inside is a folder named TEDLIUM_release-3.
I think you have duplicated  name it's may due to you already have a pre-existed path data/audio/TEDLIUM_release-3. It's just a matter of config path. On my side, the correct path after created will be: data/audio/TEDLIUM_release-3/legacy for train, test and dev dataset.


feat(core): accept mock class instantiation args in constructor

PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our guidelines: https://github.com/graycoreio/daffodil/blob/master/CONTRIBUTING.md#commit
[x] Tests for the changes have been added (for bug fixes / features)
[x] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the current behavior?

No way to instantiate mock classes with constructors
What is the new behavior?
DaffModelFactory will pass additional constructor args to the constructor of the mock class during create.
Does this PR introduce a breaking change?
[ ] Yes
[x] No

Other information
Relies on #1776
There are a few issues:

typing the create partial causes a lot of type errors in poorly written tests
returning a class instance rather than a plain object causes jasmine's expect(transformedObject).toEqual(aClassInstance) to fail. This can be fixed with jasmine.objectContaining

@damienwebdev I removed both the partial typing and the class instance as these are separate features and did not come for free as I had hoped


flashing LPC55S69 can fail if board not in a good state

I've somehow 'bricked' a board and than try and re-flash, I get:
[galak@localhost build]$ pyocd flash zephyr/zephyr.bin
Secure
0001090:CRITICAL:main:uncaught exception:
Traceback (most recent call last):
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/main.py", line 338, in run
self._COMMANDSself._args.cmd
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/main.py", line 475, in do_flash
file_format=self._args.format)
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/flash/loader.py", line 160, in program
self._loader.commit()
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/flash/loader.py", line 504, in commit
keep_unwritten=self._keep_unwritten)
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/flash/flash_builder.py", line 449, in program
sector_erase_count, page_program_time = self._compute_sector_erase_pages_and_weight(fast_verify)
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/flash/flash_builder.py", line 625, in _compute_sector_erase_pages_and_weight
self._analyze_pages_with_partial_read()
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/flash/flash_builder.py", line 561, in _analyze_pages_with_partial_read
self._enable_read_access()
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/flash/flash_builder.py", line 221, in _enable_read_access
self.flash.init(self.flash.Operation.VERIFY)
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/flash/flash.py", line 235, in init
self.target.reset_and_halt(Target.ResetType.SW)
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/core/coresight_target.py", line 333, in reset_and_halt
return self.selected_core.reset_and_halt(reset_type)
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/target/builtin/target_LPC55S69JBD100.py", line 295, in reset_and_halt
xpsr = self.read_core_register('xpsr')
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/coresight/cortex_m.py", line 987, in read_core_register
regValue = self.read_core_register_raw(regIndex)
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/coresight/cortex_m.py", line 1001, in read_core_register_raw
vals = self.read_core_registers_raw([reg])
File "/home/galak/.local/lib/python3.7/site-packages/pyocd/coresight/cortex_m.py", line 1061, in read_core_registers_raw
assert dhcsr_val & CortexM.S_REGRDY
AssertionError

Was also seeing this when flashing:
[galak@localhost zephyr]$ pyocd flash zephyr.bin
Secure
[====================] 100%
Secure
0006128:CRITICAL:__main__:uncaught exception:
Traceback (most recent call last):
  File "/home/galak/.local/lib/python3.7/site-packages/pyocd/__main__.py", line 338, in run
    self._COMMANDS[self._args.cmd](self)
  File "/home/galak/.local/lib/python3.7/site-packages/pyocd/__main__.py", line 475, in do_flash
    file_format=self._args.format)
  File "/home/galak/.local/lib/python3.7/site-packages/pyocd/flash/loader.py", line 160, in program
    self._loader.commit()
  File "/home/galak/.local/lib/python3.7/site-packages/pyocd/flash/loader.py", line 504, in commit
    keep_unwritten=self._keep_unwritten)
  File "/home/galak/.local/lib/python3.7/site-packages/pyocd/flash/flash_builder.py", line 472, in program
    self.flash.target.reset_and_halt()
  File "/home/galak/.local/lib/python3.7/site-packages/pyocd/core/coresight_target.py", line 333, in reset_and_halt
    return self.selected_core.reset_and_halt(reset_type)
  File "/home/galak/.local/lib/python3.7/site-packages/pyocd/target/builtin/target_LPC55S69JBD100.py", line 295, in reset_and_halt
    xpsr = self.read_core_register('xpsr')
  File "/home/galak/.local/lib/python3.7/site-packages/pyocd/coresight/cortex_m.py", line 987, in read_core_register
    regValue = self.read_core_register_raw(regIndex)
  File "/home/galak/.local/lib/python3.7/site-packages/pyocd/coresight/cortex_m.py", line 1001, in read_core_register_raw
    vals = self.read_core_registers_raw([reg])
  File "/home/galak/.local/lib/python3.7/site-packages/pyocd/coresight/cortex_m.py", line 1061, in read_core_registers_raw
    assert dhcsr_val & CortexM.S_REGRDY
AssertionError

Here's the image I'm trying to flash that causes the failure:
zephyr.bin.gz

I'm able to reproduce this.

I'm able to reproduce this.

Now the question is can you fix it? :)


fix(frontend): networkId always undefined in SendModal

Motivation
The networkId prop  that is passed down from SendModal is actually only defined when targetNetwork is there. Because of that, BtcSendForm was never receiving networkId even though it is relying on it to identify whether it's a regtest, testnet or mainnet btc.

/cc @lmuntaner


new hook useCreateFormik

🚀 Feature request
Create a new hook useCreateFormik returning an element to use inside de jsx to envelop all form like the element <Formik/> does, alongside of the items returned in the useFormikContext. That way, it's possible to have access to formik inside all the function and have the power of context formik hooks
Current Behavior
useFormik returns an attribute that should be referenced in every element of the form. It's not practical and have problem using with another components that use formik context hooks.
Formik limits scope inside of the element. Attributes like values or errors it's not accessible outside of the element.
Desired Behavior
It should work as the following example:
const ExampleFunc = () => {

  const { setFieldValue, values, Formik } = useCreateFormik({
    initialValues: { usename: '' }, onSubmit: () => {
    },
  });

  //now I have access to values, errors and other attributes of my formik, without the necessity of create another function.

  return (
    <Formik>
      <Field name="username">
      <Button type="submit">Submit</Button>
      </Formik>
    );
};

Hey! Thanks. It’s a really good idea. This is a pattern we’ve been thinking about for v3/4. See #2474. Closing as it’s duplicative of that conversation


Improve FinalityCertificate Tests

Addresses feedback from https://github.com/filecoin-project/go-f3/pull/266#pullrequestreview-2089396736

[x] power gets added to the right entry
[x] change in key is processed with/without a change in power
[x] sequences like deletion and re-addition of an entry (requiring key to be re-specified)

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 84.60%. Comparing base (62c5ab9) to head (2b405bc).
Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #293   +/-   ##
=======================================
  Coverage   84.60%   84.60%
=======================================
  Files          14       14
  Lines        1416     1416
=======================================
  Hits         1198     1198
  Misses        134      134
  Partials       84       84

On top of #291.


executable generation with pyinstaller

I can't generate a working executable with pyinstaller on a windows10 machine. Could you please provide an executable? Thanks a lot for your work.

Which error do you get? I'll provide a new executable for the next release (in ca. 2  ... 3 weeks), please use the executable for the last release in the meantime.

Sorry, maybe is a very noob question, but I can't find any executable between the provided files. Am I missing something?


Docs: Add top-level index page for KRM functions.

Fixes https://github.com/GoogleContainerTools/kpt/issues/1066

I don't see changes to this page:
https://googlecontainertools.github.io/kpt/guides/consumer/function/
Which is the point of entry for catalog at this point.


Fingerprint dist/wc.min.css to prevent cache issues

Purpose
We are using Cloudflare Pages to host a website built with Hugo Blox. When updating the CSS, e.g. after adding new Tailwind classes, Cloudflare's cache keeps serving the old version, which breaks the website's layout. This problem can be mitigated by fingerprinting dist/wc.min.css.
Documentation
N/A

Very relevant improvement, which we already use on mreye.nl for our own custom CSS and JS files


fix: Force sun.jnu.encoding to be UTF-8 in images

Images pick up sun.jnu.encoding from the host build environment, which thus needs to be UTF-8 to be able to access paths with non-ASCII characters in them on Linux. Since Bazel does not inherit LC_CTYPE from the host, the JVM would otherwise default to an ASCII locale.

@sgammon Looks like the sha for GraalVM 23 changed, do you happen to know why?

@fmeum GraalVM posts "archive" and "latest" URLs; the archive URLs are not available immediately after release. It is expected as I don't think we've had a release since v23 initially came out.


Memory Leak on iOS

On iOS I am seeing the memory for my NativeScript application continue to grow as I execute SQL queries using this plugin. I have written a very basic sample app that demonstrates this problem.
@NathanaelA do you have any idea why I would be seeing this memory leak?
Sample App: https://github.com/thomasconner/nativescript-sqlite-memory-leak

@thomasconner - Awesome; I'll check out your issue.    Please note if your app is using angular (which it appears you are); there are several known ios memory leaks that are being resolved right now in both the TNS core modules and Angular.
I haven't looked at your test app; but if your test app is navigating between screens to do its work; you will see leaks because of the above mentioned mention leaks, without even using ns-sqlite as resources aren't being freed properly.
If your test is just loading data and console logging it on each click and still leaking memory this; it actually might be in ns-sqlite.   :-)

This seems to be fixed by https://github.com/NativeScript/ios-runtime/pull/1019.


mungegithub moved to test-infra

Leaving behind a readme to redirect in case we've linked to it from elsewhere
cc @kubernetes/contrib-maintainers @grodrigues3 @apelisse @spxtr

This change is

/approve
/lgtm

/joke


Please provide evidence for performance claims

The readme claims "Typical speedups over Python are on the order of 10-100x or more, on a single thread."
Where do these numbers come from?
Please provide the benchmarks used, and any additional information needed to reproduce the result.

Hi @markshannon — see https://exaloop.io/benchmarks.html for performance results and exact setups used. All the code is in bench/ along with detailed descriptions if you wanted to try it yourself. A couple of these use multi-threading; you can disable that by commenting the @par line to see single-threaded performance. We also have broader benchmarks in some of our older papers, e.g. here.

It seems unreasonable to claim "typical speedups" from such a small set of benchmarks.
Why not use the standard benchmark suite?
Which version of Python are you comparing against?
What is your methodology? Do the Codon times include the full clock time, or are you discarding compilation time?

Many of the benchmarks from the pyperformance suite use various libraries that aren't supported by Codon; we've included the ones that Codon can (as of today) compile with minimal or no changes. If you have other benchmarks in mind we'd be happy to take a look and incorporate them.
The Python versions are given in the benchmark link I shared (3.10.8 and 3.8.2). I know 3.11 includes performance improvements -- we'll be using it when we run the benchmarks next.
You can look at the bench.sh script to see how the benchmarks are run. Compile times for Codon (and C++) are not included in the reported runtimes, since Codon is an ahead-of-time compiler in general.
Lastly I'll mention that the speedup over Python can be arbitrarily high depending on the program. For example there is also a bench/sum benchmark where Codon is something like 107 times faster because it optimizes the entire loop away and just precomputes the answer. We didn't include this one in the results as it can be misleading in this regard.

"Codon is a high-performance Python compiler..." rather implies that Codon supports Python.
If it that isn't true, why claim to be a Python compiler?
Perhaps make it clearer that Codon isn't Python, but a statically compiled subset that shares syntax; like RPython?

To be honest I think we are pretty upfront and clear about that: we say it in the README (What isn't Codon?) and in the docs (FAQ and Differences with Python). I don't think calling something a "Python compiler" implies it can handle 100% of Python; for example Numba is described as a "NumPy aware dynamic Python compiler using LLVM" on GitHub, and Codon can handle a lot more of Python than Numba.

The performance of codon is truly impressive. However, for the long-term success of the project, I think it would be more beneficial to focus on ensuring compatibility and stability in addition to performance.

From top of the numba home page:
"Numba is an open source JIT compiler that translates a subset of Python and NumPy code into fast machine code." (Emphasis mine)
Contrast that with Nuitka, which claims to be a "Python compiler with full language support and CPython compatibility"

Codon can handle a lot more of Python than Numba.

Can it? Again, where is your evidence?
Sure, Numba has its limitations, but Codon doesn't support strings or long integers.

The performance of codon is truly impressive. However, for the long-term success of the project, I think it would be more beneficial to focus on ensuring compatibility and stability in addition to performance.

Definitely -- that's the direction we want to push towards. Still a lot more we can do to close the gap and make it more compatible :)

@markshannon The Numba GitHub has the description I shared with the first sentence in the README being "Numba is an open source, NumPy-aware optimizing compiler for Python". It makes sense that they also characterize its limitations ("subset" etc.); so does the Codon README.
I guess I'm not really sure what it is you're asking for here?

I'm asking that you make claims backed up by evidence, and link to that evidence.
You claim that Codon is a general-purpose Python compiler, but it can only handle a few very simple programs.
You claim that Codon is 10 to 100x faster than Python, but your evidence is a very small number of hand picked examples.

I'm asking that you make claims backed up by evidence, and link to that evidence.
You claim that Codon is a general-purpose Python compiler, but it can only handle a few very simple programs. You claim that Codon is 10 to 100x faster than Python, but your evidence is a very small number of hand picked examples.

Codon is not fully compatible with CPython.  It is an acceptable trade-off as we require a pythonic static compiler that prioritizes both good performance and compatibility

@markshannon There’s not a whole lot more I can offer you here unfortunately. The relative performance of two languages/implementations varies greatly based on the program; sometimes Python and Codon will be about the same speed, sometimes Codon will be orders of magnitude faster, like in the example I gave above. We’ve found through these benchmarks and in general that Codon is typically 10-100x faster on most programs (which as expected is roughly on par with Python vs. C++; e.g. https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/python3-gpp.html). Like I said, we’re more than happy to incorporate other benchmarks (and plan to as we increase our compatibility). I’m not sure what you mean by “hand picked”—we did exactly what I said in my previous comment: took the benchmarks that Codon can compile with minimal/no changes, and certainly did not exclude any where Codon was not as fast, if that’s what you mean.
I think I answered the “Python compiler” question previously; being a “Python compiler” doesn’t imply 100% compatibility, which is why e.g. Numba is usually referred to, and advertises itself as, a “Python compiler”, even though its not a drop-in replacement for CPython.

Yes, I think it is fine to refer to Codon as a "Python compiler" despite its lack of full compatibility with CPython.

PL benchmarks are always a whack-a-mole game: you can always come up with individual cases that either prove or disprove the main point. Our benchmarks were chosen by taking commonly used benchmarks and adapting them for Codon. If you think we missed anything, please let us know and we will gladly add them. The speed-up claims are also backed up by quite a lot of real-world applications we used with Codon: lots of examples are available in https://www.nature.com/articles/s41587-021-00985-6 (see Supplementaries for details).
As for compatibility: while Codon currently covers lots of Python, there are still gaps that need to be filled in. This is work in progress, mind you— Python itself is 30+ year old and it takes some time to catch up, especially considering the attack strategy we chose.
Closing this issue as nothing else can be said. Feel free to submit benchmark PRs, especially if they disprove our claims: we are indeed looking for cases where Codon under-performs as compared to Python or pypy :)

Feel free to submit benchmark PRs

Use the standard benchmark suite. It is freely available (unlike the article you linked to).


Copy deploy script to server for execution

There seems to be an issue with our current approach where docker compose exec swallows all the rest of the script. Hopefully this will avoid that.

Based on my testing, I'm pretty sure that this will indeed fix the issue. But there's only one way to find out for sure... deploy!


Remove use of deprecated gen_server callback format_status/2

The callback format_status(Opt, StatusData) is deprecated from OTP 27 and give build warnings.
Removing them since they are optional and no-op's in our case.
https://www.erlang.org/doc/apps/stdlib/gen_server.html#c:format_status/2

Good idea. Should we release it in a patch version? It's not a documented part of the API, so it's not really a breaking change. Just fix a build issue, which can be OK for a patch release?


python create new blank jupyter notebook error

VSCode Version: 1.49
OS Version: win10

Steps to Reproduce:
F1>Python: Create New Blenk Jupyter Notebook
無法開啟 'Untitled-3.ipynb': [UriError]: If a URI contains an authority component, then the path component must either be empty or begin with a slash ("/") character。

Does this issue occur when all extensions are disabled?:
Requires dependent python

I'm having the same problem

Hi mate,
I may have found a work around. Installed VS code- insiders and it works now.
https://github.com/microsoft/vscode-python/issues/11865

I'm having the same problem.
Additionally, the Jupyter Notebook editor is not loading - don't know if these two issues are related.

Good day to you,
I am facing the same problem with creating or opening .ipnyb files(Jupyter). I tried these methods

Reinstalling the Extension of Python.
Installing the VS code- insiders.

And the problem continues still now.
Kindly please resolve this issue as soon as possible.
Thank you in advance

same here. I spent the whole day trying to solve this problem, but still got nowhere.
Unable to open 'Untitled-1.ipynb': [UriError]: If a URI contains an authority component, then the path component must either be empty or begin with a slash ("/") character.

I have the same problem too

same here. I spent the whole day trying to solve this problem, but still got nowhere.
Unable to open 'Untitled-1.ipynb': [UriError]: If a URI contains an authority component, then the path component must either be empty or begin with a slash ("/") character.

you're not alone

Looks like we've got a live one. Also no success trying to resolve...

I managed to resolve this error by following these steps:

Uninstall vscode completely - https://www.logicbig.com/how-to/visual-studio-code/remove-vs-code-completely.html (Restart the machine)
Install vscode Insiders user- https://code.visualstudio.com/insiders/
Install extensions - Python (Microsoft),  Remote-WSL
select base environment

Open a folder and create file and name it "whatever.ipynb" that should automatically ask to you install more dependencies follow through the errors and select the local kernel

This took me a whole day too, but its working now. Hopefully this works too.

Good day to you,
My dear colearners, The method refered by Mr. harishmohan60 works perfectly.
I would like to thank him and every one else.
Thank you, Happy learning

5. Open a folder and create file and name it "whatever.ipynb" that should automatically ask to you install more dependencies follow through the errors and select the local kernel

Followed every step of you, but it seems no work

I download only insider (https://code.visualstudio.com/insiders/) and python extension, but now I have a different VS code (VS Insider ) that works.

Open a folder and create file and name it "whatever.ipynb" that should automatically ask to you install more dependencies follow through the errors and select the local kernel

Followed every step of you, but it seems no work

make sure delete code and .vscode, it works for me here. Good luck

Followed each and every step, still not working

Yep, Insider did the trick.

same problem here since this morning: I would try to use the insider, but I believe this is not the end-solution
Might it be a conflict with the latest VS-Code Update

Extension version: 2020.8.108011
VS Code version: Code 1.49.0 (e790b931385d72cf5669fcefc51cdf65990efa5d, 2020-09-10T13:22:08.892Z)
OS version: Windows_NT x64 10.0.18363

Good day to you,
My dear co-learners, The method refered by Mr. harishmohan60 works perfectly for some time and after reloading the window for the other extensions. It failed again . Kindly suggest any suggestions.
Thank you, Happy learning

I'm having the same issue trying to create a new jupyter notebook.

Trying to open a notebook just hangs.  Same behavior with both VSCode 1.48.2 and 1.49.0

Facing same issue
Unable to open 'Untitled-1.ipynb': [UriError]: If a URI contains an authority component, then the path component must either be empty or begin with a slash ("/") character.

VSCode Version: 1.49
OS Version: win10

Steps to Reproduce:
F1>Python: Create New Blenk Jupyter Notebook
無法開啟 'Untitled-3.ipynb': [UriError]: If a URI contains an authority component, then the path component must either be empty or begin with a slash ("/") character。

Does this issue occur when all extensions are disabled?:
Requires dependent python

please fix this..ASAP

I too have this issue. Installing vscode insider is not an option for me, I develop in multiple languages, connected to multiple Salesforce orgs, and github repos.
I really need this fixed ASAP

Hi all,
it seems that, in my case the problem has been resolved on its own.

Try to open a folder with VS and run a python script (whatever): while running, VS has asked me to restart itself two times because "some extensions have been modified on the disk".
After the two applications restart, I could run a jupyter notebook inside of it: its a real interesting bug

Same as cslongo. Create a sample.ipynb, open VSC - Insiders from raw several times. It took like three restarts of VSC - Insiders and finally kernel worked and Jupyter notebook shows results.

Same as cslongo. It definitely works!!!
Thank you very much for your advice.

Hi all,
it seems that, in my case, the problem has been resolved on its own.

Try to open a folder with VS and run a python script (whatever): while running, VS has asked me to restart itself two times because "some extensions have been modified on the disk".
After the two applications restart, I could run a jupyter notebook inside of it: its a real interesting bug

I really worked, Thanks cslongo!

It really worked, thanks cslongo!

Hi all,
it seems that, in my case, the problem has been resolved on its own.

Try to open a folder with VS and run a python script (whatever): while running, VS has asked me to restart itself two times because "some extensions have been modified on the disk".
After the two applications restart, I could run a jupyter notebook inside of it: its a real interesting bug

Tried ...but still facing the same issue

Hi all,
it seems that, in my case, the problem has been resolved on its own.

Try to open a folder with VS and run a python script (whatever): while running, VS has asked me to restart itself two times because "some extensions have been modified on the disk".
After the two applications restart, I could run a jupyter notebook inside of it: its a real interesting bug

Tried ...but still facing the same issue

Try to run a .py file and then the notebook.
I'm using the default version of VS.

After trying cslongo's , my problem  solved by itself, Now I can able to work with Jupyter notebook

Hi all,
it seems that, in my case, the problem has been resolved on its own.

Try to open a folder with VS and run a python script (whatever): while running, VS has asked me to restart itself two times because "some extensions have been modified on the disk".
After the two applications restart, I could run a jupyter notebook inside of it: its a real interesting bug

Tried ...but still facing the same issue

Try to run a .py file and then the notebook.
I'm using the default version of VS.

After trying cslongo's , my problem  solved by itself, Now I can able to work with Jupyter notebook
Thank You


rand: configuration based DRBG selection.

Checklist

[x] documentation is added or updated
[ ] tests are added or updated

How do we test configuration settings???

How do we test configuration settings???

Probably we will need to reach into the internals and check we got the right type of EVP_RAND instances created.

Added some words to config.pod.  I'll try to figure out the test tomorrow.
I'm half tempted to change the random=name to drbg=name in the random section.

I'm half tempted to change the random=name to drbg=name in the random section.

No strong view on this.

Test case added, ready for review now.

Merged to master (an hour or so early).


Don't try to look up anonymous users

Fixes an issue where views with RedHatRolePermission would fail when unauthenticated users tried to connect.

@RedHatProductSecurity/corgi-devs FYI

@RedHatProductSecurity/corgi-devs Please review.


[Feature Request]: Webpage Styling

I am new at this and I am not sure if someone has a solution. When I pull the information from the Google Calendar and display it on the webpage it is working. What I want to do it be able to style it and make it fit the rest of the page as far as font size, bold, color, etc... Is there a solution to this? Thanks for all of the help and support.

@covertmichael The solution is CSS. Just inspect DOM of the events list in dev tools in your browser to get know what selectors you need to use in your CSS file.


EOL incorrectly rendered in README.md on Windows

When using nbdev on Windows, I got incorrect line endings in my README:
.... CR
CR LF
....

This creates extra empty lines that sometimes break the Markdown format (e.g. quote blocks, numbered bullet list, ...).
However, this works fine on Linux

Should be fixed in v2


Revises build to eliminate deprecated API in Package.swift and other improvements

Issue #, if available:
#1809
Description of changes:

replaces deprecated API in Package.swift
replaces @available statements with canImport statements to simplify build
uses XCTSkip in tests which are not yet impelemented
explicitly declares types in closures which could not handle type inference

Check points: (check or cross out if not relevant)

[ ] Added new tests to cover change, if needed
[ ] Build succeeds with all target using Swift Package Manager
[ ] All unit tests pass
[ ] All integration tests pass
[ ] Security oriented best practices and standards are followed (e.g. using input sanitization, principle of least privilege, etc)
[ ] Documentation update for the change if required
[ ] PR title conforms to conventional commit style
[ ] If breaking change, documentation/changelog update with migration instructions

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

I haven't reviewed yet, but one immediate thought. I don't think we're ready to bump the Swift version to 5.6 in our manifest yet. That immediately cuts off anyone using < Xcode 13.3 while we're working hard to continue support for >= Xcode 13.2.1 through our gates.


Revert "Revert "[LTO] Build lld on Windows CI""

This reverts commit 8c9649eb95751ba5f3cede15976b9c60e034eed3.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci please test

Build failed
Swift Test OS X Platform
Git Sha - 470332c7a257769d1f09c0c48158f9c7f2338e2c

@swift-ci please test macOS platform

Build failed
Swift Test OS X Platform
Git Sha - 470332c7a257769d1f09c0c48158f9c7f2338e2c

Hmm, please try again CI 🙏

@swift-ci please test macOS platform

Build failed
Swift Test OS X Platform
Git Sha - 470332c7a257769d1f09c0c48158f9c7f2338e2c

@swift-ci please test

@swift-ci please test

Obrigado

@swift-ci please test Windows platform


Rename generate-loc-drop and consume-loc-drop

to:

generate-localization-drop / consume-localization-drop
generate-localization-archive / consume-localization-archive
generate-localization-zip / consume-localization-zip

Votes?

I like archive


fix: false positives for typescript type + value export

Fixes #1318

This seems like it might overlap with #923?

Coverage decreased (-0.1%) to 97.764% when pulling abb354a70ae13646176dfb4011812ca0e62696ba on bradzacher:export-ts-type-value into 70a59fe1880b4061cd7c0f032c0b05c728223a0f on benmosher:master.

@ljharb to prevent double commenting - https://github.com/benmosher/eslint-plugin-import/issues/1318#issuecomment-482433011


Windows: Import normalization can incorrectly resolve as both in and out of root

Describe the bug
On Windows, vite can incorrectly resolve the svelte runtime as both in and out of root, which causes two instances of the svelte runtime to be loaded. This issue does not seem to appear in Unix systems.
In the reproduction logs below, we can see:

In root: importing svelte/internal from @svitejs/vite-plugin-svelte and from svelte-hmr is resolved as /node_modules/svelte/internal/index.mjs
"Out of root": importing ./internal/index.mjs from inside svelte/index.mjs is resolved as<EMAIL_ADDRESS>
In the browser's network tab:
http://localhost:3000/@fs/Users/GrygrFlzr/Documents/projects/vite-windows-imports/node_modules/svelte/internal/index.mjs?v=d0b9e871
http://localhost:3000/node_modules/svelte/internal/index.mjs?v=d0b9e871

The problem is likely in the import normalization:
https://github.com/vitejs/vite/blob/76f35352dbfc4e16cb65005e57e2757a97675bfd/packages/vite/src/node/plugins/importAnalysis.ts#L182-L192
Reproduction
Tested with npm, yarn, and pnpm - all reproduce the bug.
https://github.com/GrygrFlzr/vite-windows-imports
The reproduction makes use of svelte's setContext to better illustrate the bug causing two svelte runtimes to be loaded.
npm
npm i
npm run dev

Browser error:
Failed to init component
<App>
Error: Function called outside component initialization
    at get_current_component (http://localhost:3000/node_modules/svelte/internal/index.mjs?v=ad9e2cfc:649:15)
    at setContext (http://localhost:3000/node_modules/svelte/internal/index.mjs?v=ad9e2cfc:679:5)
    at instance (http://localhost:3000/src/App.svelte?import:69:2)
    at init (http://localhost:3000/@fs/Users/GrygrFlzr/Documents/projects/vite-windows-imports/node_modules/svelte/internal/index.mjs?v=ad9e2cfc:1474:11)
    at new App (http://localhost:3000/src/App.svelte?import:83:3)
    at createProxiedComponent (http://localhost:3000/node_modules/@svitejs/vite-plugin-svelte/node_modules/svelte-hmr/runtime/svelte-hooks.js:245:9)
    at new ProxyComponent (http://localhost:3000/node_modules/@svitejs/vite-plugin-svelte/node_modules/svelte-hmr/runtime/proxy.js:240:20)
    at new Proxy<App> (http://localhost:3000/node_modules/@svitejs/vite-plugin-svelte/node_modules/svelte-hmr/runtime/proxy.js:340:11)
    at http://localhost:3000/src/entry-client.js:3:13

System Info

vite version: 2.0.5
Operating System: Windows 10 10.0.19042
Node version: 14.15.3
Package managers: npm 7.5.2 / yarn 1.22.10 / pnpm 5.18.1

Yes, I approved it, after updating svelte@next to newer version, it falls.

I've located the specific file and line which causes the stripping of C: from the path:
https://github.com/vitejs/vite/blob/76f35352dbfc4e16cb65005e57e2757a97675bfd/packages/vite/src/node/utils.ts#L121-L126
Line 122's usage of the deprecated url.parse method when used on<EMAIL_ADDRESS>produces:
{
  protocol: 'c:',
  slashes: null,
  auth: null,
  host: '',
  port: null,
  hostname: '',
  hash: null,
  search: null,
  query: null,
  pathname<EMAIL_ADDRESS>  path<EMAIL_ADDRESS>  href<EMAIL_ADDRESS>}

Incorrectly assuming that c: is a protocol.
For the specific case of the minimal reproduction, it can be fixed using the recommended url.pathToFileURL...
-import { parse as parseUrl } from 'url'
+import { pathToFileURL } from 'url'

 // ...

 export function injectQuery(url: string, queryToInject: string) {
-  const { pathname, search, hash } = parseUrl(url)
+  const { pathname, search, hash } = pathToFileURL(url)
   return `${pathname}?${queryToInject}${search ? `&` + search.slice(1) : ''}${
     hash || ''
   }`
 }

...but this causes a lot of other tests to fail.


Abbreviated blobs as a Json string of their size

Instead of a shortened encoded string (e.g., "encoding=base64;blah"), blobs are a json string containing their size e.g., "{\"bytes\":123}". Formatting it like this also allows for adding in more blob info to the json later if necessary.
Fixes https://github.com/iTwin/itwinjs-backlog/issues/702
itwinjs-core: https://github.com/iTwin/itwinjs-core/tree/mattbjordan/update-abbreviated-blobs-tests

/azp run imodel-native


chore: bump versions of the containerd-shim-wasm to 0.4 and shims to 0.3

Waiting for a new release of containerd-shim crate https://github.com/containerd/rust-extensions/pull/216#event-10811692644 which adds timestamps for logs.

Bump containerd-shim to 0.6.0 to include structured logging. This PR will close #349

it won't close it but enable it.  The PR will include timestamps on our logs though 🥳

LGTM


Errors with multiline comments in haml_lint 0.18.0 (regression)

When running haml_lint 0.17.1 on this project, there are no errors:
https://github.com/braingourmets/crud
However, when I update haml_lint from 0.17.1 to 0.18.0, I get errors for (valid) multiline comments:
haml-lint app/views/
app/views/admin/dogs/edit.html.haml:3 [W] RuboCop: unexpected token tCONSTANT
(Using Ruby 2.3 parser; configure using `TargetRubyVersion` parameter, under `AllCops`)
app/views/admin/dogs/edit.html.haml:3 [W] RuboCop: unexpected token tCONSTANT
(Using Ruby 2.3 parser; configure using `TargetRubyVersion` parameter, under `AllCops`)
app/views/admin/dogs/edit.html.haml:6 [W] RuboCop: unexpected token kDO_COND
(Using Ruby 2.3 parser; configure using `TargetRubyVersion` parameter, under `AllCops`)
app/views/admin/dogs/edit.html.haml:6 [W] RuboCop: unexpected token kEND

etc.
This is one example of the affected code:
https://github.com/braingourmets/crud/blob/master/app/views/admin/dogs/edit.html.haml#L3

Thanks for the report, @oliverklee!
Duplicate of #148


Add option for customizable product

I am trying to implement this. the only problem is that cart is grouping two products as same even those two have different customization request

Do they have different options? If so, this would be a bug as product with different options are not supposed to be grouped together.

no, I don't think they have a different option. I added extra field "customization_info". content of this field is diffrent

Take a look at https://github.com/mirumee/saleor/blob/e9bf520560094fb9817887cbf44834ecf3e0d758/saleor/cart/models.py#L164-L171

I tried to change the value of data for the different order of same product variant. Now they aren't grouped together. But when trying to delete a line from the cart or update the quantity value, nothing happens. because front end logic is unable to differentiate between two line of the same variant. as there are only two parameters in the ajax request, variant id, and quantity.

@vikas-0 it sounds like you're trying to do something similar to what I am in #1080
I've made some progress on this and will have a PR up shortly if you're interested...

@amorphic I can't see any fork in your account

@amorphic @vikas-0 I'm starting to take a look at saleor to implement it for my business and, wow, this issue (together with #1080 ) is something that I will need.
You both seems to have been doing some progress towards this. Can you link your changes to me? I want to take a look at it and see if I can contribute in some way so we can get this feature in saleor ASAP!

@amorphic can you guide me how to store data after the check out as the cart is cleared after chekout.

@hackedbellini https://github.com/vikas-0/saleor/tree/data-passing in this branch, I did create a form in product detail which take the json input of customization detail. and finally saved in the order. still there are front end issues like can't delete an item-line from cart if custom data is given.

@vikas-0 hrm, after seeing your changes, I think what i need is actually something different from that. I'll probably have to discuss that in another issue. But anyway, good luck in your journey here!

@vikas-0 @hackedbellini In this branch I've successfully added a new "custom" attribute class to Saleor. I've followed Saleor conventions as closely as possible and custom line items are treated separately in the shopping cart, both for stock management and on deletion.
It all works, I just need to add some tests before submitting a PR for consideration. In the meantime if you want to check out what I've done and provide any feedback please go ahead.
I still have to add some tests and then I'll submit a PR for consideration.


Revert "WIP Store screenshots and log in a volume"

Reverts fabric8io/fabric8-ui#1165
oops I merged to early

There were the following issues with your Pull Request

Commit: a01711e5ef7565d3a2980f2eaad50e11baeaceed
Commits must be in the following format: %{type}(%{scope}): %{description}
Your subject line is longer than 50 characters

Guidelines are available at https://github.com/fabric8io/fabric8-ui

This message was auto-generated by https://gitcop.com


Update order-status.md

I have corrected errors in the text.
Scenario F Over Shipment is strange. Quantity Shipped and Invoice is bigger than Ordered and Produced.
How should a quantity taken from stock be reported?
Should it be included in Produced or should it be reported with a separate context?

You should put changes in a new folder with a new expected version. According to what I have seen, we are talking about patch changes => 1.0.1.

The idea of our versioning scheme is to keep the history of the standard within the "master" branch in folder structure where each (semantic) version has its own folder. Once such a folder has been committed/merged into the "master" branch, it must never change. So your change should be within a folder 1.0.1 which should contain the entire standard including your changes.

Closed because included in PR #30


Add a missing CHECK line to OSLogFullOptTest.swift

This test was sensitive to arbitrary block ID's because it was missing
the CHECK line that sets a regex used by subsequent checks.
This is also an example of why it helps to use CHECK-LABEL at the end
of a function scope.

@swift-ci smoke test and merge

@swift-ci smoke test and merge


BGP advertising doesn't honor externalTrafficPolicy Local for LoadBalancer services

For service type LoadBalancer and externalTrafficPolicy: Local, Calico will advertise routes from all nodes, regardless of whether valid endpoints exist.
A calico-node running in an host which has endpoints for that service will learn the endpoint IP address:
ubuntu@deployer:~$ klo -n kube-system calico-node-nsjfq  | grep 'default/my-service'
2022-05-03 22:16:24.089 [DEBUG][78] confd/routes.go 171: getEndpointsForService: service for endpoint not found, passing key="default/my-service"
2022-05-03 22:16:24.135 [DEBUG][78] confd/routes.go 171: getEndpointsForService: service for endpoint not found, passing key="default/my-service"
2022-05-03 22:16:24.200 [DEBUG][78] confd/routes.go 417: Advertising local service svc="default/my-service"
2022-05-03 22:16:24.200 [DEBUG][78] confd/routes.go 206: Checking routes for service advertise=true svc="default/my-service"
2022-05-03 22:16:24.200 [DEBUG][78] confd/routes.go 294: Setting routes for key key="default/my-service" routes=[]string{"<IP_ADDRESS>/32"}

On the other hand, a calico-node that has no endpoints for that service will recognize it correctly, as seen in the logs:
ubuntu@deployer:~$ kubectl logs -n kube-system calico-node-zhl2r  | grep 'default/my-service'
2022-05-03 22:16:24.088 [DEBUG][69] confd/routes.go 171: getEndpointsForService: service for endpoint not found, passing key="default/my-service"
2022-05-03 22:16:24.122 [DEBUG][69] confd/routes.go 171: getEndpointsForService: service for endpoint not found, passing key="default/my-service"
2022-05-03 22:16:24.190 [DEBUG][69] confd/routes.go 422: Skipping service with no local endpoints svc="default/my-service"
2022-05-03 22:16:24.190 [DEBUG][69] confd/routes.go 206: Checking routes for service advertise=false svc="default/my-service"

However, both calico-nodes have the same bird static configuration so both of them advertise all the prefixes:
[root@node4 /]# cat /etc/calico/confd/config/bird_aggr.cfg
# Generated by confd

protocol static {
   # IP blocks for this host.
   route <IP_ADDRESS>/24 blackhole;
   # Static routes.
   route <IP_ADDRESS>/32 blackhole;
   route <IP_ADDRESS>/32 blackhole;
   route <IP_ADDRESS>/32 blackhole;
}

Peer receives routes from all nodes:
<IP_ADDRESS>/32      unicast [Node_10_0_1_74 18:28:59.532 from <IP_ADDRESS>] * (100/?) [i]
	via <IP_ADDRESS> on ens3
                     unicast [Node_10_0_1_130 18:36:22.534 from <IP_ADDRESS>] (100/?) [i]
	via <IP_ADDRESS> on ens3
                     unicast [Node_10_0_1_254 18:28:59.883 from <IP_ADDRESS>] (100/?) [i]
	via <IP_ADDRESS> on ens3

Expected Behavior
Only one calico-node should advertise the route so that trafficPolicy is honored, as the documentation does not state otherwise.
Current Behavior
All calico-nodes are advertising the routes, regarding if they have a valid endpoint for the service, similar to externalTrafficPolicy: Cluster
Steps to Reproduce (for bugs)

Setup BGP Service advertising according to documentation.
Create a simple 1-pod deployment.
Create a service of type LoadBalancer and externalTrafficPolicy Local:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  externalTrafficPolicy: Local
  selector:
    app: nginx
  ports:
    - protocol: UDP
      port: 80
      targetPort: 80
  type: LoadBalancer
  loadBalancerIP: <IP_ADDRESS>

Check the received routes on the BGP peer

Your Environment

Calico version: 3.21.2
Orchestrator version (e.g. kubernetes, mesos, rkt): kubernetes 1.21.1
Operating System and version: Ubuntu 20.04

Hi @cubagabriel , thanks for raising this. I was wondering if you could share your BGP peers and BGP configuration as well? This definitely seems like a bug, but I want to make sure that I completely understand your setup so I can figure out how we got here in the first place.

Hi @mgleung , I tried different configurations but for reference consider the following:
apiVersion: projectcalico.org/v3
kind: BGPConfiguration
metadata:
  name: default
spec:
  asNumber: 65001
  serviceLoadBalancerIPs:
   - cidr: <IP_ADDRESS>/32
   - cidr: <IP_ADDRESS>/32
   - cidr: <IP_ADDRESS>/32

apiVersion: projectcalico.org/v3
kind: BGPPeer
metadata:
  name: bgppeer-global
spec:
  keepOriginalNextHop: true
  peerIP: <IP_ADDRESS>
  asNumber: 65002

I even tried setting a route reflector topology and disabling full mesh, with the same result.

I did investigate a bit around this myself. It seems to be semi-working when pods moves around.
My test case:
Daemonset running on three workers with a service using externalTrafficPolicy: Local
Routes are advertised from all three, as expected.
When doing a change causing the pod to stop on one of the workers, the route no longer advertised from that worker.
And if you're restarting the pod, it's being advertised again.
The problem seems to be when applying a new BGP configuration to an existing service, it gets advertised from all workers.
Calico: 3.21.1

I believe the problem is including /32 addresses in the serviceLoadBalancerIPs section - the code doesn't account for that right now. @mtryfoss does your BGPConfiguration include /32 addresses as well in that section?

Yes, we're also using /32. But like I mentioned, this seems to work as long as setRouteForSvc() is triggered by a change in one of the endpoints, and not in the service itself (svc nil, ep non-nil). Tried to do a custom build with a bit more verbose debug, but some step there failed.
Earlier we had troubling getting it to advertise using the full /29 we've assigned to that cluster, so ended up using /32.
I've studied that part of the code a bit. For me it almost looks like these are being "hard added" when config is applied somehow. Since it's not been advertised by setRouteForSvc(), it's not withdrawn either when it sees that there's no endpoint locally for that given services.
Will try investigating a bit more next week.

For me it almost looks like these are being "hard added" when config is applied somehow.

This is exactly right. The semantics right now are:

Advertise the prefixes provided in serviceLoadbalanacerIPs unconditionally from all nodes (this handles Cluster type svcs)
If a service has externalTrafficPolicy=Local, advertise its address as a /32 route from nodes that have the service running.

This is correct assuming the prefixes in serviceLoadbalancerIPs are not fully qualified (e.g., /32 or /128), but incorrect when using a precisely specified service IP as you are in this case.
To handle /32 or /128 routes, I think the logic needs to be changed to this:

Advertise non-fully-qualified serviceLoadbalancerIPs from every node (e.g., not /32 or /128)
For fully-qualified serviceLoadbalancerIPs entries, advertise them from every node if the service is cluster-type, otherwise
For local-type services, advertise a /32 from every node that hosts the address.

^ This is just a slightly different expression of https://github.com/projectcalico/calico/issues/6074#issuecomment-1137477343

Please have a look at my PR.
It should work more or less as you proposed, except that it will only announce services of type loadbalancer from nodes which do have endpoints regardless if cluster or local-type. Frankly, I don't see any reason not to behave like this for that case? Loadbalancer is most likely something that is not handled on all nodes anyway - for example an ingress controller.
I got fooled by another bug regarding /32 and larger definitions in the config section. We tried different configurations (switching between /32 and /29) for existing services, and that caused the /32 route from a specific node not to be announced until something on the service was changed. That should also be sorted out in the same patch.

except that it will only announce services of type loadbalancer from nodes which do have endpoints regardless if cluster or local-type

This is expressly contrary to what the externalTrafficPolicy API is asking for with a Cluster service, though. Users who use Cluster type services want to load balance across all nodes in their cluster, and so we should be advertising those IPs from all nodes for those services.
I'll take a look at your PR - it's already on my queue - but I don't believe the behavior you're describing for "Cluster" services is correct.

Would it be ok to announce services with externalTrafficPolicy: Cluster from all nodes as /32s then? Basically, skip check for endpoints.
At the point you're currently adding global routes we've got no clue of what type that route might belong to.

Would it be ok to announce services with externalTrafficPolicy: Cluster from all nodes as /32s by the logic defined in router.go then

I'm curious why you would want this? Could you explain what you're thinking a bit more?
These are the scenarios I am thinking about:

We have a LoadBalancer CIDR in BGPconfig and several cluster-type services within that CIDR. We can cover all of those serivces by advertising a single route for the entire LB CIDR.
We have a LoadBalancer CIDR in BGPconfig, and several cluster-type services, as well as a few local-type services. The cluster-type services are handled by a single route (the whole LB CIDR) and the local-type services advertise more specific /32 routes which override the full CIDR.
We have a LoadBalancer IP (not a full CIDR) in BGPconfig, and a cluster-type service that matches it. We advertise the /32 from all nodes.
We have a LoadBalancer IP (not a full CIDR) in BGPconfig, and a local-type service that matches it. We advertise the /32 from nodes that have the service running.

So, specifically for (1) and (2), switching to always advertising a /32 route for cluster-type services would result in us advertising more routes than we need to. The tradeoff, I suppose, is that we are advertising a full range, but that should be OK given that range is by definition reserved for LB IPs anyway and should match the config given to the allocator anyway.

Maybe I'm getting this wrong since our use case is a bit limited, but my main reason was to keep things simple stupid.
Let's focus on the thing this patch is solving. Entries of single IP-adresses (/32).
We can't announce these globally, since what will break services requesting the Local type, so the simple solution here is not to announce any of them globally.
In router.go we do handle changes to the service or the endpoints of the service. Here we don't know if the assigned IP-address is a /32 or part of a bigger subnet definition. We could of course extend the check against whitelisted addresses and return if it's an exact match or part of a subnet, but it kind of feels more robust to just do the more specific announcements.
I know that answer might vary depending on the environment. In our case we also do have routers in front of the k8s cluster doing aggregation, so some extra routes should not be a big issue.

I'm in favor of the simplest solution, but within the bounds set by the requirements. In this case, we need to maintain the "aggregation" behavior for non-32 CIDRs. We do have some users relying on that behavior today and this would be a breaking change for them.
As you've said, that does mean we need some additional smarts within the code to determine if a cluster-type service IP belongs to some greater CIDR (i.e., already advertised) or if it matches an exact IP within the given serviceLoadbalancerIPs (i.e., we need to advertise a /32 from every node).
However, I think this logic should be fairly straightforward to insert into the existing advertisement logic and doesn't introduce too much complexity. We can:

Skip advertisement of any /32 CIDRs we find in serviceLoadbalancerIPs.
When we get a cluster-type service, compare its address to serviceLoadBalancerIPs and advertise if it matches a /32.
When we get a local-type service, maintain existing behavior (advertise from nodes that have endpoints).

The more complex part will be handling changes to serviceLoadbalancerIPs - e.g., if a /32 is removed and we need to withdraw it or a /32 is added and we now need to program it. From a code read, it looks like this might already be a gap in the existing code, so perhaps we can tackle this problem separately.

I see your point. I've updated the PR with changes that hopefully reflect the scenarios you describe.
Regarding the last part. I think this is already handled by my patch? If not, please elaborate.
I noticed the issue during testing when switching between /29 and /32 definitions.
Please have a look at this part:

As far as I understand. A config change basically writes all the entries from scratch, and then does c.rg.TriggerResync() to trigger iteration of services. The extra code above will make sure that all routes the route generator has added earlier also exists as static routes after the change.


transfers.lua:129: attempt to index local 'inboundTransfer' (a nil value)

[ERROR] addons/libk/lua/libk/3rdparty/glib/transfers/transfers.lua:129: attempt to index local 'inboundTransfer' (a nil value)
  1. unknown - addons/libk/lua/libk/3rdparty/glib/transfers/transfers.lua:129
   2. fn - addons/libk/lua/libk/3rdparty/glib/transfers/vnet.lua:1611
    3. unknown - addons/ulib/lua/ulib/shared/hook.lua:110

I'm sure the error message is better off here, as the message was apparently triggered by LibK or an addon that uses LibK (example: Pointshop 2).
It could be that the error was already fixed, since the message is already older.
A nil check is probably not the best solution in this case, since this error can only occur if messageType is not 1 or 2.

Should have been fixed in a GLib update, please reopen if you run across this again


Authentication revamp

Move the password hashing to the client, and use websockets for the communication.

This is now becoming "Finish authentication and users". The existing code has been migrated to WS.


ADD: Method to User object so that the local variables of User's move…

…ments can be updated

dope, beautiful, yes.


at workbench.desktop.main.js:sourcemap:961
    at Array.forEach (<anonymous>)
    at S.push (workbench.desktop.main.js:sourcemap:961)
    at workbench.desktop.main.js:sourcemap:1043
    at Array.forEach (<anonymous>)
    at workbench.desktop.main.js:sourcemap:1043
    at g.runner (workbench.desktop.main.js:sourcemap:5513)
    at g.doRun (workbench.desktop.main.js:sourcemap:151)
    at g.onTimeout (workbench.desktop.main.js:sourcemap:151)
add @ workbench.desktop.main.js:sourcemap:109
workbench.desktop.main.js:sourcemap:1597   ERR timeout after 500 ms: Error: timeout after 500 ms
    at t.RawDebugSession.handleErrorResponse (file:///C:/Users/User/AppData/Local/Programs/Microsoft VS Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2892:728)
    at file:///C:/Users/User/AppData/Local/Programs/Microsoft VS Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2892:159
    at async t.RawDebugSession.shutdown (file:///C:/Users/User/AppData/Local/Programs/Microsoft VS Code/resources/app/out/vs/workbench/workbench.desktop.main.js:2890:248)
workbench.desktop.main.js:sourcemap:109 Error: Trying to add a disposable to a DisposableStore that has already been disposed of. The added object will be leaked!
    at l.add (workbench.desktop.main.js:sourcemap:109)
    at S._register (workbench.desktop.main.js:sourcemap:109)
    at workbench.desktop.main.js:sourcemap:961
    at Array.forEach (<anonymous>)
    at S.push (workbench.desktop.main.js:sourcemap:961)
    at workbench.desktop.main.js:sourcemap:1043
    at Array.forEach (<anonymous>)
    at workbench.desktop.main.js:sourcemap:1043
    at g.runner (workbench.desktop.main.js:sourcemap:5513)
    at g.doRun (workbench.desktop.main.js:sourcemap:151)
    at g.onTimeout (workbench.desktop.main.js:sourcemap:151)
add @ workbench.desktop.main.js:sourcemap:109
workbench.desktop.main.js:sourcemap:2720 Failed to launch debugger for child process 6652
onDidNotificationChange @ workbench.desktop.main.js:sourcemap:2720
workbench.desktop.main.js:sourcemap:109 Error: Trying to add a disposable to a DisposableStore that has already been disposed of. The added object will be leaked!
    at l.add (workbench.desktop.main.js:sourcemap:109)
    at S._register (workbench.desktop.main.js:sourcemap:109)
    at workbench.desktop.main.js:sourcemap:961
    at Array.forEach (<anonymous>)
    at S.push (workbench.desktop.main.js:sourcemap:961)
    at workbench.desktop.main.js:sourcemap:1043
    at Array.forEach (<anonymous>)
    at workbench.desktop.main.js:sourcemap:1043
    at g.runner (workbench.desktop.main.js:sourcemap:5513)
    at g.doRun (workbench.desktop.main.js:sourcemap:151)
    at g.onTimeout (workbench.desktop.main.js:sourcemap:151)
add @ workbench.desktop.main.js:sourcemap:109

Please note that, everything works just fine under Pycharm for example. so this is very likely a vscode issue.

@Coderx7 if you want to debug files under site-library add "justMyCode": false in launch.json.
{
    "name": "Debug",
    "type": "python",
    "request": "launch",
    "program": "${file}",
    "console": "integratedTerminal",
    "justMyCode": false,
}

@karthiknadig Thanks a lot. adding "justMyCode": false to the launch.json fixed this issue


Improve Backend of Contact Page

Related Issue
Fixes #537
Description
@SUGAM-ARORA I have added the improvement of  Backend of Contact Page. You just have to enter your email address. Kindly review this.
Type of PR

[ ] Bug fix
[x] Feature enhancement
[ ] Documentation update
[ ] Other (specify): _______________

Screenshots / videos (if applicable)
[Attach any relevant screenshots or videos demonstrating the changes]
Checklist:

[x] I have performed a self-review of my code
[x] I have read and followed the Contribution Guidelines.
[x] I have tested the changes thoroughly before submitting this pull request.
[x] I have provided relevant issue numbers, screenshots, and videos after making the changes.
[x] I have commented my code, particularly in hard-to-understand areas.

Additional context:
[Include any additional information or context that might be helpful for reviewers.]

@SUGAM-ARORA Sir Branch Updation done.


Upgrade hyperscan 5.4.2

@gliwka - please take a look?

Superseded by changes on main to migrate to latest vectorscan release to enable arm64 on Linux and macOS


Refactor ICAddressBook and ICAddressBookEntry

The ICAddressBook is only a wrapper for an OrderedCollectionand could be also a collection holding e-mails instead of ICAddressBookEntryobjects.

Closed by #318


How can this flow be managed by an LLM Tooling ?

Reflexion involves resolving toward an chaining tool (or prompt engineering or anything that brings resolution, which is :  An aspiration to start from our intial intent for a creation, have LLM assistance in generating the request.  (the most promising prompt can be found at :  langsmith hub: jgwill/cmpenghelperbeta

musical.schema.v1b.json


QD hangs with CProxy_Array(gid).ckDestroy

Original author: Harshitha Menon
Original issue: https://charm.cs.illinois.edu/redmine/issues/1013

No body.

Original author: Harshitha Menon
Original date: 2016-03-16 19:25:47

When CProxy_Array(gid).ckDestroy is called, it causes all elements and location manager to be destroyed. When element is destructed, it sends a message to the home PE for reclaimRemote and since the location manager at the remote maybe destroyed, it buffers it. So to fix this, use a flag to check whether location manager is being destroyed entirely and if so dont send the reclaimRemote message.

Original author: Harshitha Menon
Original date: 2016-03-16 19:37:22

~~https://charm.cs.illinois.edu/gerrit/#/c/1114/~~ https://github.com/UIUC-PPL/charm/commit/5a92b2880d91e5108d34782a77b151143c22aa54


LLVM 11 compatibility

Still works with LLVM 10

Thanks a lot! I appreciate very much that you took the time to send a PR. Would you mind also updating the README to mention that both LLVM 10 and 11 work?

Done


Zone needs to support PATCH

To enable moving endpoints from one existing Zone to another for the PoC, we need to support PATCH functionality within the Zone object in the emulator.

Issue in wrong repo, moved to PoC Emulator Repo.


DZ2 updates

This PR does a couple of things:

[x] remove devices/existing-device/ completely, reasons below.
[x] add the activeOpacity prop to <ClickableBox>, so that <ClickableBox> can be used to back our <Icon>.
[x] add a new <KeyboardAvoidingBox> common-adapter that passes through to <KeyboardAvoidingView>, for use in PGP import.  I wanted to call the common-adapter KeyboardAvoidingView too, but couldn't find a way to export a name that you're also importing.

Adding a new common-adapter for a component that's native only felt like overkill.  If folks prefer, I can just let users import RN's <KeyboardAvoidingView> directly.  On the other hand, it was the last direct import of a RN component so it felt good to get rid of it..
After these changes, there are no unreasonable imports from 'react-native' in shared/.
Reason for removing devices/existing-device/: This was still hooked up as a responder in the kex2 flow, for provisionUi.chooseDeviceType.  But it was unused -- kex2 now calls chooseDevice, not chooseDeviceType, and the Add.. flow in the Devices tab, which also uses kex2, overloads chooseDeviceType to do something else when registering the kex2 handler.  So this code appeared to be still in use but wasn't.

Forgot to tag @keybase/react-hackers

👍 % Comments

@MarcoPolo thanks! made your changes

Closing this in favor of #4125.


Upgrade concrete5 Translation Library to v1.4.4

See

https://github.com/oscarotero/Gettext/releases/tag/v3.5.9
https://github.com/mlocati/concrete5-translation-library/releases/tag/1.4.3
https://github.com/mlocati/concrete5-translation-library/releases/tag/1.4.4

The brand new Community Translation addon is approaching :wink:, BTW people that may want to try it need this PR... @aembler What about reviewing it? Thanks!

Thank you, @aembler !


perf(balancer) use leader/followers workers on eventual

Execute costly code in only one worker (called the "leader", the one
with PID 0). This way, only the leader will fetch the list of upstreams
from the database. Since we're leveraging mlcache, once the leader
performs the db query and fetches the list of upstreams from the L3
callback, the other workers (called the "followers") will retrieve this
list from the L2 cache. This model should be eventually consistent.

IIRC, we are not planning on using this anymore, right @murillopaula?

@locao Correct. This should be addressed in the upcoming balancer eventual consistency optimizations. Closing this.