Add automated tests which validates oid returned from sys.triggers, sys.events, sys.objects, sys.all_objects same as object_id

Description
Add automated tests which validates oid returned from sys.triggers, sys.events, sys.objects, sys.all_objects same as object_id
Issues Resolved
Issues Resolved: Task: BABEL-5108, BABEL-3927
Test Scenarios Covered

Use case based -

Boundary conditions -

Arbitrary inputs -

Negative test cases -

Minor version upgrade tests -

Major version upgrade tests -

Performance tests -

Tooling impact -

Client tests -

Check List

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>1
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 73.661%

Totals

Change from base Build<PHONE_NUMBER>9:
0.0%

Covered Lines:
44139

Relevant Lines:
59922

💛  - Coveralls


Add OSGi bundle instructions to MANIFEST.MF. #811.

添加OSGi的Bundle声明，是该jar文件可以用于OSGi环境中。对现有的jar的使用不会有影响。

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Codecov Report

Merging #3438 into master will decrease coverage by 0.02%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master    #3438      +/-   ##
============================================
- Coverage     80.37%   80.34%   -0.03%
+ Complexity     7468     7463       -5
============================================
  Files           157      157
  Lines         27912    27912
  Branches       6731     6731
============================================
- Hits          22434    22427       -7
- Misses         3460     3465       +5
- Partials       2018     2020       +2

Impacted Files
Coverage Δ
Complexity Δ

...com/alibaba/fastjson/serializer/SerialContext.java
82.69% <0.00%> (-3.85%)
21.00% <0.00%> (-1.00%)

...main/java/com/alibaba/fastjson/util/FieldInfo.java
85.95% <0.00%> (-1.01%)
97.00% <0.00%> (-2.00%)

...ava/com/alibaba/fastjson/parser/JSONLexerBase.java
69.94% <0.00%> (-0.07%)
811.00% <0.00%> (-2.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 124fb5e...0a26985. Read the comment docs.

不会为osgi改变打包方式，osgi技术也不再流行


[RESOLVED] S100-Primario has been restarted (uptime < 10m)

Problem started at 14:56:59 on 2023.08.23
Problem name: S100-Primario has been restarted (uptime < 10m)
Host: S100-Primario
Severity: Warning
Operational data: 00:06:30
Original problem ID: 86440
Event details in Zabbix: http://<IP_ADDRESS>/tr_events.php?triggerid=23452&eventid=86440

Problem has been resolved in 3m 31s at 15:00:30 on 2023.08.23
Problem name: S100-Primario has been restarted (uptime < 10m)
Host: S100-Primario
Severity: Warning
Original problem ID: 86440


[uss_qualifier] scd sub-interactions: also search for any active subs at cleanup

Adds a missing search and deletion of possible active subscriptions in the setup and cleanup of SubscriptionInteractions.
This corresponds to both the original intent of how the scenario should be set and cleaned up, and to how the scenario was originally documented.
Resolve #809

The active subs cleanup may be a useful thing to do, but I don't think it does anything to address #809.  I think the root cause of #809 is that this query is not recorded in the scenario.

Oh my, I got a bit confused there. Will open a separate PR to address the recording of the query, and will leave this PR open (but remove the mention of the issue) as the cleanup step does document a subscription search query, so it still makes sense to add it.


Block non-local connections on mdns server

We create a tcp server for listening to mdns connections, but theoretically this could receive connections from outside the local network.
We should reject/block incoming connections from non-local addresses. We can maybe do this by checking socket.remoteAddress against private IP ranges with something like https://github.com/mafintosh/bogon
The reason for doing this is a security precaution, particularly if we are sending invites to peers discovered through mdns, and we want to be sure that incoming connections are local network only.

I was wondering how can we test this. Should we have a publicly accessible node that does mdns and connects to a predefined topic, so we can reject it on a test?

This should be fixed in the latest implementation


transform toTensor fails with PIL image

🐛 Bug

The torchvision.transformer function ToTensor raise a TypeError when receiving a image loaded with PIL.
The error origins from a call to np.array and contains the message:
__array__() takes 1 positional argument but 2 were given
To Reproduce
Steps to reproduce the behavior:
import io
import requests
import torchvision.transforms as T
from PIL import Image

resp = requests.get('https://picsum.photos/200')
img = Image.open(io.BytesIO(resp.content))

preprocess1 = T.Compose([
   T.ToTensor(),
])

x = preprocess1(img)
print(x.shape)

Pip Freeze Log:
certifi==2021.5.30
chardet==4.0.0
idna==2.10
numpy==1.21.0
Pillow==8.3.0
requests==2.25.1
torch==1.9.0
torchvision==0.10.0
typing-extensions==<IP_ADDRESS>
urllib3==1.26.6

Full Traceback:
Traceback (most recent call last):
  File "/Users/sebastianlettner/Desktop/tmp/test.py", line 13, in <module>
    x = preprocess1(img)
  File "/Users/sebastianlettner/Desktop/tmp/.venv/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 60, in __call__
    img = t(img)
  File "/Users/sebastianlettner/Desktop/tmp/.venv/lib/python3.8/site-packages/torchvision/transforms/transforms.py", line 97, in __call__
    return F.to_tensor(pic)
  File "/Users/sebastianlettner/Desktop/tmp/.venv/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 129, in to_tensor
    np.array(pic, mode_to_nptype.get(pic.mode, np.uint8), copy=True)
TypeError: __array__() takes 1 positional argument but 2 were given

Expected behavior
I expect that the conversion works and I reveive a torch.Tensor with shape (200, 200, 3)
Environment
Collecting environment information...
PyTorch version: 1.9.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: macOS 10.15.7 (x86_64)
GCC version: Could not collect
Clang version: 12.0.0 (clang-1<IP_ADDRESS>)
CMake version: Could not collect
Libc version: N/A

Python version: 3.8.10 (default, May  4 2021, 03:05:50)  [Clang 12.0.0 (clang-1<IP_ADDRESS>)] (64-bit runtime)
Python platform: macOS-10.15.7-x86_64-i386-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.21.0
[pip3] torch==1.9.0
[pip3] torchvision==0.10.0
[conda] Could not collect

PyTorch / torchvision Version (e.g., 1.0 / 0.4.0): 1.9.0 / 0.10.0
OS (e.g., Linux): macOS (10.15.7 (19H15))
How you installed PyTorch / torchvision (conda, pip, source): pip
Build command you used (if compiling from source): n/A
Python version: 3.8.10
CUDA/cuDNN version: n/A
GPU models and configuration: n/A
Any other relevant information:

Additional context
n/A

Thanks for the report, I can reproduce.
The issue seems to come from the PIL version. This passes with PIL 8.2, but fails with  PIL 8.3.
import numpy as np
from PIL import Image
img = Image.fromarray(np.random.randint(0, 256, (10, 10, 3), dtype=np.uint8))
np.array(img, np.int16)

As a temporary workaround for you, downgrade PIL to 8.2. However it would be nice to get this fixed.
Would you mind checking if a similar issue was raised in the PIL repo, and if not, submit one and ping me there? Thanks!!

lol: https://github.com/python-pillow/Pillow/issues/5571

Hi Nicolas, nice to hear that you can reproduce the issue. Let's see how they proceed in the PIL repo.
Cheers, Basti

@NicolasHug We should change
https://github.com/pytorch/vision/blob/993325dd82567f5d4f28ccb321e3a9a16984d2d8/setup.py#L70
to pillow_ver = ' >= 5.3.0, != 8.3.0'. Depending on how fast they fix it, this patch could make it into torchvision==0.10.1 (or are we not going to release a bug fix patch?). This way users can get the normal behavior back out of the box fairly soon.

It's up to PIL to submit a bugfix patch, not us :)
8.3 was released 8 hours ago and they're already working on a fix https://github.com/python-pillow/Pillow/pull/5572 which should be in soon.

True, but we know that our stuff will not work with 8.3.0. Meaning, if we don't exclude it pip install torchvision pillow==8.3.0 is valid.

Is this a common practice? I've never seen it.
I understand in this case this is a question of compatibility between torchvision and PIL, but if we pushed the logic further, we might as well exclude all PIL versions that have known bugs, a.k.a only accept the very latest PIL?

Is this a common practice? I've never seen it.

I've seen it a few times, for example here

but if we pushed the logic further, we might as well exclude all PIL versions that have known bugs, a.k.a only accept the very latest PIL?

If we know that there are Pillow versions that will break our code, than yes, we should explicitly exclude them. Most of the time Pillow bugs will only degrade our quality.

thanks for the details, that sounds good then. I opened  #4148

The alternatives are:

Don't pass dtype to np.array,
detect the problem and monkey-patch array. Should be low risk to check for Image.__array__ and replace it with a wrapper.

@t-vi

Don't pass dtype to np.array,

I don't know for sure, but the extra stuff suggests that we need to do that:
https://github.com/pytorch/vision/blob/993325dd82567f5d4f28ccb321e3a9a16984d2d8/torchvision/transforms/functional.py#L126-L130

detect the problem and monkey-patch __array__. Should be low risk to check for Image.__array__ and replace it with a wrapper.

If pillow publishes a release with your patch before we do, this would be irrelevant. I imagine they will release a small patch a lot sooner than we do.

Just ran in to this with adjust_hue
Traceback (most recent call last):
  File "testhue.py", line 6, in <module>
    F.adjust_hue(im, 0.1)
  File "/Users/jamesthewlis/miniconda3/envs/unitary/lib/python3.8/site-packages/torchvision/transforms/functional.py", line 844, in adjust_hue
    return F_pil.adjust_hue(img, hue_factor)
  File "/Users/jamesthewlis/miniconda3/envs/unitary/lib/python3.8/site-packages/torchvision/transforms/functional_pil.py", line 96, in adjust_hue
    np_h = np.array(h, dtype=np.uint8)
TypeError: __array__() takes 1 positional argument but 2 were given

pillow                    8.3.0            py38hee640a0_0    conda-forge

Re-opening temporarily as other users might bump into this issue, this will hopefully help avoid duplicated entries like https://github.com/pytorch/vision/issues/4152

Pillow 8.3.1 should now be released with a fix for this.

Pillow 8.3.1 should now be released with a fix for this.

We can close this now. Unless someone explicitly specifies Pillow==8.3.0, pip install torch torchvision will now install a compatible Pillow version.


Document that read_manifest command is deprecated

I believe we intended to deprecate read_manifest command. I am not sure what a deprecation process for Cargo commands should be, but I guess it should involve mentioning somewhere that the command is deprecated :)

r? @alexcrichton
(rust_highfive has picked a reviewer for you, use r? to override)

@bors: r+

:pushpin: Commit fdbca1c has been approved by alexcrichton

:hourglass: Testing commit fdbca1c with merge 6428537...

:sunny: Test successful - cargo-cross-linux, cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-gnu-32, cargo-win-gnu-64, cargo-win-msvc-32, cargo-win-msvc-64
Approved by: alexcrichton
Pushing 642853783da66d19efe5b4d3305cb424218f49ee to master...


for range loops one time more than it should

Thanks for the library! Just to let you know, while playing with the code I encountered some problems regarding closing when using the channel in a for range loop:
A for range loop iterates once on an empty channel when that channel is closed in another thread.
More general: If you rely on the end iterator to finish a loop it comes one iteration too late.
The reason for this behaviour is that blocking while waiting for data happens in operator*. When the block is released upon closing of the channel (not because data has arrived), an undefined value is retrieved and returned.
This can be avoided when the range loop checks for channel validity like this:
1  for (auto in : incoming) {
2      // Check if we are done
3      if (incoming.empty() && incoming.closed()) {
4          break;
5      }
6      ...
7  }

But that a) looks ugly and b) can still cause errors (data not processed) when closing happens between line 1 and 3 in a different thread.

Great feedback, thank you!
I have plans to continue working on the library next year. This issue will remain open until I find a solution.
If suggestions come to your mind, they are more than welcome (even as PRs).

I thought about it and the only solutions that seems to work is kinda weird. I assume that the for range loop works as follows:

see if operator!= is true and if so break
call operator* and use its result in loop
call operator++
repeat
Since only after operator<< it is clear if we have a valid result or are in a closed&&empty state, that must be called in operator!=.

This works:
template <typename Channel>
class BlockingIterator {
public:
    using value_type = typename Channel::value_type;
    explicit BlockingIterator(Channel& ch) : ch{ch} {}
    BlockingIterator<Channel> operator++() const noexcept { return *this; }
    value_type operator*() const noexcept { return value; }
    bool operator!=(BlockingIterator<Channel>) noexcept {
        value << ch;
        return !(ch.closed() && ch.empty());
    }
private:
    Channel& ch;
    value_type value{};
};

But I don't think it is a good solution, because the semantics are mixed up. operator!= cannot be called multiple times without messing with the queue's content. And there is still a race condition here:
value << ch;
return !(ch.closed() && ch.empty());

I will give this a thought when I will work on the issue. Thanks!

Could you test the fix-iterator-off-by-one-issue branch and see if you can still reproduce the issue?

Hey, thanks! I will test that early January. Happy holidays! :)

If there is nothing else to add, I will close the issue soon and merge #9 .

Thank you for the feedback!


TimeoutError: timed out during handshake  |

I am facing below very frequently, mostly 1 out 20 times
    ws = connect(
/usr/local/python/python-3.10/std/lib64/python3.10/site-packages/websockets/sync/client.py:289: in connect
    connection.handshake(
/usr/local/python/python-3.10/std/lib64/python3.10/site-packages/websockets/sync/client.py:85: in handshake
    raise TimeoutError("timed out during handshake")
E   TimeoutError: timed out during handshake

The way I am connecting to websocket:
from websockets.sync.client import connect, ClientConnection
# Maximum size of incoming messages in bytes.
WS_MAX_SIZE = 2**30
# open_timeout: Timeout for opening the connection in seconds.
WS_OPEN_TIMEOUT = 30

        try:
            ws = connect(
                url,
                additional_headers=headers,
                max_size=WS_MAX_SIZE,
                open_timeout=WS_OPEN_TIMEOUT,
            )
            info(f"Connection Established Successfully for instance: {instance_id}")
        except Exception as e:
            error(f"Error connecting to websocket {url}: {e}")
            raise e

What is the right way to make a websocket connection? I mean what are appropriate arguments that I can supply so that it happens mostly with ease and successfully, I am seeing lot of Timeout failures in handshake. Do we not have internal retry mechanisms etc.?
The version of websockets library I am using.
✗ pip show websockets
Name: websockets
Version: 11.0
Summary: An implementation of the WebSocket Protocol (RFC 6455 & 7692)
Home-page: None
Author: None
Author-email: Aymeric Augustin<EMAIL_ADDRESS>License: BSD-3-Clause
Location: /usr/local/python/python-3.10/std/lib/python3.10/site-packages
Requires:
Required-by: pyppeteer, gs-quant

Sharing the Debug logs on where it is breaking..
[20230728 04:28:02:258 protocol.py:167 DEBUG] = connection is CONNECTING
[20230728 04:28:02:259 client.py:307 DEBUG] > GET /bifrost HTTP/1.1
[20230728 04:28:02:259 client.py:309 DEBUG] > Host: <myUrl>
[20230728 04:28:02:259 client.py:309 DEBUG] > Upgrade: websocket
[20230728 04:28:02:259 client.py:309 DEBUG] > Connection: Upgrade
[20230728 04:28:02:259 client.py:309 DEBUG] > Sec-WebSocket-Key: im4HXckXkAvff+mYYfXeTg==
[20230728 04:28:02:259 client.py:309 DEBUG] > Sec-WebSocket-Version: 13
[20230728 04:28:02:259 client.py:309 DEBUG] > Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
[20230728 04:28:02:259 client.py:309 DEBUG] > X-ANTICSRF-HEADER: DESCO
[20230728 04:28:02:259 client.py:309 DEBUG] > Authorization: Negotiate YIICrwYJKoZIhvcSAQICAQBuggKeMIICmqADAgEFoQMCAQ6iBwMFACAAAACjggGmYYIBojCCAZ6gAwIBBaERGw9VTklYLkRFU0hBVy5DT02iTDBKoAMCAQOhQzBBGwRIVFRQGzl1LWd1YXN0ZXN0LWtoZXJhLWJpZnJvc3Qtc2NoZW1hLWJpZnJvc3QtYS5zZGxiLmRlc2hhdy5jb22jggE0MIIBMKADAgESoQMCAQGiggEiBIIBHiAzDO5vOoALUVHPKJKVyPiI62eBxjenjbqhdtlwnHMLNnW6Mil2TLkoAuQahs7PTQc4MtxdjNGb450+sfSWHbPIhApqo+3gNjzgKVmvDAmgaeeHziVEmMElouzOdD7/kD6qr2ol8ufyNfyxxeExSSCcA/39CSadr4dMmlmY3awVelgD/ZIcBO3xxbs1BSI6lQBA6nAbUz/tOuMm3I1kAAINrt8k1HlAs2foohxqFRe49tG7+LdglBO2GDhBAFrjXO5suV7Lifg74ip2iT4wJMKEySY6RvAapI2r58KH+9JgY5AnxqRTsQg4n2Xv2nMEraf/Oa1IAVv8OA+XNCG/ow7SqXRW4J2VTtcqXI5acptPxzSYG2kVGUrABx9SfgqkgdowgdegAwIBEqKBzwSBzCHnIyGAHc281q9M378o9snTeLoVh5R0+wPnNJd//sfrHwxbYuwFlrdqmnnKcGpaoU7eDcTKvtVu7zDmQX6jlr/paV8ZIPpd0mFvH43ECSSmEARSxw5WZEBKa85zpqt2kiHV5uPy+FiYAAiyrMKXsNLzeA7YCKqQUXCtTpN+jbuS7m6cZVzk9h/NNbEE2AWVTswY3C8JatsYDaQi29uKE+PINIPWLToyGfACycYcxhxDBYqkX6pdsyR2Yzxml923Vttaccl3vOP8bBxSZA==
[20230728 04:28:02:259 client.py:309 DEBUG] > User-Agent: Python/3.10 websockets/11.0
[20230728 04:28:32:249 protocol.py:596 DEBUG] < EOF
[20230728 04:28:32:249 protocol.py:706 DEBUG] > EOF
[20230728 04:28:32:250 protocol.py:167 DEBUG] = connection is CLOSED
[20230728 04:28:32:250 bifrost_client.py:337 ERROR] : timed out during handshake

Just curious to know, I was checking the logs for the case when we are able to open the connection properly:
[20230728 04:28:02:218 bifrost_client.py:79 INFO] Primary Bifrost Server URL: wss://u-guastest-khera-bifrost-schema-bifrost-a.sdlb.deshaw.com/bifrost, Service Prefix: u-guastest-khera-bifrost-schema-bifrost, Primary Instance ID: a
[20230728 04:28:02:233 protocol.py:167 DEBUG] = connection is CONNECTING
[20230728 04:28:02:234 client.py:307 DEBUG] > GET /bifrost HTTP/1.1
[20230728 04:28:02:234 client.py:309 DEBUG] > Host: u-guastest-khera-bifrost-schema-bifrost-a.sdlb.deshaw.com
[20230728 04:28:02:234 client.py:309 DEBUG] > Upgrade: websocket
[20230728 04:28:02:234 client.py:309 DEBUG] > Connection: Upgrade
[20230728 04:28:02:234 client.py:309 DEBUG] > Sec-WebSocket-Key: GtcfBek8eLkedheDG89RGQ==
[20230728 04:28:02:234 client.py:309 DEBUG] > Sec-WebSocket-Version: 13
[20230728 04:28:02:234 client.py:309 DEBUG] > Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
[20230728 04:28:02:234 client.py:309 DEBUG] > X-ANTICSRF-HEADER: DESCO
[20230728 04:28:02:234 client.py:309 DEBUG] > Authorization: Negotiate YIICrwYJKoZIhvcSAQICAQBuggKeMIICmqADAgEFoQMCAQ6iBwMFACAAAACjggGmYYIBojCCAZ6gAwIBBaERGw9VTklYLkRFU0hBVy5DT02iTDBKoAMCAQOhQzBBGwRIVFRQGzl1LWd1YXN0ZXN0LWtoZXJhLWJpZnJvc3Qtc2NoZW1hLWJpZnJvc3QtYS5zZGxiLmRlc2hhdy5jb22jggE0MIIBMKADAgESoQMCAQGiggEiBIIBHiAzDO5vOoALUVHPKJKVyPiI62eBxjenjbqhdtlwnHMLNnW6Mil2TLkoAuQahs7PTQc4MtxdjNGb450+sfSWHbPIhApqo+3gNjzgKVmvDAmgaeeHziVEmMElouzOdD7/kD6qr2ol8ufyNfyxxeExSSCcA/39CSadr4dMmlmY3awVelgD/ZIcBO3xxbs1BSI6lQBA6nAbUz/tOuMm3I1kAAINrt8k1HlAs2foohxqFRe49tG7+LdglBO2GDhBAFrjXO5suV7Lifg74ip2iT4wJMKEySY6RvAapI2r58KH+9JgY5AnxqRTsQg4n2Xv2nMEraf/Oa1IAVv8OA+XNCG/ow7SqXRW4J2VTtcqXI5acptPxzSYG2kVGUrABx9SfgqkgdowgdegAwIBEqKBzwSBzMNHKFoDf3XETQ8iNyRRwzZv0oQX5XBayq0rCbHEv1+q1wBOB5TDN26uyx99HiDO/Wk2V0U5l83f6OTVOQ362nqAMV52xqbrnBRpg0my2aUFfphqRtUMekcVqLOFmXF6SoLwD94Zr2wR7q79/4fbZ7zXXye1CbQOZJ2Pq/ZcXrrlr2HELqI8HMOB4vCtABUZf0ZuNhGwGCm8SnYga3LVH9TZ+brjvm4wOJUIsjFzSiykTVNljF7wTg8y1YNglohgF6kEniSmWKpeHEtZMA==
[20230728 04:28:02:234 client.py:309 DEBUG] > User-Agent: Python/3.10 websockets/11.0
[20230728 04:28:02:237 client.py:329 DEBUG] < HTTP/1.1 101
[20230728 04:28:02:237 client.py:331 DEBUG] < WWW-Authenticate: Negotiate YIGYBgkqhkiG9xIBAgICAG+BiDCBhaADAgEFoQMCAQ+ieTB3oAMCARKicARuVsWm95CiCPy8y0/KJwzv6Wy25Gi8QUW9mOWvkD18JJCeICLQy5Vce+Q16lkYHZ2pqGEcpNUrHJgmm1rALCF/DLVKNP5vbHg1fTN+iULOs/KsaUoHorFtGQssuxbYYAxBbAc8eAKYDUIgjV8HSPA=
[20230728 04:28:02:238 client.py:331 DEBUG] < Upgrade: websocket
[20230728 04:28:02:238 client.py:331 DEBUG] < Connection: upgrade
[20230728 04:28:02:238 client.py:331 DEBUG] < Sec-WebSocket-Accept: PRGNia9zZpf2MfeHq6N+BAopLmY=
[20230728 04:28:02:238 client.py:331 DEBUG] < Sec-WebSocket-Extensions: permessage-deflate;client_max_window_bits=15
[20230728 04:28:02:238 client.py:331 DEBUG] < Date: Fri, 28 Jul 2023 08:28:02 GMT
[20230728 04:28:02:238 protocol.py:167 DEBUG] = connection is OPEN

Do we not have internal retry mechanisms in place?

Yes, there is an automatic reconnection feature, check the docs.

I didn't find in the docs, can you please point me to the docs.

https://websockets.readthedocs.io/en/stable/reference/sync/client.html#
There is no reference of reconnection in this page, it would be great if you can point me to that?

Sorry I didn't realize that you were using the sync client. This one doesn't have a reconnection feature.
The details of which implementations support which features is described here: https://websockets.readthedocs.io/en/stable/reference/features.html

Sorry I didn't realize that you were using the sync client. This one doesn't have a reconnection feature.

Any plans to add it ?

Sorry I didn't realize that you were using the sync client. This one doesn't have a reconnection feature.

Any plans to add it?

I am also interested in it, and overall the sync version of the library, which seems great and easy to use.

Is it possible to combine async with sync?
Say the websockets implementation runs inside an async rountine that can send messages to a synchronous multiprocessing environment via a message queue?
What would be the best way for such a scenario? Maybe the developers of websockets could provide more patterns and tutorials for this?
Here's what i've been thinking:
import asyncio
import multiprocessing
import websockets
import redis

import time, datetime, json, os, sys, uuid, pathlib, collections, logging

logging.root.setLevel(0)

redis_conn = redis.Redis(host='localhost', port=6379, db=0)

class WebSocketServer:
    def __init__(self, host, port, incoming_queue_name, outgoing_queue_name):
        self.host = host
        self.port = port
        self.incoming_queue_name = incoming_queue_name
        self.outgoing_queue_name = outgoing_queue_name
        self.active_connections = set()

    async def receive(self, websocket):
        async for message in websocket:
            start_time = datetime.datetime.now()
            redis_conn.rpush(self.incoming_queue_name, message)
            end_time = datetime.datetime.now()
            duration = (end_time - start_time).total_seconds()
            logging.info(f"Message received at {start_time}, processed in {duration} seconds")

    async def broadcast(self):
        while True:
            if redis_conn.llen(self.outgoing_queue_name) > 0:
                message = redis_conn.lpop(self.outgoing_queue_name).decode('utf-8')
                for websocket in self.active_connections:
                    if not websocket.closed:
                        await websocket.send(message)
            await asyncio.sleep(0.1)  # Avoid busy waiting

    async def server(self, websocket, path):
        # Add the new websocket connection to the set of active connections
        self.active_connections.add(websocket)
        try:
            receiver_task = asyncio.create_task(self.receive(websocket))
            broadcaster_task = asyncio.create_task(self.broadcast())  # This will send messages to all connected clients
            await asyncio.gather(receiver_task, broadcaster_task)
        finally:
            # Remove the websocket connection from the set of active connections
            self.active_connections.remove(websocket)

    async def start_server(self):
        async with websockets.serve(self.server, self.host, self.port, ssl=None):
            await asyncio.Future()  # This will keep the server running indefinitely.

    def run(self):
        asyncio.run(self.start_server())

class Message_Handler:
    def __init__(self, i, incoming_queue_name, outgoing_queue_name):
        self.i = i
        self.incoming_queue_name = incoming_queue_name
        self.outgoing_queue_name = outgoing_queue_name

    async def process_incoming_message(self):
        while True:
            message = redis_conn.lpop(self.incoming_queue_name)
            if message is not None:
                message = message.decode('utf-8')
                logging.info(f"Processed message: {message}")
                redis_conn.lpush(self.outgoing_queue_name, message.encode('utf-8'))
            else:
                await asyncio.sleep(self.i)

    def run(self):
        asyncio.run(self.process_incoming_message())

def main():
    incoming_queue_name = "incoming_messages"
    outgoing_queue_name = "outgoing_messages"

    ws_url = 'ws://example.com:1234'
    host = '<IP_ADDRESS>'  # Listen on all network interfaces
    port = 8765  # Replace this with your desired port number

    ws_server = WebSocketServer(host, port, incoming_queue_name, outgoing_queue_name)
    message_handler = Message_Handler(0.1, incoming_queue_name, outgoing_queue_name)

    process1 = multiprocessing.Process(target=ws_server.run)
    process2 = multiprocessing.Process(target=message_handler.run)

    process1.start()
    process2.start()

    process1.join()
    process2.join()

if __name__ == "__main__":
    main()

This is really a question about asyncio, not about websockets, and the answer is https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor

Ok, but how to communicate between cpu_bound and async? I mean, how to pass the messages through the different methods, asynio and multiprocesses, with multiprocssing.queue() it will not work, because this is blocking the asynchronous part.


ElasticSearch URL has changed

The URL for ElasticSearch has changed from elasticsearchservice to es

also, can you add a description on how the .alfredworkflow release is generated? I wanted to test this PR, but I have no idea what to do with the modified info.plist

@thisismana i need to look more, but there doesn't appear to be an easy way to actually automate a generation of an .alfredworkflow. as such, this is something i have to manually do and why i don't have any documentation for it 😞
thanks for the update to this service though, i'll go ahead and create a release now


Fallback writer in case of error

Hi Nate,
Thanks again for Lumberjack. Been using it for years now in Caddy!
We have some users (including myself, heh) who have incorrect permissions on their log folder/files. This leads to errors like write error: error getting log file info in the logs, but the actual log message is lost to oblivion. :sweat_smile:
Since Lumberjack hides the underlying io.Writer, I can't figure out a way to wrap the Write() method and, if there's an error, dump the log message to stderr as a fallback.
Would this be a feature you could support?
It would be very helpful if we didn't lose log messages on an external issue like permissions.
Thanks!

Hi Matt!  This is a tricky one.  This sounds like something that should be addressed for v3, which uses a constructor method specifically to do some sanity checks on startup.
What's your timeline for getting this done?

Sounds good to me. It's not urgent. Over the years we've had a number of questions about this, so I've learned it's not an isolated problem for sysadmins to have the wrong permissions configured. :sweat_smile:
Thank you for Lumberjack


with-editor.elc failed to provide feature ‘with-editor’

/home/kon5oul/.emacs.d/.local/straight/build/with-editor/with-editor.elc failed to provide feature ‘with-editor’)
Failed to load org package incrementally, because: (error Loading file /home/kon5oul/.emacs.d/.local/straight/build/with-editor/with-editor.elc failed to provide feature ‘with-editor’)
Failed to load org-agenda package incrementally, because: (error Loading file /home/kon5oul/.emacs.d/.local/straight/build/with-editor/with-editor.elc failed to provide feature ‘with-editor’)
Failed to load org-capture package incrementally, because: (error Loading file /home/kon5oul/.emacs.d/.local/straight/build/with-editor/with-editor.elc failed to provide feature ‘with-editor’)
Failed to load org package incrementally, because: (error Loading file /home/kon5oul/.emacs.d/.local/straight/build/with-editor/with-editor.elc failed to provide feature ‘with-editor’)

I get these messages on starting Emacs. I am using feature/native-comp branch of Emacs 28. And doom sync && doom build returns this.

rm -rf ~/.emacs.d/.local/straight/build/with-editor && doom sync fixes it.

This happened to me after a fresh doom upgrade. Thank you @DoubleClick7 for posting this question and @ncihnegn for answering it.

I had to do rm -rf ~/.emacs.d/.local/straight/build-27.1/with-editor && doom sync to fix this.

Same,

rm -rf ~/.emacs.d/.local/straight/build/with-editor && doom sync fixes it.

no change

I had to do rm -rf ~/.emacs.d/.local/straight/build-27.1/with-editor && doom sync to fix this.

this worked

rm -rf ~/.emacs.d/.local/straight/build*/with-editor && doom sync fixes it.

It didn't for me today. However, the following did (thanks Hendrik!)
rm -rf ~/.emacs.d/.local/straight && doom sync


Requirements not updated for pyvenv -- requires forceupdate

Affected Puppet, Ruby, OS and module versions/distributions

Puppet: 6.22.1
Ruby: ruby 2.0.0p648 (2015-12-16) [x86_64-linux]
Distribution: CentOS 7.9
Module version: 6.0.0

How to reproduce (e.g Puppet code you use)
  file { '/opt/requirements.txt':
    ensure  => present,
    mode    => '0755',
    source  => 'puppet:///modules/venv/requirements.txt',
  }

  python::pyvenv { "${virtual_env}":
    ensure     => present,
    version    => '3.6',
    systempkgs => false,
    venv_dir   => "${virtual_env}",
  }

  python::requirements { '/opt/requirements/requirements.txt':
    virtualenv   => "${virtual_env}",
    pip_provider => 'pip3',
  }

What are you seeing

Virtual environment is created.
requirements.txt is not applied.

What behaviour did you expect instead

Virtual environment is created.
requirements.txt is applied.

Any additional information

When setting forceupdate to true for python::requirements, the requirements file is applied. But it is applied every puppet run, and aside from this being noisy, with dependencies of packages, this may lead to future problems.
I think there may be a bug in manifests/requirements.pp:

Now:
    $local_subscribe = File[$requirements]
  } else {
    $local_subscribe = undef
  }

I think that could be so that $local_subscribe is always set to File[$requirements]:
  }
  $local_subscribe = File[$requirements]

This fixed the issue for me, but I don't know what the rationale is behind the current behaviour...

Experiencing the exact same issue, though it's worth noting that it only seems to occur on puppet 6.x hosts. When I run my bolt plan that leverages this module against a host with puppet 4.8 I do not encounter this issue.
Puppet: 6.18.0 (client) / 6.19.1 (bolt host)
Ruby: ruby 2.4.10p364 (2020-03-31 revision 67879) [x86_64-linux] (client) / ruby 2.6.3p62 (2019-04-16 revision 67580) [universal.x86_64-darwin19] (bolt host)
Distribution: CentOS 7.7
Module version: 6.0.0 / 6.1.0
Requirements file is not enforced, verbose puppet logs indicate python::requirements class never ran.
The fix @siebrand indicated worked for me as well. What's the purpose of setting subscribe to undef in this instance?
If there isn't one, I'm happy to open a PR for this fix. Tested against hosts running Centos 7.7, 7.9 and against Puppet Agent versions 4.8.2, 6.18.0

I'm going to resolve to forking because I can't really wait for this to get attention. Could a committer please comment what the issue is with this ticket and the proposed solution? It's unknown to me how to try and get more attention for this ticket...

Hi @siebrand

It's unknown to me how to try and get more attention for this ticket...

The most efficient way to move on is probably to open a PR with a first commit that add a failing test that reproduce the problem you encounter.  Then add a second commit to implement a fix.
This make it easier for contributors to reproduce the issue and tinker with the issue to fix it the best possible way.
Thanks!


Clean up YAML build scripts

No functional changes.

Pull Request Test Coverage Report for Build 95231

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.6%) to 78.531%

Totals

Change from base Build 95141:
0.6%

Covered Lines:
13154

Relevant Lines:
16750

💛  - Coveralls

:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.AI.Luis.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.AI.QnA.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.ApplicationInsights.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.Azure.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.Dialogs.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.Integration.ApplicationInsights.Core.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.Integration.AspNet.Core.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.TemplateManager.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Builder.Testing.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Configuration.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Connector.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Schema.dll compared against version 4.6.3
:heavy_check_mark: No Binary Compatibility issues for Microsoft.Bot.Streaming.dll compared against version 4.6.3


hls issue

hls is not working on ip

Hello @rumbis,
Sorry for taking long to reply.
Can you describe in details the issue.
If you want to confirm HLS fragments are generated:

go into the docker container: docker exec -it <container_name> bash
list the content of mnt/hls  ls mnt/hls/
if you find steams .m3u8 and .ts files, it means HLS is working.
you can play the HLS content using http://<server IP>:8080/hls/<stream-key>.m3u8
If you were referring to playing hls using HLS player provided. It is hard-coded to play the content from http://<IP_ADDRESS>:8080/hls/test.m3u8. You can download the player from github and modify it then attach to the docker container using
  docker run -d -p 1935:1935 -p 8080:8080 -v custom_players:/usr/local/nginx/html/players alqutami/rtmp-hls

Thank you very much I was able to find that now I'm trying to make it work on https  with domain name it's fine to add the port 443?

It is possible to have secured HLS. You will need to modify nginx.conif. You can mount HTML/certificates and the modified config file to the container using -v. Don't forget to expose port 443 using -p 443:443


Transition lifecycle suite to ginkgoless

Similar to: #1616

Marked WIP until I finish up the ginkgo removal from the suite.go.


Drag & Drop support (for images)

Before Submitting Your Feature Request

[X] Check that there isn't already a similar feature request to avoid creating a duplicate.
[X] I have seen the FAQ.

Problem
It would be great to have drag & drop support for images to pull them into websites or other apps for example.
Solution
No response

It's not currently possible to implement it due to the limitations of NSMenu. I'll keep that in mind whenever I start working on re-writing Maccy from NSMenu.

@p0deje is this something that would be more doable in the 2.0 version? Not sure whether that still uses NSMenu.

@gibfahn Yes, it would be possible in 2.0 since it doesn't use NSMenu.

It doesn't seem to work on 2.0 (at least on my end). Can anyone confirm that it is working on? If so, I will look into my installation.

It's not implemented yet.

May or may not be useful for anyone else, but my workaround for this is this zsh function:
# Save the current clipboard to a png file, copy the path to the clipboard, and show the file
# in Finder. This allows you to easily drag the file, or paste it with a ⇧⌘g in a file
# picker window.
clipboard_image_save() {
  local screenshot_path
  screenshot_path=$HOME/tmp/screen_shot_recording/screenshot_$(date "+%Y-%m-%d_%H-%M-%S").png
  osascript -e "set png_data to the clipboard as «class PNGf»
	set the_file to open for access POSIX path of (POSIX file \"$screenshot_path\") with write permission
	write png_data to the_file
	close access the_file"

  echo "Created $screenshot_path"
  pbcopy <<<$screenshot_path
  # Reveal in Finder.
  open -R $screenshot_path
}

Also just thought of this. Would be nice :)


Flutter 2 web development , How to proxy  backend service ?

Flutter 2 web development , How to proxy  backend service ?
For Example:
My real service running on: http://web-server:8080
flutter run -d chrome:
http://localhost:57900/
For CORS  ,
how to access http://web-server:8080/login in flutter app ?

Closing this as duplicate of https://github.com/flutter/flutter/issues/51530

@darshankawar  When will the feature be ready ?


Explicit reference new package in devpack.ps1

Closing as @brettsam handled this scenario in his changes recently in a better and simpler way -- https://github.com/Azure/azure-functions-dotnet-worker/commit/ac6e7034c335226eb6a816d2c4fc9f29ac24e184


Fix speaker list icon bug

a User has to be able to see the speaker list icon if they are allowed to see the speaker list OR if they are allowed to put themselves onto the list.
At the moment: they can only see the icon if they are allowed to see the speaker list

The user is still only able to see the speaker-list icon if the are allowed to see the speaker-list.
If they are only allowed to put themselves onto the list they cannot see the icon


feat: Canto EVM

Description
Add support for Canto EVM-compatible chain
How to test

Types of changes

Checklist

[ ] Create pull request as draft initially, unless its complete.
[ ] Add tests to cover changes as needed.
[ ] Update documentation as needed.
[ ] If there is a related Issue, mention it in the description.

If you're adding a new blockchain

[ ] I have read the guidelines for adding a new blockchain.

Hi @Amurmurmur, could you please fix the merge conflicts?

Not merging because of: https://twitter.com/CantoPublic/status/1703809950290718871


Ranking Metrics Question

Hi, I am trying to calculate the NDCG@10, MRR@10, and HR@10 on my dataset. My dataset is quite small, and some users may not rate up to 10 items in the test set. Usually, we look at the top 10 predictions and check to see if the predictions are in the ground truth for the given user. But, if the user in the test set does not rate that many items, how does RecBole-CDR handle this case?

I think you can change topk to a lower value in your config file.


OTAR045 -Tractability: Review of project data CSV file

Final version of csv to be received by 10Aug2018.
csv reviewed by Andrew and Michaela

Currently waiting for an updated CSV from Chris - expected to receive within the next few days. Once received, will review with @MichaelaEBI and develop data specification

Updated CSV file has been received from Chris and put into the OT Platform Google Drive in /OT Projects / OTAR042:Tractability / 2. Data

I have updated the spreadsheet that describes the tractability buckets, categories, and supporting information. I will send a link to Chris to verify that categories and descriptions are correct.
Once approved by Chris, the data in the spreadsheet can be used for docs/training and for front-end tooltips.

Chris has verified that the spreadsheet that describes the tractability buckets, categories, and supporting information is correct.
Review of CSV file is complete.


Doesn't work on osx terminal

Each log entry writes to a new line on mac... any way around this?

I'm on osx and I don't see this. Can you try to reproduce and show an example?

also haven't had any problems on mac myself


Fix misspelling of cache_exceptions

Fix misspelling of cache_exceptions in debug_level/all.mlh.
That compile-time flag is used only in Cache_lib, the misspelling is not found anywhere else.
Closes #8145.

!approved-for-mainnet


It disconnects me after a few seconds

Are you using a custom Discord perhaps? I tried it without any disconnect occurring to me, seems to be an issue on your end.


Retry downloads interrupted by connnection resets

I would have thought that it is elderly #84 and #91 addressed it in 0f13cf6 which is 0.3.0~10 but I presumably (see #130 ) run 0.5.0+2.g37513a4 on smaug  and got following crash

gory details
{
  "resources": {
    "core": {
      "limit": 5000,
      "used": 88,
      "remaining": 4912,
      "reset":<PHONE_NUMBER>
    },
    "search": {
      "limit": 30,
      "used": 0,
      "remaining": 30,
      "reset":<PHONE_NUMBER>
    },
    "graphql": {
      "limit": 5000,
      "used": 6,
      "remaining": 4994,
      "reset":<PHONE_NUMBER>
    },
    "integration_manifest": {
      "limit": 5000,
      "used": 0,
      "remaining": 5000,
      "reset":<PHONE_NUMBER>
    },
    "source_import": {
      "limit": 100,
      "used": 0,
      "remaining": 100,
      "reset":<PHONE_NUMBER>
    },
    "code_scanning_upload": {
      "limit": 500,
      "used": 0,
      "remaining": 500,
      "reset":<PHONE_NUMBER>
    }
  },
  "rate": {
    "limit": 5000,
    "used": 88,
    "remaining": 4912,
    "reset":<PHONE_NUMBER>
  }
}
2021-08-11T10:00:02-0400 [INFO    ] tinuous Fetching resources from github
2021-08-11T10:00:02-0400 [INFO    ] tinuous Fetching runs newer than 2021-08-11 09:46:06+00:00
2021-08-11T10:00:02-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/benchmarks.yml (Benchmarks)
2021-08-11T10:00:04-0400 [INFO    ] tinuous Run 3482 not completed; skipping
2021-08-11T10:00:04-0400 [INFO    ] tinuous Run 3481 not completed; skipping
2021-08-11T10:00:04-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/docbuild.yml (Docs)
2021-08-11T10:00:04-0400 [INFO    ] tinuous Found run 3482
2021-08-11T10:00:05-0400 [INFO    ] tinuous Downloading logs for docbuild.yml (Docs) #3482 to 2021/08/11/pr/5883/4bf98ae/github-Docs-3482-success
2021-08-11T10:00:06-0400 [INFO    ] tinuous Found run 3481
2021-08-11T10:00:06-0400 [INFO    ] tinuous Downloading logs for docbuild.yml (Docs) #3481 to 2021/08/11/pr/UNK/5cab1c1/github-Docs-3481-success
2021-08-11T10:00:07-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/test_extensions.yml (Extensions)
2021-08-11T10:00:07-0400 [INFO    ] tinuous Run 3482 not completed; skipping
2021-08-11T10:00:07-0400 [INFO    ] tinuous Run 3481 not completed; skipping
2021-08-11T10:00:07-0400 [INFO    ] tinuous Fetching runs for workflow  (Docs)
2021-08-11T10:00:08-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/test_crippled.yml (CrippledFS)
2021-08-11T10:00:08-0400 [INFO    ] tinuous Run 1948 not completed; skipping
2021-08-11T10:00:08-0400 [INFO    ] tinuous Run 1947 not completed; skipping
2021-08-11T10:00:08-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/update-contributors.yml (allcontributors-auto-detect)
2021-08-11T10:00:09-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/test_macos.yml (Test on macOS)
2021-08-11T10:00:10-0400 [INFO    ] tinuous Run 1719 not completed; skipping
2021-08-11T10:00:10-0400 [INFO    ] tinuous Run 1718 not completed; skipping
2021-08-11T10:00:10-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/shellcheck.yml (Shellcheck on scripts)
2021-08-11T10:00:10-0400 [INFO    ] tinuous Found run 1669
2021-08-11T10:00:10-0400 [INFO    ] tinuous Downloading logs for shellcheck.yml (Shellcheck on scripts) #1669 to 2021/08/11/pr/5883/4bf98ae/github-Shellcheck on scripts-1669-success
Traceback (most recent call last):
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/http/client.py", line 1345, in getresponse
    response.begin()
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/http/client.py", line 1345, in getresponse
    response.begin()
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/datalad/miniconda3/envs/tinuous-dev/bin/tinuous", line 33, in <module>
    sys.exit(load_entry_point('tinuous', 'console_scripts', 'tinuous')())
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/mnt/datasets/datalad/ci/tinuous/src/tinuous/__main__.py", line 126, in fetch
    paths = obj.download(Path(path))
  File "/mnt/datasets/datalad/ci/tinuous/src/tinuous/github.py", line 318, in download
    r = self.client.get(self.logs_url)
  File "/mnt/datasets/datalad/ci/tinuous/src/tinuous/base.py", line 80, in get
    r = self.session.get(url, **kwargs)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/requests/sessions.py", line 677, in send
    history = [resp for resp in gen]
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/requests/sessions.py", line 677, in <listcomp>
    history = [resp for resp in gen]
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/requests/sessions.py", line 237, in resolve_redirects
    resp = self.send(
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))

and do not see any retry log messages

@yarikoptic #91 only affected downloads that used the APIClient.download() method, while GitHub logs were just fetched directly with requests.get().  The code changes in #132 mean that GitHub logs are now downloaded via APIClient.download(), so this issue should be resolved by that.

thanks!  added Closes statement within that PR to close this issue

nope - #132 didn't affect this anyhow -- I don't see any retries

full protocol
{
  "resources": {
    "core": {
      "limit": 5000,
      "used": 50,
      "remaining": 4950,
      "reset":<PHONE_NUMBER>
    },
    "search": {
      "limit": 30,
      "used": 0,
      "remaining": 30,
      "reset":<PHONE_NUMBER>
    },
    "graphql": {
      "limit": 5000,
      "used": 2,
      "remaining": 4998,
      "reset":<PHONE_NUMBER>
    },
    "integration_manifest": {
      "limit": 5000,
      "used": 0,
      "remaining": 5000,
      "reset":<PHONE_NUMBER>
    },
    "source_import": {
      "limit": 100,
      "used": 0,
      "remaining": 100,
      "reset":<PHONE_NUMBER>
    },
    "code_scanning_upload": {
      "limit": 500,
      "used": 0,
      "remaining": 500,
      "reset":<PHONE_NUMBER>
    }
  },
  "rate": {
    "limit": 5000,
    "used": 50,
    "remaining": 4950,
    "reset":<PHONE_NUMBER>
  }
}
2021-08-11T13:00:01-0400 [INFO    ] tinuous tinuous 0.5.0+6.gba81776
2021-08-11T13:00:02-0400 [INFO    ] tinuous Fetching resources from github
2021-08-11T13:00:02-0400 [INFO    ] tinuous Fetching runs newer than 2021-08-11 14:22:32+00:00
2021-08-11T13:00:02-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/benchmarks.yml (Benchmarks)
2021-08-11T13:00:03-0400 [INFO    ] tinuous Found run 3484
2021-08-11T13:00:04-0400 [INFO    ] tinuous Downloading logs for benchmarks.yml (Benchmarks) #3484 to 2021/08/11/pr/5883/6ba978e/github-Benchmarks-3484-success
2021-08-11T13:00:04-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/docbuild.yml (Docs)
2021-08-11T13:00:05-0400 [INFO    ] tinuous Found run 3484
2021-08-11T13:00:05-0400 [INFO    ] tinuous Logs for docbuild.yml (Docs) #3484 already downloaded to 2021/08/11/pr/5883/6ba978e/github-Docs-3484-success; skipping
2021-08-11T13:00:05-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/test_extensions.yml (Extensions)
2021-08-11T13:00:05-0400 [INFO    ] tinuous Found run 3484
2021-08-11T13:00:05-0400 [INFO    ] tinuous Downloading logs for test_extensions.yml (Extensions) #3484 to 2021/08/11/pr/5883/6ba978e/github-Extensions-3484-success
2021-08-11T13:00:07-0400 [INFO    ] tinuous Fetching runs for workflow  (Docs)
2021-08-11T13:00:08-0400 [INFO    ] tinuous Fetching runs for workflow .github/workflows/test_crippled.yml (CrippledFS)
2021-08-11T13:00:09-0400 [INFO    ] tinuous Found run 1950
2021-08-11T13:00:09-0400 [INFO    ] tinuous Downloading logs for test_crippled.yml (CrippledFS) #1950 to 2021/08/11/pr/5883/6ba978e/github-CrippledFS-1950-success
Traceback (most recent call last):
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/http/client.py", line 1345, in getresponse
    response.begin()
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
ConnectionResetError: [Errno 104] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/util/retry.py", line 532, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/datalad/miniconda3/envs/tinuous-dev/lib/python3.9/site-packages/urllib3/packages/six.py", line 734, in reraise


[TwigComponent] Add 'host' context when rendering embedded components

Q
A

Bug fix?
no

New feature?
yes

Tickets

License
MIT

Problem
It's not possible to access properties and methods of a component that's using embedded components from within the embedded content (or other embedded blocks).
Reproduce
Create a wrapper component, which is including some components that have embedded content
Component.php
final class Component
{
    public function getSomething(): string
    {
        return 'sf-ux rules!';
    }
}

component.html.twig
<twig:foo>
    <twig:bar :deepVar="this.something" />
</twig:foo>

foo.html.twig (given a Foo.php, with empty class, not important)

    {% block content %}{% endblock %}

bar.html.twig (given a Bar.php, with empty class, not important)
{{ dump(deepVar) }}

Expected behavior
The template renders like (with some dump markup of course)

    sf-ux rules!

Actual behavior

    null

Cause
While rendering an embedded component the generated template will contain something like
$props = $this->extensions["Symfony\\UX\\TwigComponent\\Twig\\ComponentExtension"]->embeddedContext("foo", twig_to_array([]), $context);
        $this->loadTemplate("foo.html.twig", "foo.html.twig", 10, "732315364")->display($props);

The $props is the context used when rendering the embedded content, which is being composed by https://github.com/symfony/ux/blob/2.x/src/TwigComponent/src/ComponentRenderer.php#L92.
And there's the problem: the original $context contains this, but when rendering the Foo component this becomes a reference to Foo and the reference to the original Component is lost.
Imo this behavior is perfectly normal after all, knowing how the rendering works now. I.e. this should reference Foo, because the block is being embedded in the template of foo.html.twig, so while rendering Foo, but it makes sense that in the above template you want to be able to access properties from the "host" component, the component which is embedding some other component.
Solution
Add a host entry to the context referencing the component which is embedding the nested components.

PS: I'm not sure whether I ought to classify this as a bug or as a feature. It's perhaps not a bug in the sense that it's just not possible right now. On the other hand, it is imo something that you'd expect to be able to do, so then it might be a bug after all. You tell me 🙂
For now I classified it as a feature. If it shouldn't be I would have to remove the CHANGELOG entry.

parent instead of host maybe? I didn't immediately understand what host was referring to.

I would expect:
<twig:foo>
    <twig:bar :deepVar="this.something" />
</twig:foo>

To work though, since you are in Component's template.

I actually agree with @kbond:
{# component.html.twig #}
<twig:foo>
    <twig:bar :deepVar="this.something" />
</twig:foo>

I would imagine that this is referring to the Component on the outside. So the proper behavior for me is that this works. To say it differently: when we're inside Component.html.twig, regardless of whether we're rendering other components, we should not do anything special with the variables. If there is some component framework that does it differently, let me know.  But conceptually, the above Twig code should work the same as the below PHP code: $this === Component:
class Component

public function render()
{
    $this->renderComponent('foo', [
        'content' => function() {
            $this->renderComponent('bar', [
                'deepVar' => $this->something(),
            ]
        }
    ]);
}

And yep! @kbond I'm 99% sure this is related to #844.

parent instead of host maybe? I didn't immediately understand what host was referring to.

@kbond I purposely stayed away from parent, since the (content) block(s) are embedded and not normal extended blocks. I.o.w. there is no parent relationship, if looked at it from an embed perspective. That's actually why it's understandable that this is not working, or not referring to the host/parent Component.
However, I has to ask ChatGPT to come up with a word for "the thing where something is being embedded into". 😄
So, suggestions for a better term are welcome of course!

I would imagine that this is referring to the Component on the outside.

@weaverryan @kbond I understand the expectation that this ought to refer to the Component of the template you're in, as it was what I was expecting initially as well, and I can still see that that perhaps ought to be the desired behavior, especially considering Ryan's example of rendering in PHP.
The problem is of course that internally embedded blocks are being used, and a new template is being rendered "independently" from the parent/host context.
That being said, walking away from using embedded blocks, and using a separate lexer mechanism as suggested in #844 could be a solution, but then it's not possible to refer to the Embedded Component itself.
See the example in my test cases in the commits. I'll repeat it here:
<twig:embedded_content_foo>
    host from embedded content in Foo element: {{ host.something }}
    this from embedded content in Foo element: {{ this.something }}

    <twig:embedded_content_bar :propGivenThis="this.something" :propGivenHost="host.something" id='1' >
        <twig:embedded_content_bar :propGivenThis="this.something" :propGivenHost="host.something" id='2' />
    </twig:embedded_content_bar>
</twig:embedded_content_foo>

So, you could look at the problem as: "it's not a bug, it's a feature!" 🙂

I purposely stayed away from parent, since the (content) block(s) are embedded and not normal extended blocks. I.o.w. there is no parent relationship

Yeah, I thought about this. In the context of html, parent is the correct term I think:

foo

div is the parent of p...
How do other front end frameworks handle this issue (both logically and naming)?

Parent is perfectly fine by me, but I guess we first have to make sure that the compilation is still using the embed flow (see https://github.com/symfony/ux/issues/844#issuecomment-1545845334).

Seeing @weaverryan's latest comment in the related issue (https://github.com/symfony/ux/issues/844#issuecomment-1545885490) and his & @kbond's desire to have this refer to the originating component, I could adapt this PR to simply not override the this-context. Or are you guys preferring to be able to refer both parent ànd this? See example above

Tbh, I'm not super motivated to make changes to the existing ComponentRenderer stuff, because we've found that it is lacking in other ways. I'm thinking we should port https://github.com/giorgiopogliani/twig-components more directly over to UX named under a new tag like {% twig_component and make THAT work exactly like we want it to. Then we would update the PreLexer to lex into the new {% twig_component and eventually deprecate the current ComponentRenderer / {% component syntax.

@weaverryan sounds good!

we should port https://github.com/giorgiopogliani/twig-components more directly over to UX named under a new tag like {% twig_component and make THAT work exactly like we want it to

Is it something I can help with? I have been trying some things the last days, based on https://github.com/giorgiopogliani/twig-components. I'll open a new PR for that if you want to (when it's finished).
In essence we move from an embed node, which uses blocks, to an include node, which subcompiles the slots within the "host/parent" template. That way this always resolves to the expected component, solving the issue addressed in this PR.

I'm not super motivated to make changes to the existing ComponentRenderer stuff, because we've found that it is lacking in other ways

Can you elaborate a bit? It's best to know what should be avoided in the new version.

Sorry guys, but working with it in a real project this Monday, I'm afraid I once again have to change my mind on the embed vs include stuff.
I'll try to recap both options, and give the real example afterwards to support my preference (spoiler: it's embed after all).
Using slots
Mechanism

Content is rendered together with the template it's being used in. This means it can use the context of that template.

Syntax (concept)
{# someComponent.twig #}
{% twig_component Foo %}
  some content {# This is used for `content` var in Foo.twig #}
  {{ this.methodCall }} {# This will use SomeComponent::methodCall(), also part of  `content` var in Foo.twig #}
  {% slot namedSlot %}foobar{% slot %}
{% endtwig_component %}

{# Foo.twig #}

{{ content ?? 'default content' }} {{ namedSlot }}

PRO

Clear intuitive expectations: this refers to the Component of the template it's used in.

CON

No access to context of the component you're overriding/providing the slot of/for.

Using embed (current)
Mechanism

Content is rendered inside the template where the block is defined. This means it is using the context of that template. It also gets the context of the parent, but the this reference is overridden, changing it to the new current component. Optionally, the addition of a parent/host context could solve this, which is what this PR is about.

Syntax
{# someTemplate.twig #}
{% component Foo %}
  some content {# This is used for `content` block in Foo.twig #}
  {{ this.methodCall }} {# This will use Foo::methodCall(), also part of  `content` block in Foo.twig #}
  {{ parent.methodCall }} {# (optional) This will use SomeComponent::methodCall(), also part of  `content` block in Foo.twig #}
  {% block namedBlock %}foobar{% endblock %}
{% endcomponent %}

{# Foo.twig #}

{% block content%}default content{% endblock %} {% block namedBlock %}foobar{% endblock %}

PRO

Access to context of the component/template you're overriding/providing the slot of/for.

CON

Confusing context. this refers to the Component where the block is defined, since it's being rendered in that context, but using the overridden block definition from another (the host/parent) template.
Confusing block rules, due to embed relation and not extend relation. (see #844)
No dynamic block names (iianm)

The example to show the benefit of keeping embed
{# Table.twig #}

    {% block content %}{% endblock %}

{# Body.twig #}

{% for row in rows %}

    {% block content %}
        {% for cell in row %}
        <twig:cell>{{ cell }}</twig:cell>
        {% endfor %}
    {% endblock %}

{% endfor %}

{# Cell.twig #}
{% block content %}{% endblock %}

{# MyTable.twig #}
<twig::table>
    <twig:head :headers="headers" :withAction="true" :sortableColumns="sortableColumns" />
    <twig:body :rows="host.rows">
        {% for cell in row %}
            <twig:table:cell>{{ cell }}</twig:table:cell>
        {% endfor %}
        <twig::table:cell>
            doing something special with the action column
            <twig:block name="td_class">action</twig:block>
        </twig:table:cell>
    </twig:body>
</twig:table>

With an include (slot) syntax, it wouldn't be possible to access the row variable inside MyTable.twig, which is actually from the context of Body.twig. Imho this is a huge pro for the embed flow. I also believe that the con 'confusing context' can be solved with proper documentation.
PS: #844 is imo a case of not properly grasping embed vs include, because it is confusing, but shouldn't require any changes, except for perhaps adding documentation making that difference more clear leading to proper expectations.

Hey @sneakyvv!
I appreciate the objective detailed discovery that you're doing 👍. I admit that I am "partial" towards the "slot" idea, because I find the block hierarchy very confusing. But I'll do my best to stay objective also ;). By the way, I also think there are 2 separate "things" that are being discusses: (A) the embed block hierarchy and (B) the context inside of {% component (i.e. is this the parent component or the child component).
Let me challenge you on one point:

With an include (slot) syntax, it wouldn't be possible to access the row variable inside MyTable.twig, which is actually from the context of Body.twig. Imho this is a huge pro for the embed flow

This refers to:
    <twig:body :rows="host.rows">
        <twig:block name="row">
            {% for cell in row %}
                 ...
            {% endfor %}
        </twig:block>
    </twig:body>

That looks really confusing to me - way too magical. The row variable is coming specifically from the row block of Body.twig. I understand what you're trying to do - you want the code inside of <twig:block name="row"> to function as if it has been copied and pasted directly into THAT block in Body.twig. You're right that Twig DOES allow this and works this way. But it's very non-obvious - the row variable seems to come out of nowhere.
However, I admit that I can't think of a decent solution to this use-case. About #844, if things were kept as they are now, how would the correct code look for doing this:
{# DangerButton.html.twig #}

{% component 'Button' with { type: 'danger' } %}
    {% block content %}{{ block('content') }}{% endblock %}
{% endcomponent %}

That is where someone is calling <twig:DangerButton>Danger</twig:DangerButton> and you want to "forward" the content block (containing Danger) through to the final Button component. Would you document to "Never use the block() function inside of {% component" and instead document this?
{# DangerButton.html.twig #}

{% set outerContent = block('content') %}
{% component 'Button' with { type: 'danger' } %}
    {% block content %}{{ outerContent }}{% endblock %}
{% endcomponent %}

It's not super elegant... /cc @WebMamba gotta include you here, as you're already working on the slot-type. I would still prefer the slot type - but the table/row use-case above deserves a nice solution.

Hey, @sneakyvv thanks to raised all of this!
We can work on some tests in #873, and see if we can find solutions to convince you! 😁
For the Table example, we can simply do a test or build a separate repo using the slot PR.
I follow this discussion from the beginning, and I see the pro and con you talking about. But I just feel even if the slot has cons, it is much more intuitive. I think it's just too early to take a decision, let's play a bit with both approaches! 😁

@weaverryan I just add this test to illustrate your DangerButton example: https://github.com/symfony/ux/blob/4f95295caabd21471b8b6c913d5edc337ec91a9a/src/TwigComponent/tests/Integration/TwigComponentExtensionTest.php#LL86C1-L89C1
I don't want to annoy anybody, but for this specific example, the slot approach is way more simple. Here is the template:
https://github.com/symfony/ux/blob/4f95295caabd21471b8b6c913d5edc337ec91a9a/src/TwigComponent/tests/Fixtures/templates/components/DangerButton.html.twig#LL1C1-L1C1

Thanks @WebMamba - that IS way nicer. Any idea on the table for situation above? I was thinking about this situation this morning, and though I love the simplicity of slots, the current solution is VERY powerful.
so the question is: keep that power and document or try to find a solution to “forwarding” blocks into a component (setting the block to a variable above the component works, but isn’t great… but works) or continue the slot idea and try to find a solution for the table row problem.

Ok so here is my solution for the Table problem.
Let's keep it simple at first, we just want to make a simple table component with, which we give some data and it works:
// index.html.twig

{% extends 'base.html.twig' %}

{% block body %}
    <twig:Table :rows="{foo: ['bar', 'baz']}"/>
{% endblock %}

// /components/Table.html.twig

    <twig:TableBody :rows="rows"/>

// /components/TableBody.html.twig

    {% for row in rows %}

            {% block row %}
                {% for cell in row %}
                    <twig:Cell>{{ cell }}</twig:Cell>
                {% endfor %}
            {% endblock %}

    {% endfor %}

// components/Cell.html.twig
{{ slot }}

Ok now we have our simple component render. But now let's add some difficulties and take into account the @sneakyvv problem. 😁 So we want to add more flexibility to the Table component and give the ability to choose how to render the cell. Here is my solution for it:
// index.html.twig

{% extends 'base.html.twig' %}

{% block body %}
    <twig:Table :rows="{foo: ['bar', 'baz']}">
        <twig:slot name="cell">
            <twig:Cell>
                <twig:slot-value name="cell_value"/>
            </twig:Cell>
        </twig:slot>
    </twig:Table>
{% endblock %}

So here we put the cell component into a slot inside the table component. And we use a new syntax <twig:slot-value, lets see later how it work.
// components/Table.html.twig

    <twig:TableBody :rows="rows">
        <twig:slot name="cell">
            {{ cell }}
        </twig:slot>
    </twig:TableBody>

Here we just push again the cell component through an other slot
// component/TableBody.html.twig

    {% for row in rows %}

            {% block row %}
                {% for cell in row %}
                    {{ cell.withContext({'cell_value': cell}) }}
                {% endfor %}
            {% endblock %}

    {% endfor %}

And here we are! We render the slot that contains the cell, but with the function withContext. This function takes an array containing the key: cell_value and the value of the cell. And remember, we used: <twig:slot-value name="cell_value"/> so this gonna be replaced by the cell value, and everything will be rendered as expected.
@weaverryan tell me what you think and I can push a commit to add this withContext method.
This idea came from Reactjs they do something like that when you want to push props to children components, by using the function cloneElement on your child.

Hey @weaverryan, @WebMamba,
Sorry for the slow response, we're having a 4-day weekend here in Belgium and good weather so... 😎 😅
Anyhow... Here's my (once again, long (sry)) feedback:

That looks really confusing to me - way too magical. [...] you want the code inside of <twig:block name="row"> to function as if it has been copied and pasted directly into THAT block [...]

In my mind that's always how you'd work with blocks, whether in an extend/include or embed way. That is, you'd look at the template you're using/overriding and replace some content within it, so indeed as if it's copied right in there. Most of the time those blocks don't really use the context where they're being replaced into, so the "magic" usage of those context variables isn't that common, but as you also noted it is a VERY powerful feature. Perhaps, again referring to the component's own "universe" helps to understand the magic (or not being confused by it).
Now, about the content-in-content block issue (for now leaving @WebMamba's comments out of the equation):

Would you document to "Never use the block() function inside of {% component" and instead document this?

I love the simplicity of slots, the current solution is VERY powerful.
so the question is: keep that power and document or try to find a solution to “forwarding” blocks into a component (setting the block to a variable above the component works, but isn’t great… but works) or continue the slot idea and try to find a solution for the table row problem.

I was thinking that the fact that a content block is being use inside a component could perhaps be detected and that the variable assignment in that case could be automized? Not sure if would be that easy though...
I wouldn't necessarily document "Never use the block() function inside of {% component", or at least not as such.
The thing #844 is trying to do is wrap some (content) blocks with some HTML which is already done by some other "atom" component. Generally speaking it boils down to
{# Template A #}

<twig:B>
    My inner content for B {# content defined at top level #}
</twig:B>

{# Template B (component) #}

<twig:C>{%- block content -%}{%- endblock -%}</twig:C> {# passing content to a C component #}

{# Template C (component) #}

{%- block content -%}{%- endblock -%}

The problem is actually that the content of a twig block is automatically used as a content block. It's from there that the logical consequence ensues that you shouldn't use a content block inside a twig component instance. So therefore, I would argue that you don't need to document the "logical thing". However, since this kind of (atom) composition will be a very common use case, I would document it perhaps within that frame of reference. That in those cases, i.e. cases where you have a twig component instance (C) inside a twig component's (B) template, and you want to define the content of C from template A, that in those cases you need to pass the content along via a new variable because (indeed) you cannot ever use a content block inside a component, because that is being translated into content-in-content, which is illegal. Either that or just use another name for the inner-content block, but then again that's not elegant to use in template A, because you can't write it as the body of a component (using the default content block).
Ok, then finally, I want to address @WebMamba's approach:
First of all, let me start with saying that I love all the effort that everyone is putting into this. A token of appreciation once in a while never hurts 😉
Second of all,

take into account the @sneakyvv problem

I just love having a problem being named after me 😂
Ok, let's dive into the proposed solution. Right off the bat... Let me confess, I'm having troubles understanding it 🙈
I get that you're overriding the cell slot in index.html.twig and you're just passing it along in Table.html.twig to TableBody.html.twig, but how cell_value is getting it's value and what withContext does isn't clear to me (without having looked at the code).
A few things also came to mind seeing this example:

The first thing isn't really related to the specific example, but the withContext function made me think: How would you be able to see in a template whether some variable is a "slot variable" or a regular variable, either from the template or Component scope?
The (for me) normal workflow of looking at a template and immediately seeing what parts can be overridden is not possible, right?
The table solution seems to me like it's reversing the way components are composed. I assumed that you would always have some kind of atomic design, where smaller components are being used to build bigger and bigger blocks, reusing the smallest components many times. If I understand the solution to the @sneakyvv problem (😉) correct, then the index.html.twig & Table.html.twig are altered to make the solution possible, while they should be kept agnostic. At most they provide some blocks/parts that CAN be overridden, but not for example that the Table component's template is defining a cell slot, just so that some higher component can override the lower component's slot. That would quickly become unmaintainable as it would have to provide any slot that may be overridden at some point.

PS: I don't have any stakes in either solution, so don't take my comments as me trying to pull down the slot solution. I'm probably just missing the key part about it :-)
And to reiterate, I would love the simplicity without losing the power of blocks, keeping all of its benefits and not introducing a new layer of complexity to allow slots to do what blocks already can do.

Here is a comment to resume my thought about all of this.
I just feel like Slot is way simpler and more intuitive to use, and this is even more true for people not familiar with Twig. But of course yes, it's less powerful by default than the block approach.
Your Table problem @sneakyvv is totally legit, and if you choose the slot approach we need to address a fix for it. But this problem those not represent how people gonna use the TwigComponent in their daily tasks. If we just look at Reactjs or Svelte, everything works more like Slot.
So it sounds better for me to remove this complexity not needed most of the time and to give some solutions for it when people need it. With slot we can let people choose when they want this complexity or not. 😁

Hi!
Apologies for being a bit slow/absent. @WebMamba while I agree with what you said (I really like the simplicity of slots), I've become convinced of the power of the block case... and I think we should try to make it work first. But, it would require 3 things:

A way to get the "parent" context (this PR)
Documentation update to teach people that, when you are writing content inside of a <twig:Component>, you should pretend that you are now writing in a totally separate template that extends the component's template.
We need a decent way to "forward" blocks. The current work-around is:

{# anyTemplate.html.twig #}

<twig:SuccessAlert>You did it!</twig>

{# SuccessAlert.html.twig #}

{# the workaround: you must set the block to a variable *outside* of the block first #}
{% set content = block('content') %}
<twig:Alert type="success">
    {{ content }}
</twig>

This last one is a serious problem and I think we need to come up with a decent solution for it in order to move forward with the the block-based system. Any ideas on what the proper syntax should look like?
Or... do both?
Btw, another idea would be to combine the idea of slots and blocks where "slots" are the default mode, but you could also use blocks for advanced use-cases. Something like:
{# anyTemplate.html.twig #}

<twig:SuccessAlert>You did it!</twig>

{# SuccessAlert.html.twig #}

<twig:Alert type="success">
    {{ slot }}

    {% block footer %}
        I can print the {{ type }} var from Alert.html.twig
    {% endblock %}
</twig>

{# Alert.html.twig #}

    {{ slot }}

    {% block footer %}{% endblock %}

I kinda like this :). I think it would just require that the new twig_component from #873 be built "inside" of a structure that is very similar to the current {% component - i.e. one that leverages the "embed" type of magic.

No worries! That was for a good cause, I m eager to try the AssetComponent!
I like the idea of combining slot and block, let's try that! 😁
My only concern is about LiveComponent, I think if we only used slot, the embedded live component should work. But by adding the embed magic I think we bringing back the blocking point 🤔

Hi all,
A lot has happened meanwhile. So a quick recap:

Indeed #913 has been added, which makes live embedded components possible.
#920 is a solution to #844 — it makes passing blocks down to nested embedded components possible, in an intuitive way. Or, defined the other way around, and more correctly imo: it allows you to override a block that belongs to a nested component from within the body of a parent component, and infinitely deep.

Making that last change made me realize that this PR is insufficient, if #920 is to be merged, because it would bring the need to access the nested component hierarchy more than one level up (which is now host/parent). You can imagine you would need to have access to the grandparent variable for example, or even higher up, which actually (probably but not necessarily) refer to the template which contains the block override.
Anyhow, I have already experimented with a Hierarchy object that acts as a proxy object for a Component, which makes something like this.parent.parent.parent.someProp possible.
I'll await the feedback about #920 and this proposal to push that into this PR, or perhaps create a separate PR altogether.

Making that last change made me realize that this PR is insufficient, if #920 is to be merged, because it would bring the need to access the nested component hierarchy more than one level up (which is now host/parent).
[...]
I have already experimented with a Hierarchy object that acts as a proxy object for a Component, which makes something like this.parent.parent.parent.someProp possible. I'll await the feedback about #920 and this proposal to push that into this PR, or perhaps create a separate PR altogether.

I created a separate PR for this => #936 (which continues on the changes of #920)

I just stumbled over this PR and it's discussing an exact problem I have and I didn't find a solution yet.
I have quite a complex setup with blocks and components and for some reason, even tho there is the outerBlocks variable, I can't pass normal twig blocks down as content/named block of a component. Example:
List.html.twig

{% block content %}{% endblock %}

ListItem.html.twig

  {% block content %}{% endblock %}
  {% if hasSubList %}
    {% block sublist %}{% endblock %}
  {% endif %}

Label.html.twig
{% block content %}{% endblock %}

Now I wanna use it in a block based template (think about a form theme for example, or something selfcooked):

{% block list %}
<twig:List>
  {{ block('children') }} // doesn't work
  {{ block(outerBlocks.children }} // seems to work... but
</twig:List>
{% endblock %}

{% block children %}
  {% for child in children %}
    {{ block('item' }} // also still seems to work... but
  {% endfor %}
{% endblock %}

{% block item %}
<twig:ListItem>
   {{ block(outerBlock.label }} // doesn't render anything
   <twig:block name="sublist">
     {{ block(outerBlocks.list) }} // also doesn't render anything
   </twig:block>
</twig:ListItem>
{% endblock %}

{% block label %}
<twig:Label>{{ label }}</twig:Label>
{% endblock %}

What am I missing, I would have expected this to work, but it seems it's not working, so there is no way to build components in such a nested way? Any help would be really appreciated 🙏

@CMH-Benny it would be really easier for us if you could create an issue (and there link to this PR if you need) instead of reviving old PR/issues, because we can miss them, and their original author receives notifications ;)
I let you reopen one if this issue is still not resolved ? thx


Add haml 4.1 to supported versions

See #228
Adding this allows haml-lint to start with haml 4.1.0.beta.1.
I didn't have time to check for breaking changes in that version, so it may be that it breaks something.

Did you figure out a way to try this out locally?

I am currently using this branch on a private project with ~350 haml files, using haml 4.1.0.beta.1 and it  works without problems. But note that the project is not using any specific 4.1 features.
I will also try to get the test suite running locally but I will probably not get around to that until Monday.

Merged in 02ef5c2d and added the beta to our appraisal configuration in 04906a63 so we run tests against it. Thanks for the PR!


Handles nested fields in field options.

Would you mind adding a test for this also? :)

Will send a different PR.


[$1000] No consistency in margin left for green tick besides amount

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open the app
Open any report
Send money to report
Click on send money message and using console window, observe margin-left to green tick icon besides amount on top and in message below
Again click on message displayed in that report to sent report
Observe margin left to tick icon in that report

Expected Result:
App should maintain consistent margin between amount and green tick throughout all the reports in the flow
Actual Result:
App maintains different margin for all the green tick instances throughout all the reports in the flow
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: n/a
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://github.com/Expensify/App/assets/93399543/6bcb7d53-e8f2-440c-a680-4510ad425f68
https://github.com/Expensify/App/assets/93399543/00fd7b75-e068-4c55-b11a-d42c5c8feb4b
Expensify/Expensify Issue URL:
Issue reported by: @dhanashree-sawant
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1685201350349499
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~01628393c0c27440ce
Upwork Job ID:<PHONE_NUMBER>338699264
Last Price Increase: 2023-06-12

Reproducible

Proposal
Please re-state the problem that we are trying to solve in this issue.
No consistency in margin left for green tick besides amount
What is the root cause of that problem?
As stated in Issue. both are using different margin
What changes do you think we should make in order to solve the problem?

we should remove styles.moneyRequestHeaderCheckmark and use styles.iouPreviewBoxCheckmark as its introduced 2 years back. it will help in maintaining code for same thing.
both have same proprties except difference of 1px.
We should update MoneyRequestHeader.js and TaskHeader.js to use styles.iouPreviewBoxCheckmark instead of styles.moneyRequestHeaderCheckmark.
What alternative solutions did you explore? (Optional)
None

Proposal
Please re-state the problem that we are trying to solve in this issue.
Inconsistency between checkmark icon margin with the label
What is the root cause of that problem?
For IOUPreview we have left margin of 4px in the message whereas for the MoneyRequestHeader the margin of 8px(styles.ml2) set within this latest PR.
https://github.com/Expensify/App/blob/71017e42617bedefa04628bef5eb18d1b28df1fb/src/components/ReportActionItem/IOUPreview.js#L203
https://github.com/Expensify/App/blob/71017e42617bedefa04628bef5eb18d1b28df1fb/src/styles/styles.js#L2528-L2529
What changes do you think we should make in order to solve the problem?
We can update the style to styles.ml1 in the MoneyRequestHeader MenuItem.
https://github.com/Expensify/App/blob/71017e42617bedefa04628bef5eb18d1b28df1fb/src/components/MenuItem.js#L179-L180
OR update the margin for the class iouPreviewBoxCheckmark to 8px.

We need to clarify the expected result before selecting the proposal.

Money Request Report
Money Request Detail Header
Money Request Action

@shawnborton Would you help to determine what margin should be set between the amount and checkmark icon from the above places? Thanks!

Just to clarify with @trjExpensify and @JmillsExpensify - this screenshot here shouldn't exist right? I thought we fixed this elsewhere:

Then for the other screens, I think 8px makes sense but we're going to end up redoing the middle screenshot anyways, so I'm not really sure what we should do here.

Whatcha' mean by shouldn't exist? Right now the ReportPreview in the workspace chat looks like that. We haven't made it a "card" style with all those changes just yet, scan receipt is where those are.

The report action mockup I shared shouldn't have a green checkmark. The report preview card, yes. But the report action line, no.

Yeah, the report action line in the iou/expenseReport view shouldn't have a checkmark. The report preview (which right now is just in a message style) in the chat did, but given that we're killing it, may as well get rid of it now if you want. 👍

@shawnborton @trjExpensify We can change the margin for the first screenshot and remove the checkmark from the report action. Do you think this sounds good to you?

That works for me.

@mollfpr do any of these proposals work with the above suggestion? Thank you!

@grgia Not specifically, but closest and first is @Ollyws proposal. We can set the value to 8px and removed the checkmark from the report action.

That works for me.

Same!

@grgia let's go with @Ollyws proposal which it does what requested before.
🎀 👀 🎀 C+ reviewed!

Thanks for your patience, good to go @Ollyws!

Thanks for the correction, Olly! Melvin's bad math issue was brought up internally yesterday, I'm not sure if we've created an issue to fix this yet but we're aware and I expect we will fix it 👁 (Seems to do something with how the API is reading the Z in the timezone)

[@mollfpr] The PR that introduced the bug has been identified. Link to the PR:
[@mollfpr] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:

No offending PR, another improvement on the IOU/money request design.

[@mollfpr] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:

Since this is improvement on the visual, the regression step should be enough.

[@mollfpr] Determine if we should create a regression test for this bug.
[@mollfpr] If we decide to create a regression test for the bug, please propose the regression test steps to ensure the same bug will not reach production again.

Regression step:

Open the app
Open any report
Send money to report
Click on send money message
Verify both the left margin of the checkmark in the header and in the preview box message are a consistent 8px.
Verify that the confirmation text (e.g paid $123.00 ...) does not have a checkmark.
Navigate to IOU Chat.
Verify that the confirmation text (e.g paid $123.00 ...) does not have a checkmark.
👍 or 👎

@sonialiap Accepted, Thank you!


WARNING: Piper was not compiled with pcaudiolib. Output audio will be written to the current directory.

I've downloaded amd64 release, I can launch it without any problem, but there's a warning:
WARNING: Piper was not compiled with pcaudiolib. Output audio will be written to the current directory.

And, as it suggests, I can't use --output-dir or --output-raw arguments (probably --output-file doesn't work either)
There's pcaudiolib-1.2-2 installed in my system, if it is related in any way.
I'm running Arch Linux (not WSL).

I don't compile the released version of Piper with pcaudiolib in order to reduce the number of dependencies. This means you need to specify one of the --output-* arguments or look for the WAV file written in the cwd.
You can also recompile Piper with pcaudilib present if you want it to play the output through the sound system automatically.

I specify the --output-dir argument, it still saves the audio in current directory

Is compiling it with pcaudlib difficult? I'm willing to give it a go, but I'm not sure how to add it in.
I want to be able to use json to generate a bunch of wavs without reloading the model every time, but when I set the "output_file" in the json line, I get the same warning and piper spits out the wav in the cwd with automatically generated names.


Support sending HTTP headers

Addresses issue: https://github.com/spiral/php-grpc/issues/28
Functionality: Adds a metadata array reference to the PHP service classes which will be used by the go proxy to set server response headers

This will cause major BC change on PHP end. Golang seems ok, let me think how to deal with it. I think the easiest way is to carry response metadata as reference inside the context.

@wolfy-j changed it like you suggested.

Hi, I would like not to extend the ContextInterface but rather carry mutable outgoing headers object in one of the cotext values. Are you OK if I'll merge your request and then do such change by myself?

Okay. Sure!

I'll work on it and hopefully few other updates later today and this weekend.

Sounds great.
btw some thoughts on that:
Currently any object-reference in Context::values will get lost by calling Context::withValue().
This is fine because Context is meant to be immutable.
In RPC-methods (e.g. authenticate) we're currently not able to return a new context instance. So calling Context::withValue() there, wouldn't affect the outgoing context.
I see 3 options:

either we define that headers can only be set in interceptors (which return a ctx instance)
or we make the ctx instance partly mutable
or we change the RPC methods to support returning an ctx instance.

I think it's a valid requirement to be able to set outgoing headers also in RPC-methods and not only in interceptors.

WithValue will only overwrite one value, so you can mutable object if no one overwrites it. The ability to overwrite value is pretty useful as you can "mock" outgoing headers object for the testing or other purposes.

$context->getValue(Context::METADATA_OUTPUT)->set('key', 'value');

I've seen such an approach in PSR-7 middleware collection.


Administrative corrections

Preface - ESR is already completed (not "is developing")
Glossary:
SaaS - remove first sentence, as the headline already spells that out (same with PaaS and IaaS)
"CSPs hardware" below the figure should be CSP's
PaaS - "GP_OS_PP" is not the short name for NIAP's PP. Presumably this and others will be defined in references
"applied to the existing TOE Boundary and Cloud Cloud Operating Environment" - duplicate word (also in IaaS)

PP_GPCP_V1.0

Closed.