Consumer can't receive message from topic

Describe the bug
I create a topic with 4 partitions.When I use key shared mode to consumer message,sometimes, I can't receive any message from a specify partition.In this scenes，I create two consumers, one of it subscribe the topic(C1), one of it subscribe two partitions of the same topic(C2). I start C1 first, then C2. At first, C1 can receive all message,C2 can't receive any message.When I close C1, C2 can receive message from specify partitions. But when I start C1 again, all consumer can't recevie from the partitions those C2  subscribed.
The code of produce
import pulsar
import time
from multiprocessing import Process

def produce_test(partition, key, message):
    client = pulsar.Client('pulsar://localhost:6650')
    producer = client.create_producer(partition)
    print(client.get_topic_partitions("test10"))
    for i in range(240):
        time.sleep(2)
        message_mix = str(int(i)) + message
        print(message_mix)
        producer.send((message_mix).encode('utf-8'), partition_key=key)
    time.sleep(1000)
    client.close()

if __name__ == "__main__":
    p1 = Process(target=produce_test, args=("test10", "10000", "aaaa",))
    p2 = Process(target=produce_test, args=("test10", "20000", "bbbb",))
    p3 = Process(target=produce_test, args=("test10", "30000", "cccc",))
    p4 = Process(target=produce_test, args=("test10", "40000", "dddd",))
    p1.start()
    p2.start()
    p3.start()
    p4.start()

The code of C1:
import pulsar
import time
from _pulsar import ConsumerType
from multiprocessing import Process

def consumer_data(topic, process_index):
    client = pulsar.Client('pulsar://localhost:6650')
    print(len(client._consumers))
    consumer = client.subscribe(topic, 'my-subscription', consumer_type=ConsumerType.KeyShared)
    flag = True
    start_time = int(time.time() * 1000)
    while True:
        msg = consumer.receive()
        print(
            "Process {} Received message '{}' id='{}' partition={}".format(process_index, msg.data(), msg.message_id(),
                                                                           msg.topic_name()))
        consumer.acknowledge(msg)

if __name__ == "__main__":
    p1 = Process(target=consumer_data, args=("test10", "p1", ))
    p1.start()

The code of C2
import pulsar
import time
from _pulsar import ConsumerType
from multiprocessing import Process

def consumer_data(topic, process_index,):
    client = pulsar.Client('pulsar://localhost:6650')
    print(len(client._consumers))
    consumer = client.subscribe(topic, 'my-subscription', consumer_type=ConsumerType.KeyShared)
    flag = True
    start_time = int(time.time() * 1000)
    while True:
        msg = consumer.receive()
        print(
            "Process {} Received message '{}' id='{}' partition={}".format(process_index, msg.data(), msg.message_id(),
                                                                           msg.topic_name()))
        consumer.acknowledge(msg)

if __name__ == "__main__":
    p2 = Process(target=consumer_data, args=(["test10-partition-2", "test10-partition-3"], "2", ))
    p2.start()

The output of C1:
Process p1 Received message 'b'236dddd'' id='(15,236,0,-1)' partition=persistent://public/default/test10-partition-0
Process p1 Received message 'b'236cccc'' id='(17,472,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'236aaaa'' id='(17,473,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'237dddd'' id='(15,237,0,-1)' partition=persistent://public/default/test10-partition-0
Process p1 Received message 'b'237cccc'' id='(17,474,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'237aaaa'' id='(17,475,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'238dddd'' id='(15,238,0,-1)' partition=persistent://public/default/test10-partition-0
Process p1 Received message 'b'238cccc'' id='(17,476,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'238aaaa'' id='(17,477,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'239dddd'' id='(15,239,0,-1)' partition=persistent://public/default/test10-partition-0
Process p1 Received message 'b'239cccc'' id='(17,478,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'239aaaa'' id='(17,479,1,-1)' partition=persistent://public/default/test10-partition-1
2020-08-03 15:54:37.531 INFO  [140699652921088] ConsumerStatsImpl:65 | Consumer [persistent://public/default/test10-partition-0, my-subscription, 0] , ConsumerStatsImpl (numBytesRecieved_ = 1535, totalNumBytesRecieved_ = 1535, receivedMsgMap_ = {[Key: Ok, Value: 233], }, ackedMsgMap_ = {[Key: {Result: Ok, ackType: 0}, Value: 233], }, totalReceivedMsgMap_ = {[Key: Ok, Value: 233], }, totalAckedMsgMap_ = {[Key: {Result: Ok, ackType: 0}, Value: 233], })
2020-08-03 15:54:37.531 INFO  [140699652921088] ConsumerStatsImpl:65 | Consumer [persistent://public/default/test10-partition-1, my-subscription, 1] , ConsumerStatsImpl (numBytesRecieved_ = 3070, totalNumBytesRecieved_ = 3070, receivedMsgMap_ = {[Key: Ok, Value: 466], }, ackedMsgMap_ = {[Key: {Result: Ok, ackType: 0}, Value: 466], }, totalReceivedMsgMap_ = {[Key: Ok, Value: 466], }, totalAckedMsgMap_ = {[Key: {Result: Ok, ackType: 0}, Value: 466], })
2020-08-03 15:54:37.531 INFO  [140699652921088] ConsumerStatsImpl:65 | Consumer [persistent://public/default/test10-partition-2, my-subscription, 2] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {}, totalAckedMsgMap_ = {})
2020-08-03 15:54:37.531 INFO  [140699652921088] ConsumerStatsImpl:65 | Consumer [persistent://public/default/test10-partition-3, my-subscription, 3] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {}, totalAckedMsgMap_ = {})

The output of C2:
2020-08-03 15:49:46.326 INFO  [140382617937728] ConnectionPool:85 | Created connection for pulsar://localhost:6650
2020-08-03 15:49:46.326 INFO  [140382567016192] ClientConnection:335 | [<IP_ADDRESS>:50638 -> <IP_ADDRESS>:6650] Connected to broker
2020-08-03 15:49:46.329 INFO  [140382567016192] AckGroupingTrackerEnabled:53 | ACK grouping is enabled, grouping time 100ms, grouping max size 1000
2020-08-03 15:49:46.329 INFO  [140382567016192] HandlerBase:53 | [persistent://public/default/test10-partition-2, my-subscription, 0] Getting connection from pool
2020-08-03 15:49:46.329 INFO  [140382567016192] AckGroupingTrackerEnabled:53 | ACK grouping is enabled, grouping time 100ms, grouping max size 1000
2020-08-03 15:49:46.329 INFO  [140382567016192] HandlerBase:53 | [persistent://public/default/test10-partition-3, my-subscription, 1] Getting connection from pool
2020-08-03 15:49:46.331 INFO  [140382567016192] ConsumerImpl:199 | [persistent://public/default/test10-partition-2, my-subscription, 0] Created consumer on broker [<IP_ADDRESS>:50638 -> <IP_ADDRESS>:6650]
2020-08-03 15:49:46.331 INFO  [140382567016192] ConsumerImpl:199 | [persistent://public/default/test10-partition-3, my-subscription, 1] Created consumer on broker [<IP_ADDRESS>:50638 -> <IP_ADDRESS>:6650]
2020-08-03 15:49:46.331 INFO  [140382567016192] MultiTopicsConsumerImpl:99 | Successfully Subscribed to Topics
2020-08-03 15:59:46.330 INFO  [140382567016192] ConsumerStatsImpl:65 | Consumer [persistent://public/default/test10-partition-2, my-subscription, 0] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {}, totalAckedMsgMap_ = {})
2020-08-03 15:59:46.330 INFO  [140382567016192] ConsumerStatsImpl:65 | Consumer [persistent://public/default/test10-partition-3, my-subscription, 1] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {}, totalAckedMsgMap_ = {})

I wrote a simple C++ consumer but could not reproduce the problem:
// SampleConsumer.cc in examples directory
#include <iostream>
#include <pulsar/Client.h>

using namespace std;
using namespace pulsar;

int main(int argc, char* argv[]) {
    std::string topic = "ParTopic";  // a topic with 4 partitions
    std::vector<std::string> partitions;
    for (int i = 1; i < argc; i++) {
        partitions.emplace_back(topic + "-partition-" + argv[i]);
    }
    size_t index = partitions.size();

    Client client("pulsar://localhost:6650");

    ConsumerConfiguration consumerConfig;
    consumerConfig.setConsumerType(ConsumerKeyShared);

    std::string subName = "SubscriptionName";
    Consumer consumer;
    if (partitions.empty()) {
        client.subscribe(topic, subName, consumerConfig, consumer);
    } else {
        client.subscribe(partitions, subName, consumerConfig, consumer);
    }
    Message msg;

    while (true) {
        Result result = consumer.receive(msg);
        cout << "[" << index << "] " << msg.getDataAsString() << ", key: " << msg.getPartitionKey()
             << ", id: " << msg.getMessageId() << endl;
        consumer.acknowledge(msg);
    }
    client.close();
}

First run a producer, and a consumer subscribing the partitioned topic (C1):
$ ./SampleConsumer

Then run a consumer subscribing two partitions (C2), eg, 0 and 2:
$ ./SampleConsumer 2 3

The log:
partition 0 and 1
C1:
[0] 11bbbb, key: 20000, id: (42970,476,3,0)
[0] 11aaaa, key: 10000, id: (43000,950,1,0)

C2:
[2] 11dddd, key: 40000, id: (43001,472,-1,0)
[2] 11cccc, key: 30000, id: (43000,949,-1,0)

partition 2 and 3
C1:
[0] 0cccc, key: 30000, id: (43000,955,1,0)
[0] 0aaaa, key: 10000, id: (43000,956,1,0)
[0] 0dddd, key: 40000, id: (43001,475,0,0)

C2:
[2] 0bbbb, key: 20000, id: (42970,479,-1,0)

You should just close the main process and don't close you client by code,just close the process.

The point is, at the first step:

I start C1 first, then C2.

C2 could consume messages as the log says. Also I use Ctrl+C to kill the process, Even without it, the client.close() could never been reached because there's no way to break my loop.
The full log of C2 (while C1 has started before) is:
$ ./examples/SampleConsumer 2 3
2020-08-05 10:34:23.403 INFO  [140077698222624] ConnectionPool:85 | Created connection for pulsar://localhost:6650
2020-08-05 10:34:23.405 INFO  [140077624579840] ClientConnection:343 | [<IP_ADDRESS>:59990 -> <IP_ADDRESS>:6650] Connected to broker
2020-08-05 10:34:23.415 INFO  [140077624579840] HandlerBase:53 | [persistent://public/default/ParTopic-partition-3, SubscriptionName, 0] Getting connection from pool
2020-08-05 10:34:23.417 INFO  [140077624579840] HandlerBase:53 | [persistent://public/default/ParTopic-partition-2, SubscriptionName, 1] Getting connection from pool
2020-08-05 10:34:23.417 INFO  [140077624579840] ConsumerImpl:199 | [persistent://public/default/ParTopic-partition-3, SubscriptionName, 0] Created consumer on broker [<IP_ADDRESS>:59990 -> <IP_ADDRESS>:6650]
2020-08-05 10:34:23.418 INFO  [140077624579840] ConsumerImpl:199 | [persistent://public/default/ParTopic-partition-2, SubscriptionName, 1] Created consumer on broker [<IP_ADDRESS>:59990 -> <IP_ADDRESS>:6650]
2020-08-05 10:34:23.418 INFO  [140077624579840] MultiTopicsConsumerImpl:99 | Successfully Subscribed to Topics
[2] 160bbbb, key: 20000, id: (45337,159,-1,0)
// ...
[2] 176bbbb, key: 20000, id: (45337,175,-1,0)
^C

Could you attempt it in python client?

Here're the outputs of Python client 2.6.0 with broker 2.6.0, using the same code as provided. First start Producer.py and C1.py, then start C2.py.
C1:
$ python3 C1.py
0
2020-08-05 09:55:18.379 INFO  [140237624133440] Client:88 | Subscribing on Topic :test10
2020-08-05 09:55:18.381 INFO  [140237624133440] ConnectionPool:85 | Created connection for pulsar://localhost:6650
2020-08-05 09:55:18.383 INFO  [140237353412352] ClientConnection:343 | [<IP_ADDRESS>:50298 -> <IP_ADDRESS>:6650] Connected to broker
2020-08-05 09:55:18.412 INFO  [140237353412352] HandlerBase:53 | [persistent://public/default/test10-partition-0, my-subscription, 0] Getting connection from pool
2020-08-05 09:55:18.413 INFO  [140237353412352] HandlerBase:53 | [persistent://public/default/test10-partition-1, my-subscription, 1] Getting connection from pool
2020-08-05 09:55:18.413 INFO  [140237353412352] HandlerBase:53 | [persistent://public/default/test10-partition-2, my-subscription, 2] Getting connection from pool
2020-08-05 09:55:18.413 INFO  [140237353412352] HandlerBase:53 | [persistent://public/default/test10-partition-3, my-subscription, 3] Getting connection from pool
2020-08-05 09:55:18.582 INFO  [140237353412352] ConsumerImpl:199 | [persistent://public/default/test10-partition-3, my-subscription, 3] Created consumer on broker [<IP_ADDRESS>:50298 -> <IP_ADDRESS>:6650]
2020-08-05 09:55:18.582 INFO  [140237353412352] ConsumerImpl:199 | [persistent://public/default/test10-partition-1, my-subscription, 1] Created consumer on broker [<IP_ADDRESS>:50298 -> <IP_ADDRESS>:6650]
2020-08-05 09:55:18.582 INFO  [140237353412352] ConsumerImpl:199 | [persistent://public/default/test10-partition-2, my-subscription, 2] Created consumer on broker [<IP_ADDRESS>:50298 -> <IP_ADDRESS>:6650]
2020-08-05 09:55:18.582 INFO  [140237353412352] ConsumerImpl:199 | [persistent://public/default/test10-partition-0, my-subscription, 0] Created consumer on broker [<IP_ADDRESS>:50298 -> <IP_ADDRESS>:6650]
2020-08-05 09:55:18.582 INFO  [140237353412352] PartitionedConsumerImpl:302 | Successfully Subscribed to Partitioned Topic - persistent://public/default/test10 with - 4 Partitions.

Process p1 Received message 'b'9dddd'' id='(13,9,0,-1)' partition=persistent://public/default/test10-partition-0
Process p1 Received message 'b'9cccc'' id='(15,18,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'9aaaa'' id='(15,19,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'10cccc'' id='(15,20,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'10dddd'' id='(13,10,0,-1)' partition=persistent://public/default/test10-partition-0
Process p1 Received message 'b'10aaaa'' id='(15,21,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'11dddd'' id='(13,11,0,-1)' partition=persistent://public/default/test10-partition-0
Process p1 Received message 'b'11cccc'' id='(15,22,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'11aaaa'' id='(15,23,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'12dddd'' id='(13,12,0,-1)' partition=persistent://public/default/test10-partition-0
Process p1 Received message 'b'12cccc'' id='(15,24,1,-1)' partition=persistent://public/default/test10-partition-1
Process p1 Received message 'b'12aaaa'' id='(15,25,1,-1)' partition=persistent://public/default/test10-partition-1

C2:
$ python3 C2.py
0
2020-08-05 09:57:05.193 INFO  [139998113343296] ConnectionPool:85 | Created connection for pulsar://localhost:6650
2020-08-05 09:57:05.195 INFO  [139997834229504] ClientConnection:343 | [<IP_ADDRESS>:50328 -> <IP_ADDRESS>:6650] Connected to broker
2020-08-05 09:57:05.206 INFO  [139997834229504] HandlerBase:53 | [persistent://public/default/test10-partition-3, my-subscription, 0] Getting connection from pool
2020-08-05 09:57:05.210 INFO  [139997834229504] HandlerBase:53 | [persistent://public/default/test10-partition-2, my-subscription, 1] Getting connection from pool
2020-08-05 09:57:05.215 INFO  [139997834229504] ConsumerImpl:199 | [persistent://public/default/test10-partition-3, my-subscription, 0] Created consumer on broker [<IP_ADDRESS>:50328 -> <IP_ADDRESS>:6650]
2020-08-05 09:57:05.221 INFO  [139997834229504] ConsumerImpl:199 | [persistent://public/default/test10-partition-2, my-subscription, 1] Created consumer on broker [<IP_ADDRESS>:50328 -> <IP_ADDRESS>:6650]
2020-08-05 09:57:05.221 INFO  [139997834229504] MultiTopicsConsumerImpl:99 | Successfully Subscribed to Topics
Process 2 Received message 'b'9bbbb'' id='(14,9,-1,-1)' partition=persistent://public/default/test10-partition-3
Process 2 Received message 'b'10bbbb'' id='(14,10,-1,-1)' partition=persistent://public/default/test10-partition-3
Process 2 Received message 'b'11bbbb'' id='(14,11,-1,-1)' partition=persistent://public/default/test10-partition-3
Process 2 Received message 'b'12bbbb'' id='(14,12,-1,-1)' partition=persistent://public/default/test10-partition-3

Besides, I noticed your log has AckGroupingTrackerEnabled, this feature was improved from #6534 but the log level was reduced to debug from #7373 , could you try with latest client or just rollback to 2.6.0?

Closed as stale.
The development of the Python client has been permanently moved to https://github.com/apache/pulsar-client-python. If it's still relevant, please open an issue there.


[BUG] Some mysterious forces crashes bloxstrap on launch (Roblox failed to launch). And all the others windows that i requested to launch are "waiting for other instances"

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.
[X] I have thoroughly looked through the available Wiki articles and could not find a solution to my problem.
[X] I am using the latest version of Bloxstrap.
[ ] I did not answer truthfully to all the above checkboxes.

Bloxstrap Version
v2.8.1
What problem did you encounter?
its in title. and its getting stuck like that cuz it needs to be closed (idk why its made like that, cuz the normal roblox doesnt require you to close "failed to launch" windows, in fact, he doesnt even create them).
 You actually suck, git gud

What do you mean all of the other windows?

In my case it doesn't work, when I open Bloxstrap and Seliware starts to inject Bloxstrap ends up closing

dont use detected exploits

Client issues are no longer accepted.

can you at least have this prompt not showing (and yeah its not client sided, because bloxstrap on his own decides to block other instance's launch before you get rid of crashed window)


Feature request: Allow to use VisibilityHelper for DataGridColumn (and possibly other)

Current state
Now VisibilityHelper allow to simpler use without converters for every element which is a child of FrameworkElement. But there are classes which are not descendants of FrameworkElement like DataGridColumn, but have Visibility property and VisibilityHelper do not work for them.
Expected
VisibilityHelper should work also for DataGridColumn and possibly other WPF elements which are not descendants of FrameworkElement.
Environment

MahApps.Metro 1.5.0 (current stable)
Windows OS - any
Visual Studio 2015
.NET Framework - any

@SylwesterZarebski You can use a BindingProxy for these elements which doesn't inherit the DataContext through the visual tree.
[WPF] HOW TO BIND TO DATA WHEN THE DATACONTEXT IS NOT INHERITED

Yes, i know - i do this similar way now, but this request is not for context binding, but using VisibilityHelper instead of converters for elements which are not descendant of FrameworkElement.


Error while targetting Android 14

The following error is shown when uploading the app to the play store.
This app targets Android 14 or above, and your current Play Billing Library version doesn't support Android 14.
To target the latest Android version, please update your PBL version to either 5.2.1, or 6.0.1 and above.

Please look into this issue.

Fixed in 0.1.0


[Question]: Parsing is too slow and cannot be done in parallel

Describe your problem

Is there a way to improve the parsing speed of documents？it is too slow, the gpu memory only ran more than 2GB
Can it be parsed in parallel？

You could start multiple task executors in docker container by python rag/svr/task_executor 1

You could start multiple task executors in docker container by python rag/svr/task_executor.py 1

when try to exec command，got following error:
root@ragflow:~/ragflow# python3 rag/svr/task_executor.py 1
Traceback (most recent call last):
  File "/root/ragflow/rag/svr/task_executor.py", line 22, in <module>
    from api.utils.log_utils import initRootLogger
ModuleNotFoundError: No module named 'api'

You could start multiple task executors in docker container by python rag/svr/task_executor.py 1

it seems like the new executor will close after one file parsing is finished, how can i config it at startup ?

Increase the value of WS in entrypoint.sh.


Why?

Why this vs just using the dep paragonie/random_compat ?

paragonie/random_compat is a library and not a command line tool, so it isn't an alternative. This tool uses that library but provides a command line interface on top.
The actual alternative to this tool is the OpenSSL command line utility. I've documented this alternative in the README (including instructions on how to use it) and recommended its use if available.
To answer the more general 'Why?' question - because I recently contacted two separate library authors regarding concerns over the entropy of the cryptographic keys used in their project examples and documentation.
The main purpose of the README for this repository is to provide clear instructions on how to generate secure keys for cryptographic purposes. I'll accept any pull requests or advice that helps accomplish this :)
The ability for it to be bundled with other PHP projects is secondary, but something I still feel is important.
I'll close this issue as I believe that I have now answered the question asked. If you have a response to what I've said here then I will reopen this. If you have any further questions they will probably be better suited to a new issue.


Added TTGSnackbar

Added TTGSnackbar

Please resolve the conflicts so we can get this merged. (:

I'm so sorry that i made a mistake on this request. I will close this request and push another one.


Create contributing.md

I created a contributing.md file based on this discussion. Thanks!

It looks good.
Can you

rename it CONTRIBUTING.md? There is currently a typo in the name
add a sentence to give credit to the source? As it is mostly a copy of the one in OLS
Thanks

Thanks, @bebatut! I have modified the file.

Thanks for rephrasing @bebatut.

It looks good. I merge. Thanks

Thanks for merging, I will update the readme file now


cli.nix: improve documentation, including arguments

Description of changes
Following on from #309517 / #308500 / #264893, this improves the documentation around cli.nix
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

@infinisil does this look ok to you?


tdm64-gcc problem

After downloading tdm64-gcc onto a fresh machine, it will not create a new instance because a newer version is available.

This has been solved by having a zip of the compile in the repo


Add PDS API Specs.

Choose a PR Template
Switch to "Preview" on this description then select one of the choices below.
Click here to open a PR for a Data Plane API.
Click here to open a PR for a Control Plane (ARM) API.

Hello, There are two checks failing in this PR and I am not able to fix these.
Can you please help me in trying to figure out the root cause of these:

PoliCheck: I have added some words in the custom words txt file. Although the words I have added do not have any undesirable words, the file contains some words already for which the poli-check is throwing exceptions. I don't know how to fix this problem as removing the specified words may cause a breaking change.

I am getting some errors in SDK Track2 Validation related to the change of tag.
Although I have am not changing any tags in this PR but introducing a new tag: https://github.com/Azure/azure-rest-api-specs/blob/14296bb806b4d1ea6f8372c543d264684018c567/specification/purview/data-plane/readme.md#tag-package-2021-01-01-preview

/azp run

@pansharm-ms , can you resolve branch conflicts then re-run CI checks so that PoliCheck check should pass?


Add "Outline tool" for quickly outlining shapes.

The Outline Tool works similarly to the paint bucket, but only paints pixels along the edges of shapes. Hold Ctrl/Cmd to have it fill corners too.
I didn't create a tool icon. I copied-and-pasted the paint bucket icon as a stand-in, but I'm not so sure I did that correctly.
The tool works quite nicely, however. Try it!

I ran the Travis-CL checks locally and they pass. The log above indicates that Phantom crashed, apparently of its own accord.

Thanks so much for this! I use it in my own custom build.
https://github.com/gingerbeardman/piskel-playdate/blob/dev-1047/README.md


Test multiple Python version compatibility

Before submitting

[ ] Was this discussed/agreed via a Github issue? (no need for typos and docs improvements)
[ ] Did you read the contributor guideline, Pull Request section?
[ ] Did you make sure to update the docs?
[ ] Did you write any new necessary tests?

What does this PR do?
Fixes # (issue).
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

@aniketmaurya I would merge it as it is now and add py3.12 with needed fixes un another PR


CSS minification breaks theming

Given CSS in the form of:
[data-theme="themename"] {
  --variable: value;
  .btn{
    --btn-variable: value;
  }
}

CSS minification produces the following structure:
[data-theme="themename"] {
  --variable: value;
}
.btn{
  --btn-variable: value;
}

This effectively breaks theming by "unwrapping" nested styles from the main data-theme attribute.

A temporary fix has been put in place by removing CSS minification. PR #74

This was resolved by #245.


Do not add / for html pages in rewriteDocLinks

Signed-off-by: Anastasia Rodionova<EMAIL_ADDRESS>Hey, I just made a Pull Request!
Fixes https://github.com/backstage/backstage/issues/5965

:heavy_check_mark: Checklist

[x] A changeset describing the change and affected packages. (more info)
[ ] Added or updated documentation
[x] Tests for new functionality and regression tests for bug fixes
[ ] Screenshots attached (for UI changes)
[x] All your commits have a Signed-off-by line in the message. (more info)

@backstage/maintainers Needs approval due to changeset 🙂


Instagram Class - Methods not working

Reference line of the issue
https://github.com/Clueless-Community/scrape-up/blob/c7ac4c8a842b9e2fb43f25257cd3902a0a8625f8/src/scrape_up/instagram/users.py#L25
Method to fetch number of followers and number of posts are not working.

I would like to work on this

Go ahead @karthikbhandary2

You can close this @nikhil25803


MM-20584: Migrate 'components/admin_console/team_channel_settings/team/details' module and associated tests to TypeScript

Summary
This PR migrates 'components/admin_console/team_channel_settings/team/details' module and associated tests to TypeScript
Ticket Link
Fixes https://github.com/mattermost/mattermost-server/issues/13488
JIRA: https://mattermost.atlassian.net/browse/MM-20584
Screenshots
not applicable
Release Note
NONE

Hello @sinansonmez,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.

Kindly pinging @BenCookie95 for review

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/218640

/update-branch

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/218794


Description regarding increase request in maximum replica amounts causes misleading.

[Enter feedback here]
The following descriptions gives us a bit misleading. We misunderstood that we needed to submit a support request if we would like to increase a replica count even if we want to change the default value to 50 replicas,

To request an increase in maximum replica amounts for your container app, submit a support ticket

I suggest that this description is changed like 'If you need over 300 replicas, a submission with support request is required.'

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 58fa8bbe-e0eb-4944-bc60-617aec0c0dde
Version Independent ID: eae53948-3193-a451-5963-3e6cd8ee2972
Content: Scaling in Azure Container Apps
Content Source: articles/container-apps/scale-app.md
Service: container-apps
GitHub Login: @craigshoemaker
Microsoft Alias: cshoe

@mshyatsu
Thanks for your feedback! We will investigate and update as appropriate.

Thanks for your feedback and your contribution to Azure docs.
Feedback for this repository is moving away from GitHub to a system specific to the Microsoft Learn platform. Issues for this repository will soon be disabled, and additional comments from GitHub will no longer be possible. However, we are now tracking and triaging this issue in the new feedback system.
To learn more about our feedback systems, please see Provide feedback for Microsoft Learn content.
#please-close


@sourceloop/authentication-service: Error: No factory found for sf.passport.strategyFactory.localPassword

Describe the bug
when using authentication service APIs to login getting following error:
Error: No factory found for sf.passport.strategyFactory.localPassword
API checked: auth/login(Error: No factory found for sf.passport.strategyFactory.localPassword)
auth/login-token(Error: No factory found for sf.passport.strategyFactory.ResourceOwner)
To Reproduce
Steps to reproduce the behavior:

setup authentication microservice using sourceloop CLI.
create user through migration or manual
hit Login API

Expected behavior
user should be able to login successfully and it should return token to access to other resources
Screenshots

Additional context
loopback cli: 6.0.0
sourceloop cli: 9.0.2
node:18.20.4
lerna 5.2.4

Hello @Sumit9669 the Local Passport strategy is not a mandatory strategy its an optional feature
so to use that you are required to bind the strategy to its provider. ou can refer the docs here

@yeshamavani thanks for responding but i had tried this as well earlier but were getting same issue.
Apart from that in following scenario everything works fine:
sourceloop/cli: 4.2.0
node: 16.5
lerna : 5.4.2
authentication-service: v9.1.2

@yeshamavani thanks for responding but i had tried this as well earlier but were getting same issue.
Apart from that in following scenario everything works fine: sourceloop/cli: 4.2.0 node: 16.5 lerna : 5.4.2 authentication-service: v9.1.2

The strategy factory and its related binding needs to be provided after binding the component
Can you please confirm if you have done it that way.

@Sumit9669 is it working for you

it is resolved. After providing required binding.

@prernagp @yeshamavani  that time we were not able to solve it after providing binding, so we moved to lower version as we were on fast faced. For now you can close this issue, after updating dependencies if still breaks will re-open this issue.
Thanks a lot SF team for assistance.
🙏


Pyfakefs doesn't work with Python 2.6+ backport of os.scandir

pyfakefs doesn't work with the backport of os.scandir Is this possible or does it have the same limitations as lxml?
Currently the backport, just like in #119, doesn't find any directories created by pyfakefs and raises an OSError:
OSError: [Errno 2] No such file or directory: 'root'

Yes, unfortunately it has the same limitations as lxml. The backport is a C based module, and thus cannot be faked without explicit support in the library. Currently, we have no support for any non-standard libraries, and I'm not sure if that would be in the scope of this package. @jmcgeheeiv - what do you think?

If it's C/C++, it is technically infeasable to patch it and fake it.
On Thu, Dec 7, 2017 at 8:10 AM, mrbean-bremen<EMAIL_ADDRESS>wrote:

Yes, unfortunately it has the same limitations as lxml. The backport is a
C based module, and thus cannot be faked without explicit support in the
library. Currently, we have no support for any non-standard libraries, and
I'm not sure if that would be in the scope of this package. @jmcgeheeiv
https://github.com/jmcgeheeiv - what do you think?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/jmcgeheeiv/pyfakefs/issues/332#issuecomment-350014843,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADjLAREYeVjd_Jc5i2um0Ad7Rr3OHsRbks5s-A4GgaJpZM4Q5sq3
.

Well, we could fake it the same way we fake os, I'm just not sure if this is the way to go. We already have a fake implementation of os.scandir, so that could be used in this case, e.g. we would have to fake out the scandir module in case it is available.

I have thought about this a little more, and I think that we can refactor Patcher a little to be able to add other faked modules without the need to directly reference them in the patcher itself.
I may have a go at this after the pep8 stuff is merged.

@OskarPersson - please check if this works for you and reopen otherwise.
Just learned that mentioning the issue in the PR description (not in the commit message) also auto-closes it...

Hmm, no I can't get it to work :/
I cloned the master branch, created a new Python 2.7.14 virtualenv, pip installed scandir and tested with python all_tests.py. This resulted in 8 errors, all with the same traceback:
======================================================================
ERROR: testIndexAccessToStatTimesReturnsInt (fake_filesystem_test.FakeScandirTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/oskar/git/pyfakefs/fake_filesystem_test.py", line 4832, in setUp
    from fake_scandir import scandir
ImportError: No module named fake_scandir

I also tried setting up a simple test in example_test.py:

from scandir import scandir

def test_scandir_backport(self):
    self.fs.CreateFile('/test/text.txt')
    self.assertEqual(len(list(scandir('/test'))), 1)

This results in the following error:
=====================================================================
ERROR: test_scandir_backport (example_test.TestExample)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/oskar/git/pyfakefs/example_test.py", line 152, in test_scandir_backport
OSError: [Errno 2] No such file or directory: '/test'

Ok, I will check this tomorrow, if I find the time.

Ok, several things here:

the from scandir import scandir will generally not work under pyfakefs, you have to import scandir (there is the possibility to enable the dynamic patcher to fix this, but this is experimental and breaks at least pytest)
the example will not work that way - you have to add the tested code to the "sut", which is example.py; I adapted the existing example to also work with the scandir module
there was a syntax error in the other test, which I did not catch, because I tested under Windows, where the test was disabled (I overlooked this)
there was a real bug in the implementation - it did not work under Python 2 due to changed syntax for next in the iterator (__next__ vs next)

From my tests this seems to work now, thanks!

Glad to hear that!


raw socket truncated packets

Gracefully handle packet truncation errors, instead of panicking. Ran into this when testing smaller MTU sizes.

Can you add a test?
Can you change the commit message so that the first line isn't just raw_socket:?

Done. Let me know if there is anything else.


SSL error

Just downloaded a fresh copy 5 minutes ago, tried it with a simple command like this:
youtube-dl https://www.youtube.com/watch?v=NK6B6WjHC68

And there ya go:
[youtube] NK6B6WjHC68: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)> (caused by URLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)'),))

That's on Windows 10. All my browsers can open the url perfectly fine without warnings or anything.

Possibly duplicate of #7309. Could you try the solution at https://github.com/rg3/youtube-dl/issues/7309#issuecomment-167414048?

Tried to no avail. In my cert store, I don't have any Google/Youtube-related certs at all...

Do you have a "GeoTrust Global CA" certificate?

Two of them as well, and both expire somewhere in 2022.

There should be exactly one. Can you check that both have exactly the name "GeoTrust Global CA"?
And, is the start time of the valid interval of the certificate covers your current system time?

I also have this issue on Linux. New cert file, openssl 1.0.2g, python 2.7.11. I also tried the 2016.03.14 version of youtube-dl, same results. It seems to be a youtube-specific problem, as CBS (Platform) works fine.

Forgot to ask. Did you use proxies? Post the verbose log with -v:
youtube-dl -v "https://www.youtube.com/watch?v=NK6B6WjHC68"

@steve5777 On Linux SSL certs handling is different from Windows. Please refer to #5225.

No, I didn't use a proxy. And here's the log. Hope it helps! BTW, with --no-check-certificate it does work fine.
[debug] System config: []
[debug] User config: [u'--continue', u'--restrict-filenames']
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=NK6B6WjHC68']
[debug] Encodings: locale ANSI_X3.4-1968, fs ANSI_X3.4-1968, out ANSI_X3.4-1968, pref ANSI_X3.4-1968
[debug] youtube-dl version 2016.03.18
[debug] Python version 2.7.11 - Linux-2.4.37-i686-with-slackware-7.1.0
[debug] exe versions: ffmpeg 2.0, rtmpdump 2.4
[debug] Proxy map: {}
ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)> (caused by URLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)'),))
File "/usr/bin/youtube-dl/youtube_dl/extractor/common.py", line 365, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/usr/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1929, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/bin/youtube-dl/youtube_dl/utils.py", line 860, in https_open
req, **kwargs)
File "/usr/lib/python2.7/urllib2.py", line 1197, in do_open
raise URLError(err)

@steve5777 Please open a new issue. Don't mix Linux logs in an issue for Windows.

OK, sorry about that. In my case, the problem turned out to be a Python update. Apparently all versions since 2.7.9 try to verify certs by default. After much searching, it turns out all I needed to do was set the variable SSL_CERT_FILE to the correct location. No idea if this helps with Windows, though.

inb4 update openssl and python https://github.com/Homebrew/homebrew/issues/38226

Don't discuss SSL issues on non-Windows systems in this thread anymore. I'm waiting for @thany's response for what's happening on his/her system. SSL things are quite different on Windows and other systems. Just don't mix them together.

Same issue here on window 10 but not with youtube but soundclound
PS > .\youtube-dl.exe -v "https://soundcloud.com/hipsterslovecrazyantics/babymetal-07"
.\youtube-dl.exe : [debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'https://soundcloud.com/hipsterslovecrazyantics/babymetal-07']
[debug] Encodings: locale cp1252, fs mbcs, out None, pref cp1252
[debug] youtube-dl version 2016.03.27
[debug] Python version 2.7.10 - Windows-8-6.2.9200
[debug] exe versions: none
[debug] Proxy map: {}
[soundcloud] hipsterslovecrazyantics/babymetal-07: Resolving id
[soundcloud] hipsterslovecrazyantics/babymetal-07: Downloading info JSON
ERROR: Unable to download JSON metadata: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
(_ssl.c:590)> (caused by URLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
(_ssl.c:590)'),))
  File "youtube_dl\extractor\common.pyo", line 366, in _request_webpage
  File "youtube_dl\YoutubeDL.pyo", line 1934, in urlopen
  File "urllib2.pyo", line 437, in open
  File "urllib2.pyo", line 550, in http_response
  File "urllib2.pyo", line 469, in error
  File "urllib2.pyo", line 409, in _call_chain
  File "urllib2.pyo", line 656, in http_error_302
  File "urllib2.pyo", line 431, in open
  File "urllib2.pyo", line 449, in _open
  File "urllib2.pyo", line 409, in _call_chain
  File "youtube_dl\utils.pyo", line 864, in https_open
  File "urllib2.pyo", line 1197, in do_open

@cc0ut As another reporter said in https://github.com/rg3/youtube-dl/issues/9046#issuecomment-204344648: you need to visit https://soundcloud.com/ and https://api.soundcloud.com/ in Internet Explorer or Edge so that Windows will fetch the necessary certificates.
If things are still, check "GlobalSign Root CA" in certmgr.msc.

That works.
Btw, I had no problem downloading video from youtube or vimeo despite never having opened the those sites in edge (only in firefox)

Two possibilities:

As Adam Goodman said in http://bugs.python.org/issue20916, Microsoft pre-installs some certificates. The one YouTube uses (GeoTrust Global CA) and the one Vimeo uses (DigiCert Global Root CA) may already be included.
A root CA issues certificates to many websites. Visiting any of them in IE/Edge brings necessary root CA certificates.

I believe @thany is also a victim of Microsoft's new policy. If anyone gets the same error, feel free to open a new issue or leave comments here.

I have got the same error and solved by accessing youtube with IE11.
Error message
PS > youtube-dl.exe "http://www.youtube.com/watch?v=6brejYnhpuc" --verbose
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['http://www.youtube.com/watch?v=6brejYnhpuc', '--verbose']
[debug] Encodings: locale cp932, fs mbcs, out cp932, pref cp932
[debug] youtube-dl version 2017.02.16
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-83507-g8fa18e0, ffprobe N-83507-g8fa18e0
[debug] Proxy map: {}
[youtube] 6brejYnhpuc: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600
)> (caused by URLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)'),))
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpnd0pf5s_\build\youtube_dl\extractor\common.py", lin
e 416, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpnd0pf5s_\build\youtube_dl\YoutubeDL.py", line 2006,
 in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 464, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 482, in _open
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmpnd0pf5s_\build\youtube_dl\utils.py", line 1060, in
https_open
  File "C:\Python\Python34\lib\urllib\request.py", line 1185, in do_open

What I did

Access youtube with Edge.
Try youtube-dl and get error again.
Check GeoTrust Global CA that will be expired at 2022-05-21
Access youtube with IE11
Try youtube-dl and get worked.

Env

Windows10 Pro: 1607
youtube-dl: 2017.02.16

But such a thing shouldn't be necessary, right?
For instance, this cannot be done on a Server Core edition of Windows which doesn't have any GUI. Furthermore, IE11 is a deprecated browser - requiring that, even if only once, feels not right.
Lastly, it's a workaround that seemingly has nothing to do with the script. I mean, why would visiting the site with a deprecated browser magically fix this script? Makes no sense to me.
But good to know there's a workaround. Because that's really what it is :)

Here's a follow-up: a pending patch at http://bugs.python.org/issue28747 enables Python the ability to update system certificates, just like IE.

Makes no sense to me

Indeed. Another Microsoft software design with no sense.

The workaround, accessing the site with IE11, doesn't work for me. Just tried with a freshly downloaded copy of youtube-dl, and on a fully up-to-date Windows 10. I've got Python 2.7.13 installed, does that matter?...

Sorry I can't help you more. There are tons of reasons in SSL errors and most of them are beyond my imagination.
If you know how to build Python on Windows, my patch at http://bugs.python.org/issue28182 can give more information for SSL errors.

I've got Python 2.7.13 installed, does that matter?...

Theoretically, no


kbs: Remove oci.kbs version and references

Our KBS logic currently checks out the git.kbs version of code, but then edits the deployment config to override the image based on the oci.kbs version.
This means that there is an assumption that the oci image of the KBS is compatible with the kustomization, which might not not always be the case and in the "always safe" case where the image is built from that exact version of code (e.g. releases), just means that we have to specify it in two places, so there isn't much advantage to this approach.
This PR removes the oci.kbs image and versioning to avoid this potential incompatibility and simplify trustee version updates.
Fixes: #2076

Hi @stevenhorsman !
I just recently I tried e2e tests with KBS and I ended up with those variables copied in many places and exported, and I didn't even know if it was all needed. So this change simplifies a lot!
The downside is that we cannot point to an arbitrary SHA-1 of trustee's repo. How likely are we of hitting that case?
There is the case of trustee releasing an outdated kustomization file, for example, https://github.com/confidential-containers/trustee/blob/v0.10.0/kbs/config/kubernetes/base/kustomization.yaml#L8 . Well, in this case we would probably ask for another release....the problem is maybe the time we take to realize the file is not synchronized.
Also perhaps drop the pinning mechanism on kata side too, so that if we test with the wrong image here then we it test wrong there too (increasing the odds of finding the problem?).

The downside is that we cannot point to an arbitrary SHA-1 of trustee's repo. How likely are we of hitting that case?

I think we should be able to point to a sha with the git.kbs.reference like we had in https://github.com/confidential-containers/cloud-api-adaptor/blob/899b8057336fcd7e2ac4491aaba22cdbe2943146/src/cloud-api-adaptor/versions.yaml#L49, or have I misunderstood your point?

There is the case of trustee releasing an outdated kustomization file, for example, https://github.com/confidential-containers/trustee/blob/v0.10.0/kbs/config/kubernetes/base/kustomization.yaml#L8 . Well, in this case we would probably ask for another release....the problem is maybe the time we take to realize the file is not synchronized.

Yeah, that is a really good point, but it looks like that not being bumped in the v0.10.0 was the reason for the v0.10.1 release: https://github.com/confidential-containers/trustee/releases/tag/v0.10.1, so hopefully it's already included in their release process

Also perhaps drop the pinning mechanism on kata side too, so that if we test with the wrong image here then we it test wrong there too (increasing the odds of finding the problem?).

Yeah - I think that makes sense. I'll create an issue for this.

I've created https://github.com/kata-containers/kata-containers/issues/10379 for the kata side of this and as part of that checked with Ding that it's a fair expectation for them to keep their images pinned and "up-to-date"

The downside is that we cannot point to an arbitrary SHA-1 of trustee's repo. How likely are we of hitting that case?

I think that's not unlikely, but then it would be handled like the coco operator dependency: you cannot pick a given kbs revision directly, you'll get whatever is the default in the kbs kustomize manifest at git.trustee.reference. for in-between-release commits that might be :latest
If a developer wants to deploy a specific oci image and also use the provisioner for their test setup, then they have to fork the trustee repo, hardcode their oci image + tag and use that fork in git.trustee.url/reference in versions.yaml.

Tobin made the comment on the kata issue: https://github.com/kata-containers/kata-containers/issues/10379#issuecomment-2391489223 that suggests maybe we do need set the kustomize image tag, but we could read it from the git reference version. That would be re-adding some of this logic back to the e2e test in I think though, or having a pre-step that does the kustomize edit set image at the same time as we clone the repo?

Tobin made the comment on the kata issue: kata-containers/kata-containers#10379 (comment) that suggests maybe we do need set the kustomize image tag, but we could read it from the git reference version. That would be re-adding some of this logic back to the e2e test in I think though, or having a pre-step that does the kustomize edit set image at the same time as we clone the repo?

oof. maybe that could be done as part of the trustee clone step in the CI?

oof. maybe that could be done as part of the trustee clone step in the CI?

Yeah, that was my thinking.

oof. maybe that could be done as part of the trustee clone step in the CI?

I've added
pushd kbs/config/kubernetes/base/
kustomize edit set image kbs-container-image=*:${KBS_VERSION}
popd

after the clone, which should hopefully do the trick

I've added
pushd kbs/config/kubernetes/base/
kustomize edit set image kbs-container-image=*:${KBS_VERSION}
popd

after the clone, which should hopefully do the trick

I've already spotted an issue with this:
https://github.com/confidential-containers/trustee/pkgs/container/key-broker-service/versions?filters[version_type]=tagged only contains released versions
and ghcr.io/confidential-containers/staged-images/kbs has the commit version, so we might need to do some logic and switch the image as well?

I've already spotted an issue with this:
https://github.com/confidential-containers/trustee/pkgs/container/key-broker-service/versions?filters[version_type]=tagged only contains released versions and ghcr.io/confidential-containers/staged-images/kbs has the commit version, so we might need to do some logic and switch the image as well?

we could use this to resolve releases to a sha:
gh api repos/confidential-containers/trustee/commits/v0.10.1 -q .sha
68607d4300dda5a8ae948e2562fd06d09cbd7eca

we could use this to resolve releases to a sha:
gh api repos/confidential-containers/trustee/commits/v0.10.1 -q .sha
68607d4300dda5a8ae948e2562fd06d09cbd7eca

Yeah, I guess we can do that and always use the staged-image, or alternative do a KBS_VERSION pattern match for vx.y.z and then use the release image in that circumstance?

I've mixed feelings about this piece of code. In one side I'm happy seeing those confusing kbs properties going away from the e2e tests framework, but on the flip side I don't like the half-in-e2e-half-manual setup required to run the KBS based tests as we end up duplicating stuffs on workflows as well as making the execution on dev workstation more obscure.
Anyway, that's going to the right direction IMO.

I've mixed feelings about this piece of code. In one side I'm happy seeing those confusing kbs properties going away from the e2e tests framework, but on the flip side I don't like the half-in-e2e-half-manual setup required to run the KBS based tests as we end up duplicating stuffs on workflows as well as making the execution on dev workstation more obscure.
Anyway, that's going to the right direction IMO.

Agreed, maybe we should look at making a script for the trustee checkout commands and then call this in the e2e framework if KBS deploy is set.


Add support for char8_t (from C++20).

Fixes rdar://39988329.
I believe I've covered all the places where the related types char16_t and char32_t also appear. One question though -- what should be done about stdlib-stable.json? That file is marked as read-only, so I have not added an entry for CChar8 there.

I think it's correct not to modify stdlib-stable.json. That describes the state of the stdlib as it was in Swift 5.0, i.e. what it needs to be compatible with.

I don't quite understand what the status of the discussion is ATM or what my next steps should be:

Do I submit a written proposal for this, specifying the semantics, particularly what happens if/when C adds char8_t and the user specifies -fsigned-char/-funsigned-char?
Wait for some more discussion/consensus between the reviewers?
Something else?

I'll run the tests for now: @swift-ci please smoke test.

I think we should probably just put it on ice, and assign the Radar over to @airspeedswift or @stephentyrone, saying "here's an implementation for if and when we decide we can do this". Sorry for putting it in the list!

Closing the PR since we're not merging this right now but someone can cherry-pick the commit if they'd like to do so.

Do I submit a written proposal for this, specifying the semantics, particularly what happens if/when C adds char8_t and the user specifies -fsigned-char/-funsigned-char?

That's the entire point of creating the char8_t type in the first place.
char16_t and char32_t were created to get around platform specific behavior with wchar_t, char8_t is defined to be unsigned so it works across platforms so it can store the 8 bit code units used by the UTF-8 encoding.


WARNING: linker: Warning: "/apex/com.android.conscrypt/lib64/libc++.so" unused DT entry: unknown processor-specific (type 0x70000001 arg 0x0) (ignoring)

Environment

OS: Windows
Scrcpy version: 2.7
Installation method: Scoop
Device model: Vivo v2043
Android version: 11

The bug
pwsh> scrcpy
scrcpy 2.7 <https://github.com/Genymobile/scrcpy>
C:\Users\RGX\scoop\apps\scrcpy\current\scrcpy-server: 1 file pushed, 0 skipped. 30.1 MB/s (71200 bytes in 0.002s)
[server] INFO: Device: [vivo] vivo V2043 (Android 11)
INFO: Renderer: direct3d
INFO: Texture: 720x1600
WARNING: linker: Warning: "/apex/com.android.conscrypt/lib64/libc++.so" unused DT entry: unknown processor-specific (type 0x70000001 arg 0x0) (ignoring)

Also, if I use --no-video-playback I get something like this:

and a mouse appears in my phone and controllable for no reason. This did not happen when I use using package from winget.

WARNING: linker: Warning: "/apex/com.android.conscrypt/lib64/libc++.so" unused DT entry: unknown processor-specific (type 0x70000001 arg 0x0) (ignoring)

That's a warning from your phone (probably when you execute anything with app_process), not related to scrcpy a priori.

Also, if I use --no-video-playback I get something like this:
…
and a mouse appears in my phone and controllable for no reason.

Yes, that's the expected behavior. By default, video, audio and control are enabled. If you disable "video playback" (which will also completely disable video in your case), then it remains audio and control.
Read the highlights of the 2.5 release for explanations (and how to get the behavior you want): https://github.com/Genymobile/scrcpy/releases/tag/v2.5

This did not happen when I use using package from winget.

Because it was an older version.

Thanks for replying. I didn't check the new features, I thought everything was as usual (because I updated with scoop and no change logs was provided).

That's a warning from your phone (probably when you execute anything with app_process), not related to scrcpy a priori.

But I didn't get this warning before in previous versions (winget)? Should I try using previous versions with scoop?

You can try with older versions, yes (you can use the old official releases, they're still available).

I just tried previous versions and getting the same error.

That's a warning from your phone (probably when you execute anything with app_process).

Can you give an example what you mean by that? My foreground app is Instagram and even though I close that and go to home screen, I still get that warning.
Also winget is providing v2.4 as the latest version.

Also winget is providing v2.4 as the latest version.

Packages on winget or others are maintained by third-party developers.
Anyway, the winget package has issues: #4027 #4389 #4956.
It is not suggested anymore in the documentation: c27ab46efbcab0b9558a91e691d799ffef496c97

Can you give an example what you mean by that?

Scrcpy executes adb shell app_process … to run a Java "executable" on Android. Your vendor (vivo) applies internal changes to AOSP, and that may cause some warnings or issues (there are a lot with vivo phones), whether they are related to scrcpy or not.

Thank you. Appreciate the help.


Consolidate network namespace handling

Fixes: #27449
This is the first step towards fixing #27449 and #27450 - Removing dependence on ip netns from the internal netns lib, and consolidating netns creation/handling into one lib/spot, while not requiring mount pins for every netns creation, the way we currently do - most things (incl. tests) rarely need this.
This PR removes a few netns libs from the repo:

containernetworking/plugins/pkg/ns
vishvananda/netns from all but one spot (pkg/datapath/linux/devices_controller.go, where a sister lib wants a typecast)

in favor of an internal lib "github.com/cilium/cilium/pkg/netns" which avoids pinning netnses, and only has a dep on the unix package.
General goals of the github.com/cilium/cilium/pkg/netns package:

To be used in all Cilium code that needs to create new netnses.
Supports creating unpinned netnses owned by Cilium code.
Supports running closures in netnses previously pinned outside of Cilium code, by other processes (needed for CNI)
No support for pinning netnses owned by Cilium code, as we never actually need to do that.
Stick to stdlib sys/unix

Looking for:

Feedback on pkg/netns API surface/impl/general approach

Generally the intent is to make sure that code that creates netnses is solely responsible for closing them.
Code that is running out-of-process from the process that is the netns owner sometimes does need to run some closure in the netns context, but should not be burdened with netns lifecycle concerns, or be responsible for closing/cleanup, in virtually all cases. The API surface is designed to discourage the current overreliance on named-pin-based disposal/cleanup.
cc @ti-mo to make sure I interpreted his intent correctly.

Should be good for review.

@ti-mo tests are good, resynced against main, PTAL.

@derailed tyvm, comments addressed.

@derailed @ti-mo courteous RFR bump, since everything is addressed/passing and this otherwise seems G2G on my end.

The PR https://github.com/cilium/cilium/pull/29360 added a new helper function ReplaceMacAddressWithLinkName to the old netns.go - like most of the helper functions that were already in the old version that I have removed, it had very little to do with netnses generally, so I moved that func  to pkg/mac/mac_linux.go, which seemed like a better fit.
This is a good sign - it makes less sense to stuff lightly-related utility functions into netns.go in the way we had all been doing, now :D

go mod tidy && go mod vendor come back clean with no git status for me, so not sure what's up with that.

@bleggett Thanks for the PR.
My apologies if this has already been mentioned and buried in this thread but could you write a description of the changes per each commit as documented here (see: item 5).

@bleggett Thanks for the PR.
My apologies if this has already been mentioned and buried in this thread but could you write a description of the changes per each commit as documented here (see: item 5).

I plan to squash them into one proper commit, for now I am leaving each commit separate so reviewers can see deltas, given the PR is a bit big.

for now I am leaving each review comment fix commit separate so reviewers can see deltas, given the PR is a bit big and it's hard to sift thru otherwise.

For any non-trivial PRs, it's generally the before -> after PR delta that matters. Nobody will look at fixup commits, they actually add mental overhead. For this PR, I've done almost a full pass on every round since things changed around so much. Carrying the state of each fixup commit in the back of in mind would be a little difficult. :) What helps is breaking large changes down into logical (not historical) commits is always most helpful.

@bleggett Heads up, I've squashed and rebased on top of main. I've also made a few changes to the core implementation of the API, building on top of os.File instead of raw fds, since that's asking for bugs. I also noticed New and OpenPinned weren't using getFromPath and friends, and I've fixed a few more ignored errors, naming, comments, etc.
I'll poke the remaining codeowners this week so they can have a final look. I'll do a final pass over all non-core changes this week as well.

/test

/test

/test

/test

Resynced w/main - I'm good if you all are.

Looks basically good; I'd like to see it split in to a few more commits. One for the new API, one to move all test code, one for trivial changes, non-trivial changes in individual commits, and lastly one to remove old code. It would make reviewing a lot easier. Thanks!

I at least split into 3, that helps a bit:

Impl + tests
Non-test changes
Deleting old

(mergability check will fail but HTH, ty)

Looks basically good; I'd like to see it split in to a few more commits. One for the new API, one to move all test code, one for trivial changes, non-trivial changes in individual commits, and lastly one to remove old code. It would make reviewing a lot easier. Thanks!

@squeed I at least split into 3, that helps a bit:

Impl + tests
Non-test changes
Deleting old

(mergability check will fail but HTH, ty)

@bleggett Thanks for picking this back up, I fell sick a few weeks ago and didn't manage to continue.
I'm taking another look at this now, there's an issue with makeNetlinkFuncs() that's causing unit tests to fail and presumably also impacts the agent. The netlinkFuncs.*Subscribe members are called lazily, so makeNetlinkFuncs needs to take out a reference to the current netns and stuff it into each of the closures. This needs to be an actual NetNS reference so it doesn't get GC'd.
Also, the Build Commits workflow requires make build works for all commits individually, so I think the middle 2 commits need to be squashed together.
I'll push this up soon.

/test

@bleggett Thanks for picking this back up, I fell sick a few weeks ago and didn't manage to continue.
I'm taking another look at this now, there's an issue with makeNetlinkFuncs() that's causing unit tests to fail and presumably also impacts the agent. The netlinkFuncs.*Subscribe members are called lazily, so makeNetlinkFuncs needs to take out a reference to the current netns and stuff it into each of the closures. This needs to be an actual NetNS reference so it doesn't get GC'd.
Edit: I've added the netns.Current() function to expose getCurrent() for use in makeNetlinkFuncs().

Ah yep, this was an overzealous attempt to fully drop the final explicit "github.com/vishvananda/netns" dep. I'd initially left it like it is currently (explicitly taking the current and wrapping it in a github.com/vishvananda/netns.NetNS handle, as I wasn't sure it would take the right netns), then removed it and didn't see any local failures. I'm fine putting it back like this, it's how it was originally anyway.

@bleggett Already done, take a look at the current pkg/datapath/linux/devices_controller.go.
Looks like CI is still failing even after fixing makeNetlinkFuncs in ci-runtime. Do you have any cycles to investigate the remaining failures?

Can I get someone to /test?
I can't repro any unit test failures vs a vis main but need to check the conformance suites.

/test

@ti-mo et al - tests seem good to me locally, and the conformance tests are passing now.
I note that one each of the gateway API and ginkgo subtests are failing, I do not think that's related to this, but am happy to be corrected.
This should be good to merge if everyone is happy.

@bleggett Thanks for taking a look! What did you end up changing to address the failures?
Rebasing onto main to pick up https://github.com/cilium/cilium/pull/31017 for https://github.com/cilium/cilium/actions/runs/8072988990/job/22055814369.

@bleggett Thanks for taking a look! What did you end up changing to address the failures?
Rebasing onto main to pick up #31017 for https://github.com/cilium/cilium/actions/runs/8072988990/job/22055814369.

Absolutely nothing, just resynced to main :sweat_smile:

TY all - apologies this took a bit of time :sweat_smile:


Add sniff for calling WP_Query->get_posts()

Is your feature request related to a problem?
Calling get_posts() directly from WP_Query may be mistaken as a getter for the posts property, however, it is not. Calling this outside of the constructor will add an additional query.
$args = [
  'posts_per_page'      => '10',
  'post_type'           => 'post',
  'post_status'         => 'publish',
  'no_found_rows'       => true,
  'ignore_sticky_posts' => true,
  'cat'                 => 123
];
$my_query = new WP_Query( $args );
$my_posts = $my_query->get_posts();

Describe the solution you'd like
Call the posts property rather than the method.
$my_posts = $my_query->posts;

Just some notes:

This is only the case if new WP_Query() is called with the $query parameter. If the optional $query parameter is not passed, no extra database query is executed.
As the WP_Query::get_posts() function itself does not take a $query parameter, this would mean that the $query would either need to be set ahead of the function call by setting the property, by calling the query() method or by hooking into one of the query filtering hooks.

More than anything, I seriously wonder why the WP_Query::get_posts() method is public in the first place, but that is a discussion for WP Core.
I think it may be more appropriate to see if this can be solved in WP Core by checking if the $query is the same as previously used and if so, short-circuiting the function to return the previously retrieved results as saved in the $posts property.

Just some notes:

This is only the case if new WP_Query() is called with the $query parameter. If the optional $query parameter is not passed, no extra database query is executed.
As the WP_Query::get_posts() function itself does not take a $query parameter, this would mean that the $query would either need to be set ahead of the function call by setting the property, by calling the query() method or by hooking into one of the query filtering hooks.

More than anything, I seriously wonder why the WP_Query::get_posts() method is public in the first place, but that is a discussion for WP Core.
I think it may be more appropriate to see if this can be solved in WP Core by checking if the $query is the same as previously used and if so, short-circuiting the function to return the previously retrieved results as saved in the $posts property.

It does seem to do with user error though. For example, if say someone really wanted to call a new WP_Query(), they should be doing the below:
$my_query = new WP_Query();
$my_posts = $my_query->query( $args );

However, I do agree with your sentiment behind questioning why WP_Query::get_posts() is a public method, so I've opened a Trac ticket for it.

It does seem to do with user error though. For example, if say someone really wanted to call a new WP_Query(), they should be doing the below:
$my_query = new WP_Query();
$my_posts = $my_query->query( $args );

However, I do agree with your sentiment behind questioning why WP_Query::get_posts() is a public method, so I've opened a Trac ticket for it.


GITHUB_SHA

How can we use GITHUB_SHA when using with Github Actions? Are you going to release an adapter for it?

        KNAPSACK_PRO_COMMIT_HASH: ${{ github.sha }}
        KNAPSACK_PRO_BRANCH: ${{ github.ref }}

The above needs to be added to the env section to set environment variables needed for Knapsack Pro

Nevermind. Upgraded to the latest version of the gem.


[WIP][SCB-1735] Support Alpha call compensation timeout strategy

Use property reverseTimeout of @Compensable  to set the timeout of the compensation method

Coverage decreased (-0.8%) to 80.013% when pulling 8200f3c063e938a1d6af2402e98995af2e26862c on coolbeevip:SCB-1735 into d2b843433dc316f2532e334b7bcf1a521ee3a6b8 on apache:master.


Add missing semi colon

Fixes problem with concatenating javascript files.

FWIW the motion module in Foundation has the same API but is more robust. We'll deprecate this one eventually.

Thanks, will pass on the info to FoundationPress which includes it in their default config.


Add support for serde

This is an initial PR that adds support for the yet-to-be-released serde version 0.5.0, so please don't merge this in until that lands. I wanted to submit it to get your input on the design and if you see if there are any gaps in the implementation.

Wow!  This will take me a while to read through, but it looks great!  The one thing that I noticed right off the bat is that we could probably unify most of the unit tests.  Instead of having test and test_serde, we could just have test, with proxy functions that use both implementations.
For example
fn proxy_encoded_size<T: ...>(obj: &T) -> usize {
    let ser_size = bincode::encoded_size(obj);
    let serde_size = bincode::serialized_size(obj);
    assert_eq(ser_size, serde_size);
    return ser_size;
}

I noticed that you already did that for the the_same function where you compare the existing implementation with the serde implementation.
I might hold off on reading the rest of the code until I can build and run it myself.  At the very least I'll get a refresher on Serde tonight.

I am worried about how both of the implementations will be exposed though.
encode and to_vec are identical in terms of functionality, and so is decode and from_slice.
While they should be interchangeable, I think it'll be confusing to library users.

@TyOverby Ping. Servo really wants this. Is there a way I can help get this landed?

I won't be able to merge this until serde hits 0.5.0 on crates.io, but I will review the code later tonight.  It would be great if using serde also sidestepped #41!

@erickt: I downloaded your patch and wired it up the path for Serde to your github repo so I could check it with 0.5.0, but there are a set of errors that indicate that serde can't be used with rust-1.0.  I thought this was not the case?
$ cargo build
   Compiling aster v0.4.0
   Compiling quasi v0.3.1
   Compiling advapi32-sys v0.1.2
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\item.rs:233:24: 233:48 error: failed to resolve. Could not find `Constness` in `syntax::ast`
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\item.rs:233             constness: ast::Constness::NotConst,
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\item.rs:233:24: 233:48 error: unresolved name `ast::Constness::NotConst`
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\item.rs:233             constness: ast::Constness::NotConst,
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\item.rs:247:16: 247:30 error: use of undeclared type name `ast::Constness`
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\item.rs:247     constness: ast::Constness,
                                                                                ^~~~~~~~~~~~~~
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\item.rs:261:26: 261:47 error: failed to resolve. Could not find `Constness` in `syntax::ast`
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\item.rs:261         self.constness = ast::Constness::Const;
                                                                                ^~~~~~~~~~~~~~~~~~~~~
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\item.rs:261:26: 261:47 error: unresolved name `ast::Constness::Const`
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\item.rs:261         self.constness = ast::Constness::Const;
                                                                                ^~~~~~~~~~~~~~~~~~~~~
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\method.rs:28:16: 28:30 error: use of undeclared type name `ast::Constness`
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\method.rs:28     constness: ast::Constness,
                                                                                ^~~~~~~~~~~~~~
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\method.rs:50:24: 50:48 error: failed to resolve. Could not find `Constness` in `syntax::ast`
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\method.rs:50             constness: ast::Constness::NotConst,
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\method.rs:50:24: 50:48 error: unresolved name `ast::Constness::NotConst`
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\method.rs:50             constness: ast::Constness::NotConst,
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~~
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\method.rs:72:26: 72:47 error: failed to resolve. Could not find `Constness` in `syntax::ast`
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\method.rs:72         self.constness = ast::Constness::Const;
                                                                                ^~~~~~~~~~~~~~~~~~~~~
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\method.rs:72:26: 72:47 error: unresolved name `ast::Constness::Const`
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\aster-0.4.0\src\method.rs:72         self.constness = ast::Constness::Const;
                                                                                ^~~~~~~~~~~~~~~~~~~~~
error: aborting due to 10 previous errors
Build failed, waiting for other jobs to finish...
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\quasi-0.3.1\src\lib.rs:11:43: 11:66 error: unstable feature
c:/Users/Ty\.cargo\registry\src\github.com-1ecc6299db9ec823\quasi-0.3.1\src\lib.rs:11 #![cfg_attr(not(feature = "with-syntex"), feature(rustc_private))]
                                                                                ^~~~~~~~~~~~~~~~~~~~~~~
note: this feature may not be used in the stable release channel
error: aborting due to previous error
Could not compile `aster`.

To learn more, run the command again with --verbose.

@erickt Ping. If you don't have time for this, would you prefer that I pick up the patch and push it through?

@pcwalton: Just finally released 0.5.0 and updated this PR to build with it!

@erickt @pcwalton: Is this going to build with stable release?  I thought the syntex crate was supposed to build on stable?

Ok, I got it building with stable (testing requires nightly, but that's acceptable).
There's a few changes that I'd like to make though, so I'll take this PR from here.

Yay merged! :tada:


Show the filename on unsupported shebang error

Motivation for this change
Simplifies debugging packages with large set of scripts.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Fits CONTRIBUTING.md.

This should go to staging.

You mean the staging branch?

Yes

@Ericson2314 target changed to staging

ping. LGTM. Looks useful to me.


Replicate enter in loop calling a simple model with run

Hi,
I'm looking at my logs and I noticed that a simple call to Replicate sometimes causes an infinite loop until the process is finished.
Code:
prompt_input = "give me a hello world in python"

new_prompt = replicate.run(
    "justmalhar/meta-llama-3.2-3b:5c11ba722650525b0ab1a877d1fde44c5dab7445072e9e8e1a6a47ec2026a4d4",
    input={
        "top_k": 50,
        "top_p": 0.9,
        "prompt": prompt_input,
        "max_tokens": 1024,
        "min_tokens": 0,
        "temperature": 0.6,
        "presence_penalty": 0,
        "frequency_penalty": 0,
    },
)

# get output
output = ""
for item in new_prompt:
    output += str(item)

It call INFO 2024-10-23 14:00:18,443 _client HTTP Request: GET https://api.replicate.com/v1/predictions/br55y80wx5rgp0cjq6pr9m5k9m "HTTP/1.1 200 OK"  repeatedly.

I think that he do it while the machine still in booting mode, is this?

This is expected behaviour. The client is polling the status of the prediction until the prediction completes.


Segmentation Fault (stack-overflow) on a crafted PDF file

A crafted pdf file causes a segmentation fault, stack-overflow reported by LLVM ASan, causing a denial of service and there is a chance of code execution.
qpdflibsegfault.zip
Details are attached inside the zip  called 'gdb log'.
I have tested with libqpdf and qpdf itself, debugged and used LLVM ASan to identify the possible root cause.
A pdf POC is inside the zip.
Versions tested: 6.0.0 and 8.0.2.
c code to test:
QPDF pdf;
pdf.processFile("crafted pdf.pdf"); //segmentation fault here.

running:

== simple source code using libqpdf ==
$ cat openpdf.cc
#include
#include <string.h>
#include <stdlib.h>
#include <qpdf/QPDF.hh>
#include <qpdf/QUtil.hh>
#include <stdint.h>
int main(void)
{
int pageno = 2;
try
{
QPDF pdf;
pdf.processFile("fuzzed.pdf");
std::vector pages = pdf.getAllPages();
if ((pageno < 1) || (static_cast<size_t>(pageno) > pages.size()))
{
exit(1);
}
}
catch (std::exception& e)
{
return 0;
}
return 0;
}
$ clang++-6.0 -L/usr/local/lib -lz -ljpeg -lqpdf -lpthread *.cc -g -o openpdf
$ ./openpdf
WARNING: fuzzed.pdf (trailer, offset 191702): expected dictionary key but found non-name object; inserting key /QPDFFake1
WARNING: fuzzed.pdf (trailer, offset 191700): expected dictionary key but found non-name object; inserting key /QPDFFake1
WARNING: fuzzed.pdf (trailer, offset 191698): expected dictionary key but found non-name object; inserting key /QPDFFake1
(...)
WARNING: fuzzed.pdf (trailer, offset 328678): unknown token while reading object; treating as string
WARNING: fuzzed.pdf (trailer, offset 328700): unexpected EOF
WARNING: fuzzed.pdf (trailer, offset 328700): parse error while reading object
Segmentation fault (core dumped)

Testing qpdf 6.0.0:

[pushdword@localhost qpdflib]$ qpdf fuzzedpdf.pdf test.pdf
Segmentation fault (core dumped)
[pushdword@localhost qpdflib]$ qpdf --version
qpdf version 6.0.0
Copyright (c) 2005-2015 Jay Berkenbilt
This software may be distributed under the terms of version 2 of the
Artistic License which may be found in the source distribution.  It is
provided "as is" without express or implied warranty.

testing qpdf 8.0.2

$ qpdf fuzzed.pdf test.pdf
WARNING: fuzzed.pdf (trailer, offset 191702): expected dictionary key but found non-name object; inserting key /QPDFFake1
WARNING: fuzzed.pdf (trailer, offset 191700): expected dictionary key but found non-name object; inserting key /QPDFFake1
WARNING: fuzzed.pdf (trailer, offset 191698): expected dictionary key but found non-name object; inserting key /QPDFFake1
WARNING: fuzzed.pdf (trailer, offset 191696): expected dictionary key but found non-name object; inserting key /QPDFFake1
(... a very long warning repetition ...)
WARNING: fuzzed.pdf (trailer, offset 54774): expected dictionary key but found non-name object; inserting key /QPDFFake1
WARNING: fuzzed.pdf (trailer, offset 328678): unknown token while reading object; treating as string
WARNING: fuzzed.pdf (trailer, offset 328700): unexpected EOF
WARNING: fuzzed.pdf (trailer, offset 328700): parse error while reading object
Segmentation fault (core dumped)

Use CVE-2018-9918

Thanks. I'll take a look when I can. Pretty soon.

How depressing. I put a lot of effort into getting rid of recursion in parsing to avoid falling for this kind of trap. Now qpdf recovers well enough that it is actually able to successfully parse the file, and then it all blows up because delete ends up recursively calling destructors, which is what is blowing out the stack. What's happening is basically the same as in this program:
class A
{
  public:
    A(A* a)
    {
        this->a = a;
    }
    ~A()
    {
        delete a;
    }
  private:
    A* a;
};

int main()
{
    A* a = new A(0);
    for (int i = 0; i < 300000; ++i)
    {
        a = new A(a);
    }
    delete a;
    return 0;
}

Even though the recursive data structure is created iteratively, destruction is recursive. This happens to be an invalid file because there are dictionaries without keys that qpdf is supplying fake keys for, but you could create this exact problem using a syntactically valid PDF that just has a very deeply nested object. For example, changing each << and >> to [ and ] and putting it in a valid spot in the file, such as as the value of a unused dictionary key in trailer, would produce a valid file that would cause this same problem. A core feature of qpdf is that it uses smart pointers and STL data structures so that the user almost never has to think about memory allocation. It is not clear to me how I'm going to fix this other than perhaps imposing some arbitrary limit on the depth that any object is allowed to go to.

I'm going to fix it by just imposing an arbitrary limit on how deeply nested the data structure expressed in a direct object can be. This is a very easy fix.

Index: qpdf/ChangeLog
===================================================================
--- qpdf.orig/ChangeLog
+++ qpdf/ChangeLog
@@ -1,3 +1,8 @@
+2018-04-15  Jay Berkenbilt<EMAIL_ADDRESS>+
+	* Arbitrarily limit the depth of data structures represented by
+	direct object. This is CVE-2018-9918. Fixes #202.
+
 2018-03-06  Jay Berkenbilt<EMAIL_ADDRESS>
 	* 8.0.2: release
Index: qpdf/libqpdf/QPDFObjectHandle.cc
===================================================================
--- qpdf.orig/libqpdf/QPDFObjectHandle.cc
+++ qpdf/libqpdf/QPDFObjectHandle.cc
@@ -1487,12 +1487,26 @@ QPDFObjectHandle::parseInternal(PointerH

 	  case QPDFTokenizer::tt_array_open:
 	  case QPDFTokenizer::tt_dict_open:
-            olist_stack.push_back(std::vector<QPDFObjectHandle>());
-            state = st_start;
-            offset_stack.push_back(input->tell());
-            state_stack.push_back(
-                (token.getType() == QPDFTokenizer::tt_array_open) ?
-                st_array : st_dictionary);
+            if (olist_stack.size() > 500)
+            {
+		QTC::TC("qpdf", "QPDFObjectHandle too deep");
+                warn(context,
+                     QPDFExc(qpdf_e_damaged_pdf, input->getName(),
+                             object_description,
+                             input->getLastOffset(),
+                             "ignoring excessively deeply nested data structure"));
+                object = newNull();
+                state = st_top;
+            }
+            else
+            {
+                olist_stack.push_back(std::vector<QPDFObjectHandle>());
+                state = st_start;
+                offset_stack.push_back(input->tell());
+                state_stack.push_back(
+                    (token.getType() == QPDFTokenizer::tt_array_open) ?
+                    st_array : st_dictionary);
+            }
 	    break;

 	  case QPDFTokenizer::tt_bool:
Index: qpdf/qpdf/qpdf.testcov
===================================================================
--- qpdf.orig/qpdf/qpdf.testcov
+++ qpdf/qpdf/qpdf.testcov
@@ -335,3 +335,4 @@ QPDFObjectHandle numeric non-numeric 0
 QPDFObjectHandle erase array bounds 0
 qpdf-c called qpdf_check_pdf 0
 QPDF xref loop 0
+QPDFObjectHandle too deep 0
Index: qpdf/qpdf/qtest/qpdf/issue-146.out
===================================================================
--- qpdf.orig/qpdf/qtest/qpdf/issue-146.out
+++ qpdf/qpdf/qtest/qpdf/issue-146.out
@@ -1,7 +1,5 @@
 WARNING: issue-146.pdf: file is damaged
 WARNING: issue-146.pdf: can't find startxref
 WARNING: issue-146.pdf: Attempting to reconstruct cross-reference table
-WARNING: issue-146.pdf (trailer, offset 20728): unknown token while reading object; treating as string
-WARNING: issue-146.pdf (trailer, offset 20732): unexpected EOF
-WARNING: issue-146.pdf (trailer, offset 20732): parse error while reading object
+WARNING: issue-146.pdf (trailer, offset 695): ignoring excessively deeply nested data structure
 issue-146.pdf: unable to find trailer dictionary while recovering damaged file


QE ArduCam Mini 5MP Plus

Hi,
Does anyone have an idea about the Quantum Efficiency of this camera module?
I'm looking at QE behavior at different wavelengths in order to theoretically convert the photons hitting the image plane into generated photoelectrons.
Thanks in advance,
giogrimo

The Omnivision didn't give QE information for their cameras. You might need to check other senosr vendor like Aptina.


How to use dedicated Browserless Docker container instance with PuppeteerSharp?

I would like to completely isolate all things related to hosting/running Chrome from my core application runtimes. So I'm hosting our own Browserless.io Docker container (https://hub.docker.com/r/browserless/chrome/dockerfile). But this results in various web socket connections dropped during sessions.
Can you confirm this should work at all? Maybe I'm not using the right version/tag? See https://hub.docker.com/r/browserless/chrome/tags.
Any guidance on this, would be very much appreciated.

Which kind of app is your puppeteer app? Console/Web Core/Framework...

Sorry about this! I'm using ASP.Net Core 3.0.
I got it working after setting AppMode = false.


Request Methods

A Response Method is an RPC method that is attached to an RPC result by the SDK for convenience. This allows for results that represent their own focused APIs. For example:
const pet = House.getPet()
pet.walk()

In the example above, pet.walk() is an RPC method that takes a pet object as a parameter. Calling pet.walk() via the Response Method syntax allows contextual APIs to be called with simpler method signatures. These methods may or may not be hidden via the rpc-only tag to avoid Apps calling them without valid parameters, if desired.

home come transforms has to happen within the Transport module?
const transforms = {"methods":{"aMethod":{"type":"string","x-method":"Simple.method","x-this-param":"accessory"}}}
return Transport.send('simple', 'methodWithMethodAttribute', { title }, transforms)

vs.
const transforms = {"methods":{"aMethod":{"type":"string","x-method":"Simple.method","x-this-param":"accessory"}}}
return Transport.send('simple', 'methodWithMethodAttribute', { title }).then(x => Results.transform(x, transforms))

Then you don't have to pass transforms through a bunch of methods and it stays localized to the thing that needs the transforming , the module

:tada: This PR is included in version 1.4.1-next.1 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.6.2-next.1 :tada:
The release is available on:

npm package (@next dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


feat: New Avro entrypoint

Here is the new proposed avro entrypoint 🥳
Goal
Provide an unified, easy and extensible way of encoding and decoding avro. Unified and easy as it should only have methods on the Avro class (no streams, no builders, ...). Extensible as avro can be used in multiple contexts, and some requires adding some metadata (data files) or schema info (confluent schema registry, single object).
Closes #163
Requirements to merge:

Encode everything at root (not only records)
Encode natively to bytes

Breaking changes:

No more AvroInputStream and AvroOutputStream, thanks to the encodeSequenceXxx and decodeSequenceXxx replacements with AvroSerializationDelegate
No more encoding to and from Generic data, as the final goal of this library is to encode and decode in avro, not in Generic data, and also another step of removing the java avro library (no more public api in java, except for the Schemas)
encodeToByteArray and decodeFromByteArray will now be pure avro (no header, just the encoded data itself without decorators) while before it was in Data mode (avro data file format)

Tickets to open after that:

Create the necessary serializers to encode and decode Generic data (from bytes to generic, and from generic to bytes)

The methods
Each base method having a Schema and a SerializerStrategy or DeserializerStrategy has its related extension method to automagically get the SerializerStrategy or DeserializerStrategy and the schema from the reified type.
We now fully support java streams and ByteArray.
We are now able of streaming the avro contents with the new encodeSequence and decodeSequence methods, useful for reading avro files by example!

Example of usage

Just encode a string payload

val bytes = Avro2.encodeToByteArray("a value to be encoded")
val value = Avro2.decodeFromByteArray<String>(bytes)

Just encode a record payload

val bytes = Avro2.encodeToByteArray(ARecord(stringField = "something", intField = 42))
val value = Avro2.decodeFromByteArray<ARecord>(bytes)

Encode to the avro data file format

val avro = Avro {
    serializationDelegate = Builtins.DataFile
}
File("file.avro").outputStream().use { outputStream ->
    avro.encodeSequenceToStream(sequenceOf("a", "b", "c"), outputStream)
}
File("file.avro").inputStream().use { inputStream ->
    val sequence = avro.decodeSequenceFromStream<String>(inputStream)
}

Just generate a schema from a type:

val schema = Avro2.schema<MySerializedType>()

Write kafka events using confluent schema registry (⚠ this is an example. It doesn't match exactly what should be done during serialization. It only shows how extensible is the new api)

val avro = Avro {
    serializationDelegate = ConfluentSchemaRegistrySerializationDelegate(schemaRegistryClient)
}
val kafkaEventBytes: ByteArray = TODO()
val event = avro.decodeFromByteArray<MyEvent>(kafkaEventBytes)
val bytes = avro.encodeToByteArray(MyEvent(...))

// ...

@OptIn(ExperimentalSerializationApi::class)
class ConfluentSchemaRegistrySerializationDelegate(private val schemaRegistryClient: SchemaRegistryClient) : CustomAvroSerializationDelegate() {
    override val encodedAs: EncodedAs
        get() = EncodedAs.BINARY

    override fun <T> encodeValue(avro: Avro2, writerSchema: Schema, serializer: SerializationStrategy<T>, value: T, outputStream: OutputStream) {
        val schemaId = schemaRegistryClient.registerOrGetId(writerSchema)
        outputStream.write(0x0) // magic byte
        outputStream.write(ByteBuffer.allocate(4).putInt(schemaId).array()) // schema id
        newEncoder(avro, writerSchema, serializer, outputStream)
            .encodeSerializableValue(serializer, value)
    }

    override fun <T> decodeValue(avro: Avro2, readerSchema: Schema, deserializer: DeserializationStrategy<T>, inputStream: InputStream): T {
        check(inputStream.read() == 0x0) { "Not a valid confluent avro format, bad magic byte" }

        val schemaId = ByteBuffer.allocate(4).put(inputStream.readNBytes(4)).getInt()
        val writerSchema = schemaRegistryClient.getSchemaById(schemaId)
        return newDecoder(avro, readerSchema, writerSchema, deserializer, inputStream)
            .decodeSerializableValue(deserializer)
    }

    override fun <T> encodeSequence(avro: Avro2, writerSchema: Schema, serializer: SerializationStrategy<T>, sequence: Sequence<T>, outputStream: OutputStream) = fail()
    override fun <T> decodeSequence(avro: Avro2, readerSchema: Schema, deserializer: DeserializationStrategy<T>, inputStream: InputStream) = fail()
    private fun fail(): Nothing = throw UnsupportedOperationException("ConfluentSchemaRegistrySerializationDelegate does not support sequences")
}

Seems like you are aiming for static access, while in the current version, there is a default instance (Avro.default) to use, but also the option to create a custom instance. This is quite important when working with custom logical types .. am I missing st. or would we loose the option of configuring a custom instance?


Chargepad does not work in all situations. (1.12.1)

Please use the search functionality before reporting an issue. Also take a look at the closed issues!
Issue description:
Chargepad does not load static devices, although it has a connection, even if the Chargepad itself is connected to the power supply and should work as a relay/transmitter.
In the http://wiki.aidancbrady.com/wiki/Chargepad statement, "Any rechargeable item you have in your inventory will be charged while standing on it."
Unfortunately, when standing on a charger with a discharged Configurator, it is not charged.
Steps to reproduce:

Check why the rear Chargepad connector does not have power.
Check if the Configurator is in the inventory or in the hand is not charging while standing on Chargepad.

Version (make sure you are on the latest version before reporting):
Forge: 1.12.1
Mekanism: 1.12.1
**Other relevant version:
If a (crash)log is relevant for this issue, link it here: (It's almost always relevant)
[gist/pastebin/etc link here]

I cannot replicate this in the latest version. The code also hasn't changed for the whole release of 1.12.1.
My steps to reproduce:

Place a chargepad & a full energy cell on the back
Have a bunch of empty configurators in my inventory (hotbar, and some at random places in my inventory)
Stand on the chargepad
Look at inventory and see them charged one by one, starting at the hotbar on the left, to the right, main inventory from top left to bottom right.

Are you having an (empty) chargepad in your inventory? That also stores energy, you only don't see the damage indicator indicating it's storing energy.
Try standing on a charge pad for atleast one minute. It should atleast give some particles indicating it's active when you stand on it.


Support Annotations

Feature Request
The original package (the master branch) had support for annotations so that it was possible to configure entities like
use SimpleThings\EntityAudit\Mapping\Annotation as Audit;

/**
 * @ORM\Entity()
 * @Audit\Auditable()
 */
class Entity {
    /**
     * @ORM\Column(name="updated_at", type="datetime", nullable=true)
     * @Audit\Ignore()
     */
    protected $updatedAt;
}

Are there any plans to merge that functionality at some point?

I'm not sure any new maintainer are aware about the changes made to the previous master branch.
Any improvement is welcomed if you can find the PR with this feature and if you want to make the new PR.

These are the commits that added that feature: https://github.com/sonata-project/EntityAuditBundle/commits/legacy_master/src/SimpleThings/EntityAudit/Mapping/Annotation

@AlexeyKosov i created a different audit bundle that also uses annotations and has extra features, https://packagist.org/packages/drjele/doctrine-audit
(if my comment should not be here, let me know and i will delete it)


*: ignore unsound openssl::bio::MemBio::get_buf

What is changed and how it works?

Issue Number: Close https://github.com/tikv/tikv/issues/17291

What's Changed:
Ignore RUSTSEC-2024-0357 (unsound `openssl::bio::MemBio::get_buf`) as we
have banned relevant APIs in the TiKV codebase.

Also, TiKV temporarily pins OpenSSL 1.1.1 because OpenSSL 3.0.0 has
performance regressions. See details in deny.toml.

Related changes

Need to cherry-pick to the release branch

Check List
Tests

No code

Release note

None

Why not update openssl's version to a unaffected version instead?

/run-cherry-picker