panic: no valid local IP address found

(NOTE: This is used to report product bugs:
To report a security vulnerability, please visit https://istio.io/about/security-vulnerabilities
To ask questions about how to use Istio, please visit https://discuss.istio.io)
Bug description
Some of my pods are crashing with an error saying: "panic: no valid local IP address found"
The full description is:
`panic: no valid local IP address found
goroutine 1 [running]:
istio.io/istio/tools/istio-iptables/pkg/cmd.constructConfig(0xc000a7dd88)
istio.io/istio/tools/istio-iptables/pkg/cmd/root.go:120 +0xaf4
istio.io/istio/tools/istio-iptables/pkg/cmd.glob..func1(0x3c58c20, 0xc000b10200, 0x0, 0x10)
istio.io/istio/tools/istio-iptables/pkg/cmd/root.go:48 +0x26
github.com/spf13/cobra.(*Command).execute(0x3c58c20, 0xc000b10100, 0x10, 0x10, 0x3c58c20, 0xc000b10100)
github.com/spf13/cobra@v1.0.0/command.go:846 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0x3c58ec0, 0x0, 0x0, 0x0)
github.com/spf13/cobra@v1.0.0/command.go:950 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
github.com/spf13/cobra@v1.0.0/command.go:887
main.main()
istio.io/istio/pilot/cmd/pilot-agent/main.go:487 +0x31`
[ ] Docs
[ ] Installation
[X] Networking
[X] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
Expected behavior
Pod should start up successfully.
Steps to reproduce the bug
Kinda hard to reproduce directly, but it happens frequently right now on my cluster.
Version (include the output of istioctl version --remote and kubectl version --short and helm version --short if you used Helm)
Istio version: 1.8.2
Kubernetes server: 1.19.7
How was Istio installed?
Istio was installed using Helm.
Environment where the bug was observed (cloud vendor, OS, etc)
Cloud vendor: AKS,
OS: Ubuntu 18.04.5 LTS
Kernel: 5.4.0-1040-azure
Runtime: containerd://1.4.4+azure
Additionally, please consider running istioctl bug-report and attach the generated cluster-state tarball to this issue.
Refer cluster state archive for more details.

What kind of CNI is this? The code is:
	addrs, err := net.InterfaceAddrs()
	if err != nil {
		return nil, err
	}

	for _, a := range addrs {
		if ipnet, ok := a.(*net.IPNet); ok && !ipnet.IP.IsLoopback() {
			return ipnet.IP, nil
		}
	}
	return nil, fmt.Errorf("no valid local IP address found")

So it seems like there is no pod IP provisioned for the pod somehow?

We are using Azure CNI, and there is some limitations on how many IP's the Azure CNI can propagate, but we are nowhere near that limit. (AFAIK).
We are looking into the Azure CNI limitation is upon node level. The default limit on node level seems like it is 40, but is way higher on cluster level.
Thank you @howardjohn for pointing into the CNI direction.


TODO Items

For collaborations

Please check my work so far.

Hey @ohguitar58 I'll take a look.

GitHub should have a capital H.


fix: support POST-only API of go-ipfs 0.5

TL;DR
This PR ensures orbit-db remains compatible with HTTP API exposed by go-ipfs 0.5 (https://github.com/ipfs/go-ipfs/issues/7109) by switching to<EMAIL_ADDRESS>in tests and adding interop note in README.
Changes

added interop note to README about compatible http client version

note about version limitation can be removed when #767 lands

bumped versions of http-client and go-ipfs used in tests to ensure it works with latest go-ipfs 0.5.0-rc2

Why?
go-ipfs 0.5  will block GET commands on the API port (https://github.com/ipfs/go-ipfs/pull/7097), requiring every  command (RPC) to be sent as HTTP POST request.
Why v41.0.1? It sends only POST requests while maintaining the old JS API
(breaking changes that introduced support for async iterators landed in 42.x)
cc @aphelionz @stebalien @achingbrain

Thank you @lidel!!

The go-ipfs 0.5.0 release will be cut later today. Any chance this can be merged soon?

@Stebalien Merged to master for now, before our own release. Hope that's helpful!

It does. Thanks!

@lidel @Stebalien should the practice org-wide in orbitdb be for us to utilize<EMAIL_ADDRESS>and<EMAIL_ADDRESS>

Yes (0.5.0 just shipped) :)

Is there any chance we could cut a patch release with this change so users can upgrade go-ipfs without migrating to async/await?


docs(changelog): update changelog

Update changelog for v0.2.0

Updated changelog as suggested, updated date, updated version


Add route for connecting a wallet

Context

#80

Todos

It should receive (from "Connect your wallet" form):

wallet address URL
amount
amount type

Flow:

validate request body
fetch wallet address information

if the wallet address is not valid throw an error

request a grant for quotes and outgoing payments
it should return:

interact redirect URL
continue URI
continue token

Keys are now generated when the extension is installed. The users will need to upload the public key in the wallet provider UI.


git clone failed

When I  git clone --recursive https://github.com/jts/nanopolish.git
Cloning into 'nanopolish'...
remote: Enumerating objects: 42, done.
remote: Counting objects: 100% (42/42), done.
remote: Compressing objects: 100% (29/29), done.
error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull function.
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
help me thanks

This isn't an error I can help with, it looks like a problem with your connection to github, sorry.

This isn't an error I can help with, it looks like a problem with your connection to github, sorry.


bugfix

Please merge and release ASAP

Could you also add a new field «category» in the apn array?

Yeah good point, should add category also! :)


[BEAM-13171] Support for stopReadTime on KafkaIO SDF

There is already the support for startReadTime using SDF when the Kafka version is supported.
I want to add the support for stopReadTIme so we can extract messages from Kafka only up to a point in time and then the task will be finished.
One use case: when you want to only re-process (re-read) a period of time for a Kafka topic in your pipeline.

[x] [Choose reviewer(s)] R:@kennknowles , R:@jbonofre
[x] Format the pull request title.
[x] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement. Is it?
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R:@kennknowles
R:@jbonofre

R: @lukecwik

Also
14:44:21 * What went wrong:
14:44:21 Execution failed for task ':sdks:java:io:kafka:spotlessJavaCheck'.
14:44:21 > The following files had format violations:
14:44:21       sdks/java/io/kafka/src/test/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFnTest.java
14:44:21           @@ -186,9 +186,7 @@
14:44:21            ······}·else·if·(timeToSearch·==·this.stopOffsetForTime.getValue().getMillis())·{
14:44:21            ········returnOffset·=·this.stopOffsetForTime.getKey();
14:44:21            ······}
14:44:21           -······return·ImmutableMap.of(
14:44:21           -··········topicPartition,
14:44:21           -··········new·OffsetAndTimestamp(returnOffset,·timeToSearch));
14:44:21           +······return·ImmutableMap.of(topicPartition,·new·OffsetAndTimestamp(returnOffset,·timeToSearch));
14:44:21            ····}
14:44:21
14:44:21            ····@Override
14:44:21   Run './gradlew :sdks:java:io:kafka:spotlessApply' to fix these violations.

Run Java PreCommit

@lukecwik applied the requested changes and all checks are green now - ready for final review and merge.
Thanks for the hints :+1:

In case of using stopReadTime, does it mean that a pipeline becomes bounded and it will be stopped once all records are read?

Yes, correct.

@lukecwik - could you take another look at this change please?

Thanks for the contribution.
A follow-up to this change would be to make KafkaIO SDF a bounded SDF when there is a definitive stop defined.

Seems that this change introduced Duration ambiguity and Java PreCommit started to fail with:
01:21:19 > Task :sdks:java:io:kafka:compileTestJava
01:21:19 /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Commit@2/src/sdks/java/io/kafka/src/test/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFnTest.java:155: error: reference to Duration is ambiguous
01:21:19     public synchronized ConsumerRecords<byte[], byte[]> poll(Duration timeout) {
01:21:19                                                              ^
01:21:19   both class java.time.Duration in java.time and class org.joda.time.Duration in org.joda.time match
01:21:19 /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Commit@2/src/sdks/java/io/kafka/src/test/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFnTest.java:52: error: a type with the same simple name is already defined by the single-type-import of Duration
01:21:19 import org.joda.time.Duration;
01:21:19 ^
01:21:21 /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Java_Commit@2/src/sdks/java/io/kafka/src/test/java/org/apache/beam/sdk/io/kafka/ReadFromKafkaDoFnTest.java:300: error: reference to Duration is ambiguous
01:21:21     Instant stopReadTime = startReadTime.plus(Duration.millis(2000));
01:21:21                                               ^
01:21:21   both class java.time.Duration in java.time and class org.joda.time.Duration in org.joda.time match

Interesting that for some reasons it was not caught by this PR checks,

It is likely two concurrent changes happening at the same time independently getting tested but when combined they produced this failure.

"In case of using stopReadTime, does it mean that a pipeline becomes bounded and it will be stopped once all records are read?"
"Yes, correct."
@aghajani Are you sure that the pipeline stops once it has finished?

"In case of using stopReadTime, does it mean that a pipeline becomes bounded and it will be stopped once all records are read?"
"Yes, correct."
@aghajani Are you sure that the pipeline stops once it has finished?

Yes it does, I have tested it both with local runner and flink runner. Of course you need to ensure your reader to be configured to use SDF (setting only works in SDF) mode and also no other unbounded reader (or other in progress ones) exists on your pipeline, so the pipeline runner can understand the whole pipeline job is finished.

Well I tested with Dataflow and the job was always initalized as a streaming pipeline and it never stops, and yes I don't have any other source. It's literally a simple KafkaIO.read. At one point (around the Kafka read) the bounded PCollection becomes unbounded according to this: https://github.com/apache/beam/blob/cc0b2c5f3529e1896778dddeb6c740d40c7fb977/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L1467-L1483
Maybe the DataflowRunner is wrong in calling this?
    if (containsUnboundedPCollection(pipeline)) {
      options.setStreaming(true);
    }

I don't think it would be related to the runner. Are you sure you have your reader in SDF mode? I can do it with passing:
--experiments=use_sdf_read
As a program argument.

Well we were both right and wrong. The lack of SDF is the problem, but the runner is causing it.
.... and the lack that experiment isn't the cause. For DataflowRunner you don't need that:

The if condition already returns the same value as it would have if it has that experiment enabled:
https://github.com/apache/beam/blob/163ac6a3c10c26898ad89ca8bedde8ef78ee7ee2/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java#L1309-L1322
The pipeline creation would fail without SDF if stopReadTime is set:
https://github.com/apache/beam/blob/163ac6a3c10c26898ad89ca8bedde8ef78ee7ee2/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java#L1301-L1303

The problem is this method:
https://github.com/apache/beam/blob/163ac6a3c10c26898ad89ca8bedde8ef78ee7ee2/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L478
DataflowRunner replaces the SDF with a custom implementation. This way the SDF isn't part of the graph anymore, so anything done inside that SDF gets completely ignored by the DataflowRunner. I have no idea where to escalate this, but this is a LOT of lost functionality without any warning whatsoever.
https://github.com/apache/beam/blob/163ac6a3c10c26898ad89ca8bedde8ef78ee7ee2/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L1872-L1890

@nbali Thanks for these investigations, I think it should be moved up and discussed on<EMAIL_ADDRESS>(you need subscribe to this mailing list to write there) where are much more chances to have some help from Dataflow experts.

@aromanenko-dev I will move it there if it takes longer, but right now I think I'm pretty close to the end, so a closure here might make more sense now.
So... to my own surprise I managed to make it work. I had to trigger the "unified worker" path in the DataflowRunner, and now with a withStopReadTime call the streaming job actually stops. (If the necessity of such experiment flags was obvious to everybody and assumed I already have it, then mea culpa.)
https://github.com/apache/beam/blob/163ac6a3c10c26898ad89ca8bedde8ef78ee7ee2/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L2273-L2279
So after all this what I think the remaining problem is, that if you don't use the "unified worker", and you configure your KafkaIO.read* with configurations that require KafkaIO.Read.ReadFromKafkaViaSDF and you launch it with an unbounded PCollection/as a streaming job, then the extra functionality of the SDF will be gone without any warning.
https://github.com/apache/beam/blob/163ac6a3c10c26898ad89ca8bedde8ef78ee7ee2/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java#L505-L519
So my idea is to add fail-fast checks into the corresponding PTransformOverrideFactory. We have the AppliedPTransform available, which contains the ReadFromKafkaViaSDF, which contains the Read<> kafkaRead, which contains the custom properties (checkStopReadingFn, stopReadTime, etc - I'm not sure if there is anything else) that are only being handled by the SDF. So if any of those properties are not null, we should fail. Am I on the right path here?
https://github.com/apache/beam/blob/163ac6a3c10c26898ad89ca8bedde8ef78ee7ee2/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaIO.java#L1339-L1348

That makes a lot of sense. An alternative would be to add support for stop read time to KafkaUnboundedReader.
This translation seems like it will always be brittle in that some feature/option won't be supported in KafkaUnboundedReader and people will forget to update it here and then a future person will go down this rabbit hole again.

@nbali it can be a bit confusing, but the isStreaming pipeline option is not actually part of the core Beam model. It is a runner-specific option. Spark and Dataflow have separate batch/streaming modes. The direct runner and Flink runner don't need this. Really "streaming" is the universal mode that works for everything, while batch is a special case that allows optimizations because all the data is bounded (so we can do more splitting up front, and don't need to checkpoint and pause, etc).

@lukecwik

That makes a lot of sense. An alternative would be to add support for stop read time to KafkaUnboundedReader.
This translation seems like it will always be brittle in that some feature/option won't be supported in KafkaUnboundedReader and people will forget to update it here and then a future person will go down this rabbit hole again.

PR is coming soon to detect any misuse/lost functionality with fail-fast behaviour at pipeline creation that also enforces developers to add their newly introduced properties to that detection :)
@kennknowles

@nbali it can be a bit confusing, but the isStreaming pipeline option is not actually part of the core Beam model. It is a runner-specific option. Spark and Dataflow have separate batch/streaming modes. The direct runner and Flink runner don't need this. Really "streaming" is the universal mode that works for everything, while batch is a special case that allows optimizations because all the data is bounded (so we can do more splitting up front, and don't need to checkpoint and pause, etc).

I'm completely aware that my knowledge apart from the direct and the dataflow runners are minimal as I have only used those, and I doubt it's going to change. The logic in Spark seemed similar, but it might be actually required. My issue was that with Dataflow it isn't, but let's continue this discussion at my PR aimed at addressing that issue at https://github.com/apache/beam/pull/16773

I may be getting mixed up. I had thought that https://github.com/apache/beam/pull/15951#issuecomment-990238052 was addressed. Specifically that we would have a bounded-per-input-element SDF, hence bounded PCollections, when stop read time was specified. Is this false? Are PCollections still unbounded?


64 bits of Neutronium.WebBrowserEngine.ChromiumFx.dll is not updated in v1.5.2

There's no property called AllowGpu in 64 bits of Neutronium.WebBrowserEngine.ChromiumFx.dll in v1.5.2 release.
It seems 64 bits of this file is 1.5.0 in 1.5.2 release.

Publish version 1.6.0 that correct it


How to access Spec shared memory?

Hello,
I would like using Silx instead of Oxidis/Pymca for showing live data on the beamline at the ESRF. For this I need to access the shared memory of Spec (image_data, SCAN_D, etc,)  Do you have an example showing how this could be performed?
Thanks,
Mauro
PS: I apologize if this is not the right place to ask this question.

PyMca is currently the only distribution channel of the module.
If what you want is to avoid importing it from PyMca I think you will have to copy the module from the PyMca sources and add it to your own project.

@vasole OK, I can import the spswrap module from PyMca, no problem with that.
I would like to have an example how to run a Plot1D/Plot2D widget in an application continuously showing the SCAN_D/image_data from a Spec session when the memory is updated. What would be the best way to do this?

That's an interesting question. It sounds good to move it somewhere outside PyMCA if possible.

@maurov
The question belongs then to a PyMca and not to silx
@vallsv

sps does not comply with "windows, linux, Mac" if not it would be already in silx
Probably something based on SpecClient would be more appropriate
it only makes sense to use sps on machines where SPEC is installed therefore PyMca is not a "heavy" dependency in that case as PyMca is usually installed on those machines
I do not intend to maintain multiple sps distribution channels

@maurov
In the examples directory of silx, you have a module to update a plot from a thread:
https://github.com/silx-kit/silx/blob/master/examples/plotUpdateFromThread.py
I guess it is that what you need.

@vasole
Yes, that's exactly the example I was looking for! OK, I have to admit that my initial question was not correctly stated, sorry for that.
Just for documentation, I have obtained what wanted by:

from PyMca5.PyMcaIO import spswrap
redefine the run method in UpdateThread

Thanks for the help!


Support for 2D particles to collide against SDF

Added SDF collision support for 2D particles
Changed the SDF generation to be fully signed

https://user-images.githubusercontent.com/6265307/118996788-8d159b80-b95e-11eb-9936-54cdbe174877.mp4

@akien-mga Most of the bindings in RenderingServer and broken, will fix at a later time when the API stabilizes (its on my todo), so I am not bothering updating those for now.

Thanks!

Hi. Any idea when this will be exposed? We cannot use 2D collisions in the godot 4 beta.

@LakshayaG73 2D collisions were exposed in this PR, you can already use them. I have tested and confirmed they are working fine in Beta 2

@clayjohn can you elaborate a bit on how to use them?

when will this be available to beta? its still not working on godot 4 beta 7

To generate a SDF, you need to create LightOccluder2D nodes and ensure their appropriate property is checked.


Defines Retrier polling waiting time as a constant and fixes tests

The pooling time for the Retrier was hardcoded to 1, let's at least use a constant for that.
Also, retrier::tests::test_manage_retry_while_idle was randomly failing (for Ubuntu) when checking whether the Retrier was idle after giving up on a retry. This is due to the time of running a round not being taken into account.
Turns out this is super brittle, so I'm going to modify the timers as little as possible for now.
Superseeds #182 (Close #182)

I saw some instances in the tests that wait on API_DELAY instead of POLLING_TIME.
Funny tho, the instance linked above actually waits zero seconds but yet the test succeeds the test every time (without ever waiting for polling time duration).

Oh yeah, I corrected that on the other PR and forgot about it here, let me see if I can patch it without breaking it 😅

More on why instances with API_TIME never failed in the CI:
// Init the manager
let task = tokio::spawn(async move {
    RetryManager::new(
        wt_client_clone,
        rx,
        MAX_ELAPSED_TIME,
        LONG_AUTO_RETRY_DELAY,
        MAX_INTERVAL_TIME,
    )
    .manage_retry()
    .await
});
// Send the data
tx.send((tower_id, RevocationData::Fresh(appointment.locator)))
    .unwrap();

// Wait for the elapsed time and check how the tower status changed
tokio::time::sleep(Duration::from_secs((API_DELAY / 2.0) as u64)).await;
// Check
assert!(wt_client
    .lock()
    .unwrap()
    .get_retrier_status(&tower_id)
    .unwrap()
    .is_running());

It looks like that tokio::spawn always starts the async future late enough after tx.send is performed, thus the retry manager always hits the Ok(...) branch when try_recv()ing from the channel.
To make the test fail you can try adding a little sleep before tx.send (tokio::time::sleep(Duration::from_secs_f64(0.01);), this will delay tx.send enough that the async future is started first and will fail try_recv()ing and go to the Err(...) branch and wait POLLING_TIME, then we wait (API_DELAY / 2.0) as u64) seconds which is actually zero and the assertion will fail since POLLING_TIME wasn't over yet.
We can reduce the number of cases where we sleep POLLING_TIME if we always do tx.send before tokio::spawning the retry manager task. If we send before spawning we will be sure that the spawned task won't wait POLLING_TIME before processing our sent data. Not really important for correctness, but will only lower the testing time by a few seconds or so.

We can reduce the number of cases where we sleep POLLING_TIME if we always do tx.send before tokio::spawning the retry manager task. If we send before spawning we will be sure that the spawned task won't wait POLLING_TIME before processing our sent data. Not really important for correctness, but will only lower the testing time by a few seconds or so.

Yes! That's it!
Looks like I can finally make the timings make proper sense.
Awesome catch 🎉 🎉 🎉

Addressed the comments and slightly reduced MAX_RUN_TIME (from 0.5 to 0.2).
Tested it in a loop for both OSX and low-resource Ubuntu without issues.

I tried to repro the failed test in the CI and wasn't able to (in a loop for about 12h).
But i was able to repro it when i lowered MAX_RUN_TIME to 0.01 in just a couple of runs. So it should happen a lot when MAX_RUN_TIME is decreased.
They only possible logical explanation i can think of is that CI boxes spend so much time in self.run, more than 0.2s for some reason, and I think the reason is that so many threads are run at once (each test function thread + threads spawned from inside the test functions/code) that makes the self.run thread starve sometimes and not finish quickly. Might also be network issues but not sure if that can actually take that much time without timing out.
I'm not totally sure whether the 2 explanations are actually the reasons for this failure (or even make much sense :confused:), but i am sure that tests is logically correct at this point and through experimentation it also says the same.

Might also be network issues but not sure if that can actually take that much time without timing out.

#[tokio::main]
async fn main() {
    let start = tokio::time::Instant::now();
    let res = reqwest::Client::new()
    .get(format!("{}", "http://unreachable.tower"))
    .send()
    .await;
    println!("Response: {:?}", res);
    println!("Took: {:?}", tokio::time::Instant::now() - start);
}

Looks like it's possible for a request to take that much time before getting a DNS error (at least on my slow network). So network issues might be one valid reason.

Note that it takes that much time 0.34s only in the first run, subsequent runs finish in less than 50ms. Probably the OS caches DNS results or something.


local_address doesn't connect unless interface is highest priority

I've been attempting to use the local address option but have noticed unreliable behaviour whereas curl has no issues.
Test system is Ubuntu 20.04 server.
I have two adapters, one with the address <IP_ADDRESS> and the other with <IP_ADDRESS>.
If I use the local_address option and define one of these addresses there is a 50% chance the request will work.
Example route table

Now say if the interface with <IP_ADDRESS> has a higher adapter priority, and then <IP_ADDRESS> has a lower priority, if I set the local address to <IP_ADDRESS> the request will timeout and fail.
Example route table

logs
2021-09-25T08:09:52Z DEBUG reqwest::connect] starting new connection: http://ifconfig.co/
[2021-09-25T08:12:02Z ERROR cli::request] error sending request for url (http://ifconfig.co/): error trying to connect: tcp connect error: Operation timed out (os error 110)

However with curl using the interface flag curl --interface eth1 ifconfig.co this always works successfully for any interface regardless of the adapter priority. Each of these adapters resolve to a different WAN IP, so have confirmed the request and behaviour has always been reliable and successful via curl.

Thanks for the report! Interesting... Likely the difference is in curl, you're specifying the interface, and I imagine that is using SO_BINDTODEVICE with the interface name. Whereas here (actually in hyper), it's just trying to bind a specific address, using bind(2) with just the IP address.


Update local.txt

Update django-debug-toolbar to fix https://github.com/django-debug-toolbar/django-debug-toolbar/issues/668.

Just in case you are not getting notified: @python-spain/pycones-2015.

Thanks! I understand you don't need a new deployment for this, do you?

Nope, just for dev purposes :)
On Jul 5, 2015 2:11 PM, "Yamila"<EMAIL_ADDRESS>wrote:

Thanks! I understand you don't need a new deployment for this, do you?
—
Reply to this email directly or view it on GitHub
https://github.com/python-spain/PyConES-2015/pull/4#issuecomment-118617211
.


Pillow package not installing in Python runtime

I am trying to build an automated thumbnail pipeline for images.
I need the Pillow package in the requirements.txt file
but it fails to install

Checking...

@sebgoa So, it can't install pillow because of there are some dependencies missing:
http://pillow.readthedocs.io/en/4.0.x/installation.html#external-libraries
Unfortunately zlib is not installed via pip (I tried) but apt-get install could solve it. Working around on this issue is easy, but it leads to an idea that I think we should somehow still provide custom runtime in format of docker image, as the way fission doing, as an alternative solution for this kind of issues. Because pip install doesn't always work out of the box...
Right now for the demo (if you intend to show), I could build the python runtime image which includes zlib and libjpeg, then pillow installation should work.

yes please, can you build a custom one and force push the runtime.
this is terrible, but would help with the demo.
ultimately we need to move to minideb and indeed switch to a runtime auto-build

Yes. But in order to not messing up, I am building a separate python runtime and a separate controller just for the demo. Will ping you on slack when it done.


Custom converters

Hello,
First let me thank you for this amazing project. It works well, and the error logging is incredibly accurate and informative. I have rarely seen a library designed and implemented with such care and rigor.
The only missing feature, which would make it perfect, is the ability to set custom converters. May be it's already implemented and I haven't seen it in the docs. Another JSON mapper library for TypeScript has that feature (implemented but not documented): https://github.com/jf3096/json-typescript-mapper. May be you could get some inspiration from there ?
I might try to implement something and send a pull request but I can't promise anything in the following weeks.
All the best !

Hey @gempain, interesting point. Can you provide a few code examples how this converter could work? I think I know what you mean, but just to make sure. :-)

Hey @andreas-aeschlimann ,
Well, you're on the right path. But the converter should be typed, and we also need a serializer. In my previous post I forgot to mention the use cases where this feature would be useful. Imagine you want to work with types other than String, Number, Boolean or Date (is don't think Date is supported in the current version, but forgive me if I am wrong), for example let's say you want to use Moment in place of Date objects. I would expect setting a custom converter that tells the mapper how to serialize and/or deserialize a certain property.
In json-typescript-mapper, they have defined an interface called ICustomConverter:
/**
 * When custom mapping of a property is required.
 *
 * @interface
 */
export interface ICustomConverter {
    fromJson(data: any): any;
    toJson(data: any): any;
}

I think their interface misses genericity to take advantage of TypeScript features, and I am not a fan of the I, so after a bit of refactoring, you'd get something like this:
export interface CustomConverter<T> {
    deserialize(data: any): T;
    serialize(data: T): any;
}

Now, let's see it in action:
import {Moment} from "moment";

class MomentConverter implements CustomConverter<Moment> {
    deserialize(data: any): Moment {
      let m = moment(data);
      return m.isValid() ? m : data; // *1
    }
    serialize(data: Moment): any {
      return data; // *2
    }
}

@JsonObject
export class Product {

    @JsonProperty("creationDate", Moment, MomentConverter)
    public creationDate: string = undefined;

    @JsonProperty("purchaseDate", Moment, MomentConverter)
    public purchaseDate: Moment = undefined;

}

Notes:

*1 Deciding on the return value could be the subject of a debate. Would you return undefined, the original data, or throw an error if the type if not as expected ? I think the person that implements the converter should be the one deciding.
*2 If I am right, the serialize function returns a JSON object, not a string. That obejct can be serialized via the native JSON.stringify method. In the particular use case I chose, Moment objects have embedded JSON serialization methods similar to those for native Date objects. That's why I return the moment directly, but obviously you could return a date formatted String, or a Date with data.toDate().

Looking forward to see how this will be used !
All the best;

It looks interesting, I will certainly will look into it in details when I have time.
*1 The deserializer must return the type that is given, or null (if allowed in json2typescript). After all, json2typescript is here to force you to be strict on the API side. If your custom converter fails, there must be something wrong which you should correct and an exception should be thrown.
*2 Serialize should be just JSON.stringify I guess, so return a string.

Agree 100% !

I will look into this soon. At the moment I am busy with another project, so it will take some time.
I will keep this thread open until I finished it!

After making some tests and putting some thoughts into it, I see two options how to implement the custom converter:

Extend the current decorator @JsonProperty and make it allow the converter interface as second parameter as the type instead. Your example above

@JsonProperty("creationDate", Moment, MomentConverter)
makes no sense in my eyes because the expected type (here: Moment) is obsolete, and the third parameter is currently used already. Instead, we could do something like this:
@JsonProperty("creationDate", MomentConverter)
This approach is easy to use, but has one big downside:
Somewhere in the code I will need to make IF/ELSE and check if the second parameter is a custom converter or a type (class) as usual.
I think there is no proper way in TypeScript to find out what this parameter is. I can only check if the methods deserialize/serialize exist which should be an indication that it is actually a custom converter derived from the interface.
But what if you actually have a type that contains the functions deserialize and serialize? I think this could cause a conflict.
Do you have an idea how to make a workaround for this?

We could use the mapper as fourth parameter

@JsonProperty("creationDate", Moment, false, MomentConverter)
but this would be ugly too, as expectedType (here: Moment) is obsolete and isOptional is usually false anyway.

We could introduce a second decorator @Blabla(CustomConverter) which can be used for all properties needing custom converters. This decorator could replace the original @JsonProperty or expand it.

In typescript there is currently no way to determine reliably the name of a class. The only unreliable way I know is to use obj.constructor.name.
I think we should keep @JsonProperty and pass the converter as second argument. To differentiate types from converters, we could introduce a new annotation @Converter.
@Converter
class MyConverter implements Converter {
    // serialize and deserialize implementations
}

@JsonProperty('myProp', MyConverter)

With that annotation, you can easily check if the second argument is a converter.
Obviously, this is not perfect as you both annotate the converter @Converter and implement Converter, which is redundant. So far I don't know of a better way, but I'll keep thinking about it.

I liked your idea, but it causes issues when you forget the @Converter decorator. One could say, it's the developer's problem. On the other hand, I would like to give good debugging information which is not possible like that.
If you forget the decorator instead of the class MyConverter, the code has to assume it is a type you want to map. This will lead to failure of the whole serialization/deserialization and might confuse developers because we cannot tell them whats wrong. It will only say that the serialization/deserialization to/from MyConverter failed.
I will think about it for a little more and then make a decision. Things coming to my mind are:

use abstract class instead of interface (then check is possible I think)
use another property decorator

I went with the converter decorator and the @JsonProperty decorator.
The confusion may still happen, but I added some smart error message when the case I described would appear.

Thanks for the feature ! I will have a look at it when I have some time, and give you some feedback :)

Sure thing. I'm currently updating the README because there are quite some changes.
I'm not a fan of breaking changes, but I couldn't bypass them with the change to JsonConvert instances.

Agree, but sometimes you have to come out with a major release and I feel like this one is :)

Hopefully. I'm looking forward to your feedback.
I have checked out the GitHub code and used npm link to reference it. It worked quite well, but I dunno about the guys with the enum/JS problems (from ionic I think).

Hello, please provide a non-working minimal example. I often used nested objects of a level of 5 or more without any issues. There are no custom converters necessary for that in general.

@JsonObject
export class Resource {
@JsonProperty('@id') _id: string = undefined;
@JsonProperty('@type') _type: string = undefined;
@JsonProperty('id') _resourceId: string = undefined;
@JsonProperty('description') _description: string = undefined;
@JsonProperty({jsonKey: 'Attribute', expectedJsonType: [Attribute]}) _attribute: Attribute[] = undefined;
@JsonProperty('connectedTo') _connectedTo: string = undefined;

constructor(id: string, type: string, resourceId: string, description: string, attribute: Attribute[], connectedTo: string) {
    this._id = id;
    this._type = type;
    this._resourceId = resourceId;
    this._description = description;
    this._attribute = attribute;
    this._connectedTo = connectedTo;
}

}
So this is how my template looks like. I defined the Attribute object which as other nested types. There is a three level nesting. The first level do get serialized. However, they fail when it comes to the deseriazling attribute array. Not sure if it is has something to do with my json object.

I don't think you can use a constructor that way because json2typescript calls the constructor with no arguments. It will probably set all your properties to undefined because of that.
Try your code without constructor (or comment everything out).

You probably have some other issue because this is not possible.
For example, it could be a conflict of classes. Are you sure that you import the correct class "Resource", for example?
If you can post me a small example to GitHub which I can launch with Angular CLI, I can try to help you.

Looks like I am progressing on this. I have to verify the templates. If not I will post it on GitHub. Since I have an NDA i couldn't do it at present. I will try to modify it and post if it doesn't work. I really appreciate your help. Thank you!

They are failing when I have an array of objects which are at nested level:
Is there any way of specifying this. Right now I am using the below way;
@JsonProperty('Attribute', [Attribute])
_attribute: Attribute[];
And below is my error:
Error: Fatal error in JsonConvert. Failed to map the JSON object to the JavaScript class "Resource" because of a type error.
Class property:
	_attribute

Expected type:
	[Attribute]

JSON property:
	Attribute

JSON type:
	[object,object,object,object,object,object,object,object,object,object,object,object]

Is this all the error? Sometimes there is more above or after it.
I can't see a problem here, it should work.

So, it says that deserialization is not happening due to type mismatch.
These are the two errors being displayed:
Error: Fatal error in JsonConvert. Failed to map the JSON object to the JavaScript class "Resource" because of a type error.
core.js:1350 ERROR Error: Uncaught (in promise): TypeError: Cannot read property '_attribute' of undefined
TypeError: Cannot read property '_attribute' of undefined

You have to compare your Attribute class to your actual JSON object. There is something not matching there.

So this is how my json looks and each Aspect itself is an array of that type. And this is how I defined my classes. Do you say that it should have keys on all internal objects?
"Attribute": [{
"Aspect":[ ],
"name": "...",
"description":"..."
},
{
"Aspect":[ ],
"name": "...",
"description":"..."
},
{
"Aspect":[ ],
"name": "...",
"description":"..."
}
]

I can't tell from these fragments, can you provide a minimal non-working example in GitHub?

So, this got resolved. I noticed that there are some optional types in the json and I have to declare that flag explicitly. However, while deserializing if I am getting an array of objects, I don't see a direct way of specifying the class type. For example if I get Attribute[] from the back end, and adding jsonConvert.deserialize(response[0], [Attribute]); or jsonConvert.deserialize(response[0], Attribute[]) are giving me errors.
Is there a particular way to deserialize arrays?

You can use jsonConvert.deserializeArray(data, Attribute) if your json matches the type of an array of Attribute. The method jsonConvert.deserialize(data, Attribute) does the same because it intelligently picks the appropriate case (flat object/array).


Uncharted: Golden Abyss™ [PCSA00029]

App summary

App name: Uncharted: Golden Abyss™
App serial: PCSA00029
App version: 1.00

Vita3K summary

Version: v0.1.8
Build number: 3193
Commit hash: https://github.com/vita3k/vita3k/commit/9837745e
CPU backend: Dynarmic
GPU backend: Vulkan

Test environment summary

Tested by: Gerold55
OS: Windows 11
CPU: Intel Core i5 9300H
GPU: NVIDIA Geforce GTX 1650
RAM: 8 GB

Issues
Graphical issues with the water and occasional frame drops
Screenshots

Log
Recommended labels

Ingame-

Топ

Play this game 60 fps ok but flicker issue malig68 mc4 gup this problm
https://github.com/Vita3K/compatibility/assets/97212153/35172150-860c-4e94-8b32-0de15b0d5915

Play this game 60 fps ok but flicker issue malig68 mc4 gup this problm
Screen_Recording_20240217_132152.mp4

I get this black texture issue when using OpenGL.
The fix for this is to not go above 1.75x resolution upscaling, which is an absolute shame on a 4K screen.

One question, did you finally solve it? Because it still happens to me and I did everything it says there, in the comment.


Fix clone instructions in README.md: use https instead of ssh

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.02.14. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2017.02.14

Before submitting an issue make sure you have:

[x] At least skimmed through README and most notably FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[x] Bug report (encountered problems with youtube-dl)
[ ] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

Description of your issue, suggested solution and other information
In step 2. of the "Adding support for a new site" section of README.md, the suggested command line for cloning the newly forked repository fails with the current default configuration of github repositories:
git clone git@github.com:YOUR_GITHUB_USERNAME/youtube-dl.git

I get the error:
Permission denied (publickey).
fatal: Could not read from remote repository.

One should clone the repo using https instead:
git clone https://github.com/YOUR_GITHUB_USERNAME/youtube-dl.git

In the section "Adding support for a new site", it's assumed that people comes here for writing codes and open pull requests. To do that you need a working Github account. If you got permission denied, there's something wrong in your github account settings.


[$1000] Stop using report.participants in favor of report.participantAccountIDs

We need to stop using report.participants and instead use report.participantAccountIDs. Now that https://github.com/Expensify/App/pull/24320 is done, we should be able to remove all uses of report.participants in the App.
Let's go through and find everywhere that we use the participants array from the Onyx report object and instead use the participantAccountIDs array instead.
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~0155ce6d585250b4c1
Upwork Job ID:<PHONE_NUMBER>443893760
Last Price Increase: 2023-08-23

Hey it's Lucas from Callstack. Happy to work on that ticket

I would like to work on this, I have helped find some "unknown" user issues so I think I can help with migration and testing for the cases. In case Luke wants help or he is OOO I'm able to help with this issue.

@puneetlath Hey it's Lucas from Callstack. Happy to work on that

Thanks @lukemorawski. Mind providing on ETA for when you think you'll be able to get a PR up?

Should be today :) @puneetlath

Ok, so https://github.com/Expensify/App/pull/26102 got us one step further, but we need to fully remove any participants usage in the app from all report types: reports, IOUs, tasks, etc.
@lukemorawski can you do the following:

list out every place in the app where we are currently using participants
for any of those places where there isn't an obvious way to use participantAccountIDs instead, start a discussion either here or in slack about how we can go about removing them

This is a very high priority for us, so let me know if this is something you're able to focus on this week or whether we need to recruit more help. Thanks!

Here are my findings:

src/components/MoneyRequestConfirmationList.js

We use iou.participants quite a lot -> we’re adding new properties here: selected (boolean) and searchText. Selected is saved in Onyx, but selected ids could be stored as another Onyx key inside iou as an array of ids.
isPolicyExpenseChat -> could it be moved up to iou instead of of being a property of each participant object?
[]

src/pages/iou/MoneyRequestMerchantPage.js

https://github.com/Expensify/App/blob/5f668c669a501a94caebbb434a656580e3bcfb24/src/pages/iou/MoneyRequestMerchantPage.js#L61
Could be easily replaced by ids (we only need to check if array is empty)

src/pages/iou/MoneyRequestDatePage.js

https://github.com/Expensify/App/blob/2163c549de291cfdf305ade59015fda1dd34fc3f/src/pages/iou/MoneyRequestDatePage.js#L64
Could be easily replaced by ids (we only need to check if array is empty)

src/pages/iou/MoneyRequestDescriptionPage.js

https://github.com/Expensify/App/blob/2163c549de291cfdf305ade59015fda1dd34fc3f/src/pages/iou/MoneyRequestDescriptionPage.js#L68
Could be easily replaced by ids (we only need to check if array is empty)
-> we have some repetitive code here - maybe it also could be refactored
https://github.com/search?q=repo%3AExpensify%2FApp if (!isDistanceRequest %26%26 (_.isEmpty(iou.participants) || (iou.amount %3D%3D%3D 0 %26%26 !iou.receiptPath) || shouldReset))&type=code

src/pages/iou/SplitBillDetailsPage.js

https://github.com/Expensify/App/blob/2163c549de291cfdf305ade59015fda1dd34fc3f/src/pages/iou/SplitBillDetailsPage.js#L64C6-L64C6
Here we have similar logic with selected (ids could be moved to a separate array)

src/pages/iou/steps/NewRequestAmountPage.js

https://github.com/Expensify/App/blob/5f668c669a501a94caebbb434a656580e3bcfb24/src/pages/iou/steps/NewRequestAmountPage.js#L118
Could be easily replaced by ids (we only need to check if array is empty)

src/pages/iou/steps/MoneyRequstParticipantsPage/MoneyRequestParticipantsSelector.js

https://github.com/Expensify/App/blob/2163c549de291cfdf305ade59015fda1dd34fc3f/src/pages/iou/steps/MoneyRequstParticipantsPage/MoneyRequestParticipantsSelector.js#L150
Doesn’t use iou.participants directly but sets new participants for iou
If we could move isPolicyExpenseChat and keep selected ids in another array, we could  use only ids here

src/pages/iou/steps/MoneyRequstParticipantsPage/MoneyRequestParticipantsSplitSelector.js

https://github.com/Expensify/App/blob/c0363b79139fb73743170976e2bb9e7df3b2a907/src/pages/iou/steps/MoneyRequstParticipantsPage/MoneyRequestParticipantsSplitSelector.js#L132
We toggle participant selection here. We also have searchText property for each participant. Could we possibly not store it Onyx but create an util function for that?

src/pages/iou/steps/MoneyRequestConfirmPage.js

https://github.com/Expensify/App/blob/2163c549de291cfdf305ade59015fda1dd34fc3f/src/pages/iou/steps/MoneyRequestConfirmPage.js#L69
Same with isPolicyExpenseChat

src/pages/iou/steps/MoneyRequstParticipantsPage/MoneyRequestParticipantsPage.js

https://github.com/Expensify/App/blob/c0363b79139fb73743170976e2bb9e7df3b2a907/src/pages/iou/steps/MoneyRequstParticipantsPage/MoneyRequestParticipantsPage.js#L101
We pass iou.participants to MoneyRequestParticipantsSplitSelector as a prop

src/pages/iou/propTypes/index.js

The propTypes for participants could be replaced with participantAccountsIDs

src/libs/actions/IOU.js

A lot of participant object usage (participant.login, participant.isPolicyExpenseChat, participant.reportID) - could reportID be possibly moved up to iou?
The data like login, name could be derived from util function like getDisplayNameForParticipant

src/pages/home/sidebar/SidebarLinksData.js

https://github.com/Expensify/App/blob/c0363b79139fb73743170976e2bb9e7df3b2a907/src/pages/home/sidebar/SidebarLinksData.js#L122
I think it can be removed

src/types/onyx/Report.ts

https://github.com/Expensify/App/blob/2163c549de291cfdf305ade59015fda1dd34fc3f/src/types/onyx/Report.ts#L49
I think it can be removed

src/types/onyx/OriginalMessage.ts

https://github.com/Expensify/App/blob/106abf25c3e6f85e22d44aa8abea8c0c2ac60001/src/types/onyx/OriginalMessage.ts#L16
I think it can be removed

src/types/onyx/IOU.ts

https://github.com/Expensify/App/blob/e25a27f5250bc941c206cd92a7534757e47b4cd6/src/types/onyx/IOU.ts#L21
It’ll be changed for participantAccountIDs, the Participant type could be removed then
—————
I only could find these occurrences of iou.participants. For tasks and reports we already use participantAccountIDs.
We have a several usages of PropTypes.objectOf(participantPropTypes) - i.e. for personalDetails (https://github.com/Expensify/App/blob/106abf25c3e6f85e22d44aa8abea8c0c2ac60001/src/pages/home/report/ReportActionItemCreated.js#L31) or users (https://github.com/Expensify/App/blob/106abf25c3e6f85e22d44aa8abea8c0c2ac60001/src/pages/home/report/ReactionList/BaseReactionList.js#L22) (in this case we should probably rename this prop?)
—————
We have a few occurrences of participantsList
https://github.com/search?q=repo%3AExpensify%2FApp+participantsList&type=code
and participantList
https://github.com/search?q=repo%3AExpensify%2FApp+participantList&type=code
For consistency I propose to rename the latter one to participantsList. For Report related utils and files they already work with participantAccountIDs.
After replacing participants with participantAccountIDs these util function will require some refactor: https://github.com/Expensify/App/blob/e25a27f5250bc941c206cd92a7534757e47b4cd6/src/libs/OptionsListUtils.js#L209
https://github.com/Expensify/App/blob/e25a27f5250bc941c206cd92a7534757e47b4cd6/src/libs/OptionsListUtils.js#L1084
##Conclusion:
The only thing to discuss is how we should deal with the other data stored in the participants object (like: selected, isPolicyExpenseChat) if we wanted to remove it. Here are the examples how this object looks like i.e. for split and request

Great findings! I'll dig into them individually more, but for the ones that you think can be updated easily to use IDs instead, can you go ahead and open a PR (or multiple, whatever you prefer).

@puneetlath First PR opened 🙂 https://github.com/Expensify/App/pull/27369
what I've done here:

added participantAccountIDs to iuo object (saved in Onyx) - temporarily we have both participants and participantAccountIDs for iou
replaced participants with participantAccountIDs where it was possible without additional changes

Now I can start working with getting participant data (like login, displayName, etc.) using id from this new array. When you decide what to do with isPolicyExpenseChat, selected and searchText properties, please let me know 🙂

So I'm a bit confused about when participant.isPolicyExpenseChat is used. I tried looking at a workspace chat and an IOU report linked to a workspace chat and neither of them seemed to have this field. Were you able to find any example of that in-use in Onyx?
For the "iou" key in Onyx, it looks like that is not something that interacts with the back-end and is only used locally for draft IOUs. So I think if the back-end stopped sending report participants for all the scenarios it currently does, this would still work fine, right? In which case, perhaps we can leave it as-is. Since the main goal here is to make sure everything still works when the back-end stops sending the participants array.

@puneetlath it's used for iou and it's usedin i.e. getMoneyRequestInformation function. https://github.com/Expensify/App/blob/06e8d3ac13d9b45cae26e0967ff787998d5a4820/src/libs/actions/IOU.js#L374
In IOU actions there are several functions that receive participant object as a param.
As you've mentioned iou object doesn't come from the backend nor interacts with the backend in any other way. It looks like we don't need any changes here if we're ok with the current participant objects stored for iou in the participants array.
I'll check it once again, but it seems that we don't need to change anything and the backend can safely stop sending the participants array for the report. I think we can also revert the changes I've made in my last PR.
Sorry for my misunderstanding here - I was convinced that you want to remove participants usage no matter if it came from the backed or not.

Ah interesting. So you're saying that if we go ahead and stop sending the participants key in the reports_ object in Onyx today, everything should still work fine? If so, that's great!

@puneetlath As I mentioned - I'll check it once again just to make sure. So can we postpone removing participants from the report until at least tomorrow? 😃

Sounds good. I'll prepare the PR.

@puneetlath I've found one more usage of report participants in the getChatByParticipantsByLoginList function here. (I wonder why I couldn't find it before) The function is marked as deprecated, but it's still being used by i.e. navigateToAndOpenReport or in Task.js. I will create a PR to remove these usages

Ok sounds great thank you!

@puneetlath while working on this PR I found that for OpenReport request we're sending emailList which is an array of user's email logins. Should we also remove that? Is the backend ready for not sending this array?
I also saw that accountIDList array is also added to params - but for now it's always sent as an empty array. I think we can now replace emailList with accountIDList.

Good catch. Yes, you can use accountIDList instead of emailList. However, if you chat with someone new that you don't yet have in personalDetails, you'll need to send the email and use emailList.
Which 🤔 the back-end right now will accept either emailList or accountIDList. So actually I think for this we need to update the back-end to accept both. So if you start a group DM, for example, with:

1 user you've never chatted with before - pass the email address
1 user you "know" (meaning you have both login and accountID in personalDetails) - pass the accountID
1 user you don't "know" (meaning you have only accountID in personalDetails) - pass the accountID

So I'll update OpenReport so that it can accept either emailList or accountIDList or both. Does that make sense?

I think we can leave the current logic then. For now we have navigateToAndOpenReport and navigateToAndOpenReportWithAccountIDs the first one is used more frequently (i.e for creating a new chat) and operates with userLogins, while the latter is used only in one place. For creating the chat we would have to differentiate whether user's data is already in personal details or not (we can't do it just by accountID, because for new users we build optimisticAccountID). That means introducing some new logic to use either navigateToAndOpenReport or navigateToAndOpenReportWithAccountIDs.
So if it's ok to still send emailsList I would rather leave creating a new chat with navigateToAndOpenReport and sending user logins no matter if we have their real IDs or not.
The only additional change that comes to my mind now is to refactor both methods into one (to accept userLogins and accountIDs), but please let me know if it's necessary.
For now I'll push my changes with removing deprecated getChatByParticipantsByLoginList method which was using report.participants.

For now I'll push my changes with removing deprecated getChatByParticipantsByLoginList method which was using report.participants.

Sounds good.

The only additional change that comes to my mind now is to refactor both methods into one (to accept userLogins and accountIDs), but please let me know if it's necessary.

Yeah, good question. I'm trying to figure out if it's actually possible to start a group chat with someone you "know" and someone you "don't know" in NewDot today. It doesn't seem like there is from what I can find. So maybe this isn't necessary.

ok, PR opened: https://github.com/Expensify/App/pull/27828 🙂

@koko57 I've merged that PR. We should be good to stop sending report.participants from the back-end now, correct?

@puneetlath yeah, I think we're all set now. And as we decided to use participants for iou (as a temporary object in the Onyx) I think we can revert https://github.com/Expensify/App/pull/27369

Ok great! I'll get the back-end PR prepared.
Mind going ahead and creating the revert PR? And we can ask @burczu to review that one as well. Thank you!

@puneetlath yeah, of course! I'll take care of it

There are no contributors to pay out here, so I'm going to go ahead and close this.
Thanks again for your help getting this across the finish line @koko57 @burczu and @lukemorawski!


demtsh branch cannot be read with uproot

Currently, attempts to read the demtsh branch with uproot causes the following error:
ValueError: 'read beyond' in AwkwardForth runtime: tried to read beyond the end of an input

I have opened https://github.com/scikit-hep/uproot5/issues/1221 to better understand the issue

A fix has been found and merged into uproot. It should become available in uproot v5.3.8. Will confirm and close this issue when it is available for testing

Uproot has released v5.3.8 so this should now work


remove unproven flag

Encoding works fine as for me.
I think we should prove it and remove unproven option.
Alternative:
Split this crate on several like svd-base, svd-parser, svd-encoder, svd-utils, tests.

I think we should prove it and remove unproven option.

This means writing some tests, right? :)
In my project I added a test in which I test this library.
In the test I decode an xml, I encode it and then I decode it again.
The test asserts that the two decoded xml are equal.
Here is the test:
https://github.com/MarcoIeni/svdtools/blob/riir/tests/svd_parser.rs
Maybe we could start by writing a (integration?) test like this.


KeyError in admin when editing group permissions

There seems to be a bug in the admin when editing permissions for groups (Wagtail 1.4.1).
To reproduce the error:

Go to Admin -> Settings -> Groups -> Add a Group
Select "Add a page permission"
There is a delete button, but clicking it does nothing (problem 1)
If you do not select a page with "Choose a Page", and proceed to "Save", there is a Python KeyError, for 'page' because there was no page selected (problem 2)

Problem 2 confirmed on Wagtail 1.4.3. The delete button is working fine for me though...


Please remove pycocotools-windows on requirements since this lib doen't actively maintained caused Python3.9 users unable install on wndows!

Please remove pycocotools-windows on requirements since this lib doen't actively maintained caused Python3.9 users unable install on windows!

Thanks for your suggestion.  But in order to be compatible with Python3.7/8 users, we intend to keep this setting. Maybe you could propose an issue for pycocotools-windows.

I got the same error with Python 3.9.

sam problem, how to solve it, I really feel mmdetection is much less user friendly than detectron2....

it works by modifying the requirement directory


Fixed bug #200

Proposal to fix bug #200

Hi @luismanez, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by Microsoft and real humans are currently evaluating your PR.
TTYL, MSBOT;

Thanks Luis and Ole, good stuff.

hahahahaha!! awesome Vesa, awesome!
give me a shout if you visit Valencia! :))


Demographic survey

This adds the code to ask 30% of people who visit the site to answer a few questions for us so we can learn more about who they are and how they use the site.
The survey is hosted by mySociety and they will compare the results to data collected on other similar sites from around the world.

@henare could you please review this for me.

@equivalentideas looks OK to me. Do you want this merged and deployed?

@henare Thanks for that. Let's hold on the merge and deploy, I've got some questions out to @jacksonj04 in an email about changing the content of the survey.

@henare could you merge and deploy this please.


Create runtime helpers to parse user pool id from Lambda event object

Gabriel suggested this:
import { cognitoUtils } from '@serverless-stack/runtime'

[...]

const userIds = cognitoUtil.extractUserPoolInfoFromIdentityContext(context)
const userRole = cognitoUtil.extractAssumedRoleNameFromIdentityContext(context)

Requests:

Gabriel https://serverless-stack.slack.com/archives/C01HQQVC8TH/p1638967652013700?thread_ts=1638964681.012100&cid=C01HQQVC8TH

We are transition this repo to https://github.com/sst/v2 and are closing this issue for now. If it is still relevant please reopen it there - sorry for the trouble


Consolidate makefile and justfile into justfile

We have a Makefile and a justfile which are redundant, and some of the recipes are broken (e.g. make pip-compile). It's a bit of a maintenance problem as well as unclear during onboarding which is to be used.
This should be merged into one justfile and the commands tested (where possible).
Documentation should be updated to reflect the changes.

👍 I was confused by this


Abuse-resistant domain registrar for community seeder NS and website domain

Advice needed
Would be nice to have some crypto payment methods there as we will have public & transparent foundation, based on the blockchain. At least I suppose this a nice concept for the public-collected foundation reserves.
Related issues
https://github.com/twisterarmy/infrastructure/issues/3

At this moment, we use dot.tk / freenom.com for twister-seeder.
The official website still working, so this subject in low priority.


FORE period in Report.sso for some tables are filled with zeros for Pacific Hake

Starting with NUMBERS_AT_AGE in Report.sso to BIOMASS_AT_LENGTH the rows that pertain to the FORE period are populated with zero values. See example report file from the stock assessment of Pacific Hake below. I checked the Report.sso files from the stock assessment we did last year and they were also populated with zeros. So, it is not due to recent changes in the code. The next table, i.e., F_AT_AGE has actual values in the forecast period. I checked the results of a run using the "simple" model available in {r4ss} and this problem doesn't exist. So, I thought maybe it was a function of settings in forecast.ss and the only major difference I could find is Pacific Hake uses 2 #Bmark_relF_Basis, whereas simple uses 1 #Bmark_relF_Basis. I am happy to poke and prod the model or code to attempt to determine what is going on here but some guidance on where to start would be greatly appreciated. Thanks.

I see that the forecast output exists in report.sso, but not report_mceXX.sso.  So, this should be an easy fix.

fixed by turning on save_for_report before calling bigreport in mceval section.  Fix implemented on branch #218

Added @kellijohnson-NOAA as assignee just as a reminder to test if this is working


Docs say apiVersion is optional, but it is required  to connect

When you leave out the api-version, you get a type-error
TypeError: Cannot create data source "elasticsearch": Cannot initialize connector "es": Cannot read property 'indexOf' of undefined
at ESConnector.connect (/diekeure/platformso/API/node_modules/loopback-connector-es/lib/esConnector.js:105:37)
at Object.module.exports.initialize (/diekeure/platformso/API/node_modules/loopback-connector-es/lib/esConnector.js:30:30)
at DataSource.setup (/diekeure/platformso/API/node_modules/loopback-datasource-juggler/lib/datasource.js:339:19)
at new DataSource (/diekeure/platformso/API/node_modules/loopback-datasource-juggler/lib/datasource.js:117:8)
at Registry.createDataSource (/diekeure/platformso/API/node_modules/loopback/lib/registry.js:355:12)
at dataSourcesFromConfig (/diekeure/platformso/API/node_modules/loopback/lib/application.js:440:19)
at EventEmitter.app.dataSource (/diekeure/platformso/API/node_modules/loopback/lib/application.js:235:14)
at /diekeure/platformso/API/node_modules/loopback-boot/lib/executor.js:190:9
at /diekeure/platformso/API/node_modules/loopback-boot/lib/executor.js:281:5
at Array.forEach (native)
at forEachKeyedObject (/diekeure/platformso/API/node_modules/loopback-boot/lib/executor.js:280:20)
at setupDataSources (/diekeure/platformso/API/node_modules/loopback-boot/lib/executor.js:180:3)
at execute (/diekeure/platformso/API/node_modules/loopback-boot/lib/executor.js:38:3)
at bootLoopBackApp (/diekeure/platformso/API/node_modules/loopback-boot/index.js:154:3)
at Object. (/diekeure/platformso/API/server/server.js:28:1)
at Module._compile (module.js:556:32)

Tested with version 1.0.7

Yes correct, we added some code to decide how to operate for v1 vs v2 and that looks at the apiVersion, will think about it more and either fix the docs or the code.

Overtime the default_api_branch": "2.3" and apiVersion will continue to be a moving target.
Since we don't have quick access to that property in the code when we need to make decisions based on version, for example: if (self.settings.apiVersion.indexOf('2') === 0) { ... lets just make apiVersion required in the docs.

v1.0.8 has been released with a documentation update. Thank you for reporting the problem @diekeure


Trying to suggest to fix casing to lower again

Following up on #451, PR in three commits to separate different levels of relevance.
Some code in the hooks directory was introduced which had Sirupsen in uppercase again. This causes problems for projects with vendoring like glide and govendor. This PR is putting everything to lowercase completing #384.
Links to godoc and travis are validated.

Seems like decision was made to stick to Uppercase, bummer .....

@iheitlager bringing this back up...
@sirupsen what if the airbrake hook was moved out to its own repo? Therefore logrus could be vendored in whatever case the user likes.


Implement Kabsch algorithm in Fusee.Math

Implement Kabsch algorithm in Fusee.Math and SVD along the way.

Use case e.g.: Finding optimal rotation and translation between corresponding 3D points

ref "Least-squares estimation of transformation parameters between two point patterns", Shinji Umeyama, PAMI 1991, DOI: 10.1109/34.88573

Currently not needed


Can not access list of Amplify apps

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

App Id
https://us-west-2.console.aws.amazon.com/amplify/home?region=us-west-2#/
Region
us-west-2
Amplify Console feature
Access control
Describe the bug
I am trying to login to Amplify console, but I am getting the error below
List apps call failed
Network Failure
Expected behavior
I expect to see the list of apps I have created in Amplify
Reproduction steps
Create an Account
Create an app in Amplify
Logout and try to login
Build Settings
No response
Additional information
No response

Hello, we are also experiencing the same issue? What is the fix for this?

troubleshooting FAQ

Hello.. have you looked into this? also, its not possible to submit an App id as this issue isn't specific to a particular app but rather the entire console. It doesn't even load my apps and I have already submitted details for region

Hi @zamargab and @gnsslrrcs can you confirm that your user has permissions to list apps: https://docs.aws.amazon.com/amplify/latest/userguide/security_iam_permissions-reference.html

Yes , the user has permissions to list apps, Everything was working just a couple of days ago and I didn't change any settings at all.

I have this same issue.
I used to have 5 apps on the Amplify Console. Just yesterday I logged in and all my apps were gone.
I even tried creating a new app but immediately I navigate to "All apps" the app is gone. Yes, I know Amplify is region specific. I created the app in "us-east-1" and tried to query it in "us-east-1".
I have attached a drive video link to better explain.
https://drive.google.com/file/d/1IGwUMN8XDFDRkWZzTrXM83NC5PTXdY6Y/view?usp=sharing

@zamargab @McLeroy @gnsslrrcs we are investigating this currently. I will update you all shortly.

@zamargab @McLeroy @gnsslrrcs if you're still experiencing this issue can you please provide an AppID and region so we can further investigate what could be preventing you from listing your apps?

@hloriana I don't know how to get an AppID when I can't even create apps. I just noticed this happens in only "us-east-1" region

@hloriana I don't know how to get an AppID when I can't even create apps. I just noticed this happens in only "us-east-1" region

see if you can run this command via the CLI: https://docs.aws.amazon.com/cli/latest/reference/amplify/list-apps.html

@hloriana This is the result of aws amplify list-apps
{
"apps": []
}
But I get all the apps I created in us-east-2 when I run aws amplify list-apps --region us-east-2

@McLeroy and @zamargab Please email<EMAIL_ADDRESS>with your information and the link to this GitHub issue. We would need an AppID and region or an account number to further investigate since this doesn't appear to be a policy issue. Alternatively, you can also open up a ticket with AWS Support: https://console.aws.amazon.com/support/home#/


Add support in Launcher to filter out folders and tasks by assignment

Changelog Description
This adds a toggle in the Launcher to filter out the folders and tasks shown by whether the user has it assigned or not:
With filter

Without filter

Additional info
The next step would be to also filter out the tasks by whether they are assigned or not Done here https://github.com/ynput/ayon-core/pull/622/commits/61870650d8911c09393663047a96183a6ec638b5
Testing notes:

Open the launcher
Check that only the folders and tasks that you have assigned show up

I'm realizing this needs more work as the FoldersWidget is reused in other tools (i.e. workfile, loader...)

This PR made me realize that I never assign myself any tasks. I also have one user AYON Admin 😄.
anyways, I started assigning myself some tasks and I had to restart the launcher to see the updates.
I tried again and said to myself why don't I get back to project and re-select the same project to update the tasks list.
and then I faced this error.

Traceback (most recent call last):
  File "E:\Ynput\ayon-launcher\.venv\lib\site-packages\ayon_api\server_api.py", line 199, in raise_for_status
    self._response.raise_for_status()
  File "E:\Ynput\ayon-launcher\.venv\lib\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://<IP_ADDRESS>:5000/api/projects/None/hierarchy

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "E:\Ynput\ayon-core\client\ayon_core\lib\events.py", line 333, in process_event
    callback(event)
  File "E:\Ynput\ayon-core\client\ayon_core\tools\utils\folders_widget.py", line 577, in _on_folders_refresh_finished
    self.set_project_name(event["project_name"])
  File "E:\Ynput\ayon-core\client\ayon_core\tools\utils\folders_widget.py", line 456, in set_project_name
    self._refresh_folder_assignments(project_name)
  File "E:\Ynput\ayon-core\client\ayon_core\tools\utils\folders_widget.py", line 572, in _refresh_folder_assignments
    assigned_folder_paths = self._controller.get_assigned_folder_paths(project_name)
  File "E:\Ynput\ayon-core\client\ayon_core\tools\launcher\control.py", line 68, in get_assigned_folder_paths
    return self._hierarchy_model.get_assigned_folder_paths(
  File "E:\Ynput\ayon-core\client\ayon_core\tools\common_models\hierarchy.py", line 310, in get_assigned_folder_paths
    folder_items = self.get_folder_items(project_name, sender)
  File "E:\Ynput\ayon-core\client\ayon_core\tools\common_models\hierarchy.py", line 279, in get_folder_items
    self._refresh_folders_cache(project_name, sender)
  File "E:\Ynput\ayon-core\client\ayon_core\tools\common_models\hierarchy.py", line 573, in _refresh_folders_cache
    folder_items = self._query_folders(project_name)
  File "E:\Ynput\ayon-core\client\ayon_core\tools\common_models\hierarchy.py", line 577, in _query_folders
    hierarchy = ayon_api.get_folders_hierarchy(project_name)
  File "E:\Ynput\ayon-launcher\.venv\lib\site-packages\ayon_api\_api.py", line 785, in get_folders_hierarchy
    return con.get_folders_hierarchy(*args, **kwargs)
  File "E:\Ynput\ayon-launcher\.venv\lib\site-packages\ayon_api\server_api.py", line 3553, in get_folders_hierarchy
    response.raise_for_status()
  File "E:\Ynput\ayon-launcher\.venv\lib\site-packages\ayon_api\server_api.py", line 203, in raise_for_status
    raise HTTPRequestError(message, exc.response)
ayon_api.exceptions.HTTPRequestError: 404 Client Error: Not Found for url: http://<IP_ADDRESS>:5000/api/projects/None/hierarchy
WARNING:EventCallback:Failed to execute event callback < EventCallback - _on_folders_refresh_finished > E:\Ynput\ayon-core\client\ayon_core\tools\utils\folders_widget.py
Traceback (most recent call last):
  File "E:\Ynput\ayon-launcher\.venv\lib\site-packages\ayon_api\server_api.py", line 199, in raise_for_status
    self._response.raise_for_status()
  File "E:\Ynput\ayon-launcher\.venv\lib\site-packages\requests\models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://<IP_ADDRESS>:5000/api/projects/None/hierarchy

Looks good.
Is there any reason it's enabled by default in the launcher and disabled in the rest ?
Is there a way to specify whether or not to enable that check box by default ?

During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "E:\Ynput\ayon-core\client\ayon_core\lib\events.py", line 333, in process_event
callback(event)
File "E:\Ynput\ayon-core\client\ayon_core\tools\utils\folders_widget.py", line 577, in _on_folders_refresh_finished
self.set_project_name(event["project_name"])
File "E:\Ynput\ayon-core\client\ayon_core\tools\utils\folders_widget.py", line 456, in set_project_name
self._refresh_folder_assignments(project_name)
File "E:\Ynput\ayon-core\client\ayon_core\tools\utils\folders_widget.py", line 572, in _refresh_folder_assignments
assigned_folder_paths = self._controller.get_assigned_folder_paths(project_name)
File "E:\Ynput\ayon-core\client\ayon_core\tools\launcher\control.py", line 68, in get_assigned_folder_paths
return self._hierarchy_model.get_assigned_folder_paths(
File "E:\Ynput\ayon-core\client\ayon_core\tools\common_models\hierarchy.py", line 310, in get_assigned_folder_paths
folder_items = self.get_folder_items(project_name, sender)
File "E:\Ynput\ayon-core\client\ayon_core\tools\common_models\hierarchy.py", line 279, in get_folder_items
self._refresh_folders_cache(project_name, sender)
File "E:\Ynput\ayon-core\client\ayon_core\tools\common_models\hierarchy.py", line 573, in _refresh_folders_cache
folder_items = self._query_folders(project_name)
File "E:\Ynput\ayon-core\client\ayon_core\tools\common_models\hierarchy.py", line 577, in _query_folders
hierarchy = ayon_api.get_folders_hierarchy(project_name)
File "E:\Ynput\ayon-launcher.venv\lib\site-packages\ayon_api_api.py", line 785, in get_folders_hierarchy
return con.get_folders_hierarchy(*args, **kwargs)
File "E:\Ynput\ayon-launcher.venv\lib\site-packages\ayon_api\server_api.py", line 3553, in get_folders_hierarchy
response.raise_for_status()
File "E:\Ynput\ayon-launcher.venv\lib\site-packages\ayon_api\server_api.py", line 203, in raise_for_status
raise HTTPRequestError(message, exc.response)
ayon_api.exceptions.HTTPRequestError: 404 Client Error: Not Found for url: http://<IP_ADDRESS>:5000/api/projects/None/hierarchy
WARNING:EventCallback:Failed to execute event callback < EventCallback - _on_folders_refresh_finished > E:\Ynput\ayon-core\client\ayon_core\tools\utils\folders_widget.py
Traceback (most recent call last):
File "E:\Ynput\ayon-launcher.venv\lib\site-packages\ayon_api\server_api.py", line 199, in raise_for_status
self._response.raise_for_status()
File "E:\Ynput\ayon-launcher.venv\lib\site-packages\requests\models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://<IP_ADDRESS>:5000/api/projects/None/hierarchy

Oh yeah, I have fixed that on my fork, let me push it here as well

Looks good. Is there any reason it's enabled by default in the launcher and disabled in the rest ?

No reason really, we should probably just expose it as addon settings to let each studio choose their defaults

Oh yeah, I have fixed that on my fork, let me push it here as well
EDIT: done here 6e0de41

The error vanished. Thanks.
I wonder why the update doesn't work.. I had to return and reselect my project to make it work.

Oh yeah, I have fixed that on my fork, let me push it here as well
EDIT: done here 6e0de41

The error vanished. Thanks. I wonder why the update doesn't work.. I had to return and reselect my project to make it work.

Good catch, but the reason is because I'm only calling the function _refresh_folder_assignments on the set_project_name function, which only happens on first load. I can add it on the refresh function too and that should do it

Good catch, but the reason is because I'm only calling the function _refresh_folder_assignments on the set_project_name function, which only happens on first load. I can add it on the refresh function too and that should do it

Fixed here https://github.com/ynput/ayon-core/pull/622/commits/6dde360d71afbba05da72276b81cc2f9e3a781cb

The error is no longer occurs however a new bug shows up.
When closing the launcher after selecting a project and re-opening the launcher, the list becomes empty.


The attract menu does not appear for individual developer account in Partner Center

Type of issue
Outdated article
Feedback
The attract menu does not appear for individual developer account in Partner Center. Is this intentional? How can we enable it?
Page URL
https://learn.microsoft.com/en-us/windows/apps/publish/generate-promotional-codes
Content source URL
https://github.com/MicrosoftDocs/windows-dev-docs/blob/docs/hub/apps/publish/generate-promotional-codes.md
Author
@GrantMeStrength
Document Id
4a9cc8a8-0396-e184-f559-c7fd51169dc3

I was on the wrong page.


Improve ffi::specialize_for

Now the specialization for all integer types looks like this:
#[ffi::specialize_for(
    types = "u8",
    types = "u16",
    types = "u32",
    types = "u64",
    convention = "rust",
    name = "doublets_constants_*"
)]

I see two ways of development:
1. Remove convention and add explicit annotation with more luxury style:
#[ffi::specialize_for(
    types(
        u8  => "u8",
        u16 => "uint16",
        u32 => "uint",
        u64 => "ll",
    )
    name = "doublets_constants_*"
)]

#[ffi::specialize_for(
    u8(u8), u16(uint16), u32(uint), u64(ll),
    name = "doublets_constants_*"
)]

2. Use a more limited solution with embedded annotations:
#[ffi::specialize("doublets_constants_*")]

Improve in #10