test-race: podman pod rm -a doesn't remove a running container

/kind bug
Description
Test occasionally in CI fails, when running as a user (rootless testing).
Steps to reproduce the issue:

Re-run tests until it fails :frowning_face:

Describe the results you received:
Error: pod 15d...74f contains containers and cannot be removed: container already exists
Describe the results you expected:
Test consistently passes
Additional information you deem important (e.g. issue happens only occasionally):
Code is: test/e2e/pod_rm_test.go line 97
Output of podman version:
Happening on master, v1.1.2-269-gb83d0572
Additional environment details (AWS, VirtualBox, physical, etc.):
Two examples:

https://cirrus-ci.com/build/6328612444700672
https://cirrus-ci.com/task/5367713881915392

That is expected.
man podman pod rm
...
  -f, --force    Force removal of a running pod by first stopping all containers, then removing all containers in the pod.  The default is false

Correction: It's line 119 that fails.
@rhatdan That's what the test appears to be exercising: Confirm that rm -a fails.  This part is passing, it's the final check that fails, and only when running as a user (works fine when run as root):
    ...cut...
        // one pod should have been deleted
        result = podmanTest.Podman([]string{"pod", "ps", "-q"})
        result.WaitWithDefaultTimeout()
        Expect(len(result.OutputToStringArray())).To(Equal(1))

sigh so this rings some bells...it fails whenever there is NOT exactly one line of output :cry:
This is the worst setup to debug since we have NO idea what the multiple lines of output contain :confounded:   Maybe the first step in fixing this is to add some helpful output...

Ha! I think I caught it.  This is the ps output when the test is expecting 1 pod:
e67b1480769c   cool_jones        Created   1 second ago   0
d78c9ca2baf9   elastic_hermann   Running   1 second ago   1

So...since we're running tests in parallel, I think maybe this is another one for #2733 ?

This can't be  #2649 because separate --root and --runroot guarantee separation.  Therefor the unexpected pod must be coming from somewhere else.

So on my initial look, this test seems to make two pods, only one running, and then do an rm -a to remove only the not-running pod.
Initial diagnosis: the not-running pod is either not being deleted at all (really bad) or we're racing and it's still there when we do a ps (more likely, not as bad)

@mheon That's helpful, thanks.  I believe this is a race, sometimes it passes.  Sounds like the rm -a is exiting before the pod is actually removed.  Okay, I'll see if I can reproduce that manually (as a user).

@mheon Okay I'm with you now, having run the commands manually.  I agree your second theory seems more probable.  I used a defer() to produce the output above.  So I think that means there really were two pods still existing after the 'rm -a' happened (at least briefly).
Can you confirm that the 'rm -a' can return before all removals are complete?  If so, then all we need here is a retry/timeout loop on the final check.  I may even be able to manage writing one (or at least copying one) :grinning:

I don't think that should be possible... Part of rm that always happens before return is wiping the pod(s) in question from the database, so we should never see them again.

@mheon dang, well, it's def. happening or my eyeballs are lying to me.  I'll double-check, but I'm pretty sure the defer() I added ran a podman pod ps -a AFTER the test failed (count 2)...

...Yep, sure'nuff, from the run with the defer(podman pod ps -a) the failure came from line 126 (the final check).  Here's the full output of the test:

Running: /var/tmp/go/src/github.com/containers/libpod/bin/podman --root /tmp/podman_test083479571/crio --runroot /tmp/podman_test083479571/crio-run --runtime /usr/bin/runc --conmon /usr/libexec/podman/conmon --cni-config-dir /etc/cni/net.d --cgroup-manager cgroupfs --tmpdir /tmp/podman_test083479571 --storage-driver vfs pod ps -q
e67b1480769c
d78c9ca2baf9
output: e67b1480769c d78c9ca2baf9
Running: /var/tmp/go/src/github.com/containers/libpod/bin/podman --root /tmp/podman_test083479571/crio --runroot /tmp/podman_test083479571/crio-run --runtime /usr/bin/runc --conmon /usr/libexec/podman/conmon --cni-config-dir /etc/cni/net.d --cgroup-manager cgroupfs --tmpdir /tmp/podman_test083479571 --storage-driver vfs pod ps
POD ID         NAME              STATUS    CREATED        # OF CONTAINERS   INFRA ID
e67b1480769c   cool_jones        Created   1 second ago   0
d78c9ca2baf9   elastic_hermann   Running   1 second ago   1
output: POD ID NAME STATUS CREATED # OF CONTAINERS INFRA ID e67b1480769c cool_jones Created 1 second ago 0 d78c9ca2baf9 elastic_hermann Running 1 second ago 1
podman pod ps:
POD ID NAME STATUS CREATED # OF CONTAINERS INFRA ID e67b1480769c cool_jones Created 1 second ago 0 d78c9ca2baf9 elastic_hermann Running 1 second ago 1
[AfterEach] Podman pod rm
  /var/tmp/go/src/github.com/containers/libpod/test/e2e/pod_rm_test.go:31
Running: /var/tmp/go/src/github.com/containers/libpod/bin/podman --root /tmp/podman_test083479571/crio --runroot /tmp/podman_test083479571/crio-run --runtime /usr/bin/runc --conmon /usr/libexec/podman/conmon --cni-config-dir /etc/cni/net.d --cgroup-manager cgroupfs --tmpdir /tmp/podman_test083479571 --storage-driver vfs pod rm -fa
d78c9ca2baf993c0ed3924c6c04c8f3d18479a3c4f34d41f2dfa25ed83f29237
e67b1480769c431af35354e9be331df95ecdfd6b55a077d16f0abab384a21b2f
• Failure [4.302 seconds]
Podman pod rm
/var/tmp/go/src/github.com/containers/libpod/test/e2e/pod_rm_test.go:14
  podman pod rm -a doesn't remove a running container [It]
  /var/tmp/go/src/github.com/containers/libpod/test/e2e/pod_rm_test.go:98
  Expected
      <int>: 2
  to equal
      <int>: 1
  /var/tmp/go/src/github.com/containers/libpod/test/e2e/pod_rm_test.go:126
  Full Stack Trace

There was def. a "created" pod and a "running" pod :frowning_face:
I'll attempt to add a retry/timeout loop...

So we still have 2 pods here, after rm finished. Alright, that's properly wierd.

@mheon yeah, and I'm assuming this test owns both of them. i.e. we're NOT seeing a pod from another, parallel test, or a leftover from prior test.  I'm also assuming my understanding of the output is correct (i.e. it may not be) :grin:

It looks like they're the same pods the test is trying to remove, so that sounds accurate

@mheon Eww, a retry-loop doesn't fix this either - also the ginkgo test fails...but somehow gets marked as a pass??? (ignoring this for now).  Remember: This is the rootless test (running this as root works fine).  What other data can I grab that would be helpful debugging this?
Maybe @giuseppe do you have any ideas here, how a user's podman pod rm -a would sometimes return before removing an empty pod?
...cut...
Retry 10, pods 2Running: /var/tmp/go/src/github.com/containers/libpod/bin/podman --root /tmp/podman_test430270252/crio --runroot /tmp/podman_test430270252/crio-run --runtime /usr/bin/runc --conmon /usr/libexec/podman/conmon --cni-config-dir /etc/cni/net.d --cgroup-manager cgroupfs --tmpdir /tmp/podman_test430270252 --storage-driver vfs pod ps -q
da42f5236eb8
73f729419d1c
output: da42f5236eb8 73f729419d1c
...cut...
Retry 1, pods 2Running: /var/tmp/go/src/github.com/containers/libpod/bin/podman --root /tmp/podman_test430270252/crio --runroot /tmp/podman_test430270252/crio-run --runtime /usr/bin/runc --conmon /usr/libexec/podman/conmon --cni-config-dir /etc/cni/net.d --cgroup-manager cgroupfs --tmpdir /tmp/podman_test430270252 --storage-driver vfs pod ps -q
da42f5236eb8
73f729419d1c
output: da42f5236eb8 73f729419d1c
[AfterEach] Podman pod rm
  /var/tmp/go/src/github.com/containers/libpod/test/e2e/pod_rm_test.go:32
Running: /var/tmp/go/src/github.com/containers/libpod/bin/podman --root /tmp/podman_test430270252/crio --runroot /tmp/podman_test430270252/crio-run --runtime /usr/bin/runc --conmon /usr/libexec/podman/conmon --cni-config-dir /etc/cni/net.d --cgroup-manager cgroupfs --tmpdir /tmp/podman_test430270252 --storage-driver vfs pod rm -fa
73f729419d1ce60e6a53f4f0d772022cdb6a9ac0de84e8a87df77937b1a06798
da42f5236eb8e7ce302d6a827b29bf5ddb80c350f3c0c4c2a5e0d3b06569ff66
• Failure [60.155 seconds]
Podman pod rm
/var/tmp/go/src/github.com/containers/libpod/test/e2e/pod_rm_test.go:15
  podman pod rm -a doesn't remove a running container [It]
  /var/tmp/go/src/github.com/containers/libpod/test/e2e/pod_rm_test.go:99
  Expected
      <int>: 2
  to equal
      <int>: 1
  /var/tmp/go/src/github.com/containers/libpod/test/e2e/pod_rm_test.go:130
  Full Stack Trace

OOOOOHHH! I see how the test is passing:
SUCCESS! -- 1 Passed | 0 Failed | 1 Flaked | 0 Pending | 632 Skipped

Ginkgo is retrying it, so maybe something about the setup or cleanup is causing the problem then fixing it?

@mheon @baude Okay, adding this line at the beginning of the test appears to help.  I need to do a few more re-runs to be sure though.  So, to recap this seems to be the behavior:

As root, all tests pass normally.
As a user, this line SOMETIMES fails.
When it fails, podman pod rm -a exits 0, but podman pod ps always shows the same two pods that were started
Adding the timeout/retry loop doesn't alter the behavior, ps shows the empty pod never gets removed.
Ginkgo attempts to to re-run it up to 3 more times.   Presumably calling cleanup after each..
Sometimes this works, sometimes not.
Adding a call at the start of the test to podmanTest.CleanupPod() appears to help, though one run isn't enough to be sure.

Questions:

How is it possible to have a race with podman pod rm -a which would SOMETIMES not remove empty pods but still exit 0.
Assuming step 7 (above) pattern persists, does that suggest CleanupPod() is removing or resetting some state which is shared, and should not be?
Is it possible multiple tests are using the same /tmp/blah directory?
What other resources could be mistakenly shared across multiple tests, that could explain this behavior?

update: Ugg, nope, step 7 doesn't affect behavior.  It just failed on the first run w/ CleanupPod() at the start.  So it's still demonstrating Step 3 (above) randomly :confounded:

@mheon is there any way I can instrument the test to give us more info on what's happening?  e.g. if the test detects this condition, how can I confirm the database matches reality (the removed pod exists or doesn't).?

@cevich I think you might want to try debug logging on podman pod rm -a first - see if it actually sees the pods and tries to remove them

@mheon dangit...shoulda seen this coming:  Adding debug seems to prevent the race:   As many races are...timing-dependent :face_with_head_bandage:    So, we'll have to do this somewhat blind, but maybe not completely.  The condition does seem to persist once it occurs.  I s'pose we don't yet know if this is a problem with the rm or with the podman pod ps.  Let me try just removing the --log-level=debug from the rm and see if it reproduces with only having it on the ps.

Confirmed in this run the problem reproduces without the debug on the podman pod rm -a  and with it on the podman pod ps.  @mheon I don't see anything incriminating in the output from the two failed run (a flake re-run was successful).
Since debugging the rm masks the problem, is there some easy-ish way to verify the directory structure and/or database contents are correct?  Basically, make sure this is a 'pod rm' problem and not a 'pod ps' problem.

If it still shows up in ps and rm did not error I think it's safe to assume that rm never fired on the pod in question - if we had DB issues subsequent commands would be blowing up.

If we throw a, say, 1 second sleep before the rm -a, does it resolve the race? I'm thinking we have some sort of race where rm doesn't see the pods and doesn't try to remove them

Ahh yes, we can see that in the logs:
  /var/tmp/go/src/github.com/containers/libpod/test/e2e/pod_rm_test.go:99
Running: /var/tmp/go/src/github.com/containers/libpod/bin/podman --root /tmp/podman_test703242098/crio --runroot /tmp/podman_test703242098/crio-run --runtime /usr/bin/runc --conmon /usr/libexec/podman/conmon --cni-config-dir /etc/cni/net.d --cgroup-manager cgroupfs --tmpdir /tmp/podman_test703242098 --storage-driver vfs pod rm -fa
Running: /var/tmp/go/src/github.com/containers/libpod/bin/podman --root /tmp/podman_test703242098/crio --runroot /tmp/podman_test703242098/crio-run --runtime /usr/bin/runc --conmon /usr/libexec/podman/conmon --cni-config-dir /etc/cni/net.d --cgroup-manager cgroupfs --tmpdir /tmp/podman_test703242098 --storage-driver vfs --log-level=debug pod ps -q

It does NOT show the expected normal Error: pod 783ea2e51f5da239b65201d5aacf0bf29bb6a29bcc486f804438c3fcde067554 contains containers and cannot be removed: container already exists
In fact, that's something the test can look for (also).  Thanks @mheon I'll try adding the delay.

@mheon okay, 2-out-of-3 re-runs passed, the third one failed as before but after sleeping :cry:   Since that really sucks, I'm going to do a few more re-runs to see if I can make it happen again...

@mheon sigh...second occurrence (after 2-3 more re-runs).  So adding the sleep did not change the behavior substantially, it still flakes AFTER also emitting the expected error message :frowning:

trying with the podman pod rm -a inside a retry loop.  Perhaps re-running it multiple times will force eventual consistency.  note: I don't see this as a fix, just another debugging step.  At this point the race seems likely  to be in podman pod rm -a (somehow, possibly specific to CI)...

We're getting the error message, but are we printing the ID of the other container? If we found the running pod but not the stopped pod that could explain things.

There's absolutely no output regarding the other pod. It's not even trying to remove it.
@cevich If we retry, does it always go to 10? Or does the number of failures vary? I have some ideas about the cause now...

@mheon It only took 2 attempts to get it to fail.  My guess is that it always retries 10-times, but I will let you know if I see anything different in a re-run.

Wait a goddamn... I know what this is

Rootless Podman cannot remove multiple pods at the same time. For some reason it's not throwing an error when we try to do this.

How does this test even pass?
@giuseppe we're only joining one pod's userns, but somehow, sometimes, it successfully removes both pods?

Here's a run with debugging enabled on the rm and a 10-try test failure.
@mheon it DOES print the 'can't remove pod' error message.  I added a check for this.  Yet still doesn't actually remove the pod.

Re: How does it pass:  maybe this is the race, perhaps (somehow) the pod which is suppose to be removed, removes itself coincidentally with the rm command.  That would represent a false-positive condition this test would miss.

@cevich The error message is 100% expected. This test expects one pod to fail.
The issue is that pod 2, the not-running pod, is not even touched. We're not even attempting to remove it.

I think it has to be an ordering thing in the DB - one of the pods will be returned first by the iterator. I think we succeed if the one that's not running is returned first... maybe because we don't need to join its userns because not running?

(Iteration order is based on ID, so there's our randomness - if the pod that is not running has a lower ID, we succeed, else we fail...)

Ohhh okay, I follow.  That explains why retrying the removal fails also.  So is the only thing special in rootless-vs-root, somehow when running as root, the IDs always are the "correct" way around?
(It's possible this problem is happening when run as root, I'm just not seeing it - I also didn't look super closely)

The multiple-pod-removal restriction is rootless only - root pods work as advertised.

Excellent, so it sounds like the problems are fully-known here then.  Whew! What a ride!
Is there anything I can do to help fix this and/or should I just close #2737 then?

Should be fixed by: #2706

Looks like this is fixed.


Design the homepage for mobile screens

The homepage for HDU looks great and is responsive for most screens, but an unconventional ordering of information and long scrolling can confuse visitors. A previous issue asked graphic designers to design a better homepage view for mobile screens.
For this issue, we're looking for web developers to code the design. You can have a look at the design as a guide from here contributed by another contributor.

Hi @Sayed-Afnan-Khazi, can you assign this to me? Can i make a PR and work on this??

Sure, @FaithNjah. I've assigned this to you. Feel free to send a PR with your work linked to this issue. Good luck!

Thank you @Sayed-Afnan-Khazi


CodeFix import for lodash does not work

Not sure if it's the issue with lodash typings or something else, but CodeFix import for lodash does not work:

but if I import manually then it's ok

Not sure if related: https://github.com/Microsoft/TypeScript/issues/13374

Man lodash/index.d.ts is daunting.


Add support for Base.zero

This PR adds support for Base.zero, which is helpful for initializing thermo state fields in ClimaAtmos.

bors r+


Losing an order of keys after unflatten an object

This PR fixes an order of keys in an object. Mentioned in issue #43
I make all objects and arrays to flat before it is unflattened. It fixes both issues keeping the order and adding objects and arrays in a flattened object.

Ran into this issue too. Can this be merged?

Thanks for this, apologies for delay, fixed conflict, squashed and merged in 2ac1b4dbbee9146ab8754c31a5ccae757d824c8e


test: remove conda resolver constraint

@mederka do we want to completely drop conda? One of these tests actually was triggering a pretty interesting bug https://github.com/fal-ai/fal/actions/runs/8873813532/job/24360199536 . Here it should be using isolate 0.12.9 that got released a few days ago and bumped in isolate-cloud, but the error was complaining about --force even though 0.12.9 doesn't have it (and uses --yes). There is another symptom though.

@mederka This was triggering an interesting bug. The error was about --force , but it got removed (replaced with --yes) in isolate 0.12.9 which got released on shark a few days ago. I think it was a symptom of a bug, where somehow our workers install new isolate, but somehow run older code. Wasn't able to figure it out yet...


[FancyZones editor] be able to directly debug against it

A tool can be used to start the editor without requiring to have FZ running and to pass the arguments we want.
The tool should have two modes:

execute the same code that FZ executes to build the command line arguments (this can be used when we want to debug the editor on a real configuration).

accept command-line arguments (or as an alternative read a config file) where we specify external monitors that don't exist, build the arguments for the editor adding the fake monitors.

To be crisp here, the requirement is i can directly just F5 the editor like we can with other utilities.

To be crisp here, the requirement is i can directly just F5 the editor like we can with other utilities.

FZ Editor requires data passed as command-line arguments including monitor configuration, e.g. DPI and top-left coordinates of each monitor. It would be time-consuming to manually pass arguments if someone needs to test the editor on many different configurations.

is this still in progress?

@crutkas
yes, we made progress, but we didn't focus on this since the priority was to complete the new UI.

@enricogior is this part of 0.35?

@crutkas
probably not, more likely 0.37.

I agree to close the issue.


Launching Ubuntu 20.04 LTS... The group or resource is not in the correct state to perform the requested operation.

Hi, I've been trying to install Ubuntu-20.04 on my PC.

I've just updated my WSL2 kernel, issue still persist
Here is the log:
WslLogs-2023-04-20_10-38-58.zip

Unfortunately your trace doesn't have any. Could you please take another trace?

Begin tracing
Repro issue
End tracing
/logs

it appears that running wsl --install -d Ubuntu-20.04 --web-download
in Administrator Power Shell gives me:
Launching Ubuntu 20.04 LTS... The group or resource is not in the correct state to perform the requested operation. Error code: Wsl/InstallDistro/0x8007139f
but in a standard Power Shell, it gives:
WslRegisterDistribution failed with error: 0x80370114 Error: 0x80370114 The operation could not be started because a required feature is not installed.

I think I understood the issue now:
When running on standard PowerShell:
Virtual Machine Platform was not turned on.
Once turned on, rebooted, and I was able to complete wsl --install -d Ubuntu-20.04 --web-download
When running on elevated PowerShell:
Network access is disabled by the company, hence unable to unpack the downloaded package


Alignment for streamline model

Update Decode Cmake to match streamline model.
Fix umc vc1 warning.

Just want to check . PR name says cmake update. But here two changes: vc1 warnings fix and cmake. Is it intended? If yes, please correct commit title, or consider splitting change into two commits.

Commit message updated. They are intended to be merged


doxygen warnings about parameters

There are doxygen warnings about "alloc" parameters which no longer exist

"fix doc warnings"


When certificate and key is stored in secret, do not write to disk

Use case: We run a job to initialize an index for Searchguard in Elasticsearch that authenticates using a client certificate to our cluster. This requires an "admin" client certificate that has a DN that is matched on. We store this in a secret right now to allow users to use it to manually administer the Searchguard index, and then mount the secret in the main container. So there is no need to write it to disk.
For us specifically, there's no need for the manual workflow however, so if there's an automatic approver, we can use transient keys instead.

What is the disadvantage of also writing it to disk? Is it worth the complexity of adding an option to not write to disk?

So the desire is to allow a human to easily fish out the credential for a pod and then manually act using the pod's identity? Would it be too expensive to set up a separate approval path to allow humans to get appropriate credentials in their own right? The latter would be better for audit logs.

I agree with you, it would be better for humans to get their keys in a separate process, so maybe it is a feature you can strip completely?
What I was very clumsily getting to in the end was that if you choose to store in a secret, you should only store in a secret - as then you can check whether the secret exists and exit if it does, making a very simple implementation (rather than also reading it out and writing to disk).

I was wondering whether I should rip that back out. It wasn't, however, getting in the way and I considered that, despite my not being able to figure out the use case, there must have been one for Onedata to have written that code.
If the secret exists, I don't know why the value in it would necessarily be better than the one just issued. I would think the value in the secret would be more likely to be nearing or past expiration, have an out-of-date set of SANs, etc.

That is true. Perhaps it can be stripped out? I can honestly not think of a case where you'd want to have an automatically approved key/cert pair in a secret.

Removed support for writing to a secret.


Fix CSR approving issue for existing nodes with already approved and GCed CSRs

What type of PR is this?
/kind bug
What this PR does / why we need it:
Successfully approved CSR is GC-ed after 1 hour, effectively leading to the situation where perfectly good Kubelet cert doesn't need any new CSRs will make kubeone to throw out with misleading error.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1893
Does this PR introduce a user-facing change?:

Fix CSR approving issue for existing nodes with already approved and GCed CSRs

/cherrypick release/v1.4

/unhold


pull request for e2e test at 12/03/2016 04:45:03

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

optest/docs/updated_at.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Hactoberfest Participate Request

Why don't you participate in hactoberfest ? It's great to do that. Please add this repo on hactoberfest. 🙏
By doing this, i think more people will come to contribute.

👍🏻

Hmm idk but its caspers choice also people from hacktoberfest are smart and dont use quick.db anymore i think but like i hope it does no hate do

Ill think about it when im home later today.

I've decided to go with it, feel free to open pull requests and let me know if it's for hacktoberfest so I can add the labels.
Read

Do NOT violate the rules of hacktoberfest
Do NOT spam this repo with poor pull requests

Thanks for it. 🥰


isc.txt: wrap text of the license to 80 columns

As a bonus, the line breaks are closer to the clause boundaries,
which hopefully makes the text slightly easier to read.

There are no tests to enforce, but the standard here is 78 see #74 for explanation. Feel free to argue that is the wrong decision.
Does 78 obtain the same result for ISC?
Related, maybe worth additional issues but just noting here for now:

It'd be nice to have a test enforcing wrapping at 78, ideally comparison with output of (2), but even a stupid test that no line in license text is >78 characters would be a start.
Is it possible to automatically wrap all licenses at 78 and have output make sense? Or is manual intervention required? I suspect a fair amount of special casing would be needed, but haven't tried. In addition to being used for tests, could be part of script to ingest license texts from SPDX in #489.

(re 2 above, it would be easier to get good output if wrapping as part of structured SPDX XML license to text, maybe in https://github.com/spdx/tools/blob/master/src/org/spdx/licensexml/LicenseXmlHelper.java, rather than attempting to wrap plain text.)

Feel free to argue that is the wrong decision.

Honestly, the resolution in that thread sounds more as an arbitrary choice than a pondered decision. That said, @bendiken's comment in that thread (and particularly the SO link within it) does provide some good points in favor of a max width of less than 80 chars, so I won't contest the resolution.

Does 78 obtain the same result for ISC?

No. Several lines in this PR are 79 or 80 characters long.

It'd be nice to have a test enforcing wrapping at 78

I agree that such tooling would be helpful, but I'm not able to contribute to that.

Is it possible to automatically wrap all licenses at 78 and have output make sense?

I'm not sure what you mean by "make sense". Do you have any specific example in mind?
As for this PR, feel free to close it in light of the current guideline of 78 chars.

First draft of test in #552.
By "make sense" I mean lines that shouldn't be wrapped, aren't, and wrapping maintains indentation appropriately. That requires some smarts about when lines that aren't separated by a newline shouldn't be potentially merged, and figuring out what the appropriate indentation is for each line. Licenses with lists or indented items (eg AFL-3.0, Apache-2.0) cause problems for the fmt command, so something smarter than that is required.
Closing this, thanks for the idea!


bedtools makewindows step option error

I used to be able to run with a previous version of bedtools:
bedtools makewindows -g genome.txt -n 1
Now (v2.22.1), I get an error:
ERROR: The step (-s) option must be greater than zero.
It does not mention that the step must be specified, so I don't think I should be getting this error.

Thanks for reporting this, I just pushed a fix.


Django Tabbed Admin fail when has_change_permission is false

I have an admin class like this:
from tabbed_admin.admin import TabbedModelAdmin
class mlp(TabbedModelAdmin):
    model=Book
    tab_overview=(
       ("",{"fields": ('name','status',),}),
    )
    tabs=(
        ('Overview',tab_overview),
        )

    def has_change_permission(self, request, obj=None):
        return False

When I try to open that in admin, I get:

"Key 'name' not found in 'BookForm'. Choices are: ."

Any idea what it may be? It works fine with the has_add/delete_permission.
Tks

I am having the same problem as @iwalucas .
Do you have a possible workaround for this?

Same here, any ideas?

@BestCaseManagement @JacoBezuidenhout @iwalucas
Not sure if this is correct solution but
for user with view only permissions I have to override get_form TabbedModelAdmin with change = False, here is my code snipe:
   def get_form(self, request, obj=None, change=False, **kwargs):         # just save obj reference for future processing in Inline         request._obj_ = obj         return super(RoomAdmin, self).get_form(request, obj, **kwargs)
If you'r able to find a better solution please share with us.


What is the way to declare an endpoint that returns a flask.Response object?

I'm trying to create an endpoint that returns a flask.Response (302).
Code:
from flask import redirect
def dummy():
    return redirect('/')

But I get the following error:
<Response 209 bytes [302 FOUND]> is not JSON serializable

Additionally if I try to avoid the JSON conversion setting the output for that endpoint as text/plain using the following swagger definition:
  /dummy:
    get:
      produces: "text/plain"
      operationId: controllers.dummy.dummy
      responses:
        302:
          description: OK

I get the following stack trace when I launch the server:
test_1 | ERROR:connexion.api:Failed to add operation for GET /v1/dummy
test_1 | Traceback (most recent call last):
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/api.py", line 136, in add_paths
test_1 |     self.add_operation(method, path, endpoint)
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/api.py", line 122, in add_operation
test_1 |     self.blueprint.add_url_rule(flask_path, operation.endpoint_name, operation.function, methods=[method])
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/operation.py", line 234, in function
test_1 |     produces_decorator = self.__content_type_decorator
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/operation.py", line 271, in __content_type_decorator
test_1 |     mimetype = self.get_mimetype()
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/operation.py", line 165, in get_mimetype
test_1 |     if produces_json(self.produces):  # endpoint will return json
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/utils.py", line 121, in produces_json
test_1 |     return all(is_json_mimetype(mimetype) for mimetype in produces)
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/utils.py", line 121, in <genexpr>
test_1 |     return all(is_json_mimetype(mimetype) for mimetype in produces)
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/utils.py", line 95, in is_json_mimetype
test_1 |     maintype, subtype = mimetype.split('/')  # type: str, str
test_1 | ValueError: need more than 1 value to unpack
test_1 | ERROR:connexion.api:Failed to add operation for GET /v1/dummy
test_1 | Traceback (most recent call last):
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/api.py", line 136, in add_paths
test_1 |     self.add_operation(method, path, endpoint)
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/api.py", line 122, in add_operation
test_1 |     self.blueprint.add_url_rule(flask_path, operation.endpoint_name, operation.function, methods=[method])
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/operation.py", line 234, in function
test_1 |     produces_decorator = self.__content_type_decorator
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/operation.py", line 271, in __content_type_decorator
test_1 |     mimetype = self.get_mimetype()
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/operation.py", line 165, in get_mimetype
test_1 |     if produces_json(self.produces):  # endpoint will return json
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/utils.py", line 121, in produces_json
test_1 |     return all(is_json_mimetype(mimetype) for mimetype in produces)
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/utils.py", line 121, in <genexpr>
test_1 |     return all(is_json_mimetype(mimetype) for mimetype in produces)
test_1 |   File "/usr/local/lib/python2.7/site-packages/connexion/utils.py", line 95, in is_json_mimetype
test_1 |     maintype, subtype = mimetype.split('/')  # type: str, str
test_1 | ValueError: need more than 1 value to unpack

What is the right way to return a flask.Response object using connexion?

returning flask.Response should work out of the box (I haven't tested "redirect"..), will check...
regarding the "produces" mimetype: The "produces" property needs to be a list of mimetypes, see https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#fixed-fields (obviously Connexion's error handling could be better..)

@hjacobs flask.Response is handled by Produces(BaseSerializer) class:
class Produces(BaseSerializer):
    def __call__(self, function):
        """
        :type function: types.FunctionType
        :rtype: types.FunctionType
        """

        @functools.wraps(function)
        def wrapper(*args, **kwargs):
            url = flask.request.url
            data, status_code, headers = self.get_full_response(function(*args, **kwargs))
            logger.debug('Returning %s', url, extra={'url': url, 'mimetype': self.mimetype})
            if isinstance(data, flask.Response):  # if the function returns a Response object don't change it
                logger.debug('Endpoint returned a Flask Response', extra={'url': url, 'mimetype': data.mimetype})
                return data

BaseSerializer has associated the mimetype text/plain but I'm not able to set the endpoint output to text/plain, I tried with text/plain and text/plain; charset=utf-8 as is specified in the OpenAPI spec but I get the error that I pasted in the issue.

Ok.. the problem with the text/plain is solved. It was my fault:
produces: text/plain

is not the the same than:
produces:
  - text/plain

But it is still sending a 200 response with the text conversion of the flask.Response object:
curl http://<IP_ADDRESS>:8080/v1/dummy?test=1
<Response 209 bytes [302 FOUND]>

test_1 | <IP_ADDRESS> - - [03/Feb/2016:09:21:46 +0000] "GET /v1/dummy?test=1 HTTP/1.1" 200 32 "-" "curl/7.43.0"

Ok.. It's solved. The way to go is the following:
def test_redirect_endpoint():
    headers = {'Location': 'http://www.google.com/'}
    return '', 302, headers

Instead of returning a real flask.Response object

hmm, IMHO real flask.Response should also work -- I will reopen the issue...

@hjacobs I can send you two tests showing the behaviour if it's helpful for you guys

Check https://github.com/jvgutierrez/connexion/commit/3697a5d72c40795fe5600491a609f8421d11322d
tests/test_app.py::test_redirect_endpoint PASSED
tests/test_app.py::test_redirect_response_endpoint FAILED

__________________________________________________________________________ test_redirect_response_endpoint __________________________________________________________________________

app = <connexion.app.App instance at 0x7f531002d4d0>

    def test_redirect_response_endpoint(app):
        app_client = app.app.test_client()
        resp = app_client.get('/v1.0/test-redirect-response-endpoint')
>       assert resp.status_code == 302
E       assert 500 == 302
E        +  where 500 = <Response streamed [500 INTERNAL SERVER ERROR]>.status_code

------------------------------------------------------------------------------- Captured stderr call --------------------------------------------------------------------------------
DEBUG:connexion.decorators.produces:Jsonifing http://localhost/v1.0/test-redirect-response-endpoint
DEBUG:connexion.decorators.parameter:Function Arguments: []
DEBUG:connexion.decorators.decorator:Getting data and status code
DEBUG:connexion.decorators.decorator:Got data and status code (200)
DEBUG:connexion.decorators.decorator:Getting data and status code
DEBUG:connexion.decorators.decorator:Got data and status code (200)
ERROR:test_app:Exception on /v1.0/test-redirect-response-endpoint [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/app/connexion/decorators/produces.py", line 126, in wrapper
    data = json.dumps(data, indent=2, cls=JSONEncoder)
  File "/usr/local/lib/python2.7/json/__init__.py", line 251, in dumps
    sort_keys=sort_keys, **kw).encode(obj)
  File "/usr/local/lib/python2.7/json/encoder.py", line 209, in encode
    chunks = list(chunks)
  File "/usr/local/lib/python2.7/json/encoder.py", line 442, in _iterencode
    o = _default(o)
  File "/app/connexion/decorators/produces.py", line 43, in default
    return json.JSONEncoder.default(self, o)
  File "/usr/local/lib/python2.7/json/encoder.py", line 184, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: <Response 251 bytes [302 FOUND]> is not JSON serializable

@jvgutierrez :+1: thanks for reporting and the test (I was assuming it worked fine)

@hjacobs I can take care of this issue.

fixed in https://github.com/zalando/connexion/releases/tag/1.0.52


new bricks

Created 1x2 brick and 2x1 brick

Some work on #47

:+1:

What I was aiming for was a brand new lego brick, not based on the already made one, please see #47 I know how to make the new bricks, it would just seem better to make a new version. I won't be merging this though as I have a higher quality lego brick that I will be creating a 2x1 and 1x2 brick. The current ones you made have a very slight light/dark border in the middle, just doesn't seem right.

@zahradtj Out of curiosity, where did you get those images from? Also, do they exist in SVG?


New line is truncated and new section skipped when line ends with -%}

Expected Behavior
I noticed a weird behavior with the case when the template is divided by sections and one of the serctions end with -%} then the next section is actualy becomes a part of the current section.
What I expect to be happening:
template:
section_one:
    array_of_keys:
    {% for item in section_one__array_of_keys -%}
    - some_key: "{{ item['some_key'] }}"
    {% endfor -%}

section_two:
    key_one: section_two__key_one
    key_two: section_two__key_two

expected result once generated:
section_one:
    array_of_keys:
    - some_key: 'some_value'
    - some_key: 'some_other_value'

section_two:
    key_one: 'key_one_value'
    key_two: 'key_two_value'

Actual Behavior
the actual result, see that the section_two is in the wrong spot now:
section_one:
    array_of_keys:
    - some_key: 'some_value'
    - some_key: 'some_other_value'
    section_two:
    key_one: 'key_one_value'
    key_two: 'key_two_value'

If the for loop {% endfor -%} is not at the end of the section, this doesn't happen.
Template Code
section_one:
    array_of_keys:
    {% for item in section_one__array_of_keys -%}
    - some_key: "{{ item['some_key'] }}"
    {% endfor -%}

section_two:
    key_one: section_two__key_one
    key_two: section_two__key_two

Full Traceback
N/A

Your Environment

Python version: 2.7.14
Jinja version: 2.8.1

Recreated the issue with Python 2.7.15 and Jinja 2.10
There are 4 possible combinations of where the - sign is

section_one:
List of items:
{% for item in seq -%}
- "{{ item }}"
{% endfor -%}
section_two:
- item 1
- item 2
result in

section_one:
List of items:
- "1"
- "2"
- "3"
- "4"
section_two:
- item 1
- item 2
2. ```
section_one:
    List of items:
    {%- for item in seq %}
    - "{{ item }}"
    {%- endfor %}

section_two:
    - item 1
    - item 2

result in
section_one:
    List of items:
    - "1"
    - "2"
    - "3"
    - "4"

section_two:
    - item 1
    - item 2

section_one:
List of items:
{%- for item in seq %}
- "{{ item }}"
{% endfor -%}
section_two:
- item 1
- item 2
result in

section_one:
List of items:
- "1"
- "2"

- "3"

- "4"
section_two:
- item 1
- item 2

4. ```
section_one:
    List of items:
    {% for item in seq -%}
    - "{{ item }}"
    {%- endfor %}

section_two:
    - item 1
    - item 2

result in
section_one:
    List of items:
    - "1"- "2"- "3"- "4"

section_two:
    - item 1
    - item 2

Only the last combination can separate section_one and section_two, will investigate further

In OP's case, if I replace the spaces with string, it will be clear that what happened
section_one:
    List of items:
aaaa{% for item in seq -%}
    - "{{ item }}"
    - 2nd instance of "{{ item }}"
bbbbbbbb{% endfor -%}

section_two:
- item 1
    - item 2

result in
section_one:
    List of items:
aaaa- "1"
    - 2nd instance of "1"
bbbbbbbb- "2"
    - 2nd instance of "2"
bbbbbbbb- "3"
    - 2nd instance of "3"
bbbbbbbb- "4"
    - 2nd instance of "4"
bbbbbbbbsection_two:
- item 1
    - item 2

In the first iteration
{% for item in seq -%} takes out the white spaces before     - "{{ item }}" and creates
aaaa- "1"
    - 2nd instance of "1"

In the next iteration, bbbbbbbb{% endfor -%} is expanded into
bbbbbbbb- "2"
    - 2nd instance of "2"

in the last iteration, the - in endfor removes all the new lines before section_two and created
bbbbbbbbsection_two:
- item 1

This is expected behavior. If you want to generate YAML, you should generate a data structure in Python, then use a YAML serialization library. It looks like you might be using Ansible, which has filters for converting from and to YAML rather than templating it directly with Jinja.


[etcdsnapshot&restore] Add dev tooling and docs

Add examples folder, local dev script (+Tilt changes) and documentation for setting up a local environment

closing this for now, we can open less generic issue for future developer setup tasks


feat: update formula documentation to include new fields

Provide a description of what has been changed
Checklist

[ ] Commits are signed with Developer Certificate of Origin (DCO)

Fixes #

Can you also update the "Scaling Modifiers (Experimental) " section please?

sure!


Attribute expressions

Description
Add support to GQL for receiving and writing attribute Expressions
Testing
Updated tests, added some UTs, verified everything E2E
Checklist:

[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] Any dependent changes have been merged and published in downstream modules

Part of https://github.com/hypertrace/hypertrace-ui/issues/1099

@hypertrace/graphql review please.


Fatal error: Class 'scssc' not found in [...]/vendor/kriswallsmith/assetic/src/Assetic/Filter/ScssphpFilter.php on line 47

https://github.com/leafo/scssphp/tree/v0.5.0 seems to have broken this library. This looks to be the specific commit:
https://github.com/leafo/scssphp/commit/3f87f8ba1e066f0b18880488d724ddd76d7d150b
As of 16 hours ago, composer update causes the following entries to be removed from autoload_classmap.php:
    //...
   'scss_formatter' => $vendorDir . '/leafo/scssphp/classmap.php',
    'scss_formatter_compressed' => $vendorDir . '/leafo/scssphp/classmap.php',
    'scss_formatter_crunched' => $vendorDir . '/leafo/scssphp/classmap.php',
    'scss_formatter_nested' => $vendorDir . '/leafo/scssphp/classmap.php',
    'scss_parser' => $vendorDir . '/leafo/scssphp/classmap.php',
    'scss_server' => $vendorDir . '/leafo/scssphp/classmap.php',
    'scssc' => $vendorDir . '/leafo/scssphp/classmap.php',
    //...

Consequently, reliant libraries now yield this:
Fatal error: Class 'scssc' not found in [...]/vendor/kriswallsmith/assetic/src/Assetic/Filter/ScssphpFilter.php on line 47

Update assetic to the latest version. We already stopped using the deprecated scssc class since Assetic 1.2

Whoops, my bad! Another third-party library had pegged Assetic to ~1.1.*. I will follow-up with that library. Apologies for wasting your time, good sir!


iTotalRecords is not being set correctly

https://github.com/umesh-krishna/django_serverside_datatable/blob/0da2f9b5c6f1e5da2cc9aa1e521b1993533b7cb7/django_serverside_datatable/datatable.py#L27
iTotalRecords is being set to cardinality but it should be set to be self.qs.count()
I can provide a example if necessary, but I think this error should be evident and obvious to fix.

Fixed


Document at options page that when using require hook, ignore essentially has default value of **/node_modules/**

And that if you want to additionally ignore something else, you have to re-add node_modules.

It's a bit more than just a default. It only is set to node_modules when ignore and only are not set.
See https://github.com/babel/babel/blob/91fb64eaf53c67a7ed7605aab71f8dcaa4d39ae2/packages/babel/src/api/register/node.js#L113-L119

@thejameskyle yeah, that's what I meant by "essentially". Are you saying that my description is too sloppy considering the getRelativePath() part? :smiley: Or something about where it should be documented or how it should be described?


firefish.social - 404 error when adding account to list

internal server error, also on calckey.social


Avoid forcing lowercase

The value entered by the user was forced in lowercase.
Allow for the value to remain the same as the user entered it.

includes is never set as a value, its used to filter and map search terms for case insensitivity searching.
I can only assume I'm this was an attempt to fix searching with options groups.
A more functional fix is found below.
https://github.com/monterail/vue-multiselect/pull/321

@sandbox-services This was indeed an attempt to fix a bug: when typing a value that does not exist in tag mode, when creating the value it does not keep the case, everything is lowercase.

I am not able to re-produce your issue in the demo of current version of 2.0 or my fork with a few additional bug fixes added.
Non existent tags get passed as this.search which is not modified by toString().toLowerCase(), this.search is passed to the function called by @addTag
Are you able to produce a jsfiddle displaying the problem so i can see the combination of props you're using and how you're implementing them?

I haven't updated since my last comment. It is probably fixed by now :)

This has been fixed already! Thanks for the contribution anyway. Sorry for my very late reply. :(


⚠ Bofrost.at has degraded performance

In c6a9b88, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2425 ms

Resolved: Bofrost.at performance has improved in 4e428a8.


Teletype Studies 5

Please change the reference from ~ to Tab and NumLock to get to the Tracker

i made a pull request that fixes this bug with this commit:
https://github.com/monome/docs/commit/52ff40432bdad4676f9527a9e438f42f8045fda5
...which updates all references to tilde and the tracker throughout the docs. not just the studies, but also their linked example scenes.
(not sure why this pullreq got added into my earlier pullreq that was on the ansible docs; github shouldn't have lumped them together like that!)

fixed!


Accept doctrine-module 1.* as well

Allow usage of DoctrineModule 1.*.

Please send this PR to https://github.com/API-Skeletons/zf-doctrine-querybuilder
Once merged it will be pushed to this repo.

https://github.com/API-Skeletons/zf-doctrine-querybuilder/pull/28


fix: convert ReleaserConfig JSON keys when bootstrapping

We were directly writing the ReleaserConfig to the release-please-config.json, but the keys are actually dasherized.
Fixes #1522

It would be preferable to make releaserConfigToJsonConfig a bit more ⭐ magic ⭐ , but not a blocker for me.


Error when running #GtInspector asClass openOnPlayBook: ...

After loading from Metacello running #GtInspector asClass openOnPlayBook: (#GtDocumenter asClass editorForText: #BaselineOfGToolkit asClass comment). gives me an error.

Does this mean that loading the code works now? Or did you load it manually?
It looks like the issue appears from Metacello not being able to load the code (the document provides a visualization of dependencies).

I will close it because it should work now. If you still encounter this issue, please re-open the issue.


update 1.6 rancher microservices with basic tools

example lb container doesn't have net-tools and it makes it really hard to debug anything in the container when something goes wrong.

apt-get install net-tools
There is no step 2.

@vincent99 apt install is not possible (easily) in an air-gap setup.


add some hardening compiler flags

Adds some compiler flags (as described on https://wiki.debian.org/Hardening) that are supposed to improve security without hurting performance.

The first three flags are already enabled.
The last two, I am not sure whether they are a good idea. I am not certain they don't affect ABI, I would like to see some validation on the fact that they don't hurt performance, and they don't seem entirely portable.

Regarding performance, I couldn't measure a difference before/after (tried encode and decode with vardct and modular), but this might depend on the platform/compiler/image type/encode settings. I only did a quick benchmark_xl check though, maybe a more careful measurement could show a (small) difference.
I don't know about ABI and portability impact.

Looks like this causes trouble in the wasm build... drop the flags in the wasm build?

OK, I am closing this PR then. I added the above link and a link to this discussion to https://www.bestpractices.dev/en/projects/7845?criteria_level=2#security


oni2 --extensions-dir not working / improperly documented

The documentation states that

This can be overridden with the --extensions-dir command line flag however.

I tried both
oni2 --extensions-dir ~/.vscode/extensions

and
oni2 --extensions-dir=~/.vscode/extensions

None seem to work

Whoops! Looks like this wasn't being passed over in some part of the code...I'll stick a PR up.
Thanks for raising!

@CrossR thank you for patching this.
Unfortunately it still doesn't work / not documented properly.
How is it supposed to be used? Please consider updating the documentation too.
(⎈ |dev-1:default)➜  ~ oni2 --extensions-dir ~/.vscode/extensions
(⎈ |dev-1:default)➜  ~ oni2 --list-extensions
(nothing)


Fix duplicate "Extraction" of entities leading to duplicate link tags

If you click the "Extract'" button several times it will trigger the extraction again and again. It will add an additional copy of the highlighted entity to the adventure.

The current implementation, if a user re-extracts, existing NPC's will not duplicate due to this LOC:
https://github.com/laudanum123/fantasy_fabricator/blob/bdb35565a1a2709ea91a6a135a3614754b1129d4/backend/main/models.py#L91
However the request will still be sent to the backend, do you think the user should be limited to one extract per adventure?


scheduled notification in the queue is not saved

the notification in the queue is not saved when we close and open the app again

Are you using it for Swift 3 or Swift 2.2?
Also, did you call ABNScheduler.saveQueue()?

yes I did call save queue every time I schedule a new one. I just added the savequeque in your schedule function
both 3 and 2.2 is not working

What version of iOS did you test it on?

10.2
10.1 is also the same.
is it because i was testing using emultor?

I tried on real device, still the same. the queue is empty every time i close and open the app

I have tried it on both a real device and simulators and it does not load the queue from disk on iOS 9.3+.
ABNScheduler uses NSKeyedArchiver to store and retrieve the notification queue. I have observed something weird while trying to figure out what was going on. The Bool data type is either not properly saved or read from disk, which causes the ABNQueue to fail loading the notifications. This also happens with the Int data type. Not sure about all the other data types.
I just have to make sure that this is a bug from Apple's side not from anything in the current implementation and then push the fixes.
If you need a quick workaround, just don't decode the ABNScheduled key in init?(coder aDecoder: NSCoder) and set a default value to the soundName attribute in ABNotification.

well I'm not sure if it is bug from iOS itself. but yeah just tell me when you figure it out. I will help to figure it out in my free time.
cheers :) good to see that this repo is back to be active

The library now should save and load the queue correctly. Happy holidays :)

@ahmedabadie ok cool, I will test it soon.

Hi @ahmedabadie Thanks for this project.
I have some issues while running in devices, if i run in simulator its triggering and working properly.
If i am running in devices some iphone devices its working and some of them devices its getting crashed. due to its showing identifier nil.
Help me out, i was helpless.

@ahmedabadie
Hii its getting crashed and saying that identifier is nil while running project in device.

Would you please specify where this error happens and what are the stack contents?


"Set Log Level" works as not expected

Set Log Level behaves improperly:

${log_level_tmp}   Set Log Level  NONE does not log (in log.html) previous log level, however, I expect it to be logged in case previous log level was >= INFO;

A Robot test script to reproduce observations above:
*** Test Cases ***
Test_01
   ${log_level_tmp}   Set Log Level  NONE

Command used to run script: robot --loglevel TRACE <script>.

If I understand this correctly, logging is disabled before the old level is logged. In my opinion that's just fine. If you really want to see the old level, use this:
${old} =    Set Log Level    INFO
Set Log Level    NONE


[bug] Caching can cache too much if output directories are already populated

Describe the bug

If multiple tasks use the same output directory, moon can incorrectly task output.
Steps to reproduce
Given the following abc/moon.yml (with x being an empty file):
tasks:
  test1:
    command: "bash -c 'mkdir -p out && echo hi > out/a'"
    platform: 'system'
    inputs: [/x]
    outputs: [out]
  test2:
    command: "bash -c 'mkdir -p out && echo hi > out/b'"
    platform: 'system'
    inputs: [/x]
    outputs: [out]

You can observe the following terminal session:
$ ./moon run abc:test1
▪▪▪▪ abc:test1
▪▪▪▪ abc:test1 (2ms, 3fa2ab50)

Tasks: 1 completed
 Time: 9ms

/tmp/tmp.7IyhNnxqSb $ ./moon run abc:test2
▪▪▪▪ abc:test2
▪▪▪▪ abc:test2 (3ms, ba02b2a5)

Tasks: 1 completed
 Time: 12ms

/tmp/tmp.7IyhNnxqSb $ rm -r abc/out/
/tmp/tmp.7IyhNnxqSb $ ./moon run abc:test2
▪▪▪▪ abc:test2 (cached, ba02b2a5)

Tasks: 1 completed (1 cached)
 Time: 5ms ❯❯❯❯ to the moon

/tmp/tmp.7IyhNnxqSb $ ls abc/out
a  b

Expected behavior
moon should not have cached both a and b for the test2 task.
Screenshots

Environment

$ ./moon --version
moon 0.24.2

Additional context

This is a followup on #638 after a hint that I need to provide inputs to avoid hashing the 'out' directory.
I suspect the most pessimistic way for moon to operate by default is to treat output locations as empty, and clear them by default - i.e. applying the same logic that globs are documented to follow https://moonrepo.dev/docs/config/project#outputs

This is an interesting one. moon originally wiped output folders between runs but it was a pretty noticeable performance hit (10x slower if I remember).
I wonder how often different tasks write to the same output, and if this is something worth looking into. Do you have a real-world example?

Wow, that is a lot slower! I wonder if you could speed it up by (iff the task gets run) moving output folders into a .moon/garbage folder and then deleting them in a background thread while recreating the folder from scratch.
The thing that prompted me to poke at this was thinking about how Cargo works, i.e. dumping everything in target. I've used a lot of C/C++ build scripts that do similar things and dump their output in a single directory. It's not ideal to nuke everything in an output directory before recreating it, but far better to do that than for a task to unpredictably resurrect stale output!

Do you have a real-world example?

I have typescript projects that build types into dist but then in a separate task bundle the package with esbuild, also into a dist folder. I worked around this by having the types go into dist/src and then explicitly naming the outputs of esbuild. This worked well enough in my case but not having to think about it would have been neat.

Been a while, but updated the graph to error if multiple tasks all write to the same output location.


[Bug]: Using the latest version of kubectl (1.28.1) breaks "make serve"

Installation method
Own AWS account
What happened?
Building with the latest version of kubectl (1.28.1) causes the "make serve" command to fail
What did you expect to happen?
I expected "make serve" to work
How can we reproduce it?
Run "make serve" with the latest version of kubectl (1.28.1)
Anything else we need to know?

Dropping down to 1.23.0 of kubectl will fix this as a workaround.
Editing all of the following kustomizations to add the "path:" statement for every "patch" section will also fix this.

manifests/modules/automation/gitops/argocd/update-application/kustomization.yaml
manifests/modules/autoscaling/compute/karpenter/consolidation/kustomization.yaml
manifests/modules/autoscaling/workloads/hpa/kustomization.yaml
manifests/modules/exposing/load-balancer/ip-mode/kustomization.yaml
manifests/modules/fundamentals/affinity/checkout-redis/kustomization.yaml
manifests/modules/fundamentals/affinity/checkout/kustomization.yaml
manifests/modules/fundamentals/fargate/enabling/kustomization.yaml
manifests/modules/fundamentals/fargate/scaling/kustomization.yaml
manifests/modules/fundamentals/fargate/sizing/kustomization.yaml
manifests/modules/fundamentals/mng/taints/nodeselector-w-toleration/kustomization.yaml
manifests/modules/fundamentals/mng/taints/nodeselector-wo-toleration/kustomization.yaml
manifests/modules/fundamentals/storage/efs/deployment/kustomization.yaml
manifests/modules/introduction/kustomize/kustomization.yaml
manifests/modules/networking/custom-networking/sampleapp/kustomization.yaml
manifests/modules/networking/securitygroups-for-pods/rds/kustomization.yaml
manifests/modules/security/irsa/service-account/kustomization.yaml
manifests/modules/security/pss-psa/baseline-namespace/kustomization.yaml
manifests/modules/security/pss-psa/privileged-workload/kustomization.yaml
manifests/modules/security/pss-psa/restricted-namespace/kustomization.yaml
manifests/modules/security/pss-psa/restricted-workload/kustomization.yaml
manifests/modules/security/sealed-secrets/kustomization.yaml

EKS version
n/a

Duplicate of https://github.com/aws-samples/eks-workshop-v2/issues/624


Make working files public-read only.

Uploaded files will be processed by EMR workers and will use curl-like
commands to access the files. This will require the resources to be publicly
readable. When uploading files from the front end set the ACL header to
'public-readonly'.
To test

upload an image via the UI
Log in to the AWS panel and check the resource in your S3 bucket.
Look at properties and check that it it does not have a lock icon and the permissions mark it as "Everyone - Open/Download"

+1, tested

This fix does not account for files copied to the upload bucket from another AWS bucket.


Add WorldHop shortcut

On osbuddy it is possible to press Ctrl + Shift + Left/Right arrowkey to hop one world backwards or forwards. Would it be possible to implement something like this?

Support.
The OSB one also skips pvp worlds, f2p worlds, deadman worlds and tourney worlds.

https://github.com/runelite/runelite/wiki/Rejected-or-Rolled-Back-Features

Added in #4761


Model is not properly set for Azure OpenAI if using llm.Call and not setting Model in Client

In openaiclient.go in the llms/openai/internal/openaiclient directory, there is this code:
// CreateChat creates chat request.
func (c *Client) CreateChat(ctx context.Context, r *ChatRequest) (*ChatResponse, error) {
	if r.Model == "" {
		if c.Model == "" {
			r.Model = defaultChatModel
		} else {
			r.Model = c.Model
		}
	}
	if r.FunctionCallBehavior == "" && len(r.Functions) > 0 {
		r.FunctionCallBehavior = defaultFunctionCallBehavior
	}
	resp, err := c.createChat(ctx, r)

In the createChat function, c.buildURL is called, which then calls c.buildAzureURL if the APIType is Azure or AzureAD, however the model is not passed to c.buildURL, so it references c.Model, as shown below:
func (c *Client) buildAzureURL(suffix string) string {
	baseURL := c.baseURL
	baseURL = strings.TrimRight(baseURL, "/")

	// azure example url:
	// /openai/deployments/{model}/chat/completions?api-version={api_version}
	return fmt.Sprintf("%s/openai/deployments/%s%s?api-version=%s",
		baseURL, c.Model, suffix, c.apiVersion,
	)
}

This code leaves the model ID blank if it is only provided in the llm.Call function as an option, since c.Model is never changed, nor is the model ID passed to c.buildAzureURL.

This should have been fixed now. With the latest updated in the code base for the azure part.


Not working in ios

tested ios latest version and older not working

Confirmed.  iOS Simulator doesn't support WebRTC.  We need to find an iOS 11 device to add this feature.

We are collecting details on making this work.  Sprint complete in next two weeks.

Fixed!  Use the latest version of webrtc.js here: https://stephenlb.github.io/webrtc-sdk/js/webrtc.js

getting error
webrtc.js:571 Uncaught TypeError: this.mergeJSON is not a function
at merge (v3.js:571)
at window.PHONE (v3.js:13)
at (index):624

getting error
webrtc.js:571 Uncaught TypeError: this.mergeJSON is not a function
at merge (webrtc.js:571)
at window.PHONE (webrtc.js:13)
at (index):624

Thank you!  checking....

Fixed.

Fix is applied and live: https://github.com/stephenlb/webrtc-sdk/commit/4ed5867aa97f45040678ae41a6a07252c36aea6c

Now it us working in ios great job thanks

🎉


0.11.0rc1 linux 2.7 GPU nightlies ignore CUDA 7.5 installation, wants CUDA 8.0

A couple of days ago the 0.11.0rc1 nightly depended on CUDA 7.5 #269 for example. I've just downloaded today's nightly and it seems to want CUDA 8.0. Is CUDA 8.0 the new pip binary dependency or is this a mistake?
What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?
N/A
Environment info
Operating System: Ubuntu 16.04
Installed version of CUDA and cuDNN: CUDA 7.5 and cuDNN 4.0.7
(please attach the output of ls -l /path/to/cuda/lib/libcud*):
$ ls -l /usr/lib/x86_64-linux-gnu/libcud*
-rw-r--r-- 1 root root   322936 Sep 19  2015 /usr/lib/x86_64-linux-gnu/libcudadevrt.a
lrwxrwxrwx 1 root root       16 Mar 30  2016 /usr/lib/x86_64-linux-gnu/libcudart.so -> libcudart.so.7.5
lrwxrwxrwx 1 root root       19 Mar 30  2016 /usr/lib/x86_64-linux-gnu/libcudart.so.7.5 -> libcudart.so.7.5.18
-rw-r--r-- 1 root root   383336 Sep 19  2015 /usr/lib/x86_64-linux-gnu/libcudart.so.7.5.18
-rw-r--r-- 1 root root   720192 Sep 19  2015 /usr/lib/x86_64-linux-gnu/libcudart_static.a
lrwxrwxrwx 1 root root       12 Jun  5 01:26 /usr/lib/x86_64-linux-gnu/libcuda.so -> libcuda.so.1
lrwxrwxrwx 1 root root       17 Jun  5 01:26 /usr/lib/x86_64-linux-gnu/libcuda.so.1 -> libcuda.so.364.19
-rw-r--r-- 1 root root 16963368 Apr 19  2016 /usr/lib/x86_64-linux-gnu/libcuda.so.364.19
lrwxrwxrwx 1 root root       13 Aug 29 10:59 /usr/lib/x86_64-linux-gnu/libcudnn.so -> libcudnn.so.4
lrwxrwxrwx 1 root root       17 Aug 29 10:59 /usr/lib/x86_64-linux-gnu/libcudnn.so.4 -> libcudnn.so.4.0.7
-rwxr-xr-x 1 root root 61453024 Aug 29 10:59 /usr/lib/x86_64-linux-gnu/libcudnn.so.4.0.7
-rw-r--r-- 1 root root 62025862 Aug 29 10:59 /usr/lib/x86_64-linux-gnu/libcudnn_static.a

If installed from binary pip package, provide:

A link to the pip package you installed:
link

The output from python -c "import tensorflow; print(tensorflow.__version__)".

$ python -c 'import tensorflow'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/sperkins/venv/tftest/local/lib/python2.7/site-packages/tensorflow/__init__.py", line 23, in <module>
    from tensorflow.python import *
  File "/home/sperkins/venv/tftest/local/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 60, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "/home/sperkins/venv/tftest/local/lib/python2.7/site-packages/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/home/sperkins/venv/tftest/local/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 28, in <module>
    _pywrap_tensorflow = swig_import_helper()
  File "/home/sperkins/venv/tftest/local/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow', fp, pathname, description)
ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory

Error importing tensorflow.  Unless you are using bazel,
you should not try to import tensorflow from its source directory;
please exit the tensorflow source tree, and relaunch your python interpreter
from there.

If installed from source, provide

The commit hash (git rev-parse HEAD)
The output of bazel version

If possible, provide a minimal reproducible example (We usually don't have time to read hundreds of lines of your code)
What other attempted solutions have you tried?
Logs or other output that would be helpful
(If logs are large, please upload as attachment or provide link).

Regarding the libcuda.so loading issue, this is fixed by adding /usr/lib/nvidia-367, which contains  libnvidia-fatbinaryloader.so.367.57.
It would still be good to hear if 0.11.0 depends on CUDA 8.0 though.

Yes,
/usr/local/lib/python2.7/dist-packages/tensorflow/python/_pywrap_tensorflow.so

shows the dependency. I don't know that we have a build for 7.5, so at this point I'd recommend building from source.
@gunan for potential additional insights.

Thanks, might be worth changing the issue to track a documentation change - os_setup.md hints at CUDA 7.5 as the standard.

Looks like we updated the branch for rc1, but when merging it back into master we missed that diff, and stayed with cuda7.5
@yifeif FYI


Syntax error in action.yml for specifying image

Hi, thanks a lot for the template. I ran into an error when trying to use my action with
- uses: <user>/<repo>:<tag>
# Instead of
- uses: ./

Error: 'ghcr.io/sh1nku/count-loc-action:v1' should be either '[path]/Dockerfile' or 'docker://image[:tag]'.

https://github.com/dbanty/rust-github-action-template/blob/069fdc1b20e6f22e837c7d5273d889d1c0b6c577/action.yml#L14
This should be prefixed with docker:// to work externally

You're right! I'm really annoyed that the tests consuming an action from itself didn't have the same restriction... and therefore don't catch the issue 😬. Thanks for letting me know!


⚠ Bofrost.fr has degraded performance

In 11fa8c1, Bofrost.fr (https://www.bofrost.fr) experienced degraded performance:

HTTP code: 200
Response time: 1615 ms

Resolved: Bofrost.fr performance has improved in e96eeb4 after 6 minutes.


Assistent_IDs

Hey there, thanky our for your work <3 is it possible to connect to assistent-IDs that i created on playground? Tx

Hello and thank you for your kind words! 🌟 Yes, it is indeed possible to connect to assistant-IDs that you've created on the playground. With the provided API, you can manage assistants including creating, retrieving, modifying, and deleting them. Specifically, you can use the retrieve(by assistantId: String) async throws -> ASAAssistant function to connect to an existing assistant by its ID. This allows you to interact with and manage the assistants you've created according to your needs. If you have any more questions or need further assistance, feel free to ask. Happy coding! 🚀✨


VQ-VAE & VQ-VAE2 - Vector Quantization Based AutoEncoders

https://fenghz.github.io/vector-quantization-based-generative-model/

Here you can leave a comment


error on running "make learn-vectors-redis"

Hi @jjhenkel,
A Segmentation fault when I run the make learn-vectors-redis . could  you give me some advice?

Thanks!

Hi @llnlln ,
Try getting the latest version of the arbitrary-repos branch and running that command again. I just pushed a fix.
Thanks!

Hi @jjhenkel
I ran the following command and got the same error. The difference is that the array size is<PHONE_NUMBER>4 and 510027366, respectively.
git status
git pull
make learn-vectors-redis

The output is in the output.txt file
output.txt
Thanks!

Hi @llnlln ,
I'm trying to reproduce this on my machine---the patch I pushed did fix things for me but, just to be sure, I'll run it on a fresh clone of the repository and see if the issue re-appears.
In the meantime what system are you running this on (OS/number of cores/amount of RAM)? Could you try completely removing the repository and then re-cloning, checking out the arbitrary-repos branch, and running make end-to-end-redis ; make learn-vectors-redis.
At this point the tool has generated the Abstracted Symbolic Traces---this means that, as a workaround, you could always just run your favorite word vector learner on the generated traces (the make learn-vectors-redis command is just going to run GloVe on the generated traces using default settings).
Thanks!

Follow up on this: I am no longer able to reproduce this. I just cloned a fresh copy of the repo, checked out the arbitrary-repos branch, ran make end-to-end-redis followed by make learn-vectors-redis and got reasonable output at the end:
2019-03-01 03:57:56,651 - glove2word2vec - INFO - Converted model with 6150 vectors and 300 dimensions
[GloVe] Complete!
[GloVe] Vectors saved in '/output/vectors'
[code-vectors] Learner finished.

Hi @jjhenkel ,
I just cloned a fresh copy of the repo, checked out the arbitrary-repos branch, ran make end-to-end-redis followed by make learn-vectors-redis and got the same error.
I ran this on Linux version 4.15.0-42-generic(Ubuntu 5.4.0-6ubuntu1~16.04.10)/Intel® Core™ i5-4570 CPU @ 3.20GHz × 4 /memory 3.5 GiB.
Could you give me some advice?
Thanks!

Hi @llnlln ,
Sorry for the delay---I've been trying to find a machine I can reproduce this on.
For now my advice to you is to learn vectors using your method of choice without using the version of GloVe I provide as a Docker image. You should have the generated traces on disk (since the previous steps in the pipeline worked) so you can simply run any word-vector learner using those traces as your corpus.
If you need an example of how to run one of these learners you can see the commands I use for learning vectors via GloVe in this file. Other tools like word2vec and fasttext also provide good documentation and examples of their usage.

Hi @jjhenkel ,
Thank you very much for your reply !


No Qt platform error

Description
It's possible this is a build error, but I get the following when I right click mozc in the windows system tray and hit properties:
Steps to reproduce
These are the commands I used to build:
cd C:\work\mozc\src
python build_mozc.py gyp --qtdir=C:\Qt\5.15.2\msvc2019 --msvs_version=2019
python build_mozc.py build -c Release package
I then followed the installation tutorial for 64bit (also happens on 32bit) and everything seemingly works until I try to open the preferences page.
Expected behavior
I expect the proper menu to appear.
Actual behavior
The screenshot attached ("This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.")
Screenshots

Version or commit-id
Unsure, installed & downloaded today
Environment

Windows 10 21h1

Investigations

Whether this issue happens on both Ibus and other IMF (e.g. Fcitx, uim).

Yes? - I installed both, unsure if that could cause problems

Whether this issue happens on other IMEs (e.g. Anthy, SKK).

Don't use others

Additional context
ありがとうぐざいます for any help <3

Hi viasux,
That error message comes from the Qt library.
I guess some configurations or rebuild of Qt may solve the problem.
By searching web pages with that error messages, removing or updating the following environment variables may change the situation.

QT_PLUGIN_PATH
QML2_IMPORT_PATH

However, I'm not so sure about the Qt configurations. I hope someone who are familiar with this issue would help this issue.
Thanks,

The good news is that c7265ec5010b6b552d2be4fa7850d65d5208fc45 made it possible to build installer (Mozc32.msi and Mozc64.msi) with Qt dependencies. So you should no longer have to manually copy files by yourself. See our latest build/installation instructions for details.
I'm going to close this issue as I've locally confirmed that the preference dialog can be launched without any error (in my Windows 11 environment) by installing Mozc with Mozc64.msi. Let us know if you still see some issues after installing Mozc with Mozc64.msi.
Thanks!

I close this issue.
please let us know If you still have the issue with the above possible solution.
Thanks,

I close this issue.
please let us know If you still have the issue with the above possible solution. Thanks,

I've since switched to another solution. Thank you for all the help though!


POST data field length not validated in backend

Expected behaviour
All fields submitted in a POST request are length limited by the backend
Actual behaviour
Some fields (e.g. comments) aren't length limited in the backend.
Steps to reproduce the behaviour
Send POST data for a comment field that exceed 2000 characters
Technical information
Browser
All
Operating System
All
Screenshot
N/A

It seems the frontend limit has vanished as well on master:

@hamishtaplin  ?

I suggest we check the comments size backend in the type validation, but we also set a MAX_CONTENT_LENGTH

The second issue is raised here #370

I think that both this issue and #370 have been merged and fixed. Can you confirm @collisdigital ?

Confirmed this is fixed by #366 on pre-prod. Closing.


Rework rounding shader

Describe your PR, what does it fix/add?
This optimizes the rounding part of the Textures.hpp shaders.
Also the Shaders.h file is now only included by OpenGL.cpp,
that way only OpenGL.cpp has to be recompiled if a shader was changed.
Is there anything you want to mention? (unchecked code, possible bugs, found problems, breaking compatibility, etc.)
It shouldn't change the behavior at all.
Is it ready for merging, or does it need work?
I tried it, and it is working for me, but should be tested in other configurations/ on other computers.

have you actually done any benchmarks to back your performance claim?

I didn't, but just counting the number of branches that it has to take for each if the 5 possible paths it can take
(the 4 corners, the rest), and the knowledge that branching will be the bottleneck, this is more efficient.

branching is a bottleneck? Didn't know that. I am not very proficient with GPUs, but a branch to me is just
cmp <reg> <reg2>
jne <addr>
...
jmp <addr2>

If you are sure that this is more optimized, (I'll also tag @fufexan to test) please also test that it works under GLES2 (build with LEGACY_RENDERER)

Did you ever hear about prefetching?
A branch is a totally performance killer if it can't be predicted the right way.
That means all the prefetched data is worthless and has to be newly created.

It's the same on cpu (even more important because it's less parallel)

me using a ton of ifs:
ok

You know, there are just a few places in a program where it really makes a difference.
I there is a function with lot's of branches that will execute only a few times per frame, it's not worth the effort

By the way the Multisampling thing inside that shader, it was shifting the different positions for each corner in the same direction, was that intended? because the way it's in this pr, it shifts them towards the rounding, of every corner
i think that is the right direction, but you wrote that code so i'm not shure.

By the way the Multisampling thing inside that shader, it was shifting the different positions for each corner in the same direction, was that intended? because the way it's in this pr, it shifts them towards the rounding, of every corner
i think that is the right direction, but you wrote that code so i'm not shure.

Yeah thats better.

Don't merge! I didn't test the multisampling.
I will notify you when i'm shure!

Multisamplig was the other way around.
Now it works!
Ready to merge if you want to.

Looks fine but there are some artifacts in the bottom left corners of the windows.

@fufexan I don't have that! can you give me your config?

https://github.com/fufexan/dotfiles/blob/main/home/wayland/hyprland/config.nix

Ok @fufexan  check if it's gone if you disable multisampling

And is it in all windows or only that terminal?

And did you build after  force-pushed the rework_rounding_shader branch from 78637a3 to 168a326

And did you build after fo

Yes.

And is it in all windows or only that terminal?

All windows afaict.

Ok @fufexan check if it's gone if you disable multisampling

Still there.

Me neither!
@fufexan could you please play with your settings (scale, decorations...) and see if you can
find anything that changes the artifacts size, position, color or anything?
Are you totally shure it isn't there without this change?

@fufexan and try to disable all decorations, is it still there without them?

Scaling affects it. On scaling 1 it does not show, but on more it does, even on 2.
It only happens for rounding > 6 on scale 1.6, rounding > 14 on scale 2. For scale 1.8 it happens on rounding > 3. Really weird and inconsistent.

Are you totally shure it isn't there without this change?

Yes. I would've noticed it if it were the case.

I will push another Thing today or tomorrow, that fixes lots of animation stuff (e.g. the corner rounding thing on chromium [or was it firefox?]).
It's possible that this is caused by the same problem.

I know i will change a few things in the shaders. it already works, but i have to work on the decoration too to don't have 1 off artifacts.

maybe make this a draft then for now?

The UV coordinates are not really working there, i tried that days ago.
The problem with that approach is, that during resizing animations thos UV coordinates don't change dynamicly enough.
It depends on the program that is running in that window.
I solve it in another way, once and for all!

what's a draft?

this

The problem with that approach is, that during resizing animations thos UV coordinates don't change dynamicly enough.

Noticed that, it's a tiny bit off. No clue why but oh well.

There is a proper solution for that, i have it working with the textures, but during animations white borders will look a bit red. I think i know how to fix that!

good luck and thanks

@vaxerski what you could try to fix in the meantime:
The clients window has standard uv coordinates as long as the size of the window matches the size of the texture of that client. Now if a client has a small window, and you make it bigger (e.g. by closing another window).
Hyprland will notice that client of the size change, than the client will change it's size, but not immediately,
Hyprland should change the calculated uv's at the point when the internal client texture size changes, not
when the window resizing action is executed.

That would make a more fluid animation i think!

probably. But it's not a pressing issue at the current moment, as the error is like a couple pixels at most. I'll take a look if I'll have time and nothing better to do

It's not a couple of pixels, its just that jump of scale.
Like it is now, before that changed the window looks streched and than it jumps to the right new size.
It's not pressing though, you're right there.

Damn that red looking border is alway there.

Will update in a second!

People noticed it with chromium because there it's visible without animation.
I myself noticed first that the corners of opening windows (while growing) wheren't all rounded.
But it was even more, i made it really visible here
before:
https://user-images.githubusercontent.com/10100765/193168154-96eb07bd-ea4e-4dee-b8cd-f7df3680e94c.mp4
after:
https://user-images.githubusercontent.com/10100765/193168272-3caef869-e310-48ad-8048-829494a8e1f0.mp4

By the way @vaxerski, you should look up how floating point numbers work, and why you are really asking for
trouble when you compare them for equality!

I know how they work, thats why in a bunch of places I use an epsilon of 1.

Might be, but an operator== and operator!= for Vector2d that work the way they do in Hyprland are just a bit scary to look at.

Is there a way to send pm's on github?

I don't know another place to message you, so i write it here.
There are issues concerning the performance of the program, i didn't read the whole program of course, but
seeing the render part, i know that we could have it orders of magnitude faster (i bet at least 10x faster, if not more).
But we would need to restructure it quite a bit.
How do you think about that? Are you willing to do that?

Is there a way to send pm's on github?

no afaik, join the discord

There are issues concerning the performance of the program, i didn't read the whole program of course, but
seeing the render part, i know that we could have it orders of magnitude faster (i bet at least 10x faster, if not more).
But we would need to restructure it quite a bit.
How do you think about that? Are you willing to do that?

I am if that's the difference we're talking about

It is shurely!
I would take the Renderer.cpp and Opengl.cpp + the shaders if you want me too, but you have to restructure
the data that is send to those.

What's the issue there?

We need all the data that is needed for rendering of the windows more or less together and ordered in a certain way.
And for the love of god, we need to get rid of virtual draw functions and those things.
We need to do all calculations for preparing the windows in loops, don't call those functions for every window seperately.
The windows that havent moved doesn't need to be recalculated at all. maybe it's possible to don't even redraw that window, the windows have to be drawn in an order that makes the rendering more efficient:
all the windows that are tiled and not animated at the moment first, than the ones floating, in the order they are drawn now.
than the special workspaces of course.
The border, shadow, texture, blur shaders, use same calculations all the time, so we should make it just one shader.
We could even draw all non overlapping windows in the same shader call.
to calculate shadows and borders inside the fragment shader for every window is not very efficient, since it's the same
values all the time for the same distances, we could render a texture once for those things and then just,
set the vertexes appropriately (don't scale the corners, but scale the edges, it would be enough to have a texture of one rounded corner of the actual shadow to render it (by stretching the edges and rotation the corner, shaders are fucking fast at those things).
The slow thing about a gpu render  system is the communication between CPU and GPU, so we could/should reduce that to some places. We can have texture and vertex arrays for all the windows that can be drawn at the same time, just one call to one shader program. Inside that shader we draw all the window and the decorations at once.
The only reason why we can't do the whole rendering in one, is blur, so we have to think about how to do this the best way.
Of course we would have to try, and find the best way to do it.

@fufexan : are the artifacts still there?

And for the love of god, we need to get rid of virtual draw functions and those things.

You mean the decorations' draw()?

The windows that havent moved doesn't need to be recalculated at all.

True, but shouldRenderWindow() should already get rid of a bunch of those no?

We could even draw all non overlapping windows in the same shader call.

wot
Generally you bring valid points, but nobody (literally nobody) does this afaik, because it leads to more code clutter and untimately bugs over not that much of a performance boost. Can't imagine how this would boost the performance 10x. Simply can't, considering the difference between sway, xorg or wf and Hyprland, on similiar settings, is small.

I could start by writing a big shader that combines all of the window shaders.
The shader for the "wp"/splash should probably be seperated.

try and benchmark it.

You mean the decorations' draw()?

Yes, every window should have all those decoration values together, and if you draw a window (should probably be one renderWindow function)

True, but shouldRenderWindow() should already get rid of a bunch of those no?

Just mark them as has to be recalculated, no need to check every frame for every window.
Or at least have a function that checks all the windows in a loop.

wot
Generally you bring valid points, but nobody (literally nobody) does this afaik, because it leads to more code clutter and untimately bugs over not that much of a performance boost. Can't imagine how this would boost the performance 10x. Simply can't, considering the difference between sway, xorg or wf and Hyprland, on similiar settings, is small.

Those other window managers are not dealing with overlapping or moved windows.
and if you read the issue "idle cpu usage is over 9000" you see that Hyprland is much resource intensive than e.g. sway.
They just have to check if a window has changed, thats all, we are dealing with animations/ dynamic overlapping and so on
(because Hyprland is doing nicer things!).
But we should really try to make it an window manager that doesn't eat up 10 percent of a laptops performance.
In the end a window manager is just a program that let's open other graphical programs.

Those other window managers are not dealing with overlapping or moved windows.

Every single one I mentioned does deal with it, especially xorg or wayfire, wayfire has even more effects than hl.

and if you read the issue "idle cpu usage is over 9000" you see that Hyprland is much resource intensive than e.g. sway.

It's constant. it does not scale up. Hyprland uses around 0.5-2W more on my laptop than sway regardless of what is open. It's prolly some polling or stuff

try and benchmark it.

Not shure if drawing lot's of windows in the same shader call is what we end up with, but if it's possible, it should be checked.
In general i'm talking of a approach of data oriented design, opposed to object oriented.

If you want to have something to look up, google mike acton or casey muratori (handmade hero), jonathan blow.
I really like this talk: https://www.youtube.com/watch?v=rX0ItVEVjHc .

Provide a benchmark that shows considerable gains and I'll look into this.

How should i do that without writing the whole thing?

I think you'd just have to write a part of it :P

I will start writing a general window shader (for all the things one window needs) that should make a difference right away.
But for that we need to combine at least shadow, border, blur and texture rendering functions.

Let's wait for @fufexan to test this draft though!

I really like this talk: https://www.youtube.com/watch?v=rX0ItVEVjHc .

I would really recomment to watch this!

@vaxerski If @fufexan 's issue is solved, feel free to merge!

Don't merge this yet! this seems to be indeed not faster, i will check exactly what's the problem is,
and pick and choose what should stay!

I'm closing this and will open a new pr when i checked!

I think i saw some ghost's yesterday night.
Today i tried to fix the thing that causes the chromium rounding bug without this shader change, and i felt
like i would revert something that was already there before and was taken out!
Or as if someone already prepared to do what i was doing.
Did you write the shaders yourself?
The solution with this change however is much cleaner!
I would like this to be tested, and @fufexan could maybe try if the artifact is still there.
If somebody has a good way of benchmarking shaders that would be helpful,
the benchmark tests i came up with are very limited.
After some more limited quality testing, i could notice no speed difference to the old shader, but with the new way
the corner problem is fixed.
Make with it what you think makes sense.

Did you write the shaders yourself?

yes

If somebody has a good way of benchmarking shaders that would be helpful,
the benchmark tests i came up with are very limited.

What did you try?  Prolly good idea to google. I'll see.

@Dickby the artifacts are indeed fixed. Thanks!

@vaxerski : Sorry for the late response,
what i tried was to try to have that shaders running a lot, by opening lot's windows and having as much blur passes that the framerate is not fully 60 and not totally down to 30. than i compared those framerate drops for the 2 shaders.
But because i didn't find a way to disable vsync, this might not be very good information.
It looked like the rounding part of the shaders are not changing that much at all when i tested it against the same setup with no rounding at all.
Since my version of the shader has less branching, but a bit more calculations, i would guess that it performs a bit better if
gpu cores are limited, but might be performing a bit worse on a proper graphics card with lots of cores.
Because a branch on a graphics card with SIMT (single instruction multiple threads) basicly goes both ways by using 2 cores
and deciding later which result to use.
But those cards are quite advanced and i might be wrong here.
If i'm right though, i think it would make sense to prefer a performance gain on slower systems, because for a faster
setup running a few small shaders like this shouldn't be a concern at all.
And after @fufexan 's reported artivacts are gone (thanks for testing), i would consider to merge this.
But as always it's up to you!

@vaxerski What about this change?
Is there something you want me to clarify?
Should i add some comments, or are you not happy with some of the changes?
I use it since days and it fixes the animation/chromium problem, it makes the multisampling more consistent on each corner,
the shader has to test only half the number of fragments against the radius (so computing the distance is used only half as much), and i don't see any downsides.

sorry, I forgor :skull: about it cuz it was a draft and thought you were still working on it

@vaxerski thanks!
I think this should work flawlessly, but we will know best by just let it be tested by the users.

Let's see

Might be the multi sampling that was basicly different for all corners before.


Recording env vars can leak on new test failure

While recording a new test that failed, the env vars set in envVarYmlRecording were used in subsequent tests rather than the env vars in envVarYml

Not sure I understand. Can you clarify? I'd say it's expected behaviour?

The env vars in envVarYmlRecording were used in tests that already have recordings after the test failure, the suite was as follows, tests with recordings are marked as R, the test without the recording marked as NR

Existing recording -> used envVarYml :heavy_check_mark:
No existing recording -> used envVarYmlRecording :x:
Existing recording -> used envVarYmlRecording :x: (nock mismatch because wrong env vars used)

That makes sense. The test fails and the cleanup isn't triggered as expected. Not sure how to best fix that...

I would think this is fixed with the new test failure handling. Could you confirm please?


Removed int from Distance/dot/length

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

Snapshot stored with reference name:
refs/pull/14899/merge
Test environment:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14899/merge/index.html
To test a playground add it to the URL, for example:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14899/merge/index.html#WGZLGJ#4600
Links to test babylon tools with this snapshot:
https://playground.babylonjs.com/?snapshot=refs/pull/14899/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/14899/merge
https://gui.babylonjs.com/?snapshot=refs/pull/14899/merge
https://nme.babylonjs.com/?snapshot=refs/pull/14899/merge
To test the snapshot in the playground with a playground ID add it after the snapshot query string:
https://playground.babylonjs.com/?snapshot=refs/pull/14899/merge#BCU1XR#0

Visualization tests for WebGPU (Experimental)
Important - these might fail sporadically. This is an optional test.
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14899/merge/testResults/webgpuplaywright/index.html

WebGL2 visualization test reporter:
https://babylonsnapshots.z22.web.core.windows.net/refs/pull/14899/merge/testResults/webgl2playwright/index.html