Add vhd:// HostPath prefix moniker

This allows a caller to map a vhd/vhdx directly into the container.
Signed-off-by: Justin Terry (VM<EMAIL_ADDRESS>

@kevpar - Based on what we talked about making this the official way to map a vhd. This is breaking but I do think its "better" long term.


Low local performance with single safekeeper compared to single synchronous replica

I ran pgbench tests from https://github.com/zenithdb/zenith/pull/1121 on latest zenithdb/postgres, results:
walproposer + safekeeper, single linux vm:

test_walproposer_pgbench.pgbench_init: 106.750 s
test_walproposer_pgbench.tps_pgbench: 725.7677
test_walproposer_pgbench.walmessage_size_begin: 125781 b
test_walproposer_pgbench.walmessage_size_end: 11767 b
average CPU usage: ~15% on every core

postgres master + synchronous replica, single linux vm:

test_vanilla_pgbench.pgbench_init: 139.660 s
test_vanilla_pgbench.tps_pgbench: 5,699.1125
test_vanilla_pgbench.walmessage_size_begin: 130677 b
test_vanilla_pgbench.walmessage_size_end: 563 b
average CPU usage: ~75% on every core

When running walproposer, backends are spending a lot of time in "COMMIT waiting for LSN", that should be the reason of low CPU usage and low performance.

I wonder if metrics are reported correctly: walmessage_size_end: 11767 b  for zenith and walmessage_size_end: 563 b for vanilla?

Metrics are correct, but it seems that I was running debug build locally and TPS are not fully relevant.
Here are results of the same test, but in release:
test_walproposer_pgbench.pgbench_init: 106.130 s
test_walproposer_pgbench.tps_pgbench: 1,067.7128
test_walproposer_pgbench.walmessage_size_begin: 125714 b
test_walproposer_pgbench.walmessage_size_end: 13986 b
test_vanilla_pgbench.pgbench_init: 146.680 s
test_vanilla_pgbench.tps_pgbench: 5,178.6690
test_vanilla_pgbench.walmessage_size_begin: 130573 b
test_vanilla_pgbench.walmessage_size_end: 497 b

Also, here is benchmark job in CI: https://app.circleci.com/pipelines/github/zenithdb/zenith/4081/workflows/a7638859-c007-4f3f-90e0-6b8adacc2535/jobs/36841
CI results:
test_walproposer_pgbench.pgbench_init: 36.130 s
test_walproposer_pgbench.tps_pgbench: 6,498.5651
test_walproposer_pgbench.walmessage_size_begin: 126071 b
test_walproposer_pgbench.walmessage_size_end: 15606 b
test_vanilla_pgbench.pgbench_init: 40.940 s
test_vanilla_pgbench.tps_pgbench: 3,338.4917
test_vanilla_pgbench.walmessage_size_begin: 130044 b
test_vanilla_pgbench.walmessage_size_end: 640 b

It's very strange that results on my linux cloud vm and in CI are completely different.
Also walmessage_size_end seems to match between all runs, and it's much larger for walproposer.

I enabled syncrep.c logs to see stats on releases of waiting backends.
When we receive ack from walreceiver/safekeeper, we have this line in logs. Number of procs up to flush is the exact number of released backends. I compared these stats for safekeeper and walreceiver.
released 0 procs up to write 0/A2334AA8, 1 procs up to flush 0/A2334AA8, 0 procs up to apply FFFFFFFF/FFFFFFFF

walproposer.c + safekeeper:
sort -n flush.log | uniq -c
  21376 0
  32541 1
   4178 2
   1959 3
   1109 4
    536 5
    270 6
     82 7
     28 8
      2 9
      1 14
      1 15
      1 18
      2 21

walsender.c + walreceiver.c:
sort -n flush.log | uniq -c
  53585 0
    670 1
    691 2
    650 3
    760 4
    914 5
   1061 6
   1258 7
   1544 8
   1643 9
   1762 10
   1801 11
   1796 12
   1548 13
   1428 14
   1217 15
    953 16
    741 17
    561 18
    410 19
    307 20
    163 21
    135 22
     89 23
     50 24
     23 25
     16 26
     13 27
      2 28
      4 29
      6 32

So, walreceiver generally releases more backends => they process more transactions => performance is better.
I'm not exactly sure why this is happening, but it could be differences in between safekeeper and walreceiver. Walreceiver has optimization where it reads and processes CopyData messages in non-blocking mode, and sends ack back / performs flush to disk only when the socket is empty (https://github.com/zenithdb/postgres/blob/main/src/backend/replication/walreceiver.c#L447-L491). We don't do this in safekeeper yet.
Also, this issue is still non-reproducible in CI, but probably exists on other linux vms.

Tried to run tests again, but now with disabled fsync. It looks like fsync and optimizations around it are causing this 4x difference. Without fsync the results are similar everywhere:

Test info
pgbench -i -s 200 (time)
pgbench TPS
wal_msg size
pg_wal size

local walproposer (no checkpoints, fsync=off)
90 s
6280 TPS
816 b
2720-2592 MB

local vanilla repl (no checkpoints, fsync=off)
130 s
6041 TPS
995 b
2736-2608 MB

CI walproposer (no checkpoints, fsync=off)
27 s
7969 TPS
442 b
2752-2592 MB

CI vanilla repl (no checkpoints, fsync=off)
31 s
7790 TPS
584 b
2768-2608 MB

I also did some experiments and come to the same conclusion: something is wrong with sync in safekeepers:
pgbench -f update.sql -c 100 -T 100

Test info
pgbench TPS

vanialla fsync=on
2818

vanialla fsync=off
2750

zenith
830

zenith nosync=true
2307

zenith on tmpfs
2345

zenith w/o safekeeper
2600

It is not mistyping: vanilla shows better result with fsync=on!!!
And it is similar with zenith with disabled fsyncs in safekeeper which seems to be the main bottleneck.

Vanilla use one fsync for 10 transactions:
number of transactions actually processed: 274087
tps = 2753.263153 (without initial connection time)
knizhnik@xps:~/zenith.main$ fgrep fsync ~/db/logfile.master | wc -l
28244
knizhnik@xps:~/zenith.main$ fgrep XLogFlush ~/db/logfile.master | wc -l
276407

I wonder why zenith walproposer is sending in average ~1 transaction.

With patch my test shows 2188 TPS and average write size 2241.
So twice larger size - twice higher performance.
And it is quite close to nosync results.
But still we are performing sync for less than three transactions while vanilla - 10.

On my laptop, with patch combining fsyncs (sk-try-improve-perf branch) pgbench -N tps is roughly the same:
test_walproposer_pgbench.wal_init_size: 2,592 MB
test_walproposer_pgbench.tps_pgbench: 4,713.8266
test_walproposer_pgbench.walmessage_size_begin: 125566 b
test_walproposer_pgbench.walmessage_size_end: 623 b
test_walproposer_pgbench.wal_size: 2,688 MB
test_vanilla_pgbench.pgbench_init: 68.880 s
test_vanilla_pgbench.wal_init_size: 2,608 MB
test_vanilla_pgbench.tps_pgbench: 4,574.2987
test_vanilla_pgbench.walmessage_size_begin: 125650 b
test_vanilla_pgbench.walmessage_size_end: 996 b
test_vanilla_pgbench.wal_size: 2,720 MB

walmessage size is smaller in case of walproposer, but data is glued before fsync, so we need new metric for that.

Vanialla with synchronous replica show 2400 TPS.
Ratio of  fsyncs per commit performed by replica (walreceiver) is even larger: 13
So for some reasons postgres wal receiver is grouping transaction much efficiently than our safekeeper even with patch.

We need to add metrics which shows number of fsync performed by safekeeper, because currently it doesn't take in account optimization performed by #1266
I have inserted trace and find out that with this patch number of performed fsyncs per commit is 6. Still 2 times slower than of sync replica.

Added metric flush_wal_seconds for safekeeper. Either I have made some mistake in counting, either today for some reasons zenith working much better, but according to this metrics grouping of commits by #1266 patch
works pretty well: one fsync for 19 commits in average. And achieved TPS is 3117! Faster than vanilla!
But looks like it is just "fluke" - in another run there are 16 commits per fsync and 2600 TPS.

I wrote in https://github.com/zenithdb/zenith/pull/1318#issuecomment-1050836557 that performance of local test is now comparable with vanilla replication, the main issue was using fsync instead of fdatasync, that made a difference of my machine. This issue looks resolved, closing it.


feat: add conversation context using chat history

ref chat-langchain

feature added, need to check memory


[FEATURE] Add GH workflow which checks for license headers

Is your feature request related to a problem?
Developers sometime forget to add a license header to new files. This is not detected automatically.
What solution would you like?
An automated GH workflow which will check this
What alternatives have you considered?
N/A
Do you have any additional context?
Copy implementation from another repos.

Required by https://github.com/opensearch-project/opensearch-plugins/issues/138

Note: Some OpenSearch java files have different licence formats.
Files impacted

core/src/main/java/org/opensearch/sql/expression/aggregation/StdDevAggregator.java
core/src/main/java/org/opensearch/sql/expression/aggregation/VarianceAggregator.java
core/src/test/java/org/opensearch/sql/expression/aggregation/StdDevAggregatorTest.java
opensearch/src/main/java/org/opensearch/sql/opensearch/response/agg/CompositeAggregationParser.java
opensearch/src/main/java/org/opensearch/sql/opensearch/response/agg/FilterParser.java
opensearch/src/main/java/org/opensearch/sql/opensearch/response/agg/MetricParser.java
opensearch/src/main/java/org/opensearch/sql/opensearch/response/agg/MetricParserHelper.java
opensearch/src/main/java/org/opensearch/sql/opensearch/response/agg/NoBucketAggregationParser.java
opensearch/src/main/java/org/opensearch/sql/opensearch/response/agg/OpenSearchAggregationResponseParser.java
opensearch/src/main/java/org/opensearch/sql/opensearch/response/agg/SingleValueParser.java
opensearch/src/main/java/org/opensearch/sql/opensearch/response/agg/StatsParser.java
opensearch/src/main/java/org/opensearch/sql/opensearch/response/agg/Utils.java

Each should be investigated and fixed if possible.


Hostname resolution via reverse DNS lookups broken in OKD 4.7/4.8

Describe the bug
I installed a 4.8 RC2 cluster (vsphere UPI with static ips via ignition). The nodes came up for the first time with their correct hostname from DHCP4 but after the reboot their hostname was 'localhost'. I then reinstalled the cluster and provided an ignition snippet that sets /etc/hostname and the cluster is now running more or less fine. This was different before, i.e. I didn't have to set /etc/hostname with releases up to and including 4.7.
Version
4.8.0-0.okd-2021-05-22-053824 on vsphere UPI
How reproducible
Always without the hostname ignition snippet.
Log bundle
If you really need a log bundle I can reinstall the cluster and create one on thursday. I however doubt it is really necessary as the issue appears to happen before the installer does much of its job.

I then reinstalled the cluster and provided an ignition snippet that sets /etc/hostname

I don't think its a good idea. /etc/hostname may be overwritten by NetworkManager.
If its configured as DHCP it expects to have received the hostname via DHCP too.
How was initial hostname passed to the cluster - via DHCP/kernel args/custom ignition?

FCOS docs state this as a method to set the hostname (https://docs.fedoraproject.org/en-US/fedora-coreos/hostname/), so I thought it might be ok to try. Also /usr/local/sbin/set-valid-hostname.sh stays that /etc/hostname is authoritative.  In my case the hostname is in DHCP and DNS and since I am in control of both of them they can be considered pretty static.
But don't get me wrong, I don't insist on manually setting the hostname if there are other means to get a valid hostname. It was just a check if the installation would run through if the issue (every node is 'localhost' after the first reboot') was put aside. It did.
I now installed the cluster without static ips and got a valid hostname at first boot and later. It was taken from DHCP4 in both cases and the 'transient hostname' was set. /usr/local/sbin/set-valid-hostname.sh was happy and the cluster nodes are up.

I experimented a bit with dhcp and static ips. My goal is to have nodes with ipv4 and ipv6 on the primary interface and ipv4 static ips without dhcp on the second interface that is used for storage. Please note that I didn't install a full cluster every time, so I don't know if the first experiments would have resulted in a working cluster.

dhcp on the primary interface (ipv4 only) and static ip on the second: This seems to work, I have seen mentions of this on slack, but I wanted dual stack hosts, so I didn't try it.

dhcp and dhcp6 for the primary interface via dracut cmdline rd.neednet=1 ip=dhcp,dhcp6 rd.net.timeout.dhcp=3 rd.net.timeout.ipv6dad=3 results in a primary interface with an ipv4 config from dhcp4 and nothing more than a link local address in ipv6. dhcp6 is never even attempted. same result with other combinations of dhcp and dhcp6 and without additional parameters.

dhcp for ipv4 and static ip for ipv6. this is not how this is meant to work, i suppose. I get the ipv6 address and no ipv4 besides <IP_ADDRESS> on the lo interface.

Static ips for everything like in for example
ip=<IP_ADDRESS>::<IP_ADDRESS>:<IP_ADDRESS>:worker4.example.com:ens192:off:<IP_ADDRESS>:<IP_ADDRESS> ip=[fd3f:58c7:4:D:192:168:9:44]::[fd3f:58c7:4:D::1]:64:worker4.example.com:ens192:off ip=<IP_ADDRESS>:::24::ens224:off: This seems to be doing what it is supposed to but results in a different problem. The search domain cannot be set via dracut cmdline (https://bugzilla.redhat.com/show_bug.cgi?id=1963882) and is set to . (a single dot) which is then somehow reduced to nothing in the name-resolver pods deployed by dns operator. A resolv.conf with an empty search domain makes dig unhappy and the internal registry unresolvable.

Static ips like in 3 but set the same ips and the search domain via ignition. This seems to work but creates redundancy.

ATM I have a running cluster that was installed with method 4 including dual stack hosts and ips for the storage network.

Static ips like in 3 but set the same ips and the search domain via ignition. This seems to work but creates redundancy.

Right, that's the only feasible option at the moment. If initially the interface can be configured via DHCP, then the whole static IP configuration (along with search domain) can be set in NM keyfiles.
This however won't work on all setups - but we depend on search domain configuration in dracut to close the feature gap

I have installed already a number of clusters with static IPs. At the beginning (before Afterburn/dracut) became available in FCOS, I modified the boot images (initial kernel arguments) for the initial boot and via ignition set the config via NM config file. When Afterburn was available I first only used it for the initial boot IP config and stayed with NM config files in addition tho this.
After a number of tests I found that Afterburn config worked so well that I decided to use only Afterburn Ip config and no longer set NM config files. I did also notice that I then have no way to specify the search suffix any longer. But I have not found yet that this causes problems. @gudroot, are you sure this causes problems with the internal registry? Perhaps we do not use this enough to even notice? How would I notice this?
@vrutkovs, with some recent OKD release there was also the change that after cluster deployment, the IP config is no longer on the ens192 interface but on br-ex. The Afterburn IP config ends up in the NM config file default_connection.nmconnection - so I would have to somehow modify this to add dns-search or already pre-create this via ignition, I assume.

with some recent OKD release

We didn't do OKD 4.8 releases yet, lets keep this ticket on topic - and file a new one if the change is needed

@vrutkovs I was just meaning that what @gudroot filed here is probably not new/specific with 4.8.

BTW, do you know of work already being in progress to add dns-search support to dracut?

I can confirm that the problem that @gudroot describes about the internal registry not being resolvable when "dns-search=." is not specific to 4.8 ... It already happens at least with 4.7-2021-06-04 as well. I am pretty sure it did not happen with earlier releases. But I will verify this with another deployment/test cycle later today or tomorrow. Once I have more details I will create another issue and reference this one.

@kai-uwe-rommel you seem to have found out already: to show the search domain problem simply run an image from the local registry like oc run mariadb --image=image-registry.openshift-image-registry.svc:5000/openshift/mariadb:latest. The pod will never come up because 'image-registry.openshift-image-registry.svc' can't be found.

And I have meanwhile found out, that the problem already appears with OKD 4.7 2021-06-04 as well, not only with 4.8.
The problem does not appear with OKD 4.7 2021-05-22 and older versions.
I assume that the FCOS stream has been updated and that a newer NetworkManager is causing the problem.
@vrutkovs, do you still think I should open a new issue for this?

With the new version of OKD/FCOS/NetworkManager, this leads to "search=." in resolv.conf.

What was the behaviour in previous versions? I don't think empty DNS domain is ever a correct situation

We are not talking about the DNS domain. When you specify a FQDN via dracut/Afterburn as the hostname, then the domain part of it is used as the DNS domain. The problem is the search domain.

As a workaround I re-enabled the code in my deployment scripts that also creates NetworkManager config files via ignition in addition to the Afterburn vSphere config string. This for now solves the problem and I again have a working search domain suffix. I create a default.nmconnection referring to ens192 and the OKD setup picks up the data from it correctly for the br-ex config file it generates.
(I had stopped generating NetworkManager config files as dracut/Afterburn worked so well. And a search suffix was not a problem so far - but now with the value of "." it is a problem).

My bad, I meant search domain. I don't think it ever got set previously - and I don't see why would that affect the deployment - cluster.local would be appended to search domain list anyway. If nodes have FQDN hostnames (as they must) empty (or .) search domain should be equivalent and play no role in resolution

Well that's the theory. :-) In practice, we see this difference now.
The cluster deployment is not affected.
But once the cluster is up and you try to deply something that refers to an image in the clusters internal registry, it fails.
Example: I always deploy a cron job to each cluster for LDAP group synchronization.
It uses this image:
Containers:
  ldap-group-sync:
    Image:         image-registry.openshift-image-registry.svc:5000/openshift/cli

With the new release now that fails:
Events:
  Type     Reason          Age                 From               Message
  ----     ------          ----                ----               -------
  Normal   Scheduled       73m                 default-scheduler  Successfully assigned openshift-authentication/custom-ldap-group-sync-1622970000-g2cwh to worker-02.kur-test.ars.de
  Normal   AddedInterface  73m                 multus             Add eth0 [<IP_ADDRESS>/23]
  Normal   Pulling         71m (x4 over 73m)   kubelet            Pulling image "image-registry.openshift-image-registry.svc:5000/openshift/cli"
  Warning  Failed          71m (x4 over 73m)   kubelet            Failed to pull image "image-registry.openshift-image-registry.svc:5000/openshift/cli": rpc error: code = Unknown desc = error pinging docker registry image-registry.openshift-image-registry.svc:5000: Get "https://image-registry.openshift-image-registry.svc:5000/v2/": dial tcp: lookup image-registry.openshift-image-registry.svc on <IP_ADDRESS>:53: no such host
  Warning  Failed          71m (x4 over 73m)   kubelet            Error: ErrImagePull
  Warning  Failed          63m (x42 over 73m)  kubelet            Error: ImagePullBackOff
  Normal   BackOff         58m (x63 over 73m)  kubelet            Back-off pulling image "image-registry.openshift-image-registry.svc:5000/openshift/cli"

With all previous releases this was working fine.
This is the same that @gudroot reported above.
The problem goes away as soon as I add a sensible dns-search=... to the NetworkManager config file(s) and reboot the nodes.

in my opinion the problem is in the node-resolver pods. Since I don't have the broken cluster any more, I'll have to recall this from memory:
Not setting the search domain results in a search domain of "." and a resolv.conf with the content
search .
nameserver <ip1>
nameserver <ip2>

on the host.
This is then somehow mangled and the node-resolver pods get a resolv.conf that is slightly different:
search
nameserver <ip1>
nameserver <ip2>

I am not sure a search domain "." offers any advantages over not having a search domain at all, but having an empty search domain is IMHO syntactically incorrect. It reminds me of this old bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=669163 Maybe dig and nslookup are not the only tools that stumble upon that kind of error in resolv.conf.
For the exact error messages I would have to reinstall the cluster.
@kai-uwe-rommel do you still have your broken cluster?

No, I don't have the "broken" cluster available at the moment because I already tested/implemented the workaround.
But I can deploy a new "broken" cluster easily with about 10 minutes of work, if needed.

Perhaps its similar to https://github.com/openshift/okd/issues/690, could you check if this still happens on OKD 4.8 RC3?

@vrutkovs, I think these are two different problems. The #690 issue talks about a missing /etc/systemd/resolved.conf.d directory. This case here is (also) about the "search ." problem.
I don't have time to test OKD 4.8 RC versions at the moment but I do always test new 4.7 releases as I need them for my projects. So I just installed a 4.7 2021-06-19 cluster and still see the "search ." problem that this issue here is (also) about. But the /etc/systemd/resolved.conf.d directory does exist (#690 also is about 4.7).
So the primary topic of #690 is not our problem here. But in #690 @bobby0724 and @chrisu001 were later hijacking the issue a bit also for the "search ." problem which is apparently unrelated to @fortinj66's /etc/systemd/resolved.conf.d problem.

I was hoping working resolved.conf configuration would resolve (or at least alleviate) this issue, but alas.
Looks like there isn't anything to fix in OKD right now. It appears the preferred ways to configure this is:

via kernel args - blocked by dracut feature request implementation
via NM config file, placed by MC on every node - not sure if NM allows dropins for connection files.
Perhaps we could implement this via a service which runs nmcli early in the boot?

I can confirm, that the bug search . still exists in
● pivot://registry.ci.openshift.org/origin/4.8-2021-06-19-221544@sha256:3e274ce09a9d5cb5a1fadd3927f7f068e7d3b75cf362b16745f7706e52bd7dcb
              CustomOrigin: Managed by machine-config-operator
                   Version: 48.34.202106191727-0 (2021-06-19T17:31:30Z)

  ostree://fedora:fedora/x86_64/coreos/stable
                   Version: 34.20210529.3.0 (2021-06-14T14:45:28Z)
                    Commit: 5040eaabed46962a07b1e918ba5afa1502e1f898bf958673519cd83e986c228f
              GPGSignature: Valid signature by 8C5BA6990BDB26E19F2A1A801161AE6945719A39

node resolv.conf:
search .
nameserver <IP_ADDRESS>

node-resolver pod
$ oc exec  -n openshift-dns node-resolver-55gtc -- cat /etc/resolv.conf
search
nameserver <IP_ADDRESS>

so the  node-resolver pod still fails
$oc logs  -n openshift-dns node-resolver-55gtc
dig: parse of /etc/resolv.conf failed
...

Hint: this is an automated test-setup based on UPI Virtualboxes with static ip/kernelargs
setting a search domain will circumvent the issue

I was hoping working resolved.conf configuration would resolve (or at least alleviate) this issue, but alas.

Are we talking about the search domain issue or the hostname issue now? IMO the hostname issue affects every dual stack cluster, since you cannot use both dhcp4 and dhcp6 at the same time (I tried it and only got IPv4), but you can't use static ips via afterburn either as this would trigger the search domain issue.
Last time I checked dual stack was on the roadmap for 4.8, so if it still is, there should be some kind of idea on how to set this up correctly.

Looks like there isn't anything to fix in OKD right now. It appears the preferred ways to configure this is:

via kernel args - blocked by dracut feature request implementation
via NM config file, placed by MC on every node - not sure if NM allows dropins for connection files.
Perhaps we could implement this via a service which runs nmcli early in the boot?

This would be a temporary fix for the search domain issue until dracut is capable of configuring a search domain?

https://github.com/openshift/okd/discussions/698#discussioncomment-896761 - perhaps its SELinux preventing the hostname to be set?

Currently I'm solving the "search=..." problem already with by adding a small NetworkManager file during ignition (to keep things together, static IP config also happens in this stage).
I do not have a problem with that but it was quite a learning curve to determine all this and create my own meta installer for UPI deployment on vSphere with static IPs. At least some documentation would be helpful for others that will come the same way later.

The search=dot problem seems to be more common than I thought: https://github.com/openshift/okd/discussions/694

#698 (comment) - perhaps its SELinux preventing the hostname to be set?

I can see the same selinux messages in the logs captured on a cluster that is affected by the hostname=localhost problem that was the subject of this issue before we started to discuss overly short search domains.
I'll try another installation when selinux-policy 34.11-1.fc34 hits 4.8

@gudroot, how are you assigning hostnames?  via DHCP or reverse DNS lookup?

@fortinj66 you did ask gudroot and not me but I would still like to add my $0.02 ...
I have learned that if I want reliable clusters in all configurations (e.g. DHCP or fixed IPs) I better explicitly set the hostname via a statement in the ignition file to configure the /etc/hostname file.
I'm always using UPI on vSphere, though. This gives a lot more freedom.

@fortinj66 you did ask gudroot and not me but I would still like to add my $0.02 ...
I have learned that if I want reliable clusters in all configurations (e.g. DHCP or fixed IPs) I better explicitly set the hostname via a statement in the ignition file to configure the /etc/hostname file.
I'm always using UPI on vSphere, though. This gives a lot more freedom.

But you shouldn't have too...  FCOS should be able to resolve the hostnames either by DHCP or reverse lookup.
reverse lookup seems to be completely broken in FCOS 34.  I'm going to test DHCP assignment later this morning.

In theory, many things ought to work much better than they actually do.
Not always do I spend much time to insist on this and work with the vendor to fix things.
In some cases I simply find a good and easy solution for myself and spend my time on more important things...

As @fortinj66 says, the hostname resolution via reverse DNS lookup is broken. If, on the other hand, the FQDN is set in the hostname option in the DHCP, it currently works for me without any noticeable problems.

Seems only hostname method affected is PTR records, updated the ticket title. John, please post NM bugzilla link here so we could consider reverting NM back to 1.26

https://bugzilla.redhat.com/show_bug.cgi?id=1975280

I can confirm, last not affected version is 4.7.0-0.okd-2021-05-22-050008. "search ." in resolv.conf is absent. Record with internal registries host in /etc/hosts present.
Affected versions
4.7.0-0.okd-2021-06-19-191547

I just installed a new vsphere UPI cluster with hostnames from DHCP...  Worked fine...
But I don't see any  node-resolver pods...
Also, I was able to recreate the following

@kai-uwe-rommel you seem to have found out already: to show the search domain problem simply run an image from the local registry like oc run mariadb --image=image-registry.openshift-image-registry.svc:5000/openshift/mariadb:latest. The pod will never come up because 'image-registry.openshift-image-registry.svc' can't be found.

except in my case the registry has not been enabled yet (off by default in vSphere)  Since there was not a service defined for the image registry internal DNS would not find image-registry.openshift-image-registry.svc
Once I enabled the registry the cluster found the registry and the mariadb pods were created...
so for me internal DNS seems fine

@fortinj66, yes, DHCP works fine for me with current builds also. It's static IP configuration (which I need most of the time) where due to the missing feature in Dracut/Afterburn where it fails with the "search=." problem with the latest builds.

for me also static ip fails
what i am doing is setup the ip in the fcos boot using this line
ip=<IP_ADDRESS>::<IP_ADDRESS>:<IP_ADDRESS>:::none:<IP_ADDRESS> coreos.inst.install_dev=/dev/sda coreos.inst.image_url=http://<IP_ADDRESS>:8080/okd4/fcos.raw.xz coreos.inst.ignition_url=http://<IP_ADDRESS>:8080/okd4/master.ign
using haproxy
api
api-int
*.apps
dns entries point to <IP_ADDRESS>
if i do this the hostname issue appears randomly and dns issue is always there

for me also static ip fails
what i am doing is setup the ip in the fcos boot using this line
ip=<IP_ADDRESS>::<IP_ADDRESS>:<IP_ADDRESS>:::none:<IP_ADDRESS> coreos.inst.install_dev=/dev/sda coreos.inst.image_url=http://<IP_ADDRESS>:8080/okd4/fcos.raw.xz coreos.inst.ignition_url=http://<IP_ADDRESS>:8080/okd4/master.ign
using haproxy
api
api-int
*.apps
dns entries point to <IP_ADDRESS>
if i do this the hostname issue appears randomly and dns issue is always there

OK, I can reproduce this with a static IP address
ip=<IP_ADDRESS>::<IP_ADDRESS>:<IP_ADDRESS>:worker0.poc-upi-4.os.maeagle.corp::none nameserver=<IP_ADDRESS> nameserver=<IP_ADDRESS>
[core@worker0 ~]$ cat /etc/resolv.conf | grep -v "\#"

nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>
search .

@fortinj66, that is like I do it, too. Same result since OKD 4.7 2021-06-03. No problem up until (including) 2021-05-22.

I'm planning on new 4.7 release on July 04. We should decided if we want to pin to older NM or we have a new package with the fix.
Any known problems NM fixes since 05-22 payload (afaik it has NM 1.30)?

This problem with the "search=." is the only one that I am aware of.

looks like 5-22 release has: 1.26
[core@dev-c1v4-pp647-master-0 ~]$ rpm -qa | grep Network
NetworkManager-libnm-1.26.8-1.fc33.x86_64
NetworkManager-1.26.8-1.fc33.x86_64
NetworkManager-tui-1.26.8-1.fc33.x86_64
NetworkManager-cloud-setup-1.26.8-1.fc33.x86_64
NetworkManager-team-1.26.8-1.fc33.x86_64
NetworkManager-ovs-1.26.8-1.fc33.x86_64

I'm planning on new 4.7 release on July 04. We should decided if we want to pin to older NM or we have a new package with the fix.

It would seem unlikely that there would be NetworkManager fixes prior to July 4th...  Can we create an install package with NM 1.26.8-1 for testing?

This problem with the "search=." is the only one that I am aware of.

hostname from Reverse DNS Lookup is also broken

So interestingly enough, I tried this on the latest FCOS 33 stable.   It does not create a search entry at all:
[core@worker0 ~]$ cat /etc/resolv.conf
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients directly to
# all known uplink DNS servers. This file lists all configured search domains.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver <IP_ADDRESS>
nameserver <IP_ADDRESS>

Ok, so FCOS 34 has NetworkManager 1.30+ (seems broken)
FCOS 33 has NetworkManager <IP_ADDRESS> (seems to work)
Are we going to have to go back to FCOS 33?

This might be fixed in 1.32.0+ by https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_requests/877 of which there is unfortunately no F34 build. Maybe we can get this backported to 1.30.z which is what ships in F34?

Are we going to have to go back to FCOS 33?

Instead of downgrading to FCOS 33, I would prefer a backport of the fix.

Are we going to have to go back to FCOS 33?

No, luckily NM team has COPR, so we can use 1.26 or 1.32 builds - or try 1.30.2 from fedora stable repo (latest from updates is .4). See PRs to implement these above.
I mirrored CI test releases to

quay.io/vrutkovs/okd-release:4.7-nm-1.32
quay.io/vrutkovs/okd-release:4.7-nm-1.30.2
quay.io/vrutkovs/okd-release:4.7-nm-1.26

Please give these a try.

I've tested all three versions...
search .  is still broken for static IPs on all three versions
reverse lookup for hostname works on 1.26 and 1.32.  Broken on 1.30.2

So, It seems the issue with search domains with static IPs in FCOS 34 is systemd-resolved related:
static int write_uplink_resolv_conf_contents(FILE *f, OrderedSet *dns, OrderedSet *domains) {

        fputs("# This is "PRIVATE_UPLINK_RESOLV_CONF" managed by man:systemd-resolved(8).\n"
              "# Do not edit.\n"
              "#\n"
              "# This file might be symlinked as /etc/resolv.conf. If you're looking at\n"
              "# /etc/resolv.conf and seeing this text, you have followed the symlink.\n"
              "#\n"
              "# This is a dynamic resolv.conf file for connecting local clients directly to\n"
              "# all known uplink DNS servers. This file lists all configured search domains.\n"
              "#\n"
              "# Third party programs should typically not access this file directly, but only\n"
              "# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a\n"
              "# different way, replace this symlink by a static file or a different symlink.\n"
              "#\n"
              "# See man:systemd-resolved.service(8) for details about the supported modes of\n"
              "# operation for /etc/resolv.conf.\n"
              "\n", f);

        if (ordered_set_isempty(dns))
                fputs("# No DNS servers known.\n", f);
        else {
                unsigned count = 0;
                DnsServer *s;

                ORDERED_SET_FOREACH(s, dns)
                        write_resolv_conf_server(s, f, &count);
        }

        if (ordered_set_isempty(domains))
                fputs("search .\n", f); /* Make sure that if the local hostname is chosen as fqdn this does not
                                         * imply a search domain */
        else
                write_resolv_conf_search(domains, f);

        return fflush_and_check(f);
}

This line:
fputs("search .\n", f); / Make sure that if the local hostname is chosen as fqdn this does not
* imply a search domain /

The equivalent  in FCOS 33:
        if (!ordered_set_isempty(domains))
                write_resolv_conf_search(domains, f);
        return fflush_and_check(f);

No search line is written if there are no domains

Actually, I'd even regard it as smart if in the case there is no search domain specified explicitly, it would use the domain suffix from the hostname if that was specified as a FQDN ...

Seems it was implemented by https://github.com/systemd/systemd/pull/17201, so we might want to revert systemd-* to v246

Also, this bug is like million comments long, anyone could summarize which problems we're hitting, which are the workaround and which packages need updated/downgraded?
IIUC its two bugs:

reverse DNS hostname is broken in NM 1.30+, fixed in 1.32 (we can fasttrack it)
systemd-resolved no longer allows containers to use image-registry.openshift-image-registry.svc:5000  image. This requires reverting systemd to v246, which is very risky.
Do we have a workaround for that? Does .svc.cluster.local work?

Do we have a workaround for that? Does .svc.cluster.local work?

We could workaround this in the prepender code...  Just add a line which removes "search ."  Issue would be if systemd-resolved was run again it would recreate the issue

A service that gets triggered when systemd-resolved is run...

https://github.com/openshift/okd-machine-os/pull/156

Someone will have to file this upstream with FCOS and resolved. Any fixup commits will have to contain a proper justification and links to the filed issues.

@kai-uwe-rommel @bobby0724 @gudroot
would you folks be able to test:    quay.io/fortinj66/origin-release:v4.7-search-fix
Its based off the quay.io/openshift/okd:4.7.0-0.okd-2021-06-19-191547 stable release...
It has a workaround for the search . issue.

Someone will have to file this upstream with FCOS and resolved. Any fixup commits will have to contain a proper justification and links to the filed issues.

I want to make sure the 'fix' works for folks other than me before I go through the rest of the hoops...

Will do tonight.

Seems to be fixed. No search line in resolve.conf at all (when configuring static IP purely via Afterburn, e.g. no search domain specified).

Seems to be fixed. No search line in resolve.conf at all (when configuring static IP purely via Afterburn, e.g. no search domain specified).

Were you able to deploy new Deployments and pods?  I was able to with my testing...

Yes, looks all normal.

fixes for search . and hostname via reverse DNS should be in the latest nightlies starting with nightly registry.ci.openshift.org/origin/release:4.7.0-0.okd-2021-06-29-103107

thanks for fixing this issue

Should be fixed in https://amd64.origin.releases.ci.openshift.org/releasestream/4-stable/release/4.7.0-0.okd-2021-07-03-190901

I'm running 4.7.0-0.okd-2021-07-03-190901, but it still broken in my cluster

I followed the above conversation and wanted to know is there a work around for this issue? What should resolv.conf look like to work?

What should resolv.conf look like to work?

Latest OKD now runs a service which removes search . from /etc/resolv.conf. Check that the file on the  nodes doesn't contain it.
If you're hitting an issue with similar symptoms and search . is not present, please file a new bug

Hi, just installed 4.8.0-0.okd-2021-10-24-061736 with static IP's and still have search . there. So same as:

Static ips for everything like in for example
ip=<IP_ADDRESS>::<IP_ADDRESS>:<IP_ADDRESS>:worker4.example.com:ens192:off:<IP_ADDRESS>:<IP_ADDRESS> ip=[fd3f:58c7:4:D:192:168:9:44]::[fd3f:58c7:4:D::1]:64:worker4.example.com:ens192:off ip=<IP_ADDRESS>:::24::ens224:off: This seems to be doing what it is supposed to but results in a different problem. The search domain cannot be set via dracut cmdline (https://bugzilla.redhat.com/show_bug.cgi?id=1963882) and is set to . (a single dot) which is then somehow reduced to nothing in the name-resolver pods deployed by dns operator. A resolv.conf with an empty search domain makes dig unhappy and the internal registry unresolvable.

How can I get rid of it?

I cannot reproduce this. I just installed a cluster with the same version and also static IPs and do not see this problem.


[VS Code] Joining collaboration session: Unable to join the host due to an authentication error. Verify the host is online, then sign out and sign in again and retry.

Error:
Joining collaboration session: Unable to join the host due to an authentication error. Verify the host is online, then sign out and sign in again and retry.
Steps to Reproduce:
1.
2.

Version Data

extensionName
VSLS

extensionVersion
1.0.471

protocolVersion
2.2

applicationName
VSCode

applicationVersion
1.35.1

platformName
MacOS

platformVersion
18.5.0

Duplicate of #433


dnsprobe: add small binary to probe dns servers

This PR aims to solve #108 by adding a small go binary that resolves a given name on a given (udp) server, and fails if any error is encountered in the process.
This binary can be used as a readiness or health probe in environments like Kubernetes.

Do these tests look reasonable to you? I can merge them separately or you can cherry-pick the commit to the PR. Up to you.
https://github.com/klutchell/dnscrypt-proxy-docker/tree/kyle/dnsprobe-tests

(also squash the commits if you get a chance)

Do these tests look reasonable to you? I can merge them separately or you can cherry-pick the commit to the PR. Up to you.
https://github.com/klutchell/dnscrypt-proxy-docker/tree/kyle/dnsprobe-tests

Tests look great to me, thanks for taking care of that!
I can cherrypick those, np at all.


Code Editor Extension

So it will be an extension which provides a code editor to write code on like chrome extentions work on Chromebooks

look great! go ahead @Jyo561

Can you assign GSSoC22 to it

@Jyo561 We will add labels in 1-2 days don't worry about that :)

Thank you

@Jyo561 any update on this?

yup currently working on its UI

status on this @Jyo561 ?

Yeah I am still working on its UI
had to take a week off due to exams so it will be a little late

updates on this @Jyo561 ?


Update Fundamentals/Routing doc sample code to CSPROJ

The sample code for the Routing doc is still based on project.json. Is anyone already working on upgrading it to CSPROJ? If not, I'll take care of it.
/cc: @Rick-Anderson

That would be great - Thanks.


Reuse token when certain factors are met

Expected Behavior

It'll be nice if access token could be reuse when certain factors are met. Like the AuthenticationKeyGenerator in Spring Security OAuth.
When creating OAuth2AccessToken, find access token by key generated by AuthenticationKeyGenerator first(Maybe user ID, session ID or any others):
public class DefaultTokenServices implements AuthorizationServerTokenServices, ResourceServerTokenServices,
		ConsumerTokenServices, InitializingBean {

	@Transactional
	public OAuth2AccessToken createAccessToken(OAuth2Authentication authentication) throws AuthenticationException {

		OAuth2AccessToken existingAccessToken = tokenStore.getAccessToken(authentication);
		OAuth2RefreshToken refreshToken = null;
		if (existingAccessToken != null) {
			if (existingAccessToken.isExpired()) {
				if (existingAccessToken.getRefreshToken() != null) {
					refreshToken = existingAccessToken.getRefreshToken();
					// The token store could remove the refresh token when the
					// access token is removed, but we want to
					// be sure...
					tokenStore.removeRefreshToken(refreshToken);
				}
				tokenStore.removeAccessToken(existingAccessToken);
			}
			else {
				// Re-store the access token in case the authentication has changed
				tokenStore.storeAccessToken(existingAccessToken, authentication);
				return existingAccessToken;
			}
		}

           // ...
   }
}

public class InMemoryTokenStore implements TokenStore {

    public OAuth2AccessToken getAccessToken(OAuth2Authentication authentication) {
		String key = authenticationKeyGenerator.extractKey(authentication);
		OAuth2AccessToken accessToken = authenticationToAccessTokenStore.get(key);
		if (accessToken != null
				&& !key.equals(authenticationKeyGenerator.extractKey(readAuthentication(accessToken.getValue())))) {
			// Keep the stores consistent (maybe the same user is represented by this authentication but the details
			// have changed)
			storeAccessToken(accessToken, authentication);
		}
		return accessToken;
	}

}

Current Behavior

Every time we get token using authorization code, a new access token is issued, regardless of whether it is the same user and the same browser.
Because OAuth2Authorization ID is a UUID and can not be easily replaced.
public OAuth2Authorization build() {
			Assert.hasText(this.principalName, "principalName cannot be empty");
			Assert.notNull(this.authorizationGrantType, "authorizationGrantType cannot be null");

			OAuth2Authorization authorization = new OAuth2Authorization();
			if (!StringUtils.hasText(this.id)) {
				this.id = UUID.randomUUID().toString();
			}
}

Context

In some cases, we need to get all tokens issued to a user. For example, in a concurrent session scenario, we want to kick out a user's non-latest sessions and tokens, we need to get the tokens issued to the user and do revocations. And i think the same is true in a single sign-out scenario.

@wapkch

Every time we get token using authorization code, a new access token is issued, regardless of whether it is the same user and the same browser.

Each time a client performs an authorization_grant, a new instance of OAuth2Authorization is created. This is the expected behaviour. Please see the OAuth2Authorization reference documentation for further details.

In some cases, we need to get all tokens issued to a user.

You can extend OAuth2AuthorizationService and provide additional operations required by your application, for example:
public interface CustomOAuth2AuthorizationService extends OAuth2AuthorizationService {

	List<OAuth2Authorization> findByPrincipalName(String principalName);

}

You would implement CustomOAuth2AuthorizationService and then register it as a @Bean. Then you can inject the bean and use findByPrincipalName() to perform the custom logic you require.
I'm going to close this as a new OAuth2Authorization per authorization_grant is works as designed and the custom operation/logic for findByPrincipalName() needs to be implemented in the consuming application.


Authentication of Release

Hi,
I'm using MDL in a project and am packaging it for cljsjs.  A MD5 checksum is used for validation.  Is there a way that you could publish a checksum over a secure channel?
It would also be nice, for projects more interested in security than cljsjs, to have PGP signed releases.
At minimum, could I get confirmation that 1.0.4 has an MD5 checksum of e6883041f5cf44642b1539ecf5a25f9c?
Thanks!
Randall Mason

Adding a checksum to the download sounds like a good idea to me.
And I can provide the following checksums for MDL’s 1.0.4 ZIP file

MD5 e6883041f5cf44642b1539ecf5a25f9c
SHA1<PHONE_NUMBER>123b1b05e2c323e83e1dd5957a9e1d
SHA256 03a5b2acb124bc7d389943ce8c9835e60cdad33643d949d337516da8ceb0649a

We have decided to provide these checksums on the GitHub release notes in the future.

FYI, we started including these in our release notes, starting with 1.0.5 today!


Current dtype of photometry changes the values.

Currently the dtype when setting photometry is np.float16, however this is not very precise and can change a magnitude of 23.93 to 23.94. We should change the dtype to np.float32.

Is there a test you could write which would fail using the current code but would pass with this change?


Game Request

Have you guys considered adding Apex to the list of supported games? I feel like a lot of cool effects could come out of that, like separate profiles for the legends and effects if you use their tacs and ultimates.

Having such an interactive profile is sadly not something we can achieve with Apex. We'd need to be able to get this information from the game somehow. Ideally the game would expose the data itself (such as how CS:GO does) or we'd create a mod (such as we do for Terraria). As a fallback we can also try using memory reading, but this is difficult to maintain with game updates etc.
Since Apex does not expose these values nor does it have modding we can't do that, and we won't implement memory reading for online games as it will almost certainly trigger anti-cheat mechanisms.
I believe that Apex does have Razer Lighting integration so you should be able to use it with the Razer Chroma layer, but you will only be able to have the lighting as set by the developers and won't be able to customise it.

This is correct, the game should work with the razer chroma layer as overwatch does.


[core] Remove replace branch to reduce IO

Purpose

The replace_branch incurs an expensive IO overhead for most operations in the normal code path. For HDFS, it is a namenode access, and for object storage, it is a separate billing. And for object storage, every IO cost 10ms+. It is very costly.
This is difficult to accept, and if replace_branch is not as useful, we should remove this operation.
Detail is here: https://lists.apache.org/thread/q46clxx38fz7n1xw0sgscmcslo3qrp5c

Tests

API and Format

Documentation

+1 for this.


There is a conflict dialog when pasting one file share which contains a non-empty folder

Storage Explorer Version: 1.25.0-dev
Build Number: 20220615.6
Branch: main
Platform/OS: Windows 10
Architecture ia32
How Found: From running test cases
Regression From: Previous release (1.24.2)
Steps to Reproduce

Expand one storage account -> File Shares.
Create a file share -> Click 'Upload -> Upload Files...'.
Select files -> Type '/folder1/folder2' to the 'Destination directory'.

Click 'Upload'.
Copy the file share -> Paste it to another storage account.
Check there is no conflict dialog.

Expected Experience
There is no conflict dialog.
Actual Experience
There is a conflict dialog.

Addition Context

This issue can't be reproduced consistently.
For now, we only met this issue on Windows.
Here is the screenshot if selecting 'Do not paste -> Apply to All Conflicts'.

Could you please get us the AzCopy log if you can reproduce it one more time? I suspect it's due to AzCopy ordering the transfers incorrectly.

Got it. Will share the AzCopy log if this issue reproduces again.

Given it's been long enough that this issue hasn't be reproduced. Let's close this issue. If it reproduces, we can open a new one and investigate.


is it possible to return Result<Response, impl Error> ?

Right now handlers expect a type of
Fn(Request<Body>, Captures) -> Response<Body> + Send + Sync>;
but this loses the ergonomics of the ? operator. Is it possible to change this to something like:
Fn(Request<Body>, Captures) -> Result<Response<Body>, impl std::error::Error> / etc? So we can use handlers with our own errors.

I'm not opposed to this. I think I omitted errors in the handler type since a lot of hyper examples just used Infallible but I understand that may not be the best solution for everything.

I was playing around with some changes. I.e. changing Response<Body> to AppResult<Response<Body>>.
The issue I currently get is:
error[E0271]: type mismatch resolving `for<'r> <for<'_> fn(http::request::Request<hyper::body::body::Body>, std::option::Option<smallvec::SmallVec<[&str; _]>>) -> impl std::future::Future {status2} as std::ops::FnOnce<(http::request::Request<hyper::body::body::Body>, std::option::Option<smallvec::SmallVec<[&'r str; 4]>>)>>::Output == std::result::Result<http::response::Response<hyper::body::body::Body>, errors::AppError>`

  --> src/main.rs:39:17
   |
39 |         builder.get(r"/api/health", status2);
   |                 ^^^ expected opaque type, found enum `std::result::Result`
   |
   = note: expected type `impl std::future::Future`
              found type `std::result::Result<http::response::Response<hyper::body::body::Body>, errors::AppError>`

I didn't run your example but I think that is because you didn't change the return type of H in the get signature to AppResult.

@gsquire thanks for pointing that out, I changed all the references and abstracted into a new repo. Even with those changes, the error is still the same.

Sorry I overlooked it before but you don't need to make the function asynchronous. Try removing that and it should work for you. Only your handler function needs to be asynchronous.

@gsquire ah, that fixed it! So I ran some tests with wrk:
// async removed for the modified version
pub async fn status(_: Request<Body>, c: Captures<'_>) -> Response<Body> {
    Response::builder()
        .header(header::CONTENT_TYPE, "text/plain")
        .body(Body::from("Healthy!"))
        .unwrap()
}

# default reroute
wrk -t12 -c400 -d30s http://<IP_ADDRESS>:8000
Running 30s test @ http://<IP_ADDRESS>:8000
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.64ms    2.20ms  65.11ms   93.35%
    Req/Sec     3.65k     1.95k    8.20k    65.50%
  1091379 requests in 30.11s, 94.71MB read
  Socket errors: connect 156, read 55, write 0, timeout 0
  Non-2xx or 3xx responses: 1091379
Requests/sec:  36249.98
Transfer/sec:      3.15MB

# reroute modified to use `AppResult<Response<Body>>`
wrk -t12 -c400 -d30s http://<IP_ADDRESS>:8000
Running 30s test @ http://<IP_ADDRESS>:8000
  12 threads and 400 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.32ms    1.89ms  51.01ms   93.37%
    Req/Sec     4.79k     1.17k    6.90k    64.83%
  1143737 requests in 30.10s, 99.26MB read
  Socket errors: connect 156, read 47, write 0, timeout 0
  Non-2xx or 3xx responses: 1143737
Requests/sec:  37992.65
Transfer/sec:      3.30MB

I can get to work on a PR if you want to take a look later.

Why are there so many Non-2xx or 3xx responses?
I would be glad to review a PR soon too.

That's a good question, I'm not sure if that's due to wrk or something I set up.
So the main issue here is it was easy to set up reroute for my particular error type, but what about the generic error trait? I'm looking at -> Result<Response<Body>, Box<dyn Error>> but that also has its own issues. Thoughts?

We could make a generic variant of the Error enum in the errors module. And just have a From implementation for anything that implements std::error::Error. That way all errors can just be converted with map_err.

@gsquire something like
#[derive(Debug)]
pub struct Error<T: std::error::Error>(T);
impl From for Error
where T : std::error::Error {
fn from(err: T) -> Error {
Error(err)
}
}
// Potential errors that can happen while constructing a router.
pub enum RouteError {
BadRegex(::regex::Error)
}
impl From<::regex::Error> for RouteError {
fn from(err: ::regex::Error) -> RouteError {
RouteError::BadRegex(err)
}
}
?

After reading some documentation, it seems like if you pass errors along through the handler you effectively give an empty reply. I think the typical pattern is to catch each error and make a Response from them. So I am feeling a little more hesitant to change up the API now. Here is an example from the hyper repository that shows how errors get handled.

Hey, @azah just wanted to see if there was anything else I could help with?

hey @gsquire, I've been meaning to respond but I got caught up with some other stuff.
I think you're right. I played around and came up with this:
// errors.rs
impl Into<hyper::Response<hyper::Body>> for AppError {
    fn into(self) -> hyper::Response<hyper::Body> {
        let description: String = self.description().into();

        hyper::Response::builder()
            .header(hyper::header::CONTENT_TYPE, "text/plain")
            .status(http::status::StatusCode::INTERNAL_SERVER_ERROR)
            .body(hyper::Body::from(description))
            .expect("Unable to generate error")
    }
}

// main.rs
pub async fn status() -> AppResult<Response<Body>> {
    Response::builder()
        .header(header::CONTENT_TYPE, "text/plain")
        .body(Body::from("Healthy!"))
        .map_err(|e| AppError::from(e))
}

async fn router(req: Request<Body>) -> AppResult<Response<Body>> {
    match (req.method(), req.uri().path()) {
        //(&Method::GET, "/health") => handlers::status::status(), <-- this errors
        (&Method::GET, "/health2") => {
             Response::builder()
                .header(header::CONTENT_TYPE, "text/plain")
                .body(Body::from("Healthy!"))
                .map_err(|e| AppError::from(e))

        },
        (&Method::GET, "/error") => {
            Err(AppError::Generic("test".into()))
        }
        _ => Response::builder()
            .status(StatusCode::NOT_FOUND)
            .body(Body::empty())
            .map_err(|e| AppError::from(e)),
    }
}

async fn handle_gracefully<F>(res: F) -> AppResult<Response<Body>>
where F: Future<Output = AppResult<Response<Body>>>,
{
    let result: AppResult<Response<Body>> = res.await;

    match result {
        Ok(r) => Ok(r),
        Err(e) => { Ok(e.into()) }
    }
}

#[tokio::main]
async fn main() -> AppResult<()> {
    dotenv().ok();
    pretty_env_logger::init();

    let config =
        app_config::Config::from_env().expect("Unable to load config!");

    let addr = SocketAddr::new(
        IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)),
        config.server_port,
    );

    let svc = make_service_fn(move |_| async {
        Ok::<_, AppError>(service_fn(move |req| handle_gracefully(router(req))))
    });

    let server = Server::bind(&addr).serve(svc);
    server.await?;

    Ok(())
}

The only issue here is when I try to do something like
        (&Method::GET, "/health") => handlers::status::status(),

I get the following error:
error[E0308]: mismatched types
  --> src/main.rs:57:38
   |
57 |         (&Method::GET, "/health") => handlers::status::status(),
   |                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^ expected enum `std::result::Result`, found opaque type
   |
   = note: expected type `std::result::Result<http::response::Response<hyper::body::body::Body>, errors::AppError>`
              found type `impl std::future::Future`

In any case, I'll probably end up handling things manually.


Rewrite training loop EfficientDet.Pytorch to PL

See https://github.com/toandaominh1997/EfficientDet.Pytorch/blob/master/train.py
Priorities:
1 fast-dev-run https://pytorch-lightning.readthedocs.io/en/latest/debugging.html#fast-dev-run
2. single node training command on one node
..
..
N -1 distributed training

@Borda let's closing this since now since there are no bounding boxes right now in the dataset.
Feel free to reopen when it becomes relevant.


[main] Upgrade to latest dependencies

Cron -knative-prow-robot
/cc knative-sandbox/networking-wg-leads
/assign knative-sandbox/networking-wg-leads
Produced by: knative-sandbox/knobots/actions/update-deps
Details:
bumping knative.dev/networking 8ea3045...a70f23c:
  > a70f23c Update community files (# 617)
  > bd21fc9 add component label to reduce invocations (# 616)
  > 58bc12b upgrade to latest dependencies (# 615)
  > c9e7082 Add `activator-ca` and `activator-name` keys in `config-network` (# 608)
  > 9ce389c Update community files (# 614)
bumping knative.dev/pkg b619c92...fc93ac7:
  > fc93ac7 upgrade to latest dependencies (# 2424)
  > 51be315 Update community files (# 2419)
bumping knative.dev/hack 82bfca1...7331bb1:
  > 7331bb1 rename function (# 149)
  > 9b303d6 Update community files (# 148)

/retest

/lgtm
/approve

I believe this PR needs to be regenerated


Problem kind version

This PR updated the engine to use version 2 of the ProblemKind

Hi @hstairs  ,
This PR refers to this UP PR where the ProblemKind versioning is implemented.


Tasks cannot be collapsed after they are opened.

Open the Application and navigate to the following link:
http://allready-d.azurewebsites.net/Event/12
Click on a task.
Click on it again.
The task will not collapse.

The Campaign Info expand/collapse icon has a similar flaw, the collapse doesn't happen.
Tested on
Version: <IP_ADDRESS> 8a77b3a

@calaharie  - We selected you as a prize winner from the test lab. Either try to find me before you leave the conference, or send me your email address and I will get the prize to you.
Thanks for all of your help at the lab.

@wadewachs<EMAIL_ADDRESS>This notification was hidden in my e-mail, glad I happened to see it!

@calaharie I forwarded your email over to the folks at Techwell that have your prize. If you don't get it in a couple weeks let me know.
-Wade-

@tonylambert is this the same/duplicate of #1354 we just closed?  Thanks

@tonysurma - The bug is still present at the azure website linked to above, but running the latest in master on my machine at home I do not see the issue. The build on the azure website appears to be from August timeframe.
So, in short, yes, I think this one is no longer an issue. I think you can close it.

Hmm, that site is auto deployed from master latest, I will need to see what is up.  Either the auto deploy stopped or something else.  Thanks

Well. I assumed that the number in the screenshot below was the short Git commit hash:
http://screencast.com/t/chI57n6Q
Perhaps that's not right?

@tonylambert - nope that is totally valid.  I just checked and the site is auto deploying but it looks like the git version isn't bieng updated (I will create an issue tomorrow AM for that).  However now it is a mystery of why this works locally for you but not on live.  Missing JS or something I wonder...

Just checked the original link and the Tasks do collapse, both on clicking the same task again or on another task.

thanks @slaneyrw it may have been fixed by another in the time since then.


Create worker to recover lost jobs

Automatically re-enqueue content changes and messages that have been lost.
There is a general problem with the queuing system that means in-progress jobs can be permanently lost, which leads to alerts and call-outs. We experience the most pain from the loss of jobs to process content changes. We want to recover these lost jobs automatically.
https://trello.com/c/JD9CohHz/356-incident-action-automatically-requeue-content-changes-and-messages-that-have-been-lost

closed and replaced by https://github.com/alphagov/email-alert-api/pull/1318.


az servicebus namespace authorization-rule keys list produces python traceback if the name is not found

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

Describe the bug
Using the command "az servicebus namespace authorization-rule keys list" if the name specified is not found, the CLI incorrectly produces python traceback lines.  It should just say that the name is not found.
To Reproduce
az servicebus namespace authorization-rule keys list --name {incorrect-name)--namespace-name {correct-value} --resource-group {correct value)
Expected behavior
no output should be produced, or a message saying that the name is not found.  No python tracebacks should appear.
Environment summary
Windows MSI, version 2.5.1
Additional context
The output looks like this:

Traceback (most recent call last):
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\knack\knack\cli.py", line 215, in invoke
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-cli-core\azure\cli\core\commands_init_.py", line 631, in execute
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-cli-core\azure\cli\core\commands_init_.py", line 695, in run_jobs_serially
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-cli-core\azure\cli\core\commands_init.py", line 688, in run_job
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\six\six.py", line 703, in reraise
raise value
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-cli-core\azure\cli\core\commands_init.py", line 665, in run_job
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-cli-core\azure\cli\core\commands_init.py", line 324, in call
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-cli-core\azure\cli\core_init_.py", line 574, in default_command_handler
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-mgmt-servicebus\azure\mgmt\servicebus\operations\namespaces_operations.py", line 870, in list_keys
azure.mgmt.servicebus.models._models.ErrorResponseException: Operation returned an invalid status code 'Not Found'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\runpy.py", line 193, in run_module_as_main
"main", mod_spec)
File "C:\Program Files (x86)\Microsoft SDKs\Azure\CLI2\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-cli\azure\cli_main.py", line 41, in
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-cli\azure\cli_main.py", line 30, in cli_main
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\knack\knack\cli.py", line 226, in invoke
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-cli-core\azure\cli\core_init_.py", line 151, in exception_handler
File "C:\Users\VSSADM~1\AppData\Local\Temp\pip-install-7cl6xofh\azure-cli-core\azure\cli\core\util.py", line 82, in handle_exception
TypeError: string indices must be integers

I have no idea why those line are struct through.  Some sort of github goofiness

servicebus

cc @houk-ms @chenlomis , user would like to see errors directly instead of tracestack.

fixed, please use latest release


Add Vertical Line and Justified Alignment

[ ] Add vertical line
[x] Add justified alignment.

Vertical line addition is not implementable in Qt as we have tried different approaches.

Vertical line cannot be done in QT. Justified alignment can be found on commit number 926cfa4ccaad7d33dba22cd594b0e612ad74db1a


Add search templates

Added ability to search with templates (inline or stored) resolving #240.

Appreciate the effort, it would be even more complete with an integration test, so do you mind adding one? :)

Got the unit test missed the integration test. Happy to.

From: Cihat Keser<EMAIL_ADDRESS>Sent: Thursday, December 10, 2015 11:45 AM
To: searchbox-io/Jest
Cc: Hubbard, Bobby
Subject: Re: [Jest] Add search templates (#273)
Appreciate the effort, it would be even more complete with an integration test, so do you mind adding one? :)
�
Reply to this email directly or view it on GitHubhttps://github.com/searchbox-io/Jest/pull/273#issuecomment-163699823.

CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be confidential and/or legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.

So I cannot get the integration tests to recognize mustache as a legit script_lang. I've tested it on 2.0 and 2.1 clusters directly and this works fine. However, the integration test config has something goofy where any reference to a mustache template (whether inline or id) fails for mustache not supported.
"script_lang not supported [mustache]"
mustache is built-in and sandboxed and I dont see anywhere in the integration test config where it is explicitly turned off. I've also tried manually setting the script.* settings to no avail.
@kramer Any recommendation as to why it might be rejecting mustache?

I will take a deeper look but my initial guess is mustache is expected as an additional dependency just like groovy.

You were right. Somewhere along the way (since 2.0?) they stopped shading most dependencies. Which is cool. I just didn't realize it :) Anyway, I'll continue flushing out these integration tests. Thanks for the recommendation.

From: Cihat Keser<EMAIL_ADDRESS>Sent: Friday, December 11, 2015 12:58 AM
To: searchbox-io/Jest
Cc: Hubbard, Bobby
Subject: Re: [Jest] Add search templates (#273)
I will take a deeper look but my initial guess is mustache is expected as an additional dependency just like groovy.

Reply to this email directly or view it on GitHubhttps://github.com/searchbox-io/Jest/pull/273#issuecomment-163859542.

CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the intended recipient(s) and contain information that may be confidential and/or legally privileged. If you have received this email in error, please notify the sender by reply email and delete the message. Any disclosure, copying, distribution or use of this communication (including attachments) by someone other than the intended recipient is prohibited. Thank you.

Integration tests added.. Thanks for your help @kramer

Thanks @bobbyhubbard !

@kramer Checking in. Any idea if this pull will be included in your next jest release?

@bobbyhubbard It's merged to master so yes of course it'll be included, don't know when though.


fonts: add module

Thanks for the PR! Thus far I've simply put font packages into home.packages. Do you know of any benefit of using this method over using home.packages? In principle I like the idea of keeping fonts more organized but I'm worried that it doesn't actually add anything beyond what's possible already…

Oh, are they picked up from their? I have actually not tried that and just copied the behavior from nixpkgs.

I also just assumed that adding fonts to the packages list will not work and managed all fonts in the system configuration :)

So this module would be probably only useful for non-nixos system, I suppose.

That is a very good argument! Although I personally use NixOS everywhere, a goal of mine is for Home Manager to be useful for people using other distros.
I guess an alternative may be to have a boolean option fonts.linkFontsDir (or whatever) that on true causes
home.file.".local/share/fonts".source = "${config.home.path}/share/fonts";

to happen (config.home.path is the pkgs.buildEnv of all packages in home.packages). Then a non-NixOS user could also install fonts through home.packages…
I don't know which solution is most elegant but @Mic92 convinced me that it would be nice to have a way to get the fonts into ~/.local/share. Do you have any preference?

@rycee your approach sounds better.


[Dubbo-4340]Throws exception whenever SPI extensions are in abnormal status during loading.

What is the purpose of the change
Fixes #4340
Brief changelog
Split findException into two sperate functions, with one called before loading and another called after loading.

包下载有点问题。


Precompute logbook excluded events

Description:
A proposal for reducing the logbook load time by computing the exclude flag when the event is written (I realize that there are still some details missing).
@balloob @OverloadUT I wanted to get your opinion on this approach before spending more time on it. I am not yet sure what I think about it myself.
Related issue (if applicable): fixes #
Pull request in home-assistant.io with documentation (if applicable): home-assistant/home-assistant.io#<home-assistant.io PR number goes here>
Checklist:

[X] The code change is tested and works locally.
[ ] Local tests pass with tox. Your PR cannot be merged unless tests pass
[ ] There is no commented out code in this PR.

If user exposed functionality or configuration variables are added/changed:

[ ] Documentation added/updated in home-assistant.io

If the code communicates with devices, web services, or third-party tools:

[ ] New dependencies have been added to the REQUIREMENTS variable (example).
[ ] New dependencies are only imported inside functions that use them (example).
[ ] New or updated dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[ ] New files were added to .coveragerc.

If the code does not interact with devices:

[ ] Tests have been added to verify that the new code works.

I decided that I don't like this. I think a more radical change is needed, this one has too many problems for too little benefit.

For truly radicalism, https://github.com/home-assistant/architecture/issues/110 proposes to compute the logbook line at the time of the event, and so the reader will just fetch lines.


Unable to run an app on kubernetes

Steps to reproduce:

Create and publish the container to the registry.
Create a deployment:

kubectl run viz-ui --image=<whoami>/vizceral-example:latest --replicas=1 --port=8080
kubectl expose deployment viz-ui --port=8080 --type=LoadBalancer

curl -v http://<IP_ADDRESS>:<exposed-port>
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 31865 (#0)
> GET / HTTP/1.1
> Host: <IP_ADDRESS>:31865
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< X-Powered-By: Express
< Content-Type: text/html; charset=utf-8
< Content-Length: 19
< ETag: W/"13-OxsTL6IB85fkJxv9HO8uum0slCI"
< Date: Mon, 16 Apr 2018 11:56:43 GMT
< Connection: keep-alive
<
* Connection #0 to host <IP_ADDRESS> left intact
Invalid Host header

I would start at the pod level and work my way back. Exec into the pod and curl the localhost:8080
Does it respond?
What about the Service? Does it have endpoints?

Im also having trouble getting to run
$ docker run -d -p 41911:8080 --name test-${CI_JOB_ID} -e America/New_York ${DOCKER_TEST_IMAGE}
5c55d19e734bef42eb07a23338dea94b27f62044e30bb50728068672a74a4206
$ echo "Sleeping for 60" && sleep 60
Sleeping for 60
$ export RESPONSE=$(curl --write-out %{http_code} --silent --output /dev/null localhost:41911)
$ echo "Response was $RESPONSE"
Response was 000

@denismakogon I was finally able to figure this out.
This app relies on https://github.com/webpack/webpack-dev-server. This allows you to run a local development server that will autoupdate as the files change live on the disk. By default this server only allows connections from localmachines. You can fix this several ways, they all are based on configuring a devServer block in your webpack.config.js.
module.exports = {
  devServer: {
  },

You can then set disableHostCheck: true to allow all connections or you can set one or more URL's you will allow access to with the allowsHosts option. More details can be found here https://webpack.js.org/configuration/dev-server/
I have not yet figured out how to run this in non dev move, but I'm far from that stage anyways.

thank you @shadycuz


Create LICENSE.md

Mistakenly did not commit a LICENSE.md before transferring ownership

As this PR has been open for 6 days with zero activity, and is a non-code change I'll go ahead and merge.


fix: move LogMessageVoidify into internal namespace

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (941c66f) 60.00% compared to head (af749ef) 60.00%.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #1053   +/-   ##
=======================================
  Coverage   60.00%   60.00%
=======================================
  Files          19       19
  Lines        2580     2580
  Branches      911      911
=======================================
  Hits         1548     1548
  Misses        692      692
  Partials      340      340

Files
Coverage Δ

src/glog/logging.h
73.23% <ø> (ø)


Please add .jcscrc file

I have a few ideas I want to try. Please add .jcscrc file so I can better match current syntax.
$jscs --auto-configure *.js */*.js
Checking index.js against the presets
  Preset    #Errors
1 airbnb    429
2 crockford 62
3 google    414
4 grunt     410
5 jquery    577
6 mdcs      533
7 wikimedia 836
8 yandex    49

ref jscs

:+1: doing some research on what the best approach to bringing some style guidelines into yargs would be.

@agokjr based on advice (and the fact that the codebase was already fairly close to it) I've opted to move yargs to the @feross' standard style guide:
https://github.com/feross/standard
Let me know if this does the trick for you :+1:


Move automated posting to Slack into their own channels

We have a number of automated 'bots' that post to Slack about various topics.
The bots are swamping some channels with automated messages, and hiding any real post.  It is also unnecessary to archive most of the bot postings, so we can choose which are archived.
This issue is to create #<blah>-bots channels and switch the bots to posting on there so the humans have a chance.
Biggest offenders are likely:
#infrastructure where Nagios should be posting to #infrastructure-bot (un-archived), and
#website where Localize should be posting to #website-bot (un-archived).

Also consider renaming #jenkins to show it is a bot channel

Slack channel #infrastructure-bot has been created
Nagios has been reconfigured to post there.

Looking in Slack, I can see that @gdams has added a Localize app, and has defined 1 configuration that posts to #website. I don't have permission to modify or add configurations. Can someone (@gdams, @bblondin) make me an administrator on Slack so I can fix this?

@geraintwjones I've added you to the Jenkins admin group, log out, log back in again and hopefully you'll have the right credentials

Thanks @karianna.

adoptopenjdk-website-production now posts to #website-bot.

@gdams - could you modify the Localize app on Slack please? It needs to point at #website-bot instead of #website.

This appears to be done - closing for now.


WIP: Test IBC Typed Events PR

ref: https://github.com/cosmos/cosmos-sdk/pull/7762

@jackzampolin downgrade this to a draft?


Add GOST style support

Hello,
This PR adds support for GOST file format (compliant with ГОСТ 7.1, ГОСТ 7.80, ГОСТ 7.82) and documentation with examples of GOST formatting

Thank you for the contribution!
I'm afraid that we will not be merging this as work to move to CSL-based bibliographies is in progress.This will enable us to support a whole set of styles very soon.

Hmmm. At least it was fun writing it and I got some experience in Rust.
I will close the Pr.


CHANGES for #1059.

Current coverage is 100% (diff: 100%)

Merging #1073 into master will not change coverage

@@           master   #1073   diff @@
=====================================
  Files          21      21
  Lines        1904    1904
  Methods         0       0
  Messages        0       0
  Branches        0       0
=====================================
  Hits         1904    1904
  Misses          0       0
  Partials        0       0

Powered by Codecov. Last update bb0805d...a208ee5


Add vtkAxesActor serializer?

Should we add a vtkAxesActor serializer that would generate an OrientationMarkerWidget in VTK.js?
For example:

We could but I'm not sure if we have a 1-to-1 mapping here... But if we do, then yes.

Do you think there'd be a better mapping with InteractiveOrientationWidget ? (which I think that example is broken FYI)

Maybe we could have a "dummy" mapping thing which could be ok. I would not make it a widget though. Just a sub-renderer with come axis

Fixed in

vtk.js
trame-vtk

Giving this a try and running into issues with PyVista:
File ~/Software/Pan3D/trame-course/trame-vtk/trame_vtk/modules/vtk/addon_serializer.py:468, in axesActorSerializer(parent, actor, actorId, context, depth)
    457 # sphere_radius = actor.GetSphereRadius()
    458 # sphere_resolution = actor.GetSphereResolution()
    459
   (...)
    465
    466 # Apply transform
    467 matrix = vtkMatrix4x4()
--> 468 actor.GetUserTransform().GetTranspose(matrix)
    469 user_matrix = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
    470 for i in range(4):

AttributeError: 'NoneType' object has no attribute 'GetTranspose'

Here is an example:
"""Validate axes actor serialization."""

import pyvista as pv
import numpy as np
from trame.app import get_server
from trame.ui.vuetify import SinglePageLayout
from trame.widgets import vuetify
from trame.widgets.vtk import VtkLocalView, VtkRemoteView

server = get_server()
state, ctrl = server.state, server.controller

state.trame__title = "Axes Actor Validation"

# -----------------------------------------------------------------------------

mesh = pv.Cone()

plotter = pv.Plotter(off_screen=True)
actor = plotter.add_mesh(mesh)
plotter.add_axes()
plotter.reset_camera()

# -----------------------------------------------------------------------------
# GUI
# -----------------------------------------------------------------------------

with SinglePageLayout(server) as layout:
    layout.icon.click = ctrl.view_reset_camera
    layout.title.set_text(state.trame__title)

    with layout.toolbar:
        vuetify.VSpacer()

    with layout.content:
        with vuetify.VContainer(
            fluid=True,
            classes="pa-0 fill-height",
        ):
            with vuetify.VCol(classes="fill-height"):
                view = VtkLocalView(plotter.ren_win)
                ctrl.view_update = view.update
                ctrl.view_reset_camera = view.reset_camera
            with vuetify.VCol(classes="fill-height"):
                VtkRemoteView(plotter.ren_win)

    # hide footer
    layout.footer.hide()

# -----------------------------------------------------------------------------
# Main
# -----------------------------------------------------------------------------

if __name__ == "__main__":
    server.start()

I tried the following logic to fix it, and while it doesn't error, the axes actor is not visible:
diff --git a/trame_vtk/modules/vtk/addon_serializer.py b/trame_vtk/modules/vtk/addon_serializer.py
index 18f84e3..ec6e907 100644
--- a/trame_vtk/modules/vtk/addon_serializer.py
+++ b/trame_vtk/modules/vtk/addon_serializer.py
@@ -464,13 +464,14 @@ def axesActorSerializer(parent, actor, actorId, context, depth):
     # actor.GetXAxisLabelText()

     # Apply transform
-    matrix = vtkMatrix4x4()
-    actor.GetUserTransform().GetTranspose(matrix)
     user_matrix = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
-    for i in range(4):
-        for j in range(4):
-            idx = i + 4 * j
-            user_matrix[idx] = matrix.GetElement(j, i)
+    if actor.GetUserTransform():
+        matrix = vtkMatrix4x4()
+        actor.GetUserTransform().GetTranspose(matrix)
+        for i in range(4):
+            for j in range(4):
+                idx = i + 4 * j
+                user_matrix[idx] = matrix.GetElement(j, i)

     return {
         "parent": render_window_serializer.getReferenceId(parent),

Make the user_matrix = [1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1] the identity matrix

That is related to the camera custom handling of the local view. We'll just need to figure out a way to describe the link between camera while keeping the server out of the loop. (the point of local rendering)

We'll talk more once I'm back...


Plex Integration

Add the notion of Plex integration, which can be used to support things like:

Supplying the directory to which videos can be stored in (stretch goal)
Automatically triggering a library refresh when a video has been downloaded

Some information regarding library refresh:
From looking at Python client source, the request to refresh libraries in Plex is GET /library/sections/all/refresh?X-Plex-Token=<plex-token>.
The Plex token can be found using this method.


feat(rollup.config): support multiple formats and preserve modules/ split esm bundle files

So this PR fixes how to generate esm and cjs.
This works fine with bun
# COMPILE
npm run build

# RE-WRITE TEST PROGRAMS IF REQUIRED
npm run test:template

# BUILD TEST PROGRAM
npm run build:test

# DO TEST
npm run test

Learn more about the CONTRIBUTING

If we do not use exports property in package.json you do not need merge it

Got it.


Observer not called when calling update on hasMany

Laravel Version: v9.34.0
PHP Version: 8.1.6

Description:
My expectation is that the Observer of a Model is always called when a mutation is done via an Eloquent way.
But this is not calling the Observer of Basket:
$order->baskets()->update(['amount' => 10]);

Result is fine; results are set to an amount of 10, but Observer is not called.
I have to loop through to baskets of the order, and update the amount separately :
foreach ($order->baskets as $basket) {
	$order->update(['amount' => 10]);
}

When I do this, the Observer is called.
Steps to reproduce
Make an Observer.
# EventServiceProvider
protected $observers = [
     Basket::class => [BasketObserver::class],
];

Set this inside the Observer:
public function updating(Basket $basket)
{
	if ($basket->isDirty('amount')) {
		ray('Not called');
	}
}

Update the records in the way shown before.
Is this a bug or a feature?

According to the documentation:
When issuing a mass update via Eloquent, the saving, saved, updating, and updated model events will not be fired for the updated models. This is because the models are never actually retrieved when issuing a mass update.

https://laravel.com/docs/9.x/eloquent#mass-updates
To answer your question, this was a feature since 2016.
You could override the update method by creating a custom builder and still send out the event manually.


Unimplemented system channels

Below are the channels implemented in Android but not implemented in Tizen.
These should be reviewed and implemented.

[x] flutter/accessibility: flutter-tizen/flutter-tizen/issues/151
[x] flutter/deferredcomponent: not applicable
[x] flutter/mousecursor: not generally applicable
[ ] flutter/restoration
[x] flutter/system: not needed (FlutterEngineNotifyLowMemoryWarning)
[ ] flutter/platform: has unimplemented methods

~SystemChrome.setApplicationSwitcherDescription~ (not supported)
SystemChrome.setEnabledSystemUIMode (see https://github.com/flutter-tizen/engine/pull/207)
~SystemChrome.setSystemUIOverlayStyle~  (not supported)

The restoration channel is not applicable for Tizen because there's nothing on Tizen that corresponds to Android's onSaveInstanceState or iOS's encodeRestorableStateWithCoder. (Note: OnPause or OnDestroy has nothing to do with state restoration.)
Brief overview of how restoration works on Android:

The framework always stores the latest copy of restoration data to the platform thread (through the restoration channel).
When onSaveInstanceState is called, it synchronously retrieves the stored data and saves them to a bundle.
When onRestoreInstanceState is called, the restoration data are read from the bundle passed by the platform and RestorationChannel stores them until they are requested by the framework.


@Ajax decorator and json serialization

Currently @ajax and render_to_json uses two different json encoders, one that uses a very old datetime format (the @ajax one) and another one that uses datetime isoformat (which should be fine to meet current javascript date formatting).
Would be great to have all json serialization code to use the same encoder, and even better, use the django one (DjangoJSONEncoder) that's already implemented to handle all non standard data types.

I see. Unifying them is a good idea laid aside for maybe too long.
Switching to DjangoJSONEncoder will be not backwards compatible, but maybe a good idea anyway.

Perhaps the @Ajax decorator could take an encoder in case someone really needs the old encoder. But if I'm not wrong the only difference is the datetime serialization which nowdays anyone should be using ISO format anyways.

Fixed in aa44bc80846892acf3c1e1ddea4de6f4a409dc39. Using DjangoJSONEncoder in both places now. Old behavior is removed. Anyone needing it to open new issue and justify.


Debugger 3

Description
Implementation of a customized debugger for padl transforms.

[ ] Please add unit tests for the debugger

Codecov Report

Merging #339 (cdb268a) into main (7ce07d0) will decrease coverage by 3.27%.
The diff coverage is 51.79%.

@@            Coverage Diff             @@
##             main     #339      +/-   ##
==========================================
- Coverage   89.46%   86.18%   -3.28%
==========================================
  Files          15       15
  Lines        2572     2780     +208
==========================================
+ Hits         2301     2396      +95
- Misses        271      384     +113

Impacted Files
Coverage Δ

padl/utils.py
40.38% <21.91%> (-41.44%)
:arrow_down:

padl/transforms.py
88.32% <63.84%> (-3.78%)
:arrow_down:

padl/__init__.py
100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7ce07d0...cdb268a. Read the comment docs.

I think it would be nice if we could add a short explanation and example of how to use the Debugger in the sphinx docs. Maybe a new md doc under padl/docs/src/usage/?

I also found an edge case when playing around with the debugger.
from padl import transform, pd_debug
t = transform(lambda x: x)
t2 = transform(lamba x: x + 's')

t2.infer_apply(2)
# Returns a TypeError

pd_debug() # works great!

t.infer_apply(x) # x is purposely not defined
# Returns a NameError since x is not defined

pd_debug() # Still points to the trace from t2.infer_apply(2)
debug # debugger from ipython points to the trace from t.infer_apply(x)

I found an unexpected behavior (I think).
t2.infer_apply(2)
pd_debug()
> x # This prints "x" instead of returning a "name x is not defined"
> y # This prints the expected "name y is not defined"

I'm not entirely sure why this happens. Do you know @eramdiaz ?

I see, in the code there are a couple of _pd_trace.clear() statements to prevent not cleaning previous traces in new debugging sessions. In this case you are describing, the pd_trace.clear() inside infer_apply does not work because if x is not defined, then the code inside infer_apply is not even run (I would say that we are not even reaching the padl code), so I don't know how to solve it. In other case where we get into the padl code (for example, if you run again something like `t2.infer_apply(3)' , the trace is cleaned and it works as expected.
Otherwise, I'd recommend to use pd_debug with Pipelines and avoiding using it with AtomicTransforms

Okay I see what you're saying. I think it's fine to leave as is.

@eramdiaz Please merge when you can


disables_back_slot config half working

BeansBackpacks 2.0.2
The disables_back_slot config option only works when doing a specific sequence of actions.
Working case:
If you have a backpack equipped in the backslot, and then try to shift-right click an item into your chest slot, it will stop you by saying "x cannot be worn with x backpack". In this case, the quark backpack and create back tanks.

Broken cases:

If you directly equip the banned item by moving the item into your chestplate slot, it ignores the config option entirely. Right clicking to equip from hotbar or inventory also ignores it completely.
Already wearing the banned item does not block you from putting on a bean backpack.

Awhile ago I changed disable_back_slot to only disable the rendering since it was causing a lot of compatibility issues. I must've missed something that still disables it. I'll double check that everything's working consistently.


Add CacheBuilderSpecSerializer

If a configuration has a default value for a CacheBuilderSpec (e.g. CacheBuilderSpec cacheBuilderSpec = CacheBuilderSpec.disableCaching();) and the value is not overridden in the configuration file, we need to serialize the object before parsing the configuration so that it can be properly instantiated in the Configuration class.

Coverage increased (+0.008%) to 80.966% when pulling 36ec4964effc9928fe3d860a7b741992dbda17b4 on tburch:CacheBuilderSpecSerializer into d95b8593392d2e1b411d9f97a88c013568668cd9 on dropwizard:master.

Coverage increased (+0.008%) to 80.966% when pulling 36ec4964effc9928fe3d860a7b741992dbda17b4 on tburch:CacheBuilderSpecSerializer into d95b8593392d2e1b411d9f97a88c013568668cd9 on dropwizard:master.

LGTM. Thanks! 👍


MSID: 89392 Version: 1 DOI: 2<IP_ADDRESS>4195

"msas": "Ecology"
"msid": "89392"
"preprintDoi": "10.1101/2<IP_ADDRESS>4195"
"Reviewed Preprint posted": "2023-10-06"
"Sent for peer review": "2023-05-31"
"Posted to bioRxiv": "2023-06-08" (link: "Go to bioRxiv": "https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>4195v1/")
[PLACE PDF URL HERE WHEN AVAILABLE] See step 7
Step 1. Inform bioRxiv
Who can help: @QueenKraken, @nlisgo, @scottaubrey

[x] Manuscript is in data hub index (https://data-hub-api.elifesciences.org/enhanced-preprints/docmaps/v2/by-publisher/elife/get-by-manuscript-id?manuscript_id=89392)

Step 2. Create preview of manuscript
Who can help: @fred-atherden, @nlisgo, @scottaubrey

[ ] Manuscript is available for preview (https://prod--epp.elifesciences.org/preview/89392/v1)

[ ] Trigger import for staging: https://prod--epp.elifesciences.org/import
[ ] Sync data folder with s3://prod-elife-epp-data/data
[ ] Trigger import for staging: https://staging--epp.elifesciences.org/import
[ ] Sync data folder with s3://staging-elife-epp-data/data
[ ] Pull request created to enhanced-preprint-data
[ ] Post link to pull request below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-manuscript
Step 3: Awaiting public reviews and QC
Who can help: Production team

[ ] QC passed or QC not needed

Who can help: Editorial team

[ ] Public reviews available on sciety (https://sciety.org/articles/activity/10.1101/2<IP_ADDRESS>4195)
[x] Manuscript and editor details at the top of this issue (Supplied by Editorial team)
[ ] Reviews returned by EPP - https://prod--epp.elifesciences.org/api/reviewed-preprints/89392/v1/reviews

Example
"msas": "Genetics and Genomics", "Neuroscience"
"msid": "84628"
"version": "1"
"preprintDoi": "10.1101/2<IP_ADDRESS>4241"
"articleType": "Reviewed Preprint"
"status": "Published from the original preprint after peer review and assessment by eLife."

"Reviewed Preprint posted": "2023-01-02"
"Sent for peer review": "2022-10-28"
"Posted to bioRxiv": "2022-11-21" (link: "Go to bioRxiv": "https://www.biorxiv.org/content/10.1101/2<IP_ADDRESS>4241v1")

Editors:
Reviewing Editor
Michael B Eisen
University of California, Berkeley, United States
Senior Editor
Michael B Eisen
University of California, Berkeley, United States

Step 4: Modify manuscripts.json (no PDF)

[ ] Reviewed preprint is published to EPP (https://elifesciences.org/reviewed-preprints/89392)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[ ] Request that a doi be registered

[ ] Post link to #enhanced-preprint comment below

Pull request: [PLACE LINK TO PULL REQUEST HERE]
#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]

Instructions to modify manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/publish-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/334/files
Once the pull request is merged in it should be available a few minutes later.

Request that a doi
Post the following in #enhanced-preprint:
@Fred can you register a doi for https://elifesciences.org/reviewed-preprints/89392

Step 5: Awaiting search reindex

[ ] Reviewed preprint is avaliable on journal homepage (https://elifesciences.org)

The search reindex is triggered once an hour. We need the reviewed preprint to be indexed as the search application serves the journal homepage.

Additional info
If needed, the jenkins pipeline to reindex search can be triggered sooner.
https://alfred.elifesciences.org/job/process/job/process-reindex-reviewed-preprints/

Step 6: Published! Request PDF generation

[ ] Post the link to the PDF url at the top of the issue

[ ] Post link to #enhanced-preprint comment below

#enhanced-preprint comment thread: [PLACE LINK TO COMMENT HERE]
Post the following to the #enhanced-preprint on slack:
@Ryan Dix-Peek please can you generate a PDF for https://elifesciences.org/reviewed-preprints/89392

Step 7: Introduce PDF to data folder and git repo

[ ] PDF is avaliable at https://github.com/elifesciences/enhanced-preprints-data/raw/master/data/89392/v1/89392-v1.pdf

Detailed instructions: https://github.com/elifesciences/enhanced-preprints-data#add-a-pdf
Step 8: Add PDF url to manuscripts.json

[ ] Reviewed preprint PDF is available for download (https://prod--epp.elifesciences.org/reviewed-preprints/89392)

[ ] Pull request created to enhanced-preprint-client
[ ] Pull request switched to "Ready for review"
[ ] Post link to pull request below

[PLACE LINK TO PULL REQUEST HERE]

Instructions to add PDF url to manuscripts.json

Visit: https://github.com/elifesciences/enhanced-preprints-client/actions/workflows/add-pdf-url-to-manuscript.yaml
Click: Run workflow
Complete the form and click "Run workflow"
A successful run should result in a new pull request at https://github.com/elifesciences/enhanced-preprints-client/pulls
Open the pull request and click the "Ready for review" button to trigger tests
Once the tests pass and you are happy with the changes the PR can be merged

Example pull request: https://github.com/elifesciences/enhanced-preprints-client/pull/397/files
Once the pull request is merged in it should be available a few minutes later.

Step 9: Done!

[ ] Kettle is on!

Hi @fred-atherden this one isn't showing on Sciety yet

Hi @KJT08 thanks for flagging. It looks like bioRxiv haven't minted the DOI for it yet! I'll reach out to them.
Even though this technically doesn't stop us from publishing the reviewed preprint, it will cause some problems so I think we should hold off until we hear back from them.

bioRxiv have now minted the DOI

@fred-atherden Some of the equations aren't displaying on biorxiv but they seem to be ok in EPP so I'm marking as QC ok!

Sounds good to me - thanks Naushin!


SPA prerender

Does it work with
export const ssr = false;
export const prerender = true;

The exported html files must have title and others into head

I haven't tried but you can try and see