Latency issues with services with more than one replica

Bug Description
I'm setting up Istio in a new cluster and created a basic nginx deployment to test. When the deployment only has one replica, it works perfectly, responding in less than 100ms.
When I add one replica, the response time goes up like crazy, averaging around 10 seconds.
My setup is the following:
# AWS ALB Ingress -> istio-ingressgateway (ClusterIP) -> gateway -> virtualservice -> service -> nginx

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: default
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"

---

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - "example.com"
  gateways:
  - default
  http:
  - route:
    - destination:
        host: nginx

---

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  selector:
    app: nginx
  ports:
    - port: 80
      name: http

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
    version: v1
spec:
  replicas: 3
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      app: nginx
      version: v1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
        - name: nginx
          image: nginx:latest
          ports:
            - containerPort: 80
          resources:
            requests:
              memory: 100Mi
              cpu: 100m
            limits:
              memory: 1500Mi
              cpu: 1000m

Version
$ istioctl version
client version: 1.13.2
control plane version: 1.13.2
data plane version: 1.13.2 (1 proxies)

$ kubectl version --short
Client Version: v1.21.11
Server Version: v1.21.5-eks-bc4871b

Additional Information
No response

Here are the logs from the istio-ingressgateway:
One replica:
[2022-03-31T15:40:03.794Z] "HEAD / HTTP/1.1" 200 - via_upstream - "-" 0 0 1 1 "<IP_ADDRESS>,<IP_ADDRESS>" "curl/7.68.0" "3224f12e-823c-4f1b-9634-21e1b6b5da0c" "example.com" "10.1.108. │
│ 78:80" outbound|80||nginx.my-namespace.svc.cluster.local <IP_ADDRESS>:40780 <IP_ADDRESS>:8080 <IP_ADDRESS>:16526 - -

Multiple replicas:
[2022-03-31T15:41:50.720Z] "HEAD / HTTP/1.1" 200 - via_upstream - "-" 0 0 10018 10017 "<IP_ADDRESS>,<IP_ADDRESS>" "curl/7.68.0" "a3da15be-bae1-4450-ab87-0cb9f5b9fc6f" "example.com" "1 │
│ <IP_ADDRESS>:80" outbound|80||nginx.my-namespace.svc.cluster.local <IP_ADDRESS>:40780 <IP_ADDRESS>:8080 <IP_ADDRESS>:32958 - -

When I scale from one to two replicas, I get these logs in the new pod's istio-proxy container:
2022-03-31T15:41:25.777128Z    info    FLAG: --concurrency="2"
 2022-03-31T15:41:25.777152Z    info    FLAG: --domain="my-namespace.svc.cluster.local"
 2022-03-31T15:41:25.777157Z    info    FLAG: --help="false"
 2022-03-31T15:41:25.777159Z    info    FLAG: --log_as_json="false"
 2022-03-31T15:41:25.777161Z    info    FLAG: --log_caller=""
 2022-03-31T15:41:25.777164Z    info    FLAG: --log_output_level="default:info"
 2022-03-31T15:41:25.777166Z    info    FLAG: --log_rotate=""
 2022-03-31T15:41:25.777168Z    info    FLAG: --log_rotate_max_age="30"
 2022-03-31T15:41:25.777171Z    info    FLAG: --log_rotate_max_backups="1000"
 2022-03-31T15:41:25.777173Z    info    FLAG: --log_rotate_max_size="104857600"
 2022-03-31T15:41:25.777175Z    info    FLAG: --log_stacktrace_level="default:none"
 2022-03-31T15:41:25.777181Z    info    FLAG: --log_target="[stdout]"
 2022-03-31T15:41:25.777184Z    info    FLAG: --meshConfig="./etc/istio/config/mesh"
 2022-03-31T15:41:25.777186Z    info    FLAG: --outlierLogPath=""
 2022-03-31T15:41:25.777188Z    info    FLAG: --proxyComponentLogLevel="misc:error"
 2022-03-31T15:41:25.777190Z    info    FLAG: --proxyLogLevel="warning"
 2022-03-31T15:41:25.777192Z    info    FLAG: --serviceCluster="istio-proxy"
 2022-03-31T15:41:25.777194Z    info    FLAG: --stsPort="0"
 2022-03-31T15:41:25.777196Z    info    FLAG: --templateFile=""
 2022-03-31T15:41:25.777199Z    info    FLAG: --tokenManagerPlugin="GoogleTokenExchange"
 2022-03-31T15:41:25.777202Z    info    FLAG: --vklog="0"
 2022-03-31T15:41:25.777205Z    info    Version 1.13.2-91533d04e894ff86b80acd6d7a4517b144f9e19a-Clean
 2022-03-31T15:41:25.777331Z    info    Proxy role    ips=[<IP_ADDRESS>] type=sidecar id=nginx-5847dcb4dc-qpp5r.my-namespace domain=my-namespace.svc.cluster.local
 2022-03-31T15:41:25.777379Z    info    Apply proxy config from env {}
 2022-03-31T15:41:25.778228Z    info    Effective config: binaryPath: /usr/local/bin/envoy
 concurrency: 2
 configPath: ./etc/istio/proxy
 controlPlaneAuthPolicy: MUTUAL_TLS
 discoveryAddress: istiod.istio-system.svc:15012
 drainDuration: 45s
 parentShutdownDuration: 60s
 proxyAdminPort: 15000
 serviceCluster: istio-proxy
 statNameLength: 189
 statusPort: 15020
 terminationDrainDuration: 5s
 tracing:
   zipkin:
     address: zipkin.istio-system:9411
 2022-03-31T15:41:25.778238Z    info    JWT policy is third-party-jwt
 2022-03-31T15:41:25.782631Z    info    platform detected is AWS
 2022-03-31T15:41:25.786211Z    info    CA Endpoint istiod.istio-system.svc:15012, provider Citadel
 2022-03-31T15:41:25.786241Z    info    Using CA istiod.istio-system.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem
 2022-03-31T15:41:25.786311Z    info    citadelclient    Citadel client using custom root cert: var/run/secrets/istio/root-cert.pem
 2022-03-31T15:41:25.786713Z    info    Opening status port 15020
 2022-03-31T15:41:25.805980Z    info    ads    All caches have been synced up in 31.492343ms, marking server ready
 2022-03-31T15:41:25.806354Z    info    sds    SDS server for workload certificates started, listening on "etc/istio/proxy/SDS"
 2022-03-31T15:41:25.806461Z    info    xdsproxy    Initializing with upstream address "istiod.istio-system.svc:15012" and cluster "Kubernetes"
 2022-03-31T15:41:25.808572Z    info    Pilot SAN: [istiod.istio-system.svc]
 2022-03-31T15:41:25.810553Z    info    Starting proxy agent
 2022-03-31T15:41:25.810687Z    info    Epoch 0 starting
 2022-03-31T15:41:25.810782Z    info    Envoy command: [-c etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --drain-strategy immediate --parent-shutdown-time-s 60 --local-addr
 2022-03-31T15:41:25.826129Z    info    sds    Starting SDS grpc server
 2022-03-31T15:41:25.826421Z    info    starting Http service at <IP_ADDRESS>:15004
 2022-03-31T15:41:45.789602Z    warn    ca    ca request failed, starting attempt 1 in 98.268194ms
 2022-03-31T15:41:45.889013Z    warn    ca    ca request failed, starting attempt 2 in 217.347923ms
 2022-03-31T15:41:46.107074Z    warn    ca    ca request failed, starting attempt 3 in 372.630502ms
 2022-03-31T15:41:46.201156Z    info    xdsproxy    connected to upstream XDS server: istiod.istio-system.svc:15012
 2022-03-31T15:41:46.231759Z    info    ads    ADS: new connection for node:nginx-5847dcb4dc-qpp5r.my-namespace-1
 2022-03-31T15:41:46.273401Z    info    ads    ADS: new connection for node:nginx-5847dcb4dc-qpp5r.my-namespace-2
 2022-03-31T15:41:46.480775Z    warn    ca    ca request failed, starting attempt 4 in 755.358082ms
 2022-03-31T15:41:47.253763Z    info    cache    generated new workload certificate    latency=21.447162935s ttl=23h59m59.746271874s
 2022-03-31T15:41:47.253891Z    info    cache    Root cert has changed, start rotating root cert
 2022-03-31T15:41:47.253982Z    info    ads    XDS: Incremental Pushing:0 ConnectedEndpoints:2 Version:
 2022-03-31T15:41:47.254103Z    info    cache    returned workload certificate from cache    ttl=23h59m59.745902044s
 2022-03-31T15:41:47.254176Z    info    cache    returned workload trust anchor from cache    ttl=23h59m59.745828015s
 2022-03-31T15:41:47.254401Z    info    cache    returned workload trust anchor from cache    ttl=23h59m59.745605754s
 2022-03-31T15:41:47.254799Z    info    ads    SDS: PUSH request for node:nginx-5847dcb4dc-qpp5r.my-namespace resources:1 size:4.0kB resource:default
 2022-03-31T15:41:47.254978Z    info    ads    SDS: PUSH request for node:nginx-5847dcb4dc-qpp5r.my-namespace resources:1 size:1.1kB resource:ROOTCA
 2022-03-31T15:41:47.255142Z    info    cache    returned workload trust anchor from cache    ttl=23h59m59.744863524s
 2022-03-31T15:41:47.255344Z    info    ads    SDS: PUSH for node:nginx-5847dcb4dc-qpp5r.my-namespace resources:1 size:1.1kB resource:ROOTCA
 2022-03-31T15:41:48.124918Z    info    Readiness succeeded in 22.349536684s
 2022-03-31T15:41:48.126538Z    info    Envoy proxy is ready

The response time very clearly drops as soon as the last line appears (Envoy proxy is ready). As soon as I scale back to one replica, it starts responding quickly again.

I turned off retries and am now seeing this in the logs:
[2022-03-31T16:11:52.191Z] "GET / HTTP/1.1" 503 UF upstream_reset_before_response_started{connection_failure} - "-" 0 91 10003 - "<IP_ADDRESS>,<IP_ADDRESS>" "curl/7.68.0" "6fa51be8-1441-4454-8d 1b-a03c93b257dc" "example.com" "<IP_ADDRESS>:80" outbound|80||nginx.my-namespace.svc.cluster.local - <IP_ADDRESS>:8080 <IP_ADDRESS>:21410 - -

I tried this:

Deploy 1 replica, 100% of requests are fast (Pod A)
Scale deployment to 2 replicas, all requests that hit the new replica (Pod B) are slow, Pod A still responds quickly
Delete Pod A, new pod (Pod C) was created. Now Pod C is the one that responds quickly, Pod B still is slow.

I see a lot of warnings when the istio-proxy container is starting, and it's taking a while (around 90s) to be ready. Could this be related?
2022-03-31T16:15:21.454709Z    info    Envoy command: [-c etc/istio/proxy/envoy-rev0.json --restart-epoch 0 --drain-time-s 45 --drain-strategy immediate --parent-shutdown-time-s 60 --local-addr
 ess-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --log-format %Y-%m-%dT%T.%fZ    %l    envoy %n    %v -l warning --component-log-level misc:error --concurrency 2]
 2022-03-31T16:15:21.466266Z    info    starting Http service at <IP_ADDRESS>:15004
 2022-03-31T16:15:21.472691Z    info    sds    Starting SDS grpc server
 2022-03-31T16:15:41.444827Z    warn    ca    ca request failed, starting attempt 1 in 101.719092ms
 2022-03-31T16:15:41.547633Z    warn    ca    ca request failed, starting attempt 2 in 200.361684ms
 2022-03-31T16:15:41.748473Z    warn    ca    ca request failed, starting attempt 3 in 391.163587ms
 2022-03-31T16:15:42.139878Z    warn    ca    ca request failed, starting attempt 4 in 849.443513ms
 2022-03-31T16:16:02.990382Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: des
 c = "transport: Error while dialing dial tcp <IP_ADDRESS>:15012: i/o timeout"
 2022-03-31T16:16:03.625823Z    warn    ca    ca request failed, starting attempt 1 in 92.743892ms
 2022-03-31T16:16:03.719343Z    warn    ca    ca request failed, starting attempt 2 in 193.91154ms
 2022-03-31T16:16:03.913830Z    warn    ca    ca request failed, starting attempt 3 in 418.07389ms
 2022-03-31T16:16:04.332435Z    warn    ca    ca request failed, starting attempt 4 in 735.098697ms
 2022-03-31T16:16:05.092309Z    info    cache    generated new workload certificate    latency=1.637036616s ttl=23h59m59.90770439s
 2022-03-31T16:16:05.092347Z    info    cache    Root cert has changed, start rotating root cert
 2022-03-31T16:16:05.092368Z    info    ads    XDS: Incremental Pushing:0 ConnectedEndpoints:0 Version:
 2022-03-31T16:16:05.092451Z    info    cache    returned workload trust anchor from cache    ttl=23h59m59.907561775s
 2022-03-31T16:16:22.239394Z    warning    envoy config    StreamAggregatedResources gRPC config stream closed since 40s ago: 14, connection error: desc = "transport: Error while dialing dial tcp <IP_ADDRESS>:15012: i/o timeout"
 2022-03-31T16:16:44.213393Z    warning    envoy config    StreamAggregatedResources gRPC config stream closed since 62s ago: 14, connection error: desc = "transport: Error while dialing dial tcp <IP_ADDRESS>:15012: i/o timeout"
 2022-03-31T16:17:04.533165Z    warning    envoy config    StreamAggregatedResources gRPC config stream closed since 82s ago: 14, connection error: desc = "transport: Error while dialing dial tcp <IP_ADDRESS>:15012: i/o timeout"
 2022-03-31T16:17:24.693112Z    warning    envoy config    StreamAggregatedResources gRPC config stream closed since 103s ago: 14, connection error: desc = "transport: Error while dialing dial tcp <IP_ADDRESS>:15012: i/o timeout"
 2022-03-31T16:17:25.305970Z    info    xdsproxy    connected to upstream XDS server: istiod.istio-system.svc:15012
 2022-03-31T16:17:25.343513Z    info    ads    ADS: new connection for node:nginx-59f5d6b84b-r2tbh.my-namespace-1
 2022-03-31T16:17:25.343601Z    info    cache    returned workload certificate from cache    ttl=23h58m39.656405588s
 2022-03-31T16:17:25.343862Z    info    ads    SDS: PUSH request for node:nginx-59f5d6b84b-r2tbh.my-namespace resources:1 size:4.0kB resource:default
 2022-03-31T16:17:25.384680Z    info    ads    ADS: new connection for node:nginx-59f5d6b84b-r2tbh.my-namespace-2
 2022-03-31T16:17:25.384844Z    info    cache    returned workload trust anchor from cache    ttl=23h58m39.615163344s
 2022-03-31T16:17:25.385195Z    info    ads    SDS: PUSH request for node:nginx-59f5d6b84b-r2tbh.my-namespace resources:1 size:1.1kB resource:ROOTCA
 2022-03-31T16:17:25.634148Z    info    Readiness succeeded in 2m4.213907005s
 2022-03-31T16:17:25.634721Z    info    Envoy proxy is ready

Closing this. I figured out that my whole problem was due to misconfigured security group rules for the nodes. I was not allowing node to node traffic, preventing the istio ingress gateway from communicating with pods in other nodes. :facepalm:


Problems with page titles on sidebar

Hi, I'm very enthusiastically trying to use JupyterBook to bundle together a lot my own content. Unfortunately banging my head against some bizarre problems with getting page titles to show up correctly on the sidebar.
I put together a minimal example here:
https://brian-rose.github.io/minimal-book/intro.html
with the source located at
https://github.com/brian-rose/minimal-book
You will see that the sidebar repeats the title "Brian's Homepage" for two pages, despite having different title fields set in toc.yml and setting page_titles: toc in _config.yml
(I get the same result with page_titles: infer)
Any thoughts?

We can close this since #471 ensures that all url: fields will begin with /.
Alternatively, we can keep this open and implement the "auto-add /" option. I don't have strong feelings either way.

Note: in the current pypi version, if all beginning "/" are missing, it ignores the titles and they are all blank. Adding a single "/" fixes all the titles. Very odd.

Ah yeah, it's super annoying. It has been fixed on master but we just haven't released it yet


Build libSkiaSharp.so and libHarfBuzzSharp.so entirely in make

As mentioned by @hurricanehrndz (https://github.com/mono/SkiaSharp/issues/90#issuecomment-307112384), there is not that much needed to build on linux, and the dependency on mono, dotnet and cake is a bit overkill just to run make:
Reasons for current state (https://github.com/mono/SkiaSharp/issues/90#issuecomment-307124367):

I didn't know make or linux at the time of starting
building native skia was a bit herder when I started (few extra steps)

This is the previous linux issue: https://github.com/mono/SkiaSharp/issues/90

@mattleibow,
My only concern is getting libSkiaSharp.so built. I have been able to do that with a minimal make file. I will try and write a small bash script that ensures all the build dependencies are installed and that make is called. I will post something early next week.

Why not just use Skia's GN build file?
GN is a robust solution used in Chromium project as well as some other projects, such as Skia. It's not that complicated to setup even on Windows. It also builds with harfbuzz.

@andrew-boyarshin the reason you may want libSkiaSharp is that it comes with extras, as well as less :)
The basic skia is OK, if you are using C++, but for libSkiaSharp, we have a largish C API, and link out the rest. As a result, you get a tiny little, but comprehensive, package that can do CPU and GPU drawing. In addition, we have some extra APIs that help out with managed wrappers. Finally, we also provide a managed wrapper that can talk to it.

@hurricanehrndz I may have something that might be good... I have just creating a makefile that will pop out the native bits. It is awful, should be taken out back and shot, but it works :)
I WILL be accepting ANY PRs, lol.
I will try get it up ASAP.

@mattleibow I am afraid I still don't see your point or I just misunderstood the goal of this issue.
mono/skia has BUILD.gn (of course with modifications to take winrt platform addition and C source set expansions in account). There is also plenty of skia_use_* to control what is being built in. Why make the Makefile that would duplicate the GN build file just for one platform?
I also see that now Cake is used to call GN and then call Ninja. The original build overhead discussed in #90 is about the usage of Cake and therefore the .NET platform to do that. That can indeed be removed, but in my opinion a premade args.gn is sufficient. Why make? Why even shell script? gclient sync && gn gen out && ninja -C out or sth like that is not that hard but provides better understanding what is done on each step.
The addition of alternative build file would definitely slow down the contribution back to upstream (as I can see from BINDING.md it was planned, or at least a year ago).

@andrew-boyarshin I see what you mean... simple reason - I am no Linux guy, and I barely know what I am supposed to do for this platform. (I just recently learned about sonames and stuff that make Linux tick) If someone can help out then I will gladly accept. There is also a few bits of code in the SkiaSharp repo that handle the API for managed interop.
I just committed my dodgy Makefile. This does all the work and pops out a native binary that can be used: https://github.com/mono/SkiaSharp/tree/master/native-builds/src. I now need to get into what I need to do, and how I need to do it. I am totally a Windows/MSBuild guy, but I am learning :)
The reason to split libSkiaSharp and libHarfBuzz is that I want to keep them separate as (hopefully) I will replace it with ICU for better text support. Skia doesn't do text shaping, and adding harfbuzz to it helps no one (I think it is just for the tests).
So right now, there are 3 Makefiles:

build all Linux: https://github.com/mono/SkiaSharp/blob/linux-make/Makefile
build libSkiaSharp: https://github.com/mono/SkiaSharp/blob/linux-make/native-builds/libSkiaSharp_linux/Makefile
build libHarfBuzzSharp: https://github.com/mono/SkiaSharp/blob/linux-make/native-builds/libHarfBuzzSharp_linux/Makefile

Right now, libSkiaSharp is basically a linked, C interface designed for managed P/Invoke. As skia does not support text shaping, this has no shaping engines (such as harfbuzz or icu). libHarfBuzzSharp is a "temporary" text shaping engine that can do basic shaping. It is helpful for many simple cases (a single direction and typeface) I want to replace this soonish with icu, which is a much fuller system that can to multi-direction, multi-typeface, wrapping, etc.
I need to spend some time to understand how gn works and to create a build for Linux, and maybe get this to produce the binaries for all the platforms. This way we don't need to spin up cake just to call ninja.

This bit is how I start make from cake:
https://github.com/mono/SkiaSharp/blob/linux-make/cake/BuildExternals.cake#L720-L732
    RunProcess ("make", new ProcessSettings {
        Arguments = "externals-linux" +
            " SKIA_PATH=" + SKIA_PATH.FullPath +
            " DEPOT_PATH=" + DEPOT_PATH.FullPath +
            " HARFBUZZ_PATH=" + HARFBUZZ_PATH.FullPath +
            " NATIVEBUILDS_PATH=" + ROOT_PATH.Combine("native-builds").FullPath +
            " ARCH=" + arch +
            " SKIA_VERSION=" + VERSION_FILE +
            " HARFBUZZ_VERSION=" + HARFBUZZ_VERSION_FILE +
            " HARFBUZZ_VERSION_SOURCE=" + HARFBUZZ_VERSION_SOURCE +
            " SUPPORT_GPU=" + SUPPORT_GPU,
        WorkingDirectory = ROOT_PATH.FullPath,
    });

Also want to mention that I currently have to store some of the C/C++ code in the SkiaSharp repo as that will never be merged into upstream skia. I have been trying to upstream the C API - it doesn't do anything extra, just exposes still to C, but Google is not that interested in this at the moment. Basically, what gets into upstream is what Android/Chrome need. The C API is more of a nice to have, but they haven't gotten round to it yet.
We can, or course, have our own BUILD.gn in SkiaSharp and that pulls in the skia bits - sort of what Google is doing to bull in third party libraries. So it is totally feasible, but I think that we have to keep in mind that all changes in the mono/skia repo need to be, at least considering, possible to upstream. So, the SkManagedStream will never get upstreamed, but the C API for SkStream will.
Hope this makes things a bit clearer...

@kekekeks @galvesribeiro I thought I would add you in case you were interested... You seem to have done some Linux work before 😉
This is just a Makefile (https://github.com/mono/SkiaSharp/issues/323) that should pop out a native file - it really just download/syncs the native code and then calls the individual Makefiles. I am not sure exactly how you do your own builds, but maybe you can comment. Obviously, I am the worst Linux/make guy, but hopefully it is useful in some way :skeptical-emoji:

Thanks @mattleibow! I agree with @andrew-boyarshin that if could use GN as it is the standard tooling from Google it would be great.
However, as you know (and spent days and nights helping me on that), Google tooling is great if you are doing what they want, the way they want and to use how they want. If you need anything different than that, you are doomed (as we were).
In my case, we weren't able to build Skia for ARM Linux just because their build config files (by that time, gyp files) were so tricky and hardcoded that we weren't able to make it build for ARM. If you remember Matthew, we had to hack several gyp files in order to make it build properly and I could finally use it on our ARM device.
So, if using a makefile will enable more scenarios to run SkiaSharp, I'm all up for it! What I don't like is to be caged by Google tooling (believe me, Skia is not my first adventure with Google's code) and have 0 flexibility.
I'll have a look on the new changes over the weekend.
Thank you Matthew!

Any comments/suggestions will be very welcome!

@mattleibow
ICU LayoutEngine is deprecated and has been removed since ICU 58. ICU encourages users to switch to HarfBuzz which is now more robust and bug-free text shaping engine. So, probably no point in trying ICU.

@galvesribeiro GN buildconfigs for Skia seem too small and inflexible compared to Chromium ones. I was negatively surprised by the lack of real VC tools and Windows SDK search in Skia configs. GCC and MSVC toolchains in one main BUILD.gn file. Well, no wonder it is so given GN was built specifically to suit Chromium needs and Skia only adopted it later for the sake of ecosystem.

@andrew-boyarshin the ICU LayoutEngine is deprecated in favour of HarfBuzz as a shaping engine, but not the rest. For example, HarfBuzz cannot handle mixed cultures or typefaces. ICU will first process the text, split it into text chunks and the HarfBuzz will actually do the layout/shaping

The problem with ICU is that it's huge (~30MB) so it's super inconvenient to ship as a part of the app. Is it possible to somehow build only "text shaping bits"? Pango, for example, doesn't seem to have a dependency on ICU and only takes around 1MB of disk space.

http://apps.icu-project.org/datacustom/ - seems to provide a way to exclude unneeded data

@mattleibow I might not have put emphasis correctly in my comment. ICU LayoutEngine was deprecated and removed, all of it. LayoutEngine itself used to contain the same set of features as HarfBuzz (as far as I can tell from docs, sources, samples, and ICU-LE-HB compatibility layer). LE also itself couldn't handle mixed font properties and scripts, but there was mostly complete sample with implementation of text split by script algo (so not actually part of ICU).
Of course the whole power of the rest of the ICU remains (break iterators, bidi algorithm and so on) and can be used, it's just that HarfBuzz now has little alternative as a text shaping library (only Graphite, but its a whole different ecosystem).

Just adding this comment from another issue:

This is all extra work, and I am not that up to standard with make. Obviously we accepts PRs and help. Most of the time, you should be able to install the NuGet from nuget.org, and then just include your own native build of the libSkiaSharp.so for whatever distro/architecture you are running.

This is no longer relevant as the build is now done with GN and ninja directly: https://github.com/mono/SkiaSharp/wiki/Building-on-Linux


Implement OSV format

I think we should work toward the goal of supporting multiple formats including OSV: https://ossf.github.io/osv-schema/
For the API we should probably rename useStandardFormat to useFormat. This new property take a string with possible enumeration: OSV or Standard
const vulnerabilities = await definition.getVulnerabilities(process.cwd(), {
  useFormat: "OSV"
});

I take


PE-4896: fix(drag and drop)

fixes drag and drop getting the wrong content type.
--- Releases ---
Android release: https://appdistribution.firebase.google.com/testerapps/1:305132849030:android:6cf0cd5ec064fad3ffce07/releases/0i7a84pggbof0

Merged here: https://github.com/ardriveapp/ardrive-web/pull/1453


Create 0000-01-02-asanikamu.md

i  created a branch, created a file and made a commit, and opened a pull request

I will replace file 5 and commit changes


Evaluation metrics on CORe50

Is test accuracy obtained on the remaining 3 sessions (#3，#7 and #10) after completing 8 sessions training sequentially? and is the average of 10 runs calculated?

Yes, your understanding should be correct.


mx.random.seed does not give deterministic results on multi-cpus

Description
I am trying to fix the random seed across multiple cpus and followed the example to pass ctx here : https://mxnet.apache.org/versions/1.6/api/python/docs/api/mxnet/random/index.html and the example code works but it seems order dependent.  See the output below.  If I call mx.random.seed(128, ctx=mx.cpu(0)) and mx.random.seed(128, ctx=mx.cpu(1)), it does not work.  Also is this the only way to loop over all the num_cpus and set each ctx?
To Reproduce
mx.random.seed(128, ctx=mx.cpu(0))
print(mx.nd.random.normal(shape=(2,2), ctx=mx.cpu(0)).asnumpy())
[[ 0.47400656  0.20251541]
 [ 1.3648157  -1.4962182 ]]
mx.random.seed(128, ctx=mx.cpu(1))
print(mx.nd.random.normal(shape=(2,2), ctx=mx.cpu(1)).asnumpy())
[[ 0.47400656  0.20251541]
 [ 1.3648157  -1.4962182 ]]
mx.random.seed(128, ctx=mx.cpu(0))
mx.random.seed(128, ctx=mx.cpu(1))
print(mx.nd.random.normal(shape=(2,2), ctx=mx.cpu(0)).asnumpy())
[[ 0.47400656  0.20251541]
 [ 1.3648157  -1.4962182 ]]
print(mx.nd.random.normal(shape=(2,2), ctx=mx.cpu(1)).asnumpy())
[[ 1.0954498  -0.20808885]
 [ 1.590508   -0.41777727]]

The above arrays do not match and I am not sure why because the seed is set.  We also see this when trying to use multi-processing in our algorithms and the numbers are very different even for fixed seed.  Above is a simple representative example.

mx.cpu(0) is not really different from mx.cpu(1). There is only one random resource for CPUs, whereas there is one per device-id for GPUs:
https://github.com/apache/incubator-mxnet/blob/6bbd53107aa16fc41e8d462cf5dc46fb70d592df/src/resource.cc#L394-L402

You may see also the issue: https://github.com/apache/incubator-mxnet/issues/15662

mx.cpu(0) is not really different from mx.cpu(1). There is only one random resource for CPUs, whereas there is one per device-id for GPUs:
https://github.com/apache/incubator-mxnet/blob/6bbd53107aa16fc41e8d462cf5dc46fb70d592df/src/resource.cc#L394-L402

Why is the result different than on different cpus? The seed does not seem to be fixed in my example.  Thank you!

Why is the result different than on different cpus? The seed does not seem to be fixed in my example. Thank you!

You are running  two random operations in sequence. The first operation uses the fixed seed but modifies the internal state of the random resource.

However resetting the seed of cpu(1) should give us deterministic results at CPU(1). The current behavior is not expected.

It's not expected and should be fixed. I'd just like to point out the root cause.

Is there any updates to this problem? It's kind of a big deal because models are producing slightly different results


Added specs for layer new model comparing to old one

@rafatower CR please (it's quick)

Frontend tests were OK :+1: (details)

:+1:


feat: Add support for complex objects as custom metadata

Codecov Report
Attention: 7 lines in your changes are missing coverage. Please review.

Files
Coverage Δ

Sources/Trap/Datasource/Metadata.swift
0.00% <0.00%> (ø)

Sources/Trap/Manager/Manager.swift
81.69% <0.00%> (-1.11%)
:arrow_down:

:loudspeaker: Thoughts on this report? Let us know!


Fix completion for variables initialized by get_node call

This will mostly fix https://github.com/godotengine/godot/issues/64477, the only thing I did not fix is the variables from the base class (completion does not have access to it, and I don't know how to do it yet):

Thanks!


Create a wiki

:smile:

@medensqui For what? For API docs I made a GitHub pages repo that makes a prettyful website.

@thatoddmailbox True. idk


HL-422: include calculation row id in API response

Description :sparkles:
Add id field to the calculation row element in API for easier processing in the frontend
Issues :bug:  HL-422
Testing :alembic:
backend/benefit/calculator/tests/test_calculator_api.py:test_application_retrieve_calculation_rows_hl422
Screenshots :camera_flash:
Additional notes :spiral_notepad:

Codecov Report

Merging #727 (b4e1277) into develop (42e598f) will increase coverage by 31.55%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##           develop     #727       +/-   ##
============================================
+ Coverage    63.30%   94.85%   +31.55%
============================================
  Files          776      168      -608
  Lines        19393     6901    -12492
  Branches      1878        0     -1878
============================================
- Hits         12276     6546     -5730
+ Misses        6754      355     -6399
+ Partials       363        0      -363

Impacted Files
Coverage Δ

backend/benefit/calculator/api/v1/serializers.py
96.25% <ø> (ø)

...nd/benefit/calculator/tests/test_calculator_api.py
98.93% <100.00%> (+0.04%)
:arrow_up:

backend/shared/shared/audit_log/viewsets.py
46.96% <0.00%> (-43.94%)
:arrow_down:

backend/shared/shared/azure_adfs/views.py
41.17% <0.00%> (-41.18%)
:arrow_down:

backend/shared/shared/audit_log/audit_logging.py
76.59% <0.00%> (-21.28%)
:arrow_down:

backend/shared/shared/common/tests/conftest.py
87.87% <0.00%> (-12.13%)
:arrow_down:

backend/shared/shared/oidc/auth.py
81.48% <0.00%> (-1.86%)
:arrow_down:

frontend/kesaseteli/employer/src/pages/500.tsx

...fit/handler/src/hooks/useUpdateApplicationQuery.ts

...eps/step2/accordions/EmploymentAccordionHeader.tsx

... and 605 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 42e598f...b4e1277. Read the comment docs.


Add Description for post-build-script.py

Hi, a useful note on the purpose of the file, and any relevant software installs that are needed would be excellent.  I have discovered that any software dependancies that are missing cause the build to fail.

I have updated the post-build script so that it should only run clang-format if it can find it. Please let me know if that works.

And there is a note in the main readme.md file now.

Hi, think i need to check python requirements on my machine.  I get a python failure on the POpen command (i am running 3.9).

It is 'Popen' rather than 'POpen'. It should be supportd from Python 3.5 I think.

i got the advanced wall sensor board and it seems like it doesn't work with the code where should i update and how?
thank u

Hi,
The board should work with the code.  I use the advanced board with the base code with no issues.  Do you have access to an oscilloscope?
Try the following things (in no particular order)
Check the solder joints for dry joints, and re-solder those that don’t look “shiny”
Check the orientation of the emitters / photo transistors.  If either are not the correct way, you won’t get any readings on the outputs
I can’t remember which way round (currently on a boat leaving Sardinia), but one has a very tight spectral response, so you may need to tweak the alignment of the sensors
Check that the transistors switching the emitters are the correct way around as well.
Thanks,
James
On 28 Sep 2024, at 16:39, SITHILMANETH @.***> wrote:
i got the advanced wall sensor board and it seems like it doesn't work with the code where should i update and how?
thank u
—
Reply to this email directly, view it on GitHubhttps://github.com/ukmars/ukmarsbot-mazerunner/issues/11#issuecomment-2380661056, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AOEEW2AWHOFIIOHINDDVI63ZY25SJAVCNFSM6AAAAABPAVZQ6OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGOBQGY3DCMBVGY.
You are receiving this because you authored the thread.Message ID: @.***>


Broken in 6.3 (Debian 10)

Seems like 6.3 broke it again. Here is the terminal output
kyral@XIVAlexVM:/usr/local/bin$ sudo /usr/bin/curl https://raw.githubusercontent.com/Soreepeong/XivMitmLatencyMitigator/main/mitigate.py | sudo /usr/bin/python3
% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload  Upload   Total   Spent    Left  Speed
100 68657  100 68657    0     0   145k      0 --:--:-- --:--:-- --:--:--  145k
2023-01-10 13:30:06,714 714(main)       INFO    Region filter: (None)
2023-01-10 13:30:06,715 714(main)       INFO    Extra delay: 0.075s
2023-01-10 13:30:06,715 714(main)       INFO    Use measured socket latency: no
Base: 0xf3f0a000
Oodle encode test: 256 -> 100
Oodle helper running: state=2eb400 shared=400018 window=8000
2023-01-10 13:30:10,532 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1:65535 -d <IP_ADDRESS>/16 -j REDIRECT --to 53295
2023-01-10 13:30:10,544 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1:65535 -d <IP_ADDRESS>/16 -j REDIRECT --to 53295
2023-01-10 13:30:10,550 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1025:65535 -d <IP_ADDRESS>/24 -j REDIRECT --to 53295
2023-01-10 13:30:10,555 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1:65535 -d <IP_ADDRESS>/16 -j REDIRECT --to 53295
2023-01-10 13:30:10,560 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1025:65535 -d <IP_ADDRESS>/24 -j REDIRECT --to 53295
2023-01-10 13:30:10,565 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1025:65535 -d <IP_ADDRESS>/24 -j REDIRECT --to 53295
2023-01-10 13:30:10,571 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1025:65535 -d <IP_ADDRESS>/24 -j REDIRECT --to 53295
2023-01-10 13:30:10,576 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1025:65535 -d <IP_ADDRESS>/24 -j REDIRECT --to 53295
2023-01-10 13:30:10,581 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1:65535 -d <IP_ADDRESS>/16 -j REDIRECT --to 53295
2023-01-10 13:30:10,586 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1025:65535 -d <IP_ADDRESS>/24 -j REDIRECT --to 53295
2023-01-10 13:30:10,592 714(main)       INFO    Running: iptables -t nat -I PREROUTING -p tcp -m multiport --dports 1025:65535 -d <IP_ADDRESS>/24 -j REDIRECT --to 53295
net.ipv4.ip_forward = 1
2023-01-10 13:30:10,602 714(main)       INFO    Listening on ('<IP_ADDRESS>', 53295)...
2023-01-10 13:30:10,602 714(main)       INFO    Press Ctrl+C to quit.
2023-01-10 13:30:14,247 748     INFO    Log will be saved to /tmp/xmlm.20230110133014.748.log
2023-01-10 13:30:14,249 748     INFO    New[game.JP.2<IP_ADDRESS>0.0000.json] ('<IP_ADDRESS>', 53295) ('<IP_ADDRESS>', 63412) ('<IP_ADDRESS>', 54994)
2023-01-10 13:30:55,928 750     INFO    Log will be saved to /tmp/xmlm.20230110133055.750.log
2023-01-10 13:30:55,934 750     INFO    New[game.JP.2<IP_ADDRESS>0.0000.json] ('<IP_ADDRESS>', 53295) ('<IP_ADDRESS>', 61899) ('<IP_ADDRESS>', 55023)
2023-01-10 13:30:55,932 751     INFO    Log will be saved to /tmp/xmlm.20230110133055.751.log
2023-01-10 13:30:55,936 748     INFO    D->U Read finish
2023-01-10 13:30:55,936 748     INFO    D->U Source read and target write shutdown
2023-01-10 13:30:55,937 751     INFO    New[game.JP.2<IP_ADDRESS>0.0000.json] ('<IP_ADDRESS>', 53295) ('<IP_ADDRESS>', 59889) ('<IP_ADDRESS>', 55023)
2023-01-10 13:30:56,027 748     INFO    U->D Read finish
2023-01-10 13:30:56,028 748     INFO    U->D Source read and target write shutdown
2023-01-10 13:30:56,028 748     INFO    Closed
2023-01-10 13:30:56,102 714(main)       INFO    [748   ] has exit with status code 0.
Base: 0xf3eb6000
Base: 0xf3f4e000
Oodle encode test: 256 -> 100
Oodle helper running: state=2eb400 shared=400018 window=8000
Oodle encode test: 256 -> 100
Oodle helper running: state=2eb400 shared=400018 window=8000
2023-01-10 13:30:57,622 751     INFO    D->U discarded 00
2023-01-10 13:30:57,622 750     INFO    D->U discarded 00
2023-01-10 13:31:56,296 750     INFO    D->U Read finish
2023-01-10 13:31:56,297 751     INFO    D->U Read finish
2023-01-10 13:31:56,307 751     INFO    D->U Source read and target write shutdown
2023-01-10 13:31:56,308 750     INFO    D->U Source read and target write shutdown
2023-01-10 13:31:56,413 750     INFO    U->D Read finish
2023-01-10 13:31:56,414 750     INFO    U->D Source read and target write shutdown
2023-01-10 13:31:56,414 751     INFO    U->D Read finish
2023-01-10 13:31:56,415 751     INFO    U->D Source read and target write shutdown
2023-01-10 13:31:56,426 750     INFO    Closed
2023-01-10 13:31:56,427 751     INFO    Closed
2023-01-10 13:31:58,956 714(main)       INFO    [750   ] has exit with status code 0.

Tried running with -u to  update opscode files, still no dice.

This is due to ff14 changing from Oodle UDP functions to Oodle TCP functions.
If you update the exe file as well you get this error instead:
Base: 0xf3d05000
Could not find signature
Base: 0xf3c9c000
Could not find signature
Oodle test fail

Ah so I need to update the EXE and wait for a fix to the daemon as well?

XivAlexander is also broken as well. Need to wait for a fix.

I believe I understand what architectural changes need to be made to fix XivMitmLatencyMitigator, all caused by the Oodle TCP implementation being stateful -- but I'm not too familiar with the code to be able to implement it easily:

Maintain 3 oodle_helper instances (or, 3 separately addressable oodle TCP_State instances hosted in the same process):

One for decoding Server-to-Client packets
One for decoding Client-to-Server packets
One for re-encoding Server-to-Client packets

Use a new oodle_helper instance per ff14 connection

A single instance cannot be reused, as the state must be reset at the start of a fresh connection.

Intrusively re-encode all packets going from Server->Client

This must be done since once a modified packet is sent to the client, the client's decoding state will diverge, and the original unmodified packets from the server will not decode correctly

@Soreepeong

I would like to know if at least they will try to fix it

Well...Just time to w8

lets just hope the developer is alive and doing well :)

The emeregency maintenance has rolled back the biggest compression change, now the script should be easily repairable.

soo just opcodes change is enough? or we still need to wait for the dev to debug the script?

repairable

Would this mean that we would have to get a copy of the new ffxiv.exe?

repairable

Would this mean that we would have to get a copy of the new ffxiv.exe?

I already tried new ffxiv.exe but it create new problem with oodle. For script to work, it still need to use old ffxiv.exe.
For now I'm still waiting for today's Opcodes before test it again with old ffxiv.exe.

By "old" you mean "before 6.3“?

By "old" you mean "before 6.3“?

Yes, the one before 6.3

repairable

Would this mean that we would have to get a copy of the new ffxiv.exe?

I already tried new ffxiv.exe but it create new problem with oodle. For script to work, it still need to use old ffxiv.exe.
For now I'm still waiting for today's Opcodes before test it again with old ffxiv.exe.

The trick of using an old EXE may work, but you may also have to modify two lines in the script as well:
- int htbits = 19;
+ int htbits = 17;

- std::vector<uint8_t> window(0x8000);
+ std::vector<uint8_t> window(0x100000);

someone mind sharing the new opcodes please :D

I just tested with the old .exe and it is running without any error. just need the new opcodes now.

Im getting this after modifying the code like @syldrathecat suggested

With the old code, i get this

Will the change be necessary?

Im getting this after modifying the code like @syldrathecat suggested
With the old code, i get this
Will the change be necessary?

I'm not sure if the change is neccessary, but it seems to work for me with the changes (didn't try without them) -- I am just waiting for someone to spoonfeed me new opcodes.

i only know how to get the latest opcodes using xivalexander but it's broken. Please if anyone knows any other way to retrieve the latest opcodes we would be extremely grateful

p.s. the output indicates that you didn't make the changes since it should say window=100000

I triple cheecked it and i infact did the change, he just refuses to run it i guess? I have no duplicate file either so i dunno what might be happening

PEPEGA

    "C2S_ActionRequest": "0x0249", // old: 0x00e1
    "C2S_ActionRequestGroundTargeted": "0x03dc", // old: 0x0143
    "S2C_ActionEffect01": "0x01c9", // old: 0x01e6
    "S2C_ActionEffect08": "0x024a", // old: 0x031b
    "S2C_ActionEffect16": "0x038a", // old: 0x03be
    "S2C_ActionEffect24": "0x00c8", // old: 0x02c4
    "S2C_ActionEffect32": "0x032b", // old: 0x0214
    "S2C_ActorCast": "0x029c", // old: 0x0b0
    "S2C_ActorControl": "0x0179", // old: 0x131
    "S2C_ActorControlSelf": "0x026f", // old: 0x1a8

new opcodes after 6.3 hotfix -- working with a 6.2 ffxiv.exe and the two lines of code from https://github.com/Soreepeong/XivMitmLatencyMitigator/issues/73#issuecomment-1381865654 changed

working with a 6.2 ffxiv.exe

I now regret symlinking my ffxix.exe instead of making a manual copy..

working with a 6.2 ffxiv.exe

I now regret symlinking my ffxix.exe instead of making a manual copy..

owo whats this https://mega.nz/file/gvgwTaCA#vh7uYwxvrxVSi3o2uDdNvwAcNiyLob-Z5nfunwjdZ3I

Even ruunning a local file it still downloading stuff from the repo, any help?

Even ruunning a local file it still downloading stuff from the repo, any help?

If the file is modified in the last hour it will not be replaced

lol... how do you even edit the script on linux when its loaded directly from https://raw.githubusercontent.com/Soreepeong/XivMitmLatencyMitigator/main/mitigate.py

I just used wget to download it and ran it directly

Instead of using curl <script-url>, you pipe it into a file to save it locally or you use wget <script-url> doing the same thing. Then you use python3 <script-file> to run it

Modify line 1479 to
if os.path.getmtime("definitions.json") > time.time():

I'm getting an interesting error with the updated definitions.json.
2023-01-13 12:50:09,793 4200(main)      INFO    Failed to read previous opcode definition files: string indices must be integers
Here is my definitions.json:
{ 	"C2S_ActionRequest": "0x0249", 	"C2S_ActionRequestGroundTargeted": "0x03dc", 	"S2C_ActionEffect01": "0x01c9", 	"S2C_ActionEffect08": "0x024a", 	"S2C_ActionEffect16": "0x038a", 	"S2C_ActionEffect24": "0x00c8", 	"S2C_ActionEffect32": "0x032b", 	"S2C_ActorCast": "0x029c", 	"S2C_ActorControl": "0x0179", 	"S2C_ActorControlSelf": "0x026f", 	"Server_IpRange": "<IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/24", 	"Server_PortRange": "1025-65535" }

I'm getting an interesting error with the updated definitions.json.
2023-01-13 12:50:09,793 4200(main) INFO Failed to read previous opcode definition files: string indices must be integers
Here is my definitions.json:
{ "C2S_ActionRequest": "0x0249", "C2S_ActionRequestGroundTargeted": "0x03dc", "S2C_ActionEffect01": "0x01c9", "S2C_ActionEffect08": "0x024a", "S2C_ActionEffect16": "0x038a", "S2C_ActionEffect24": "0x00c8", "S2C_ActionEffect32": "0x032b", "S2C_ActorCast": "0x029c", "S2C_ActorControl": "0x0179", "S2C_ActorControlSelf": "0x026f", "Server_IpRange": "<IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/24, <IP_ADDRESS>/24", "Server_PortRange": "1025-65535" }

The entire content of the file shouldn't be replaced, just the relevant JP opcodes.

thank you @syldrathecat for the efforts. its working again now on my server. time to go brrt brrt...

They were provided by Flawed on the Sapphire discord server

very resourceful. thanks! i was gonna go sleep but then tried to sneak for updates.

yah thats what I tried to do, used the opscode def in the main repo as a base. Thanks @syldrathecat, it works now with the thing you posted. BTW, there is a typo. The first "S2C_ActionEffect32" says "S2C_ActionEffect35" :)

yah thats what I tried to do, used the opscode def in the main repo as a base. Thanks @syldrathecat, it works now with the thing you posted. BTW, there is a typo. The first "S2C_ActionEffect32" says "S2C_ActionEffect35" :)

oof sorry that was a really bad copy-paste error lol

Daemon is up and running. I'll try it out after work (been doing this on my lunch break :D )

Just wrapped everything up in a Docker container. If anyone prefer Docker way, feel free to use it. https://hub.docker.com/r/bankja/xivlm
Modified version of mitigate.py, updated definitions.json and 6.2 EXE can also be found at https://github.com/bankjaneo/XivMitmDocker/tree/main/app

Just wrapped everything up in a Docker container. If anyone prefer Docker way, feel free to use it. https://hub.docker.com/r/bankja/xivlm
Modified version of mitigate.py, updated definitions.json and 6.2 EXE can also be found at https://github.com/bankjaneo/XivMitmDocker/tree/main/app

God bless you, thank you SO MUCH

I got this error but everything is working just fine, should i be worried about it?

I have no idea about this error too. Just start seeing this after modify mitigator.py as suggested. However, it's working find and I think it can be ignored.

Just wrapped everything up in a Docker container. If anyone prefer Docker way, feel free to use it. https://hub.docker.com/r/bankja/xivlm
Modified version of mitigate.py, updated definitions.json and 6.2 EXE can also be found at https://github.com/bankjaneo/XivMitmDocker/tree/main/app

God bless you, thank you SO MUCH

I got this error but everything is working just fine, should i be worried about it?

Oh yes, I also got this, but it seemed to not affect anything.

@Soreepeong sorry about write this just now. But thanks so much for this awesome work so far. Fort sure XIVAlexander helped a lot of players.
I wish you are healthy and happy and even if you decide to not work on XIVAlexander anymore, remember that you helped a lot of people.
@bankjaneo Thanks for all you effort to help. Good to see that are people trying to help instead of complain.

Ported the change in XivAlexander to the script so that it works with the latest ffxiv.exe:
https://raw.githubusercontent.com/xzn/XivMitmLatencyMitigator/472533cceaf6b7cccb389ddb931163cfde952ee2/mitigate.py
Unsure about the malloc/free function signatures and if they need to be changed but that's outside my skill level (don't know how to re). So far it seems to work.
On a related note anyone know if network namespace and network bridge can be used to make this script work on the same computer without vm or docker? Would be helpful on Deck.

This is great. Thanks for your contribute. I've just update my container to curl your script and use it with 6.3 EXE.
https://github.com/bankjaneo/XivMitmDocker
Cheers and happy raiding!

@xzn It would be nice if you could also test and port the proposed changes in this issue as well. Specifically they would apply to L830 and L1592. They would address the broken filename for cleanup.sh and oodle helper placed in / instead of the working directory. You can reproduce these bugs by piping the script.
cls.oodle_helper_path = os.path.join(os.getcwd(), "oodle_helper")
cleanup_filename = os.path.join(os.getcwd(), ".cleanup.sh")

Thank you so much for fixing it @bankjaneo @xzn
Lets hope Square dont fk it up that bad again

Its working now!!
Thank you @bankjaneo @steponmepls
Your work is really appreciated <3

@xzn if you've got a fix, could you please make a PR so others can pull the branch and @Soreepeong has an easier time updating? Linking to an arbitrary commit in a fork isn't really sustainable.
As an aside, no clue where this rumour of @Soreepeong abandoning the project came from but they've been active in discord and said they just haven't had a chance to download the patch. The update for xivalexander was done by someone else too.

There has been a sort of "discussion" in an issue on the main standalone xivalexander repo and a bunch of users complained about not receiving any reply on github ever or even going as far as checking @Soreepeong commits frequency to bring validity to their speculation. Not being in any discord or having a chance to directly talk to him, these users assumed the worst and here we are with people looking for workarounds and hotfixes from any source.
Just like other people I had the chance of using @xzn fork and it works but it still suffers from some errors. They don't appear to affect the game but anyone having the chance to look at the console can see something weird is going on so I don't know if it's worth making a pull request in its current state:
Jan 19 04:09:55 thinkcentre python3[9122]: Oodle encode test: 256 -> 100
Jan 19 04:09:55 thinkcentre python3[9122]: Oodle helper running: state=2eb400 shared=400018 window=8000
Jan 19 04:09:58 thinkcentre python3[9120]: 2023-01-19 04:09:58,240        9120        ERROR        unknown error Buffer size too small (48 instead of at least 52 bytes) occurred in downstream handler; skipping
Jan 19 04:09:58 thinkcentre python3[9120]: Traceback (most recent call last):
Jan 19 04:09:58 thinkcentre python3[9120]:   File "/home/user/.local/bin/xivalexander/mitigate.py", line 1179, in to_downstream
Jan 19 04:09:58 thinkcentre python3[9120]:     effect = XivMessageIpcActionEffect.from_buffer(message_data, ctypes.sizeof(ipc))
Jan 19 04:09:58 thinkcentre python3[9120]: ValueError: Buffer size too small (48 instead of at least 52 bytes)
Jan 19 04:10:09 thinkcentre python3[9119]: 2023-01-19 04:10:09,696        9119        INFO        C2S_ActionRequest: actionId=0005 sequence=0001
Jan 19 04:10:09 thinkcentre python3[9119]: 2023-01-19 04:10:09,775        9119        INFO        S2C_ActorCast: actionId=0005 type=0001 action_id_2=0005 time=5.000 target_id=10750327

Jan 19 04:15:52 thinkcentre python3[9119]: 2023-01-19 04:15:52,440        9119        INFO        U->D Read finish
Jan 19 04:15:52 thinkcentre python3[9119]: 2023-01-19 04:15:52,440        9119        INFO        U->D Source read and target write shutdown
Jan 19 04:15:52 thinkcentre python3[9119]: 2023-01-19 04:15:52,440        9119        INFO        Closed
Jan 19 04:15:52 thinkcentre python3[691]: 2023-01-19 04:15:52,674        691(main)        INFO        [8729  ] has exit with status code 0.

Jan 19 04:15:52 thinkcentre python3[691]: Message: '[8729  ] has exit with status code 0.'
Jan 19 04:15:52 thinkcentre python3[691]: Arguments: ()

@xzn if you've got a fix, could you please make a PR so others can pull the branch and @Soreepeong has an easier time updating? Linking to an arbitrary commit in a fork isn't really sustainable.
As an aside, no clue where this rumour of @Soreepeong abandoning the project came from but they've been active in discord and said they just haven't had a chance to download the patch. The update for xivalexander was done by someone else too.

He is just not answering on issues for months now, since theres no other way to talk with him, ppl just asssumed he abandoned the project.

@xzn Unsure if related only to the forked version but changing region from NA->EU or EU->NA crashes the script.
What I did to reproduce was:

Connect to a DC and login into a world
Logout
Connect to a DC in a different region
Wait for Lobby Error to happen due to script crashing

I tried looking into the logs and this is the only error I found at the end:
$ cat xmlm.20230124185555.48510.log | tail
2023-01-24 21:24:33,871 48510   INFO    S2C_ActionEffect: actionId=08d6 sourceSequence=040d wait=800ms->673ms rtt=202ms
2023-01-24 21:26:58,058 48510   INFO    C2S_ActionRequest: actionId=08d6 sequence=040e
2023-01-24 21:26:58,438 48510   INFO    S2C_ActionEffect: actionId=08d6 sourceSequence=040e wait=800ms->494ms rtt=381ms
2023-01-24 22:08:14,169 48510   INFO    D->U Read finish
2023-01-24 22:08:14,170 48510   INFO    D->U Source read and target write shutdown
2023-01-24 22:08:14,358 48510   INFO    Closed, exception occurred: <class 'BrokenPipeError'> [Errno 32] Broken pipe
Traceback (most recent call last):
  File "/home/user/.local/bin/xivalexander/mitigate.py", line 1397, in run
    target.target.send(target.outgoing)
BrokenPipeError: [Errno 32] Broken pipe

This is how I launch the script using a systemd service:
[Unit]
Description=XivMitmLatencyMitigator
StartLimitIntervalSec=500
StartLimitBurst=5

[Service]
Type=simple
User=root
WorkingDirectory=/home/user/.local/bin/xivalexander
ExecStart=/usr/bin/python3 mitigate.py
ExecStop=/bin/bash .cleanup.sh
ExecStopPost=-/usr/bin/rm .cleanup.sh
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

Sorry but I literally could not reproduce it on steam deck..

I tried reproducing the error again after closing the game and I also couldn't do it as well. I will try again tomorrow after playing for an hour or so. The reason why I had to add Restart=always is because the script would exit on its own after x hours. Maybe it's related to that?

@xzn I have maybe a bit more info. Script crashed again as I tried to switch world within the same datacenter so I don't think the region has thing has anything to do with this. I tried checking my logs in journalctl since the normal xivalexander logging wasn't showing anything at all.
These are the logs from previous day as I was about to exit the game:
Jan 26 00:26:35 thinkcentre python3[63366]: 2023-01-26 00:26:35,072        63366        INFO        S2C_ActorCast: actionId=0e00 type=0001 action_id_2=0e00 time=1.482 target_id=40014a6c
Jan 26 00:26:35 thinkcentre python3[63366]: 2023-01-26 00:26:35,819        63366        INFO        S2C_ActionEffect: actionId=0e00 sourceSequence=145f wait=100ms
Jan 26 00:26:37 thinkcentre python3[63366]: 2023-01-26 00:26:37,111        63366        INFO        C2S_ActionRequest: actionId=0e00 sequence=1460
Jan 26 00:26:37 thinkcentre python3[63366]: 2023-01-26 00:26:37,321        63366        INFO        S2C_ActorCast: actionId=0e00 type=0001 action_id_2=0e00 time=1.482 target_id=40014a6c
Jan 26 00:26:38 thinkcentre python3[63366]: 2023-01-26 00:26:38,437        63366        INFO        S2C_ActionEffect: actionId=0e00 sourceSequence=1460 wait=100ms
Jan 26 00:29:02 thinkcentre python3[63366]: 2023-01-26 00:29:02,205        63366        INFO        D->U Read finish
Jan 26 00:29:02 thinkcentre python3[63366]: 2023-01-26 00:29:02,206        63366        INFO        D->U Source read and target write shutdown
Jan 26 00:29:04 thinkcentre python3[63367]: 2023-01-26 00:29:04,369        63367        INFO        D->U Read finish
Jan 26 00:29:04 thinkcentre python3[63367]: 2023-01-26 00:29:04,431        63367        INFO        D->U Source read and target write shutdown
Jan 26 00:29:04 thinkcentre python3[63367]: 2023-01-26 00:29:04,527        63367        INFO        U->D Read finish
Jan 26 00:29:04 thinkcentre python3[63367]: 2023-01-26 00:29:04,528        63367        INFO        U->D Source read and target write shutdown
Jan 26 00:29:04 thinkcentre python3[63367]: 2023-01-26 00:29:04,528        63367        INFO        Closed

This is the supposedly unrelated `broken pipe error happening as I am closing the game:
Jan 26 00:29:04 thinkcentre python3[63366]: 2023-01-26 00:29:04,638        63366        INFO        Closed, exception occurred: <class 'BrokenPipeError'> [Errno 32] Broken pipe
Jan 26 00:29:04 thinkcentre python3[63366]: Traceback (most recent call last):
Jan 26 00:29:04 thinkcentre python3[63366]:   File "/home/user/.local/bin/xivalexander/mitigate.py", line 1397, in run
Jan 26 00:29:04 thinkcentre python3[63366]:     target.target.send(target.outgoing)
Jan 26 00:29:04 thinkcentre python3[63366]: BrokenPipeError: [Errno 32] Broken pipe
Jan 26 00:29:07 thinkcentre python3[51897]: 2023-01-26 00:29:07,436        51897(main)        INFO        [63366 ] has exit with status code 65280.

These are the logs from the next day, after around 13h from last time I played the game. I logged in and checked subs voyages on a character (lines are truncated because I forgot to disable paging):
Jan 26 13:01:15 thinkcentre python3[65764]: 2023-01-26 13:01:15,215        65764        INFO        Log will be saved to /tmp/xmlm.20230126130115.65764.log
Jan 26 13:01:15 thinkcentre python3[65764]: 2023-01-26 13:01:15,220        65764        INFO        New[game.JP.2<IP_ADDRESS>0.0000.json] ('<IP_ADDRESS>', 32633) ('<IP_ADDRESS>', 59986) ('<IP_ADDRESS>', 549>
Jan 26 13:01:56 thinkcentre python3[65781]: 2023-01-26 13:01:56,630        65781        INFO        Log will be saved to /tmp/xmlm.20230126130156.65781.log
Jan 26 13:01:56 thinkcentre python3[65781]: 2023-01-26 13:01:56,631        65781        INFO        New[game.JP.2<IP_ADDRESS>0.0000.json] ('<IP_ADDRESS>', 32633) ('<IP_ADDRESS>', 50382) ('<IP_ADDRESS>', 54>
Jan 26 13:01:56 thinkcentre python3[65782]: 2023-01-26 13:01:56,669        65782        INFO        Log will be saved to /tmp/xmlm.20230126130156.65782.log
Jan 26 13:01:56 thinkcentre python3[65782]: 2023-01-26 13:01:56,670        65782        INFO        New[game.JP.2<IP_ADDRESS>0.0000.json] ('<IP_ADDRESS>', 32633) ('<IP_ADDRESS>', 50383) ('<IP_ADDRESS>', 54>
Jan 26 13:01:56 thinkcentre python3[65764]: 2023-01-26 13:01:56,675        65764        INFO        D->U Read finish
Jan 26 13:01:56 thinkcentre python3[65764]: 2023-01-26 13:01:56,675        65764        INFO        D->U Source read and target write shutdown
Jan 26 13:01:56 thinkcentre python3[65764]: 2023-01-26 13:01:56,716        65764        INFO        U->D Read finish
Jan 26 13:01:56 thinkcentre python3[65764]: 2023-01-26 13:01:56,716        65764        INFO        U->D Source read and target write shutdown
Jan 26 13:01:56 thinkcentre python3[65764]: 2023-01-26 13:01:56,717        65764        INFO        Closed
Jan 26 13:01:56 thinkcentre python3[51897]: 2023-01-26 13:01:56,917        51897(main)        INFO        [63367 ] has exit with status code 0.
Jan 26 13:01:57 thinkcentre python3[65784]: Base: 0x
Jan 26 13:01:57 thinkcentre python3[65783]: Base: 0xf3a43000
Jan 26 13:01:57 thinkcentre python3[65784]: f3a43000
Jan 26 13:01:57 thinkcentre python3[65783]: Oodle encode test: 256 -> 100
Jan 26 13:01:57 thinkcentre python3[65783]: Oodle helper running: state=2eb400 shared=400018 window=8000
Jan 26 13:01:58 thinkcentre python3[65784]: Oodle encode test: 256 -> 100
Jan 26 13:01:58 thinkcentre python3[65784]: Oodle helper running: state=2eb400 shared=400018 window=8000

And these are the error lines showing what happened when I tried to logout and then login into another character in the same DC:
Jan 26 13:06:21 thinkcentre python3[65781]: 2023-01-26 13:06:21,157        65781        INFO        D->U Read finish
Jan 26 13:06:21 thinkcentre python3[65782]: 2023-01-26 13:06:21,157        65782        INFO        D->U Read finish
Jan 26 13:06:21 thinkcentre python3[65782]: 2023-01-26 13:06:21,531        65782        INFO        D->U Source read and target write shutdown
Jan 26 13:06:21 thinkcentre python3[65781]: 2023-01-26 13:06:21,532        65781        INFO        D->U Source read and target write shutdown
Jan 26 13:06:21 thinkcentre python3[65781]: 2023-01-26 13:06:21,571        65781        INFO        U->D Read finish
Jan 26 13:06:21 thinkcentre python3[65781]: 2023-01-26 13:06:21,572        65781        INFO        U->D Source read and target write shutdown
Jan 26 13:06:21 thinkcentre python3[65781]: 2023-01-26 13:06:21,572        65781        INFO        Closed
Jan 26 13:06:21 thinkcentre python3[65782]: 2023-01-26 13:06:21,577        65782        INFO        U->D Read finish
Jan 26 13:06:21 thinkcentre python3[65782]: 2023-01-26 13:06:21,577        65782        INFO        U->D Source read and target write shutdown
Jan 26 13:06:21 thinkcentre python3[65782]: 2023-01-26 13:06:21,578        65782        INFO        Closed
Jan 26 13:06:21 thinkcentre python3[51897]: 2023-01-26 13:06:21,966        51897(main)        INFO        [65764 ] has exit with status code 0.
Jan 26 13:06:21 thinkcentre python3[51897]: --- Logging error ---
Jan 26 13:06:22 thinkcentre python3[51897]: Traceback (most recent call last):
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/logging/__init__.py", line 1103, in emit
Jan 26 13:06:22 thinkcentre python3[51897]:     stream.write(msg + self.terminator)
Jan 26 13:06:22 thinkcentre python3[51897]: RuntimeError: reentrant call inside <_io.BufferedWriter name='<stderr>'>
Jan 26 13:06:22 thinkcentre python3[51897]: During handling of the above exception, another exception occurred:
Jan 26 13:06:22 thinkcentre python3[51897]: Traceback (most recent call last):
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/logging/__init__.py", line 1103, in emit
Jan 26 13:06:22 thinkcentre python3[51897]:     stream.write(msg + self.terminator)
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/home/user/.local/bin/xivalexander/mitigate.py", line 1619, in on_child_exit
Jan 26 13:06:22 thinkcentre python3[51897]:     logging.info(f"[{pid:<6}] has exit with status code {status}.")
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/logging/__init__.py", line 2138, in info
Jan 26 13:06:22 thinkcentre python3[51897]:     root.info(msg, *args, **kwargs)
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/logging/__init__.py", line 1477, in info
Jan 26 13:06:22 thinkcentre python3[51897]:     self._log(INFO, msg, args, **kwargs)
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/logging/__init__.py", line 1624, in _log
Jan 26 13:06:22 thinkcentre python3[51897]:     self.handle(record)
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/logging/__init__.py", line 1634, in handle
Jan 26 13:06:22 thinkcentre python3[51897]:     self.callHandlers(record)
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/logging/__init__.py", line 1696, in callHandlers
Jan 26 13:06:22 thinkcentre python3[51897]:     hdlr.handle(record)
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/logging/__init__.py", line 968, in handle
Jan 26 13:06:22 thinkcentre python3[51897]:     self.emit(record)
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/logging/__init__.py", line 1108, in emit
Jan 26 13:06:22 thinkcentre python3[51897]:     self.handleError(record)
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/logging/__init__.py", line 1021, in handleError
Jan 26 13:06:22 thinkcentre python3[51897]:     sys.stderr.write('--- Logging error ---\n')
Jan 26 13:06:22 thinkcentre python3[51897]: RuntimeError: reentrant call inside <_io.BufferedWriter name='<stderr>'>
Jan 26 13:06:22 thinkcentre python3[51897]: Call stack:
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/home/user/.local/bin/xivalexander/mitigate.py", line 1674, in <module>
Jan 26 13:06:22 thinkcentre python3[51897]:     exit(__main__())
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/home/user/.local/bin/xivalexander/mitigate.py", line 1631, in __main__
Jan 26 13:06:22 thinkcentre python3[51897]:     sock, source = listener.accept()
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/usr/lib/python3.10/socket.py", line 293, in accept
Jan 26 13:06:22 thinkcentre python3[51897]:     fd, addr = self._accept()
Jan 26 13:06:22 thinkcentre python3[51897]:   File "/home/user/.local/bin/xivalexander/mitigate.py", line 1619, in on_child_exit
Jan 26 13:06:22 thinkcentre python3[51897]:     logging.info(f"[{pid:<6}] has exit with status code {status}.")
Jan 26 13:06:22 thinkcentre python3[51897]: Message: '[65764 ] has exit with status code 0.'
Jan 26 13:06:22 thinkcentre python3[51897]: Arguments: ()
Jan 26 13:06:23 thinkcentre python3[51897]: 2023-01-26 13:06:23,141        51897(main)        INFO        Cleaning up...
Jan 26 13:06:23 thinkcentre python3[51897]: 2023-01-26 13:06:23,144        51897(main)        INFO        Running: iptables -t nat -D PREROUTING -p tcp -m multiport --dports 1:65535 -d <IP_ADDRESS>/16 -j REDIRE>
Jan 26 13:06:23 thinkcentre python3[65790]: 2023-01-26 13:06:23,145        65790        INFO        Log will be saved to /tmp/xmlm.20230126130623.65790.log
Jan 26 13:06:23 thinkcentre python3[65790]: 2023-01-26 13:06:23,145        65790        INFO        New[game.JP.2<IP_ADDRESS>0.0000.json] ('<IP_ADDRESS>', 32633) ('<IP_ADDRESS>', 50479) ('<IP_ADDRESS>', 549>
Jan 26 13:06:23 thinkcentre python3[51897]: 2023-01-26 13:06:23,463        51897(main)        INFO        [65782 ] has exit with status code 0.

Jan 26 13:06:23 thinkcentre bash[65813]: /bin/bash: .cleanup.sh: No such file or directory
Jan 26 13:06:23 thinkcentre systemd[1]: xivalexander.service: Control process exited, code=exited, status=127/n/a
Jan 26 13:06:24 thinkcentre rm[65815]: /usr/bin/rm: cannot remove '.cleanup.sh': No such file or directory
Jan 26 13:06:24 thinkcentre systemd[1]: xivalexander.service: Failed with result 'exit-code'.
Jan 26 13:06:24 thinkcentre systemd[1]: xivalexander.service: Consumed 4min 2.349s CPU time.
Jan 26 13:06:29 thinkcentre systemd[1]: xivalexander.service: Scheduled restart job, restart counter is at 1.
Jan 26 13:06:29 thinkcentre systemd[1]: Stopped XivMitmLatencyMitigator.
Jan 26 13:06:29 thinkcentre systemd[1]: xivalexander.service: Consumed 4min 2.349s CPU time.
Jan 26 13:06:29 thinkcentre systemd[1]: Started XivMitmLatencyMitigator.

The systemd service crashed, failing to execute ExecStop directives (see .cleanup.sh failing to run) and then restarted again.

hmm you can try comment out line 1619 for now. it just logs some info by printing to stderr but printing is bad inside a signal handler.
        def on_child_exit(signum, frame):
            if child_pids:
                pid, status = os.waitpid(-1, os.WNOHANG)
                if pid:
                    logging.info(f"[{pid:<6}] has exit with status code {status}.") # <--- this line right here ---
                    child_pids.discard(pid)

        signal.signal(signal.SIGCHLD, on_child_exit)

Do you think that kind of logging error would be enough to cause the script to crash?

Yes, it's what your log shows. Just comment out the line that causing the crash should work since it only does something informational and not functional.
(Specifically it's the act of logging itself that's causing the crash. I don't see any other thing crashing the script or indicating any errors from your logs).

Yesterday's commits are working as expected (e2d00c2, or 093818f for Python <3.10) so this can probably be closed. With the direct exe patching, mitm should be stable in CN and KR too.

Thanks for the fixes @ayyaruq @Soreepeong, hope u all have a great weekend.


Docker install does not work properly

Amundsen Version
Master
Expected results
When I follow Installation tutorial everything works.
Actual results
It fails with following error:
2019-04-03T15:43:02+0200.069 [ERROR] app.log_exception:1761 (40925:Thread-1) - Exception on / [GET]
Traceback (most recent call last):
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/amundsen_frontend-1.0.0-py3.6.egg/amundsen_application/api/__init__.py", line 16, in index
    return render_template("index.html")  # pragma: no cover
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/flask/templating.py", line 134, in render_template
    return _render(ctx.app.jinja_env.get_or_select_template(template_name_or_list),
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/jinja2/environment.py", line 869, in get_or_select_template
    return self.get_template(template_name_or_list, parent, globals)
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/jinja2/environment.py", line 830, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/jinja2/environment.py", line 804, in _load_template
    template = self.loader.load(self, name, globals)
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/jinja2/loaders.py", line 113, in load
    source, filename, uptodate = self.get_source(environment, name)
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/flask/templating.py", line 58, in get_source
    return self._get_source_fast(environment, template)
  File "/Users/tomsejr/Documents/03-Workspace/LMC/amundsenfrontendlibrary/venv/lib/python3.6/site-packages/flask/templating.py", line 86, in _get_source_fast
    raise TemplateNotFound(template)
jinja2.exceptions.TemplateNotFound: index.html

It is same as problem mentioned at issue #6
How to fix that
Posibilities:

Running npm run build before docker build -f public.Dockerfile -t amundsen-frontend:latest . so index.html is created and copied into Docker imege during build proccess.
Add npm into amundsen-frontend image and build everything here:

FROM python:3
WORKDIR /app
COPY . /app
RUN pip3 install -r requirements3.txt
RUN apt-get update \
    && apt-get -y install curl gnupg \
    && curl -sL https://deb.nodesource.com/setup_11.x  | bash - \
    && apt-get -y install nodejs \
    && cd amundsen_application/static \
    && npm install \
    && npm run build
RUN python3 setup.py install

ENTRYPOINT [ "python3" ]
CMD [ "amundsen_application/wsgi.py" ]

BTW thank you for this wonderful project!

cc @ttannis @danwom
@tomsej , thanks for reporting the issue. Feel free to create a pr to fix the issue :)

@feng-tao ok will send PR tomorrow. I was not sure which way is preferable, think changing installation guide is ok for now.

I believe this has the same root cause as https://github.com/lyft/amundsenfrontendlibrary/issues/6, and may share the same solution. Can you try following the updated instructions here https://github.com/lyft/amundsenfrontendlibrary/pull/77?
Thanks!

close this issue for now, @tomsej let us know if you still encounter the same issue and feel free to reopen one.


AREA_SPACE_OVERRIDE_COMBINE only combines gravity

When areas overlap, and the top one uses AREA_SPACE_OVERRIDE_COMBINE, only the gravity gets combined. The linear/angular dampenings are replaced.
Is this correct behavior?

not sure, probably should be combined too i guess

you did a PR and i merged it, so i'm closing this

Nice. Now I can write the docs accordingly.


Is there any way for using LoftQ to GPTQ or AWQ model?

I want to use LoftQ initialization for GPTQ or AWQ baseline model.
Is that possible?

Hi @LameloBally, thanks for your interest of our work.
Unfortunately the answer is no. LoftQ aims to provide a data-free initialization for all downstream task finetuning, so quantization with data calibration is out of our scope. However, we welcome you to explore more possibilities of our method: you can obtain a quantized backbone (Q) from GPTQ or AWQ and then initialize LoRA adapters, A and B, by SVD(W-Q), where W is the full-precision pre-trained weight.
Let me know if you have further questions.

@yxli2123  Thanks for your kind answer. It was very helpful for me!!
But, I have another question. How Can I initialize LoRA Adapter by SVD(W-Q) s.t. Q is GPTQ model & W is full precision.
I thought LoftQ is very innovative idea so I want to apply it to other cases to develop it further.
It will be very helpful for me if you let me know How can I find any guide or the way I can do it.
Thanks!


Implement TLV in CASE Sigma Protocol

Problem
What is being fixed?  Examples:

CASE does not currently follow the current spec with message handling.
Fixes #5934

Change overview

Replaced Sigma Message Construction using BoundBuf with TLV

Testing
How was this tested? (at least one bullet point required)

Tested using CASE Session Unit Tests

@jpk233, let us not use PacketBufferHandle where not needed. It puts extra burden on the pool. In my testing I had noticed the pool was getting depleted on repeated tests. Even though the buffer is not leaked, but it can temporarily deplete the pool enough to cause issues.

Thanks for resolving this comment!


Add phpdoc to ItemList and Database\ItemList

No code changes, I just added some phpdoc and gave a better order to the methods.

What about adding a couple of generator functions?

See my file iterator branch for what an iterator based itemlist would look like. Unfortunately the itemlist is currently very tightly coupled to arrays so a generator doesn't make much sense.

@mlocati would you be able to see what is different in the upstream compared to your pull request? I'm afraid a lot might have changed.

Too many conflicts. If @aembler isn't going to work on these ItemList classes anymore I'll create another pull request.


JavaScriptEngineSwitcher.Core.JsRuntimeException: RangeError: Maximum call stack size exceede

Hi,
I am using React.NET with following settings

JsEngineSwitcher.Instance.EngineFactories.AddV8();
IJsEngine engine = JsEngineSwitcher.Instance.CreateEngine(V8JsEngine.EngineName);
ReactSiteConfiguration.Configuration
.SetLoadBabel(false)
.SetUseDebugReact(false)
.AddScriptWithoutTransform("~/Scripts/server.bundle.js")
.SetStartEngines(25)
.SetMaxEngines(100)
.SetReuseJavaScriptEngines(true)
;

Now I am getting following error for few of the cases.

[29] Pwa-Logger-Renderengine - JavaScriptEngineSwitcher.Core.JsRuntimeException: RangeError: Maximum call stack size exceeded ---> Microsoft.ClearScript.ScriptEngineException: RangeError: Maximum call stack size exceeded
at V8Exception.ThrowScriptEngineException(V8Exception* )
at Microsoft.ClearScript.V8.V8ContextProxyImpl.Execute(String gcDocumentName, String gcCode, Boolean evaluate, Boolean discard)
at Microsoft.ClearScript.V8.V8ScriptEngine.<>c__DisplayClass63_0.b__0()
at Microsoft.ClearScript.ScriptEngine.ScriptInvoke[T](Func1 func)    at Microsoft.ClearScript.V8.V8ScriptEngine.BaseScriptInvoke[T](Func1 func)
at Microsoft.ClearScript.V8.V8ScriptEngine.<>c__DisplayClass68_01.<ScriptInvoke>b__0()    at Microsoft.ClearScript.V8.?A0x792c8756.LockCallback(Void* pvArg)    at Microsoft.ClearScript.V8.V8ContextProxyImpl.InvokeWithLock(Action gcAction)    at Microsoft.ClearScript.V8.V8ScriptEngine.ScriptInvoke[T](Func1 func)
at Microsoft.ClearScript.V8.V8ScriptEngine.Execute(String documentName, String code, Boolean evaluate, Boolean discard)
at Microsoft.ClearScript.ScriptEngine.Evaluate(String documentName, Boolean discard, String code, Boolean marshalResult)
at JavaScriptEngineSwitcher.V8.V8JsEngine.InnerEvaluate(String expression, String documentName)
--- End of inner exception stack trace ---
at JavaScriptEngineSwitcher.V8.V8JsEngine.InnerEvaluate(String expression, String documentName)
at JavaScriptEngineSwitcher.V8.V8JsEngine.InnerEvaluate[T](String expression, String documentName)
at JavaScriptEngineSwitcher.V8.V8JsEngine.InnerEvaluate[T](String expression)
at JavaScriptEngineSwitcher.V8.V8JsEngine.InnerHasVariable(String variableName)
at JavaScriptEngineSwitcher.Core.JsEngineBase.HasVariable(String variableName)
at JSPool.PooledJsEngine.HasVariable(String variableName)
at React.ReactEnvironment.EnsureUserScriptsLoaded()
at React.ReactEnvironment.CreateComponent[T](String componentName, T props, String containerId, Boolean clientOnly)
at BAL.PWA.CMS.PWACMSRenderedData.React[T](String componentName, T props, String htmlTag, String containerId, Boolean clientOnly, Boolean serverOnly, String containerClass)
at BAL.PWA.CMS.PWACMSRenderedData.GetNewsDetails(PwaNewsDetailReducer reducer, String url, String containerId, String componentName)

It's like I get it 3/4 times a day in 1000s of instances. I am not able to understand it as when I log the specific component for which this is coming and try to reproduce the same, then it does not reproduce. So it's an intermittent issue which is not consistently reproducing. Can any one of you help put more light on this?
PS: I have copied the React() function from HTMLHelpers to my class as I am caching the rendered page to memcached.

IJsEngine engine = JsEngineSwitcher.Instance.CreateEngine(V8JsEngine.EngineName);

Why did you create a separate instance of JS engine?

It's a mistake @Taritsyn . I had copied the code earlier from some online solution to make V8 work with the new libraries. Just checked that it's useless to do this. Thanks for pointing out. But do you think that this is causing the issue?

But do you think that this is causing the issue?

No, problem is not this. Very similar to the lack of memory on the server. Try to change pool settings to the following:
…
.SetStartEngines(3)
.SetMaxEngines(15)
…

Sure will try with this. But do you think that max 15 engines will be able to handle the load of 8000 render requests/ minute? I will also do the load testing to check the same. Any inputs in handling the things better would be of great help.

I believe, that 100 JS engines is too much (maximum is to use 25). Can you imagine how much memory is spending this amount of JS engines? Usually even a pool of StringBuilders is limited to 20.

I will do some load-testing with the settings you have provided and will update it here. It will be good learning and I second the thought that I should not create more than 25 engines in a pool.

I'm using JSPool directly, not React.NET, but I have also started running into this since a week and half ago. Actually looks like it was right as we added more RAM to our servers, which is pretty counterintuitive. We also updated to the latest JSPool/JavaScriptEngineSwitcher/ClearScript a week before that.
For reference, we've been running with 32 engine pools for a year now, which we arrived at based on testing at ~8000 requests/min, but your needs will vary depending on other factors, including how long a render takes.

@Taritsyn I have set the minsize to 3 and max size to 20 but I am still getting the errors (twice a day). The RAM for the Server is 8GB and is underutilized. Do you think something else is the issue?

Do you think something else is the issue?

@cw-gawdeprasad In this case, I recommend to check your JSX code on presence of recursion.

The recursion is not there for sure as when I run the code for the same component with the same rout for which issue came, it runs fine. Also the issue is intermittent, does not reproduce all the time. I wanted to know if RAM is the only possible culprit if there is no Recursion?

I wanted to know if RAM is the only possible culprit if there is no Recursion?

Thinking, that there is not, because you have enough RAM.
Also check the ~/Scripts/server.bundle.js file. See examples of such errors: “Maximum call stack size exceeded error” and ““RangeError: Maximum call stack size exceeded” Why?”.

I checked both these pages before creating an issue here. For sure there is no recursion problem in the code as I logged the component and the URL for the which the issue came up and tried reproducing the same with the same to render the same URL and component but it did not reproduce. I also did the load testing where I set the max engines to 100 and tried to fill up the RAM but even then the error did not reproduce. So I am wondering as what's the cause.

I have been getting the same error, but only in Azure after pushing changes with web deploy. This is with server side rendering enabled. The error is intermittent, affecting about 1/3 of requests. I never get this error in development.
Is it possible that the JavaScript engines are not getting destroyed+recreated when I do a web deploy? I am also using V8.

By the way, my error is resolved by restarting the App Service.

This error is coming from the JS engine, which makes it hard to debug 😕  You could try using the ChakraCore JS engine rather than ClearScript and see if that helps.

@Taritsyn Is it possible to get the JavaScript stack trace when this occurs?

@Daniel15 Yes, it is possible. I already wrote about this.

@Daniel15 But here is another problem, that the “Maximum call stack size exceeded” error may not have a stack trace.

Yes the fact that there is no indication of where in the code the exception is happening, combined with the fact that it only happens on few renders, makes this very hard to debug.

@Daniel15 @Taritsyn I changed the engine to ChakraCore (1.5.2) using JavaScriptEngineSwitcher.ChakraCore and the problem has disappeared but not I am seeing that Chakra eats memory in some cases and the memory consumption goes up to 3GB. I never had this issue with V8. Any suggestions on why this might happen? Also the consumption is not consistent. It happens 3/4 times a week when the w3wp process takes around 3GB memory.

@cw-gawdeprasad I moved part of functionality from 3.X branch to 2.4.12 and 2.4.13 versions. It is possible, that new configuration properties in the JavaScriptEngineSwitcher.V8 (HeapSizeSampleInterval, MaxHeapSize and MaxStackUsage) and the JavaScriptEngineSwitcher.ChakraCore (DisableFatalOnOOM and MemoryLimit) modules will help to solve your problems.

Closing issues older than a year, please re-open if you think this is still relevant.


An div that is not affected by the materialize css and js

I need to use the original html, css and js code in part of a page design, but all of the element has been affected by materialize. if there is a "unmaterialize" tag or attribute for this case would be wonderful.

I think you basically solved your own problem - write its' own css - so subclass it and use jquery select/find to use it and attach needed js.

found this because I have the same issue as the OP.  I'm trying to use jquery-maskmoney.  No idea how to "unmaterialize" nor "uncss" everything.  Any ideas?  Or perhaps material design has its own ideas about currency input?  But I've found nothing on that topic either.


Add devShell.mkShellArgs

Resolves #83
Details: https://github.com/srid/haskell-flake/pull/83#issuecomment-1433427743
(We need shellHook for generating .cabal from package.yaml, so HLS will work.)

This is just an intermediate solution until we have composable devshells.


init: ported Get_First_{String,Unicode} primitive function

Since a number of level 1 Hestia libraries use string functions, we have to port its primitive ones into HestiaKERNEL library package. Hence, let's do this.
This patch ports Get_First_{Char,Unicode} primitive function into HestiaKERNEL library in init/ directory.

merge in first since it does not affect the current production ones.


M2 gcs browser toolbar buttons

This PR adds a toolbar button to create new directories and some small tweaks into the name prompt window.

Thank you for the review.

Thank you for the review.


Tags to trigger release pipelines

Description :sparkles:
Release pipelines are triggered by tag instead of commit to master branch
Issues :bug:
DEVOPS-528
Testing :alembic:
Screenshots :camera_flash:
Additional notes :spiral_notepad:

Codecov Report

Merging #2022 (9a455ac) into develop (b31561d) will increase coverage by 6.73%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop    #2022      +/-   ##
===========================================
+ Coverage    34.22%   40.95%   +6.73%
===========================================
  Files          744      264     -480
  Lines        14461     4844    -9617
  Branches      3062     1023    -2039
===========================================
- Hits          4949     1984    -2965
+ Misses        9038     2730    -6308
+ Partials       474      130     -344

see 480 files with indirect coverage changes
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

API branch is deployed to platta: https://kesaseteli-pr2022.api.dev.hel.ninja/healthz :rocket::rocket::rocket:

EMPLOYER branch is deployed to platta: https://kesaseteli-pr2022.dev.hel.ninja :rocket::rocket::rocket:

YOUTH branch is deployed to platta: https://youth-pr2022.dev.hel.ninja :rocket::rocket::rocket:

HANDLER branch is deployed to platta: https://handler-pr2022.dev.hel.ninja :rocket::rocket::rocket:

TestCafe result is success for https://kesaseteli-pr2022.dev.hel.ninja :laughing::tada::tada::tada:

TestCafe result is success for https://handler-pr2022.dev.hel.ninja :laughing::tada::tada::tada:

TestCafe result is success for https://youth-pr2022.dev.hel.ninja :laughing::tada::tada::tada:


adata.X is None in resources_test

In the provided resources_test data, adata.X is None. This seems unintuitive to me.

That's because the information is stored in the layers.
>>> adata = ad.read_h5ad("resources_test/common/pancreas/dataset.h5ad")
>>> adata
AnnData object with n_obs × n_vars = 500 × 419
    obs: 'tech', 'celltype', 'size_factors', 'n_counts', 'batch'
    var: 'n_cells', 'hvg', 'hvg_score'
    uns: '_from_cache', 'data_reference', 'data_url', 'dataset_description', 'dataset_id', 'dataset_name', 'dataset_organism', 'dataset_summary', 'knn', 'normalization_id', 'pca_variance', 'var_names_all'
    obsm: 'X_pca'
    varm: 'pca_loadings'
    layers: 'counts', 'log_normalized', 'normalized'
    obsp: 'knn_connectivities', 'knn_distances'

The section "Format of a raw dataset object" describes which slots should be present in the output of a dataset loader.
After this, a raw dataset will be processed by a series of components which add on additional information and slots.
The page I just linked should contain the required and optional slots for each of the steps, similar to the "Format of a raw dataset object" page. Other than that, I wouldn't consider the fact that the dataset doesn't contain a .X slot an issue, so I'm retagging this issue as something that's needed for the documentation.
Also I'm changing the milestone, as this is not functionality that's migrating from v1 to v2.

Coming back to your earlier comment about it being unintuitive that X is none: I think it's preferable to be explicit here. When X is not none, you don't know whether the matrix you're working with is counts, normalized or something else entirely.
The dataset reference documentation contains the full spec of the common dataset. https://main--openproblems.netlify.app/documentation/reference/openproblems-v2/src-datasets.html#file-format-common-dataset
Can we close this issue? :)

I feel pretty strongly that this is confusing and frustrating for new contributors. At the very least I'd want a line here saying "Note: expected counts slot (e.g., AnnData.X) may be empty. See the api/comp_method.yml spec to understand the data specification for a given task.


Blazor server rendering upgrade

Hi,
Is there a branch for upgrading the template to .net 8 blazor web
thanks,
abdo

Yes, once .NET 8 is officially released, I'll be upgrading the template to .NET 8 Blazor Web.

Hi any update on the .NET 8 upgrade.

Hi any update on the .NET 8 upgrade?

Hello,
See #554 for the latest state of our transition to .NET 8.
Since .NET 8 doesn't bring any major breaking changes the transition should be quite smooth and easy!
Kind regards,
Bram


Post install hooks

This one goes mostly to @ncannasse who has opposed this up until now.
I'm not really against, because with macros, malicious code can be executed with simply by calling -lib evillib (which has --macro evellib.CitadelOfDoom.worldDomination()).
The one issue I see is that there are situations where users need to run haxelib with sudo, in which case a post install hook would have more permissions. But generally you shouldn't sudo if you don't know what you're doing.
NPM is a significantly bigger ecosystem and they don't seem to run into problems with this. Unless there is serious opposition on this, I'd like to add it. If need be, we can still ask for confirmation and the brave ones can just skip that with --always. Thoughts?

To quote Hugh from haxedev:

With the advent of macros, I thought we might be past caring about malicious code, since compiling with a lib can now do pretty much whatever it likes, so the post-install seems like  a good idea. (eg, haxelib dev haxelib_client my_malicious_path, effectively rootkitting you next time you do "su").

Why not but then let's make sure that it's at least written in Haxe by running it with haxe --run, so our wanabee world dominators will have to learn Haxe first, so we win.

Is this only for post-install hook or is there plans for a pre/post-update and pre-remove too?


add unit tests for tags.scm

tree-sitter v0.20.6 was just published that has a new testing mechanism for tags queries, so I've added some unit tests. I'm not very creative with the fixture, it was just some code I had locally. Are there any edge cases we want to cover? Maybe we should add some for the module+function vs. record+field stuff?

Hmm well now I wish there was a way to specify that something is not highlighted. It would be nice to have that for the record+member lines. I think we'll need to settle for only testing the module+function part for now.


Initial assembly implementation

the toString() method on NihmsSubmission has been expanded in this PR to include the new "name" field. this conflicts with the changes from another request, but should be included

@jrmartino ok, let me resolve conflicts and re-push!


Fix user-agents used in automated tests

See the commit messages for details, but in short, this makes our automated tests use real User-Agent strings for outgoing HTTP requests, rather than sending somewhat unusual HTTP requests that don't specify that header.
The net effect beyond more realistic testing is that it is removes an extra database query in the first HTTP query in each of our automated tests that has been used to create a Client object, which could be annoying when counting database queries.
@showell FYI, feel free to merge if you're happy with this and it passes CI.

Steve approved this in PMs, so I went ahead and merged this now so it's easy to rebase #13998 on top of it.


Reverted "Pinned asgiref == 3.2.5 in test requirements."

This reverts commit aa21020218a2dcd29a03444ad2a77f03f085b04e.
Regression was fixed by https://github.com/django/asgiref/commit/e4367c571fd1833bda0b7ff0bd68d924916352da.

I might suggest ~= 3.2 in future so that an asgiref 4 does not break Django?

On stable/3.0.x we pinned asgiref ~= 3.2. I think it's fine to hit breaking changes on master.


Iced Example

We need some solution to 2D rendering for doing UIs such as imgui. We have a couple of options:

Integrate with imgui directly, taking a ready-to-render frame, and rendering it.
Provide a 2D rendering interface that we could implement an imgui renderer on top of.
Don't support it at all, letting external integration deal with it. This means a user could have rend3 render to the swapchain, then they would render the UI on top of it later (see https://github.com/BVE-Reborn/rend3/issues/15)

Todo:

[x] Support external resources (#40 and #41)
[x] Imgui example (#42)
[ ] Iced example

Iced works well with WGPU rendering and is easy to setup. There is a good example here showing how to set up WGPU rendering with Iced on top. It's also a good model for #15. Maybe there could be an example showing how to use Iced, or an external Imgui WGPU renderer, but implementing an Imgui backend seems like it would be a bit out of scope.

Yeah I agree. I think I've settled on the 3rd method as the way to do things. Thank you for that link to the iced integration model, I'll take a look and see if there's anything I can glean from it.

Closing in favor of #192 as it has more up to date information.


refactor(sourcemap): lower the msrv.

#4873  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @rzvxa and the rest of your teammates on  Graphite


"lastSeen" has different semantics for active connections and database peer

For connections in NetworkController, lastSeen (in peer info) contains time of last message got from a peer. In opposite, lastSeen in peer info in PeerDatabase implementation (InMemoryPeerDatabase) contains time when last connection with a peer was established (or 0 if never).
It makes sense to have two fields for these two different use cases.

Fixed


How do you use the sidebar

I see in the documentation you have it clear how to template the sidebar, but how do you actually use it? What code makes it appear and disappear?

This beyond the scope of nativeBase.


Fix travis build for api version 26

Travis build started failing from build#41. This is because after build#40 26.0.0 was released and 26.0.0-beta2 started downloading 26.0.0 instead of the beta version.
This PR updates the .travis.yml file to use android api 26 instead of 25. As api version in travis should match that in build.gradle

Thanks!


SAML Integration for External IDP configuration ACTIVE state not persisted in UAA Database

When integrating with SAML for External IDP, the active flag is not persisted, when restarting the application, it resets the value in identity_provider table to false.
select id,name,type,active from identity_provider;
id                  |       name        |     type     | active
--------------------------------------+-------------------+--------------+--------
b750dca2-ee00-4ada-b9f4-de32088a4f2d | login-server      | login-server | f
f348fff2-44f0-49b1-a928-04ae01cf0137 | ldap              | ldap         | f
59aa1ae4-e5df-4e81-afcd-81291e0e42be | keystone          | keystone     | f
5fa3b153-53db-469d-aff3-eb73c6fc08c6 | idp-admin-sso-dev | saml         | f
ba9ae379-4c86-44e0-9c6d-92618b9cbc91 | uaa               | uaa          | f

@karthikduraisamy I believe that this can happens if you have removed the provider configuration from your Yaml manifest. Can you check if that is the case here?

@fhanik I am working with @karthikduraisamy.
Here in our application, we are creating IDP based on the UAA API and not through yaml configuration.
I believe UAA shouldn't disable the active IDP's to inactive during startup based on yaml.

@james-b-thomas We are working on handling this in a consistent manner. We will fix the issue with provide being deactivated if it no longer exists in the YML

Can you give us an estimate when this issue will be fixed ?


Oasis app version 2.x support

enable app version 2.x. in this PR, getting the public key and signing
also some error handling fixes

Looking at source code in files in src/ in this repo and source code in files in js/src/ in Ledger Oasis app repo, there is a lot of copying/duplication.
What would be the best way to avoid that?

I didn't know about there being a copy of this module in the repo of the app itself. Is this intended? How did the two copies com to be?

BTW, what is the "read public key" part of the PRs title?

This PR had originally addressed the read public key functionality. But I had since found that the signing also needed updating, and I later added that here too. I should just remove this part of the title

Looking at source code in files in src/ in this repo and source code in files in js/src/ in Ledger Oasis app repo, there is a lot of copying/duplication.
What would be the best way to avoid that?

I didn't know about there being a copy of this module in the repo of the app itself. Is this intended? How did the two copies com to be?

I also don't know the genesis of this :slightly_smiling_face:.
Can you open an issue in this repo to track this unusual code duplication and we can tackle that separately from this PR?

BTW, what is the "read public key" part of the PRs title?

This PR had originally addressed the read public key functionality. But I had since found that the signing also needed updating, and I later added that here too. I should just remove this part of the title

Aha, I see. Yeah, please, just update the title to avoid confusion :slightly_smiling_face:.

title changed, filed #288
gonna make some changes to address review comments next

weird, github sets co-authored-by line to the user who started the thread, not the author of the suggestion. gonna fix that manually

thanks for the help!