Add docs for winrm and salt

Also add flags to the options for the winrm backend can be passed from the command line
Thanks!
Daniel

I reworked some of your commits (some fixes in doc).
For options in the command line I was not satisfied because all backend options should be specified in hostspec (eg. winrm://user@IP?opt=val&opt2=val2). Also I found passing password in hostspec with the syntax user:password@host is more frequent. So I added this in parse_hostspec.
I pushed my code + your code in your master branch (github allow maintainer to do this) but I appears to close the PR automatically :D Looks buggy. Can you recreate your pull request based on your actual master branch ?

ha! weird.
Can you push them again, I think I overwrote them when I did a force push :/

Well this seems buggy, now i've a "permission denied". I recreated the PR in #239 , sorry for confusion

No worries!

Hrm, fyi I did this today and it worked great.  thanks for the tip.
https://github.com/saltstack/salt/pull/43298


Fix istioctl release binary corruption

The release acrhive scripts invoke sed to update versions fields in
several charts. This search/replace ran over the entire release
archive included istioctl binaries. The istioctl operator commands
which were introduced in
https://github.com/istio/istio/pull/16622/files include embedded
charts which matched the sed expression. This corrupted the data
portion istioctl binaries when created through the official release process.
fixes https://github.com/istio/istio/issues/16660 by reducing scope of
sed to the install/kubernetes/helm subdirectory.
Please provide a description for what this PR is for.
And to help us figure out who should review this PR, please
put an X in all the areas that this PR affects.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Thank you!

great work Jason!

@ayj: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

istio-unit-tests-master
d188679e74b09a17322f98fe5ca192d2550cf6d8
link
/test istio-unit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

istio-unit-tests-master
d188679e74b09a17322f98fe5ca192d2550cf6d8
link
/test istio-unit-tests-master

istio-racetest-master
d188679e74b09a17322f98fe5ca192d2550cf6d8
link
/test istio-racetest-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@ayj: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

istio-racetest-master
d188679e74b09a17322f98fe5ca192d2550cf6d8
link
/test istio-racetest-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test all

@ayj: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

istio-racetest-master
d188679e74b09a17322f98fe5ca192d2550cf6d8
link
/test istio-racetest-master

istio-unit-tests-master
d188679e74b09a17322f98fe5ca192d2550cf6d8
link
/test istio-unit-tests-master

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

This needs to be backported to release-1.3 ASAP. Please someone comment /cherrypick release-1.3.

/cherrypick release-1.3.

@duderino: once the present PR merges, I will cherry-pick it on top of release-1.3. in a new PR and assign it to you.

In response to this:

/cherrypick release-1.3.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/cherrypick release-1.3
On Thu, Sep 5, 2019, 10:05 PM Romain Lenglet<EMAIL_ADDRESS>wrote:

This needs to be backported to release-1.3 ASAP. Please someone comment /cherrypick
release-1.3.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/16865?email_source=notifications&email_token=AAEYGXLULUVA2NSNODQFIKTQIHQJJA5CNFSM4IUDR56KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6BXPWI#issuecomment-528709593,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEYGXLU5LO5SERK6H4K6E3QIHQJJANCNFSM4IUDR56A
.

@howardjohn: once the present PR merges, I will cherry-pick it on top of release-1.3 in a new PR and assign it to you.

In response to this:

/cherrypick release-1.3
On Thu, Sep 5, 2019, 10:05 PM Romain Lenglet<EMAIL_ADDRESS>wrote:

This needs to be backported to release-1.3 ASAP. Please someone comment /cherrypick
release-1.3.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/16865?email_source=notifications&email_token=AAEYGXLULUVA2NSNODQFIKTQIHQJJA5CNFSM4IUDR56KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6BXPWI#issuecomment-528709593,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEYGXLU5LO5SERK6H4K6E3QIHQJJANCNFSM4IUDR56A
.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@duderino: cannot checkout release-1.3.: error checking out release-1.3.: exit status 1. output: error: pathspec 'release-1.3.' did not match any file(s) known to git.

In response to this:

/cherrypick release-1.3.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@howardjohn: new pull request created: #16875

In response to this:

/cherrypick release-1.3
On Thu, Sep 5, 2019, 10:05 PM Romain Lenglet<EMAIL_ADDRESS>wrote:

This needs to be backported to release-1.3 ASAP. Please someone comment /cherrypick
release-1.3.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/istio/istio/pull/16865?email_source=notifications&email_token=AAEYGXLULUVA2NSNODQFIKTQIHQJJA5CNFSM4IUDR56KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6BXPWI#issuecomment-528709593,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAEYGXLU5LO5SERK6H4K6E3QIHQJJANCNFSM4IUDR56A
.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ayj nice work!


32bit version

Hi,
Is it possible to have a 32 bit binary version in Release archive ?
Thanks

Hi, thank you the request.
I'll add 32bit build soon.

Done.
Could you test it on your environment? I don't have one myself.

You made it ! :)
run smoothly on Debian Strech / Ubutnu Trusty
thank you very much, I found you tool very useful !
Best regards

Hi,
just to add another way of sharing terminal (like tmux) but with screen
command.

first, start the session, named GOTTY, with command top

screen -dmS GOTTY top

then start Gotty

gotty screen -x GOTTY

you can interact with the GOTTY session from local terminal by:

screen -x GOTTY

Hope it helps ;)
Best Regards
On Thu, Aug 20, 2015 at 9:56 AM, Iwasaki Yudai<EMAIL_ADDRESS>wrote:

Done.
Could you test it on your environment? I don't have one myself.
—
Reply to this email directly or view it on GitHub
https://github.com/yudai/gotty/issues/29#issuecomment-132923631.

Thanks!


problem with golang recursive performance

Please answer these questions before submitting your issue. Thanks!
What version of Go are you using (go version)?
go 1.10
Does this issue reproduce with the latest release?
What operating system and processor architecture are you using (go env)?
What did you do?
If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.
What did you expect to see?
What did you see instead?
go:
package main
func fib(n int64) int64 {
if n <= 2 {
return 1
}
return fib(n-1) + fib(n-2)
}
func main() {
println(fib(45))
}
java:
public class Bench1 {
static long fib(long n) {
return n <= 2 ? 1 : fib(n - 1) + fib(n - 2);
}
public static void main(String[] args) {
System.out.println(fib(45));
}
}
java:3.1s
go:5.2s

Can you please provide some instructions for how to build both examples and how to reproduce your benchmarks. Thank you.

Your program calls fib 2,269,806,339 times.  I don't think you are measuring a problem with recursive calls, I think you are measuring the effectiveness of the JIT of the JVM you are using.  Looking at it another way, Go function calls do have a certain overhead, because of the use of growable stacks.  Java function calls, after using a JIT, do not need to have that overhead.
I'm going to close this issue because I don't think there is anything we can do here.  If you have specific suggestions about places where Go's code generation could be improved, please let us know.  Examples like this don't help us make anything better.


That's the default value that VS Code fills it with, so I'm assuming it is false by default. Please re-open if that's not so!


[Slider] Allow generic value type to narrow onChange value type

Duplicates

[X] I have searched the existing issues

Latest version

[X] I have tested the latest version

Summary 💡
Giving a number type as value should narrow onChange value param to number, and giving a number[] type as value should narrow onChange value param to number[]
Examples 🌈
<Slider value={1} onChange={(e, value:number) => {}} />
<Slider value={[1,2]} onChange={(e, value:number[]) => {}} />
Motivation 🔦
We currently have to do something like this
<Slider value={1} onChange{(e, value) => typeof value === 'number' ? ... : ...what do i do here? throw an error? } /> to make typescript happy

@DiegoAndai @eloiqs I would like to pick this up

@gitstart sure! let me know if you need any help

Hi, I've also encountered this issue, so I understand the concern. I've tried to work on a solution.
I've made some modifications to the method proposed by @sai6855 . - #38753
Could you please review them?
It has worked on the latest master branch.


OTOTOY

I honestly couldn't find any API for OTOTOY, but since it is a Japanese store, most of the help pages aren't in English, so there might be.
https://ototoy.jp
that said, perhaps it could be scraped for data, especially since it's one of the few stores I know that shows catalog numbers (for example, here).
important note, OTOTOY does keep seperate pages for Lossless and High-Resolution releases, which would be the same MusicBrainz release (all other data being the same, of course)

also, I believe releases might only be available in Japan, but I'm not 100% certain on that... I think they could be available in Europe too? again, not sure about the borders of availability (and if they change from release to release)

OTOTOY is restricted to the Japanese market only. Furthermore, they don't appear to have an API either.
That being said, they're a decent source of cover art; sometimes, they have even higher-resolution cover art than, e.g., iTunes. If there's any interest in pulling that in, I think it's programmatically possible to find the link to the full-resolution cover art; I believe one of the "Music Hoarders" userscripts that deals with cover art has OTOTOY as a provider. (I'd find the link but am away from my computer for the time being.)

TL;DR: found what I was talking about, but it relies on a centralized service that does not appear to provide source code. Might be worth that trouble to get cover art integration; might not. In any case, it appears possible to scrape the tracklist, and given a (scraped) link to imgs.ototoy.jp, you might be able to extract the cover art image yourself.
I was able to track down the userscript I mentioned before that uses a Music Hoarders project to get high-quality cover art from OTOTOY. The Git host for it is currently experiencing some issues, but I extracted a direct link to the script from my browser's copy:
https://notabug.org/SuperSaltyGamer/ame/raw/main/dist/musicbrainz.user.js
The repo itself would most likely be found at: https://notabug.org/SuperSaltyGamer/ame
IIRC the Ame project has a number of integrations with other sites that could be useful (or not) for this project, but this particular script, designed for use on MB's cover art addition page, basically just seeds some data about the release to this closed-source project's interface, then takes the cover art the user selects (and the project feeds it) and drops it into MB's image upload form. The "integration" link near the top of that page details how to programmatically interact with the project. IMO, the project has some benefits, but the fact that it (at least by appearances) is closed-source would not make me comfortable basing an application function on it if I were in kellnerd's shoes.
That being said, for cover art scraping, it might be possible to benefit from Image Max URL or the techniques it itself uses for OTOTOY. As for tracklist scraping, there might be some benefit in adopting techniques used in this userscript.


Lg

Hi Cr1cker what's the issue you are currently having with this so I can take a look at it.
It would help if you could let me know the following information as below

Type of Issue
LG TV Model and Model Number
WebOS and Firmware Version
What you've tried already so we can see if it can be worked on

Reprodev

Closing this ticket but please open a new one if you still need with the information above so I can look into it for you


return null when token claim is not present

closes #910

@liga-oz thanks for the fix, can you please document the change in the CHANGELOG.md as it may lead to incompatibilities for our consumers.

Hi @liga-oz could you please do me a favour and evaluate whether the code or the javadoc is wrong. With this code change NullPointerExceptions may occur on customers applications side, when we update the behaviour.

Previously null was never returned instead empty array was returned, but javadoc and @Nullable annotation on interface level indicated that it can be null, so I presume that application users should have considered the fact that the null might be returned. We can change the annotation and javadoc which have been there since last year, but here for example it is actually considered to be null: https://github.com/SAP/cloud-security-xsuaa-integration/blob/b6003b7e77262c122cfaef8efe0bbd820e56bfe3/java-security/src/main/java/com/sap/cloud/security/adapter/xs/XSUserInfoAdapter.java#L410
so I would guess that this bug should be fixed to return null values instead of empty array.


chore: Merge chore_release-7.2.0 into edge

Overview
This is an incremental merge of the in-progress chore_release-7.2.0 branch into edge.
We had to make a synchronized change between this repo's chore_release-7.2.0 branch and Opentrons/oe-core's main branch. (PRs #14480 and Opentrons/oe-core#134.)
So, we also need to synchronize this repo's edge branch. Otherwise, builds with this repo's edge and Opentrons/oe-core's main — which is most of them — will be broken because this repo's edge branch is missing the change.
Test Plan
Changelog
Review requests
Is my premise about our automatic builds correct?
Risk assessment
Low.

Replacing with #14495. I think there are some overeager branch protection rules preventing me from pushing to this.


How to preprocess ShapeNet models?

Hi, thanks for your great work. But it's still not clear that how to preprocess the 3D models. Based on my understanding of the paper, the number of triangle faces will be reduced to 800 with a tool called "Blender". However, "blender" is not in the depencency list of this repo.  Could you kindly share an example demonstrating how your team processed the original mesh data?
By the way, the computational cost of Autoencoder is a lot larger than GPT in my experiments. My guess is that the Graph CNN doesn't scale well when the number of nodes grows large (i.e. too many faces in the 3D model).  Bluring the model is one solution, but is it possible to break one giant 3D model to pieces and treat each piece as an independent model?

Hi, thanks for your great work. But it's still not clear that how to preprocess the 3D models. Based on my understanding of the paper, the number of triangle faces will be reduced to 800 with a tool called "Blender". However, "blender" is not in the depencency list of this repo. Could you kindly share an example demonstrating how your team processed the original mesh data?
By the way, the computational cost of Autoencoder is a lot larger than GPT in my experiments. My guess is that the Graph CNN doesn't scale well when the number of nodes grows large (i.e. too many faces in the 3D model). Bluring the model is one solution, but is it possible to break one giant 3D model to pieces and treat each piece as an independent model?

This repo isn't the original but a recreation/improved version, lucidrains is not related to the MeshGPT paper.
Blender isn't a requirement, they explained how they simplified the mesh so it was easier to train since it's less data and they could train on more complex models.
The total parameters of autoencoder is total around 8.8M parameters, it's very light but the residual vector quantization uses a lot of VRAM.  The transformer (100-200M) is much slower, in the paper they trained the autoencoder for 2 days using 2x A80 and the transformer for 4 days using 4x A80.
Sure, you can break it down but then the output won't fit since doesn't know how the rest looks :/

Hi, thanks for your great work. But it's still not clear that how to preprocess the 3D models. Based on my understanding of the paper, the number of triangle faces will be reduced to 800 with a tool called "Blender". However, "blender" is not in the depencency list of this repo. Could you kindly share an example demonstrating how your team processed the original mesh data?
By the way, the computational cost of Autoencoder is a lot larger than GPT in my experiments. My guess is that the Graph CNN doesn't scale well when the number of nodes grows large (i.e. too many faces in the 3D model). Bluring the model is one solution, but is it possible to break one giant 3D model to pieces and treat each piece as an independent model?

This repo isn't the original but a recreation/improved version, lucidrains is not related to the MeshGPT paper. Blender isn't a requirement, they explained how they simplified the mesh so it was easier to train since it's less data and they could train on more complex models.
The total parameters of autoencoder is total around 8.8M parameters, it's very light but the residual vector quantization uses a lot of VRAM. The transformer (100-200M) is much slower, in the paper they trained the autoencoder for 2 days using 2x A80 and the transformer for 4 days using 4x A80.
Sure, you can break it down but then the output won't fit since doesn't know how the rest looks :/

Many thanks for your reply. The mesh I was using has more than 3000 faces, and the quantization phase tried to assign VRAM for all nodes at the same time. That's why my experiment failed using 4x A100.
I'll see whether there is a better quantization algorithm to use, or is it possible to only process part of a large mesh each time.

Hi, thanks for your great work. But it's still not clear that how to preprocess the 3D models. Based on my understanding of the paper, the number of triangle faces will be reduced to 800 with a tool called "Blender". However, "blender" is not in the depencency list of this repo. Could you kindly share an example demonstrating how your team processed the original mesh data?
By the way, the computational cost of Autoencoder is a lot larger than GPT in my experiments. My guess is that the Graph CNN doesn't scale well when the number of nodes grows large (i.e. too many faces in the 3D model). Bluring the model is one solution, but is it possible to break one giant 3D model to pieces and treat each piece as an independent model?

This repo isn't the original but a recreation/improved version, lucidrains is not related to the MeshGPT paper. Blender isn't a requirement, they explained how they simplified the mesh so it was easier to train since it's less data and they could train on more complex models.
The total parameters of autoencoder is total around 8.8M parameters, it's very light but the residual vector quantization uses a lot of VRAM. The transformer (100-200M) is much slower, in the paper they trained the autoencoder for 2 days using 2x A80 and the transformer for 4 days using 4x A80.
Sure, you can break it down but then the output won't fit since doesn't know how the rest looks :/

Many thanks for your reply. The mesh I was using has more than 3000 faces, and the quantization phase tried to assign VRAM for all nodes at the same time. That's why my experiment failed using 4x A100. I'll see whether there is a better quantization algorithm to use, or is it possible to only process part of a large mesh each time.

It may be the due to the face_edges, it uses a lot VRAM when it generates these during runtime. For example; it used 6 GB for 1 batch of 6000 faces.
If you were using lets say batch size of 16x 6GB = 96GB VRAM. So it's really important to generate these beforehand.
If you generate the face_edges and save the dataset, you can then restart the session to clear the VRAM and load them back, the 6000 faces mesh then only used 500 MB.
https://github.com/lucidrains/meshgpt-pytorch/pull/37/commits/68693ac560123a04931575eca8e0fcc197e2f5a6

Hi, thanks for your great work. But it's still not clear that how to preprocess the 3D models. Based on my understanding of the paper, the number of triangle faces will be reduced to 800 with a tool called "Blender". However, "blender" is not in the depencency list of this repo. Could you kindly share an example demonstrating how your team processed the original mesh data?
By the way, the computational cost of Autoencoder is a lot larger than GPT in my experiments. My guess is that the Graph CNN doesn't scale well when the number of nodes grows large (i.e. too many faces in the 3D model). Bluring the model is one solution, but is it possible to break one giant 3D model to pieces and treat each piece as an independent model?

This repo isn't the original but a recreation/improved version, lucidrains is not related to the MeshGPT paper. Blender isn't a requirement, they explained how they simplified the mesh so it was easier to train since it's less data and they could train on more complex models.
The total parameters of autoencoder is total around 8.8M parameters, it's very light but the residual vector quantization uses a lot of VRAM. The transformer (100-200M) is much slower, in the paper they trained the autoencoder for 2 days using 2x A80 and the transformer for 4 days using 4x A80.
Sure, you can break it down but then the output won't fit since doesn't know how the rest looks :/

Many thanks for your reply. The mesh I was using has more than 3000 faces, and the quantization phase tried to assign VRAM for all nodes at the same time. That's why my experiment failed using 4x A100. I'll see whether there is a better quantization algorithm to use, or is it possible to only process part of a large mesh each time.

@xuanyaoming
I don't think that the quantizer is the issue then since you are using 4x A100. It may be the due to the face_edges, it uses a lot VRAM when it generates these during runtime. For example; i convert 6000 faces using a batch size of 1, it used 6 GB VRAM. If you were using lets say batch size of 16x 6GB = 96GB VRAM. So it's really important to generate these beforehand.
If you generate the face_edges and save the dataset, you can then restart the session to clear the VRAM and load them back, the 6000 faces mesh then only used 500 MB.
Take a look at how I do it in the pull request below. Mesh dataset pull request
Really awesome that you have access to so many powerful GPU's, I raised the quantizer VRAM issue due to I'm trying to optimize it to run it using 16 GB VRAM. Don't forget to augment the data so it can generalize better, you can reuse the same face_edges for a 3D model and it's variations since you not changing the faces but just the positions.

I'm not entirely sure whether you were talking about training phase or inference phase. But if you were talking about training, then the real issue in my program is probably the configs of huggingface accelerate, as the mesh data and batch_size I was using is smaller than yours (4000 faces and 1 batch_size). I also observed the GPU VRAM usage grew 10 times larger during inference while launching with accelerate, but the inference speed didn't improve much. To be honest, I'm not an expert in multi-GPU training, so the analysis above is just my personal speculation.
However, VRAM bottleneck is definitely a major issue. Because I also tried to train the autoencoder without huggingface accelerate, and it didn't work either. The error message was "out of memory". I'll see what I can do to reduce VRAM.

I'm not entirely sure whether you were talking about training phase or inference phase. But if you were talking about training, then the real issue in my program is probably the configs of huggingface accelerate, as the mesh data and batch_size I was using is smaller than yours (4000 faces and 1 batch_size).

I was talking about the training. During the training the autoencoder needs something called face_edges, these are used for the ConvSage to connect the faces.
If face_edges isn't passed to the model's forward when training it needs to generate these by itself, this is done by calling the function derive_face_edges_from_faces .
This function is very expensive since it process the entire batch at the same time. If you have 5k faces and 16 batch size, it generates 98GB VRAM.
The faces edges won't be saved since this responsibility belongs to the dataset, so each step this happens every time.
To avoid this problem; you can preprocess the mesh and generate the face_edges and save tons of VRAM. see here

I also observed the GPU VRAM usage grew 10 times larger during inference while launching with accelerate, but the inference speed didn't improve much. To be honest, I'm not an expert in multi-GPU training, so the analysis above is just my personal speculation.

I haven't tried using accelerate when inferencing and got no experience multi-GPU so I cannot help at the moment:/

However, VRAM bottleneck is definitely a major issue. Because I also tried to train the autoencoder without huggingface accelerate, and it didn't work either. The error message was "out of memory". I'll see what I can do to reduce VRAM.

What is the max count of faces & vertices of the mesh your training on?
Are you using the face_edges in the dataset? I can train the autoencoder on mesh models with 3200 faces and 1400 vertices models using my 3060 using 9.7 GB at 4 batch size, so I don't it's a model problem (unless it's directly related to multi-gpu)
I found it helpful to also normalize and remove the duplicate vertexes when preprocessing, are you also sorting the faces by lowest vertex index & vertices by yxz (where y = vertical).
def get_mesh(file_path):
    mesh = trimesh.load(file_path, force='mesh')

    vertices = mesh.vertices.tolist()
    faces = mesh.faces.tolist()
    centered_vertices = vertices - np.mean(vertices, axis=0)
    max_abs = np.max(np.abs(centered_vertices))

    vertices = centered_vertices / (max_abs / 0.95)

    def sort_vertices(vertex):
        return vertex[1], vertex[0], vertex[2]

    seen = OrderedDict()
    for point in vertices:
      key = tuple(point)
      if key not in seen:
        seen[key] = point

    unique_vertices =  list(seen.values())
    sorted_vertices = sorted(unique_vertices, key=sort_vertices)

    vertices_as_tuples = [tuple(v) for v in vertices]
    sorted_vertices_as_tuples = [tuple(v) for v in sorted_vertices]

    vertex_map = {old_index: new_index for old_index, vertex_tuple in enumerate(vertices_as_tuples) for new_index, sorted_vertex_tuple in enumerate(sorted_vertices_as_tuples) if vertex_tuple == sorted_vertex_tuple}

    reindexed_faces = [[vertex_map[face[0]], vertex_map[face[1]], vertex_map[face[2]]] for face in faces]
    return np.array(sorted_vertices), np.array(reindexed_faces)

I'm not entirely sure whether you were talking about training phase or inference phase. But if you were talking about training, then the real issue in my program is probably the configs of huggingface accelerate, as the mesh data and batch_size I was using is smaller than yours (4000 faces and 1 batch_size).

I was talking about the training. During the training the autoencoder needs something called face_edges, these are used for the ConvSage to connect the faces. If face_edges isn't passed to the model's forward when training it needs to generate these by itself, this is done by calling the function derive_face_edges_from_faces .
This function is very expensive since it process the entire batch at the same time. If you have 5k faces and 16 batch size, it generates 98GB VRAM. The faces edges won't be saved since this responsibility belongs to the dataset, so each step this happens every time.
To avoid this problem; you can preprocess the mesh and generate the face_edges and save tons of VRAM. see here

I also observed the GPU VRAM usage grew 10 times larger during inference while launching with accelerate, but the inference speed didn't improve much. To be honest, I'm not an expert in multi-GPU training, so the analysis above is just my personal speculation.

I haven't tried using accelerate when inferencing and got no experience multi-GPU so I cannot help at the moment:/

However, VRAM bottleneck is definitely a major issue. Because I also tried to train the autoencoder without huggingface accelerate, and it didn't work either. The error message was "out of memory". I'll see what I can do to reduce VRAM.

What is the max count of faces & vertices of the mesh your training on? Are you using the face_edges in the dataset? I can train the autoencoder on mesh models with 3200 faces and 1400 vertices models using my 3060 using 9.7 GB at 4 batch size, so I don't it's a model problem (unless it's directly related to multi-gpu)
I found it helpful to also normalize and remove the duplicate vertexes when preprocessing, are you also sorting the faces by lowest vertex index & vertices by yxz (where y = vertical).
def get_mesh(file_path):
    mesh = trimesh.load(file_path, force='mesh')

    vertices = mesh.vertices.tolist()
    faces = mesh.faces.tolist()
    centered_vertices = vertices - np.mean(vertices, axis=0)
    max_abs = np.max(np.abs(centered_vertices))

    vertices = centered_vertices / (max_abs / 0.95)

    def sort_vertices(vertex):
        return vertex[1], vertex[0], vertex[2]

    seen = OrderedDict()
    for point in vertices:
      key = tuple(point)
      if key not in seen:
        seen[key] = point

    unique_vertices =  list(seen.values())
    sorted_vertices = sorted(unique_vertices, key=sort_vertices)

    vertices_as_tuples = [tuple(v) for v in vertices]
    sorted_vertices_as_tuples = [tuple(v) for v in sorted_vertices]

    vertex_map = {old_index: new_index for old_index, vertex_tuple in enumerate(vertices_as_tuples) for new_index, sorted_vertex_tuple in enumerate(sorted_vertices_as_tuples) if vertex_tuple == sorted_vertex_tuple}

    reindexed_faces = [[vertex_map[face[0]], vertex_map[face[1]], vertex_map[face[2]]] for face in faces]
    return np.array(sorted_vertices), np.array(reindexed_faces)

Thanks for your codes. I didn't realize some vertices could be duplicate. As to the size of 3D assets, the max number of faces I observed was over 6K, and the subsequent number of face edges can reach 48358116 (really big, isn't it?). The GPUs were able to process them ( with proper configurations ) at first but soon failed on smaller data ( with only 233 faces and 50K face edges ).
I managed to rewrite and optimize the encode function of MeshAutoencoder, shrinked the space complexity of generating face_coords into O(n) (See my codes below), and disable the linear attention layer in the encoder modules temporarily. None of the effort works  because the SAGEConv layer can't handle large graphs. The only way to make meshGPT support high resolution 3D assets is to replace the graph convolution layer.
face_without_pad = faces.masked_fill(~rearrange(face_mask, 'b nf -> b nf 1'), 0)
partial_flattened_face_vertices = repeat(face_without_pad.view(batch, -1), 'b x -> b x c', c=num_coors)
face_coords = torch.gather(vertices, dim=1, index=partial_flattened_face_vertices).view(batch, num_faces, 3, num_coors)

I'm not entirely sure whether you were talking about training phase or inference phase. But if you were talking about training, then the real issue in my program is probably the configs of huggingface accelerate, as the mesh data and batch_size I was using is smaller than yours (4000 faces and 1 batch_size).

I was talking about the training. During the training the autoencoder needs something called face_edges, these are used for the ConvSage to connect the faces. If face_edges isn't passed to the model's forward when training it needs to generate these by itself, this is done by calling the function derive_face_edges_from_faces .
This function is very expensive since it process the entire batch at the same time. If you have 5k faces and 16 batch size, it generates 98GB VRAM. The faces edges won't be saved since this responsibility belongs to the dataset, so each step this happens every time.
To avoid this problem; you can preprocess the mesh and generate the face_edges and save tons of VRAM. see here

I also observed the GPU VRAM usage grew 10 times larger during inference while launching with accelerate, but the inference speed didn't improve much. To be honest, I'm not an expert in multi-GPU training, so the analysis above is just my personal speculation.

I haven't tried using accelerate when inferencing and got no experience multi-GPU so I cannot help at the moment:/

However, VRAM bottleneck is definitely a major issue. Because I also tried to train the autoencoder without huggingface accelerate, and it didn't work either. The error message was "out of memory". I'll see what I can do to reduce VRAM.

What is the max count of faces & vertices of the mesh your training on? Are you using the face_edges in the dataset? I can train the autoencoder on mesh models with 3200 faces and 1400 vertices models using my 3060 using 9.7 GB at 4 batch size, so I don't it's a model problem (unless it's directly related to multi-gpu)
I found it helpful to also normalize and remove the duplicate vertexes when preprocessing, are you also sorting the faces by lowest vertex index & vertices by yxz (where y = vertical).
def get_mesh(file_path):
    mesh = trimesh.load(file_path, force='mesh')

    vertices = mesh.vertices.tolist()
    faces = mesh.faces.tolist()
    centered_vertices = vertices - np.mean(vertices, axis=0)
    max_abs = np.max(np.abs(centered_vertices))

    vertices = centered_vertices / (max_abs / 0.95)

    def sort_vertices(vertex):
        return vertex[1], vertex[0], vertex[2]

    seen = OrderedDict()
    for point in vertices:
      key = tuple(point)
      if key not in seen:
        seen[key] = point

    unique_vertices =  list(seen.values())
    sorted_vertices = sorted(unique_vertices, key=sort_vertices)

    vertices_as_tuples = [tuple(v) for v in vertices]
    sorted_vertices_as_tuples = [tuple(v) for v in sorted_vertices]

    vertex_map = {old_index: new_index for old_index, vertex_tuple in enumerate(vertices_as_tuples) for new_index, sorted_vertex_tuple in enumerate(sorted_vertices_as_tuples) if vertex_tuple == sorted_vertex_tuple}

    reindexed_faces = [[vertex_map[face[0]], vertex_map[face[1]], vertex_map[face[2]]] for face in faces]
    return np.array(sorted_vertices), np.array(reindexed_faces)

Thanks for your codes. I didn't realize some vertices could be duplicate. As to the size of 3D assets, the max number of faces I observed was over 6K, and the subsequent number of face edges can reach 48358116 (really big, isn't it?). The GPUs were able to process them ( with proper configurations ) at first but soon failed on smaller data ( with only 233 faces and 50K face edges ).
I managed to rewrite and optimize the encode function of MeshAutoencoder, shrinked the space complexity of generating face_coords into O(n) (See my codes below), and disable the linear attention layer in the encoder modules temporarily. None of the effort works because the SAGEConv layer can't handle large graphs. The only way to make meshGPT support high resolution 3D assets is to replace the graph convolution layer.
face_without_pad = faces.masked_fill(~rearrange(face_mask, 'b nf -> b nf 1'), 0)
partial_flattened_face_vertices = repeat(face_without_pad.view(batch, -1), 'b x -> b x c', c=num_coors)
face_coords = torch.gather(vertices, dim=1, index=partial_flattened_face_vertices).view(batch, num_faces, 3, num_coors)

Oh, I think I understand your issue.
I can't remember the specifics but when I created the generate_face_edges I tested with chunking the faces together and created batches e.g [4, faces,4 ]. But the face_edges that returned was way too big for all the faces.
When I padded the batches I used 0 as padding, but since 0 is a valid index to the vertex list, it thought it was a valid face edge.
Instead of padding with 0 I should have used pad_id which is -1.
If you are creating your own trainer, keep in mind to pad the batches with -1 instead of zeros.
Here is the correct face edges count:
faces  torch.Size([6206, 3])
vertices  torch.Size([3199, 3])
face_edges  torch.Size([24706, 2])

faces  torch.Size([3232, 3])
vertices  torch.Size([1630, 3])
face_edges  torch.Size([12928, 2])

Oh, I think I understand your issue. When I generated the face edges I tested with chunking the faces together and created batches e.g [4, faces, 3 ]. but the return edges was too many to make sense.
When I padded the batches I used 0 as padding, but since 0 is a valid index to the vertex list, it threaded it was a valid face edge and created padding_amount * padding_amount face edges.
Instead of padding with 0 I should have used pad_id which is -1. So take a look and see if this is the problem, I find it better to use a batch size of 1 since the data wont be padded and is using minimal VRAM to find the edges.
Here is a example of some meshes and the correct face edges count:
faces  torch.Size([6206, 3])
vertices  torch.Size([3199, 3])
face_edges  torch.Size([24706, 2])

faces  torch.Size([3232, 3])
vertices  torch.Size([1630, 3])
face_edges  torch.Size([12928, 2])

Thanks a lot for your idea. However I don't think that's the issue, because I was using the default pad_id which is -1. Additionally, I was using the "plane" category of ShapeNetV2. The data is very clean and should have already been normalized and filtered. I tried preprocessing the dataset with your code but the results were identical. The final solution I chose was simply dropping all meshes whose number of faces are larger than 4000. It works for now, at the expense of losing large portion of the dataset.
Personally, the ideal solution I was hoping for is an open-source mesh quantizer, or any other function that can replace the graph convolutional layers efficiently. A preprocessing technique that can both simplify meshes and preserve all notable features is also good enough. I'll settle with dropping complex meshes for now but open for new ideas.

Oh, I think I understand your issue. When I generated the face edges I tested with chunking the faces together and created batches e.g [4, faces, 3 ]. but the return edges was too many to make sense.
When I padded the batches I used 0 as padding, but since 0 is a valid index to the vertex list, it threaded it was a valid face edge and created padding_amount * padding_amount face edges.
Instead of padding with 0 I should have used pad_id which is -1. So take a look and see if this is the problem, I find it better to use a batch size of 1 since the data wont be padded and is using minimal VRAM to find the edges.
Here is a example of some meshes and the correct face edges count:
faces  torch.Size([6206, 3])
vertices  torch.Size([3199, 3])
face_edges  torch.Size([24706, 2])

faces  torch.Size([3232, 3])
vertices  torch.Size([1630, 3])
face_edges  torch.Size([12928, 2])

Thanks a lot for your idea. However I don't think that's the issue, because I was using the default pad_id which is -1. Additionally, I was using the "plane" category of ShapeNetV2. The data is very clean and should have already been normalized and filtered. I tried preprocessing the dataset with your code but the results were identical. The final solution I chose was simply dropping all meshes whose number of faces are larger than 4000. It works for now, at the expense of losing large portion of the dataset.
Personally, the ideal solution I was hoping for is an open-source mesh quantizer, or any other function that can replace the graph convolutional layers efficiently. A preprocessing technique that can both simplify meshes and preserve all notable features is also good enough. I'll settle with dropping complex meshes for now but open for new ideas.

That is very weird, I'm not actually sure if that's the model fault. Having 48 358 116 face edges is 100% an error.
I also have access to the shapenet dataset so could you provide me with the filename(s) for the mesh that have such high count of face edges?
The quantizer doesn't input any face edges, only the faces and it's embeddings of the ConvSage e.g [Faces x 576].
I've tried with 6k faces and it only got up to 5-7GB in VRAM when training the autoencoder.
However the ConvSage does input the face edges but it's a tiny model with 600k parameters and the VRAM barely moved when I tried a 16k face mesh.

I also have access to the shapenet dataset so could you provide me with the file/folder name(s) for the mesh that have such high count of face edges?

Sure, the 3D asset works for me is 02691156/feb0bd0dcd155af4cc34b900bb2492e/models/model_normalized.obj, with 607 faces and 367236 face edges. Another bigger example is 02691156/fdb107ea7a600f5fe3f3aec01b086ba/models/model_normalized.obj, with 3555 faces and 6687396 face edges. A more complex one is 02691156/ece25d89898c7d9090b1d6deb98feec6/models/model_normalized.obj, with 5511 faces and 30360100 face edges. The number of face edges is printed from the the shape of face_edges tensor in the encoder function of MeshAutoencoder before it's passed into the for-loop. I agree that there is an error, because in the last example it's unlikely for one face node to have 10k neighbors on average, considering there are only 5k face nodes in total.

However the ConvSage does input the face edges but it's a tiny model with 600k parameters and the VRAM barely moved when I tried a 16k face mesh.

The reason for your success is that the number of face edges is small. Pytorch has to allocate extra memories in the computational graph for each face edge in the ConvSage phase, which is why it needed so much VRAM in my experiments. I agree that the number of trainable parameters is not the issue here, because both the autoencoder and GPT2 are small models.
Anyway, I'll recheck derive_face_edges_from_faces function in data.py. That's where the problem most likely lies.

Sure, the 3D asset works for me is 02691156/feb0bd0dcd155af4cc34b900bb2492e/models/model_normalized.obj, with 607 faces and 367236 face edges. Another bigger example is 02691156/fdb107ea7a600f5fe3f3aec01b086ba/models/model_normalized.obj, with 3555 faces and 6687396 face edges. A more complex one is 02691156/ece25d89898c7d9090b1d6deb98feec6/models/model_normalized.obj, with 5511 faces and 30360100 face edges. The number of face edges is printed from the the shape of face_edges tensor in the encoder function of MeshAutoencoder before it's passed into the for-loop. I agree that there is an error, because in the last example it's unlikely for one face node to have 10k neighbors on average, considering there are only 5k face nodes in total.

Thank you, I checked the shapenet files but I'm getting different face count then you.
The only file i managed to generate face edges was the feb0bd0dcd155af4cc34b900bb2492e and I got 4868 face_edges.
I think there has been some mistake since those other meshes has 44k and 157k faces, how are you loading the meshes or counting the faces?
I checked the glb dataset and ShapeNetCore but the face count was same for both.
Here are the files I checked if you want to double check:  https://file.io/A2FtnExPHbWv
ece25d89898c7d9090b1d6deb98feec6: 44 410 face/triangles
fdb107ea7a600f5fe3f3aec01b086ba: 157 302 face/triangles
File:  feb0bd0dcd155af4cc34b900bb2492e.obj
Faces  torch.Size([606, 3])
Vertices  torch.Size([163, 3])
face_edges  torch.Size([4868, 2])

Anyway, I'll recheck derive_face_edges_from_faces function in data.py. That's where the problem most likely lies.

Fixed the issue. The version of meshGPT I installed with pip isn't up-to-date. By adding parentheses around num_shared_vertices >= face_edges_vertices_threshold in the derive_face_edges_from_faces function can reduce the number of face edges to normal. The contributors must have noticed the bug and fixed it two weeks ago, but somehow the pip version wasn't updated.

Thank you, I checked the shapenet files but I'm getting different face count then you. The only file i managed to generate face edges was the feb0bd0dcd155af4cc34b900bb2492e and I got 4868 face_edges. I think there has been some mistake since those other meshes has 44k and 157k faces, how are you loading the meshes or counting the faces?
I checked the glb dataset and ShapeNetCore but the face count was same for both. Here are the files I checked if you want to double check: https://file.io/A2FtnExPHbWv
ece25d89898c7d9090b1d6deb98feec6: 44 410 face/triangles fdb107ea7a600f5fe3f3aec01b086ba: 157 302 face/triangles

Thanks a lot for checking the 3D sasset files. But the file name is a little bit incorrect. I was using the normalized version of the same 3D asset, downloaded from this link. Maybe normalization made the number of faces smaller? By the way, where did you download the original dataset?

Have you as well used derive_face_edges_from_faces when you are processing and creating the dataset? Since when AutoEncoder is creating the face_edges, it wont save due to it uses a copy of the dataset that is created by dataloader, also; it does the entire batch the same time so that will generate a lot of VRAM.

No, I didn't do that. But in the encoder function, MeshAutoencoder will do it automatically.

Fixed the issue. The version of meshGPT I installed with pip isn't up-to-date. By adding parentheses around num_shared_vertices >= face_edges_vertices_threshold in the derive_face_edges_from_faces function can reduce the number of face edges to normal. The contributors must have noticed the bug and fixed it two weeks ago, but somehow the pip version wasn't updated.

Ah that's great!

Thanks a lot for checking the 3D sasset files. But the file name is a little bit incorrect. I was using the normalized version of the same 3D asset, downloaded from this link. Maybe normalization made the number of faces smaller? By the way, where did you download the original dataset?

Oh I extracted and renamed it, it had the same path in the zip file (ShapeNetCoreV2).
The original name was:
ShapeNetCore.v2.zip\ShapeNetCore.v2\02691156\ece25d89898c7d9090b1d6deb98feec6\models\model_normalized.obj
The data should be the same, I got access to the glb (version 2)  and the ShapeNetCore-archive (both v1 and v2). Take a look at the 3d mesh files I uploaded and let me know if it's the same or you can view yours on 3dviewer to get a accurate face count.
I got both the versions of it since the v1 is the only one that contains the labels and tags. Example:
fullId,wnsynset,wnlemmas,up,front,name,tags
3dw.a4678e6798e768c3b6a66ea321171690,"02842573,04012084,02691156","biplane,propeller plane,airplane,aeroplane,plane","0.0\,0.0\,1.0","0.0\,1.0\,0.0",Old-school Yellow Biplane,

Have you as well used derive_face_edges_from_faces when you are processing and creating the dataset? Since when AutoEncoder is creating the face_edges, it wont save due to it uses a copy of the dataset that is created by dataloader, also; it does the entire batch the same time so that will generate a lot of VRAM.

No, I didn't do that. But in the encoder function, MeshAutoencoder will do it automatically.

Correct, but since it sets the faces to the copy that exists in the dataloader, it vanishes and not saved. I'm not sure if it's removed each time it cycles but if you preprocess the face_edges you wont need to preprocess 16 or 32 meshes at same time due to the batch size.

ece25d89898c7d9090b1d6deb98feec6: 44 410 face/triangles
fdb107ea7a600f5fe3f3aec01b086ba: 157 302 face/triangles

Rechecked my codes. I mixed up the outputs of those data items because they were processed in parallel. The numbers generated by my code matches the yours. I hope that means my mesh reader is safe to use.

ece25d89898c7d9090b1d6deb98feec6: 44 410 face/triangles
fdb107ea7a600f5fe3f3aec01b086ba: 157 302 face/triangles

Rechecked my codes. I mixed up the outputs of those data items because they were processed in parallel. The numbers generated by my code matches the yours. I hope that means my mesh reader is safe to use.

Ah great, I'm guessing that you resolved your issue and no longer have the issue about huge numbers of face edges?
You should then be fine to train on 6k face meshes without any VRAM issues.

Ah great, I'm guessing that you resolved your issue and no longer have the issue about huge numbers of face edges?
You should then be fine to train on 6k face meshes without any VRAM issues.

VRAM issues always exist. Because expanding GPT sequence length also leads to out-of-memory.  But I think seperate autoencoder from transformer will help a lot, Another solution in my mind is mixed-precision training, which should be able to reduce the VRAM usage to half. If neither solution works, I will consider replace GPT with the latest varients. A lot of recent papers are relating to this topic.
Anyway, I'll close this issue for now.

Ah great, I'm guessing that you resolved your issue and no longer have the issue about huge numbers of face edges?
You should then be fine to train on 6k face meshes without any VRAM issues.

VRAM issues always exist. Because expanding GPT sequence length also leads to out-of-memory. But I think seperate autoencoder from transformer will help a lot, Another solution in my mind is mixed-precision training, which should be able to reduce the VRAM usage to half. If neither solution works, I will consider replace GPT with the latest varients. A lot of recent papers are relating to this topic.
Anyway, I'll close this issue for now
.
@xuanyaoming
But the autoencoder is already separate, after the autoencoder generates the codes it's finished and never ran again while training.
So if you generate the codes beforehand you'll never need to run the autoencoder together with the transformer.


New package: Supermicro.IMPIView version 2.21.1

Checklist for Pull Requests

[x] Have you signed the Contributor License Agreement?
[x] Resolve #191712

Manifests

[x] Have you checked that there aren't other open pull requests for the same manifest update/change?
[x] This PR only modifies one (1) manifest
[x] Have you validated your manifest locally with winget validate --manifest <path>?

[x] Does your manifest conform to the 1.6 schema?

Note: <path> is the directory's name containing the manifest you're submitting.
Manual validation

[!WARNING]
"Archive scan" feature of winget.exe detected malware in the ZIP archive. It might not pass the validation pipeline.

Microsoft Reviewers: Open in CodeFlow

The package manager bot determined there was an issue with installing the application correctly. Please check the application installs correctly. Once repaired, please push an update to your pull request.
Template: msftbot/validationError/installation/general

CC @stephengillie, could you please investigate if it's the reason already mentioned above? 🙏🏼

Related https://github.com/microsoft/winget-pkgs/issues/191712#issuecomment-2478986667

Automatic Validation ended with:

Installation failed with exit code -2145844845
2024-11-15 06:52:14.092 [CORE] DeliveryOptimization error: 0x80190193, extended error: 0x00000000
2024-11-15 06:52:14.171 [FAIL] D:\a_work\1\s\external\pkg\src\AppInstallerCommonCore\DODownloader.cpp(289)\WindowsPackageManager.dll!78E757EB: (caller: 78E75E1C) Exception(1) tid(770) 80190193 Forbidden (403).
2024-11-15 06:52:14.800 [CORE] Download request failed. Returned status: 403
2024-11-15 06:52:14.800 [FAIL] D:\a_work\1\s\external\pkg\src\AppInstallerCommonCore\Downloader.cpp(148)\WindowsPackageManager.dll!78E6FA75: (caller: 78E70E2B) Exception(2) tid(770) 80190193 Forbidden (403).
2024-11-15 06:52:14.802 [CLI ] Failed to download, waiting a bit and retry. Url: https://www.supermicro.com/Bios/sw_download/585/IPMIView_2.21.1_build.230720_Windows.zip
2024-11-15 06:52:15.456 [CORE] DeliveryOptimization error: 0x80190193, extended error: 0x00000000
2024-11-15 06:52:15.456 [FAIL] D:\a_work\1\s\external\pkg\src\AppInstallerCommonCore\DODownloader.cpp(289)\WindowsPackageManager.dll!78E757EB: (caller: 78E75E1C) Exception(3) tid(770) 80190193 Forbidden (403).
2024-11-15 06:52:15.571 [CORE] Download request failed. Returned status: 403
2024-11-15 06:52:15.571 [FAIL] D:\a_work\1\s\external\pkg\src\AppInstallerCommonCore\Downloader.cpp(148)\WindowsPackageManager.dll!78E6FA75: (caller: 78E70E2B) Exception(4) tid(770) 80190193 Forbidden (403).

Automated error analysis suggests -2145844845 may mean Forbidden (403).
Automated error analysis suggests 80190193 may mean Forbidden (403)
(Automated response - build 897.)

@wingetbot run

[!NOTE]
It seems an x64 application packaged by a x86 install wizard creator.


[Stable] Replace most occurrences of "indexes" with "indices" (#557)

(cherry picked from commit b50578374346312aa147ff061f07baf676600304)

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

11 of 11 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.002%) to 98.496%

Totals

Change from base Build<PHONE_NUMBER>:
-0.002%

Covered Lines:
11791

Relevant Lines:
11971

💛  - Coveralls


[FEATURE] Make the operation/session/engine idle timeout configurable at client side.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Search before asking

[X] I have searched in the issues and found no similar issues.

Describe the feature
The idle timeout settings can only work if they are defined in the kyuubi-default. It is better to make them cnfigurable at client side, so that different sessions/engines/operations can have different timeout settings.
If this is not achievable, we should at least update the docs to tell users the limitations.
Motivation
No response
Describe the solution
No response
Additional context
No response
Are you willing to submit PR?

[ ] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to improve.
[ ] No. I cannot submit a PR at this time.

cc @zhaohehuhu, do have time to take a look at this issue?

Oh, the engine timeout has supported on client side, right? Please point out if I am wrong.

BTW, can we rename kyuubi.session.engine.idle.timeout to kyuubi.engine.idle.timeout?

It is also worth to well explain the definition of idle in the doc

BTW. Does each Engine have a SessionManager, or there is only one SessionManager in the server?

Does each Engine have a SessionManager

Yes, the internal of Kyuubi could be illustrated by

BTW, can we rename kyuubi.session.engine.idle.timeout to kyuubi.engine.idle.timeout?

+1

cc @zhaohehuhu, do have time to take a look at this issue?

Got it.


HydroShare resource timing out b94e2da3b5094cdfa679ad31fe7fb09d

Description of the bug
The resource b94e2da3b5094cdfa679ad31fe7fb09d is timing out and giving a 404 error. you can find the listing through discover but can't open the page (or at least I can't).
Steps to reproduce the bug:

Go to '...'

I just tried, didn't time out for me

Wow I guess it is just me we can just close this issue then.

well hold on, the timings do look really slow

I'm going to check it out

I poked around a little bit and noticed that the response time for viewing the resource is much slower than in edit mode.  That should help narrow down the scope to investigate by quite a bit.  Perhaps injecting some timings on beta (it is slow on beta as well) would help determine what part of the request is slow for this resource.
https://github.com/hydroshare/hydroshare/blob/develop/hs_core/page_processors.py#L121-L214

The page template might need to be checked too.

Still poking around, the update_view_count method is taking about 20 seconds.  https://github.com/hydroshare/hydroshare/blob/develop/hs_core/page_processors.py#L122

Interesting
recent change:
https://github.com/hydroshare/hydroshare/pull/4938/files#diff-3bc7182059367a2c6cf9e96428d08a0ce66f709baec5d00090abd19683582891L2065

so content_model.save() is really slow for this resource. It has a lot of .cpg files

Thanks @sblack-usu sounds like we should try a content_model.update() as you suggested.

@devincowan and @sblack-usu I did some checking on this issue. The update_view_count() could take a while if there are too many aggregations in the resource due to solr indexing triggered as a result of resource.save(). I think we can change the code for update_view_count() to do an update query to increment view_count to bypass solr indexing. I don't think we are currently indexing view_count.

nice @pkdash I was thinking it might be something in our signals but I hadn't yet suspected SOLR -- good sleuthing, thanks!

It'd be nice to make the signal processor for solr async... perhaps something like this? https://pypi.org/project/django-haystack-channels/

not a bad idea @sblack-usu
I just dropped a breakpoint in handle_save and was surprised that updating the title of a resource, a simple change, results in 6 calls to handle_save!

<Title: Untitled Resourceaaaa>
<BaseResource: Untitled Resourceaaaa>
<CompositeResource: Untitled Resourceaaaaa>
<Date: modified 2023-03-21T13:15:33.646070+00:00>
<BaseResource: Untitled Resourceaaaaa>
<Variable: Variable object (920)>
1 and 4 are sensical enough. I can potentially see why we need 2 to update the resource index even though we update the Title metadata in 1. I don't understand why we need 3, but I can be convinced easily. 5 is perplexing...seems like a duplicate -- for a simple title change we index the BaseResource to  twice!? oh but I do see that handle_save calls itself recursively so that prob what's going on. And then 6 I believe this is an hs_tracking Variable which doesn't get indexed so I guess it's okay.

I feel like some of these calls can obviously be async as @sblack-usu suggested. But might we run into race conditions etc if we are saving the updated date (4) and the BaseResource (2, 5)? Think the answer is "NO" because we are just talking about updating the indexes async, so I would think there wouldn't be a problem (vs for instance, actually saving to the Django db).

@pkdash Scott and I took a look just now.
Seems like we might want to revise our signal processor, custom haystack processor
We are thinking there could be significant performance gain if we trigger (async?) indexing not for all saved objects but just for Raccess and Date metadata objects with type=modified
Pabitra, does that approach make sense to you? I haven't actually looked into the particular instance of a res with lots of aggregations but presumably we could make similar changes in that indexing process

@devincowan I agree with the short-term fix idea.


Errors at startup - HASS Core 2024.08.01

After updating to 2024.08.01 the system log reports these two errors on startup:
2024-08-12 10:45:48.261 WARNING (ImportExecutor_0) [py.warnings] /usr/local/lib/python3.12/site-packages/paramiko/pkey.py:100: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from this module in 48.0.0.
"cipher": algorithms.TripleDES,
2024-08-12 10:45:48.510 WARNING (ImportExecutor_0) [py.warnings] /usr/local/lib/python3.12/site-packages/paramiko/transport.py:258: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from this module in 48.0.0.
"class": algorithms.TripleDES,

I can't see this problem. Don't know how to reproduce it.


[fix][test] Using another thread instead of io-thread for bookie

Master Issue: #17520
Motivation
In the bookkeeper, the request processor should use a dedicated thread pool, not the io thread pool to process the request. Using io thread pool may cause some unexpected behaviors, such as a deadlock or something else. In the production, it doesn't make sense that use io thread pool to handle everything. So change the working thread to 1 to fix the problem.
Modifications

change AddWorkerThreads, ReadWorkerThreads, and HighPriorityWorkerThreads to 1

Verifying this change

[ ] Make sure that the change passes the CI checks.

(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API: (yes / no)
The schema: (yes / no / don't know)
The default values of configurations: (yes / no)
The wire protocol: (yes / no)
The rest endpoints: (yes / no)
The admin cli options: (yes / no)
Anything that affects deployment: (yes / no / don't know)

Documentation
Check the box below or label this PR directly.
Need to update docs?

[ ] doc-required
(Your PR needs to update docs and you will update later)

[x] doc-not-needed
(Please explain why)

[ ] doc
(Your PR contains doc changes)

[ ] doc-complete
(Docs have been already added)

Discussed with @codelipenghui and @hangc0276. We think the error happened caused by bookkeeper fix, and it doesn't impact the pulsar too much, so it is unnecessary to change the pulsar to tolerate that case.


Prisma enviorment related error

Bug description
I want to unit testing the shopify app code functions with the help of library "jest", but for this i changed test enviorment to "jest-environment-jsdom" . and see the below error :-
PrismaClient is unable to run in this browser environment, or has been bundled for the browser (running in Node.js).
so, please solved my problem
How to reproduce

Expected behavior
No response
Prisma information

// Add your schema.prisma

// Add your code using Prisma Client

Environment & setup

OS:
Database:
Node.js version:

Prisma Version

Hi @jas4f
The error you're encountering is related to using PrismaClient in a browser-like environment, which is not supported.
The problem stems from setting your Jest test environment to jest-environment-jsdom. This environment simulates a browser, which is incompatible with PrismaClient.
To solve this issue, you should change your Jest test environment to the default node environment. You can also look at this previously closed issue.


Runtime Requirements

Here it says that Avalonia requires .NET Standard 2.0
https://docs.avaloniaui.net
Here it says that Avalonia requires .NET Core 3.1
https://github.com/AvaloniaUI/Avalonia/wiki/Runtime-Requirements
Which is it?
And what are the requirements of Avalonia 11?

.NET Standard 2.0 is not an executable target.
Technically, Avalonia can run anything that supports .NET Standard 2.0. But we officially provide support to .NET Core 3.1 and newer.

Same as with Win8.1 and Win7. Technically runs on Win7, but we don't test on it, don't spend time on it, and guarantee nothing for it.

As for the third-party libraries, I would recommend targeting .NET Standard 2.0 + .NET 6 (as a multitarget)
Unless you need specific features from newer targets and can't do it with .NET Standard 2.0 apis.

Documentation in the Wiki should be updated to clarify that.


Better Support of Legal Names and Identifiers

Feature Summary
The Portal registration flow and its overall UX and UI must be compatible with our customers' legal names and identifiers.
Change Description
Business Value and Requirements
Allow all customers to register smoothly with both their legal names and identifiers:

Have legal name used only where it is needed

Use customer-chosen brand name everywhere else

Clear support and validation patterns for legal identifiers

Great UX/UI integration

High-Level Overview

Review the use of legal and brand names across the portal and modify them according to business requirements

Modify all form fields to match both legal and brand name requirements

Improve UX/UI for legal and brand names

Support and validate the legal identifiers based on the Gaia-X Digital Clearing House list

Detailed Analysis

Customer registration:

Allow the use of any legal name based on country specificity

Allow the choice of a brand name with limited characters

Allow the use of legal identifiers as per the Gaia-X Clearing House list

Use the brand name in all subsequent fields, unless the legal name is required

Where the legal name is required, adjust the UI

Where the brand name can be used, adjust the UI

For each supported legal identifier, have the correct validation rules (REGEX)

Impacted Components

BPDM can process all legal names and has no restrictions or REGEX in place

Acceptance Criteria

Portal should support countries as per the Catena-X official White List (+ Yellow List under certain conditions).

100% of all legal names in the countries we officially support are accepted in the registration process

Very long names

Country-specific with unique characters

Legal names are used only where required, brand names are otherwise used

Legal identifiers should match the Gaia-X Clearing House official list

OSP functionality should also comply with the defined requirements

Test Scenarios

For each supported country, successful registration of an edge case legal name (very long or special characters) + identifier

Use of brand name across:

Registration process

User invitation process

Company detail pages

App release process

Service release process

Risks/Dependencies

Gaia-X Clearing House

@ybidois  not sure about this feature. Is it planned to present it in tomorrow's planning? It was never presented in a Refinement meeting. Did we miss something?

Regarding identifier types, we have an overlap with https://github.com/eclipse-tractusx/sig-release/issues/605. Let's work together here.

We probably want to store the REGEX in BPDM rather than in the Portal. @nicoprow: Please confirm.
Regarding GXDCH identifier types, there is a first implementation proposal for portal-backend here: https://github.com/eclipse-tractusx/portal-backend/issues/895

Regarding "brand name": please use the term "short name" from BPDM as defined in CX-0012.

Regarding identifier types, we have an overlap with Harmonization and Enhancement of identifier types #605. Let's work together here.
* We probably want to store the REGEX in BPDM rather than in the Portal. @nicoprow: Please confirm.
* Regarding GXDCH identifier types, there is a first implementation proposal for portal-backend here: [Add new BPDM identifier types portal-backend#895](https://github.com/eclipse-tractusx/portal-backend/issues/895)

Regarding "brand name": please use the term "short name" from BPDM as defined in CX-0012.

Any validation on the identifier types eventually also needs to be performed in the Pool. Yea we will need to align on the REGEX and we can also make it available over the BPDM API to reduce duplication and reduce the risk of mismatched validations.

Hi @Sebastian-Wurm, @nicoprow, I agree with what you propose:

Using the short name → I updated the feature description accordingly
Aligning on REGEX, storing in BPDM, and making it available over BPDM API
It will reduce a lot of errors on all sides.

Let me check with @evegufy and the Open-Source Portal team the best way to work together on this and we will get back to you.

Updated with new template

The scope feels very different for short name + identifier changes.

Short name UX/UI adjustments with clear REGEX to limit length and characters
Identifier validation in general and especially with dependency to BPDM validation

Maybe a bit late to split up.
Was there already some refinement on what methods BPDM team wants to provide?

I don't think it's too late to split the scope. We could keep this one for names and have identifiers as part of the Registration Process Improvements here → https://github.com/eclipse-tractusx/sig-release/issues/966.

Some hints from Release Management (@ther3sa) and Tractus-X Project Lead (@stephanbcbauer)

Please add missing sections from the feature template, or fill them out
Please add committer


Can't mine on HashCryptos

Hi, I'm having an issue mining DogeCoin on HashCryptos.
I've activated the address, waited for few minutes, and tried but received the error One or more miner binaries are showing rapid failing...
Mining on Zergpool.com works fine.

Verthash One Click Miner 2.0 Final

Windows 11 / GTX 1050

debug.log https://pastebin.com/zNxpJMG4

Are you still having this issue?

Closing due to inactivity.


Android: bad response for v-touch="tap:.."

On multiple Android devices, I get very bad response for:

Tap me

It requires multiple, very deliberate clicks to have the app respond. The padding area around the div is large enough.
Any ideas? Thanks!

I changed the event-listners back to

Tap me

and then it works great for Android (with the 300ms delay, but response is much better that above)

This might have something to do with the old version of Hammer v-touch is
currently using. I'll take a look at this later, but in the meanwhile you
can try using a newer version of Hammer.
On Thu, Apr 2, 2015 at 4:42 AM Klaas van der Molen<EMAIL_ADDRESS>wrote:

I changed the event-listners back to
Tap me
and then it works great for Android (with the 300ms delay, but response is
much better that above)
—
Reply to this email directly or view it on GitHub
https://github.com/vuejs/vue-touch/issues/12#issuecomment-88828159.

我也遇到一个很奇怪的问题，在我的chrome模拟器上，我使用v-on="tap:function()" 是可以响应事件的，但同一份代码，我的同事就不可以，很奇怪。到底tap是一个什么支持情况呢？

Closing this for inactivity. If the problem persists, please open a new issue.


1.19.2 Fabric?

 Since the jar says it only support 1.19 or something, I'm assuming that it's not a bug. So I'm wondering when/if there will be a version released for 1.19.2? I don't mind building it myself from the source code, but I can't see any gradle tasks to do that.

Have you tried version 2.3.0?

I compiled the jar from the project files, but when I tried loading it, the server spat out an error about not being able to load some classes, so I'm assuming I didn't compile it correctly

You can download version 2.3.0 from Modrinth.

Oh you can.  Thank anyways, it solved my problem.


Improve dataset metainformation

Add the following metainformation:
scaaml_version
firmware_url
paper_url
licence
Fix issue #31

Thanks Karel! LGTM. Only thing I spotted: since license/licence are valid spellings, maybe print out an alert if you find the unsupported version in the config (I'm partial to the American spelling, but I'm ok with either, as long as we're consistent)

@invernizzi PTAL Thank you for the review and patience. I have implemented the spelling check. I left the 'licence' spelling to match issue #31

Thank you, Karel!


Can't set model directory with HF

Is there any way to be able to set the model directory similar to load_faster_whisper via download_root?

You try to use the directory instead of the mode name.
https://github.com/huggingface/transformers/blob/58e3d23e97078f361a533b9ec4a6a2de674ea52a/src/transformers/utils/hub.py#L289-L293

Sorry, but what would that look like in the context of
model = stable_whisper.load_hf_whisper('medium', device='cuda') result = model.transcribe("test.mkv", batch_size=2)?

Use the path to the directory instead of 'medium'.

Looks like using the environment variable "HF_HUB_CACHE" is the simplest solution.  os.environ['HF_HUB_CACHE'] = /foo/bar


⚠ Bofrost.at has degraded performance

In 60447c4, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1223 ms

Resolved: Bofrost.at performance has improved in 434836f after 20 minutes.


refactor: dead code removed

I found a dead code and removed it.

I prefer to keep these guards here, even if they technically can't be reached.


Two different links to glossary

The faq links to one glossary: https://doc.perl6.org/language/faq#Is_there_a_glossary_of_Perl_6_related_terms%3F
but https://doc.perl6.org/language.html links to a different glossary: https://doc.perl6.org/language/glossary .

The link should be to https://doc.perl6.org/language/glossary that is a new page that was recently added. Would you submit a PR for that?


Southeast Asia Population Growth Not Great Enough

Population growth in Southeast Asia (Indochina, Indonesia, Philippines, Malaysia, Singapore, etc.) is not great enough; population from 1946 consistently does not match countries' respective populations in 1992.

Possible solutions include raising the life rating of provinces in SEA from 30 to 35.

It should be part of the population trends. Will uses this ticket for another round of pop growth tuning.


docs: remove filter "type"

This is no longer used by code. This change removes it from the docs
and also makes the field optional in the schema so that we don't
break existing configs.

@htuch

@rshriram yeah, 1.4 included the support needed for this deprecation.

type is not removed from config examples: https://github.com/envoyproxy/envoy/blob/master/configs/envoy_service_to_service.template.json#L9

@winguse yup will try to get to this.


can't solaris_exporter on solars11.3.36

root@xxxxxxxxxx # /opt/csw/bin/pip install psutil==5.7.0
/opt/csw/bin/gcc-5.2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -m32 -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=570 -DPSUTIL_SUNOS=1 -DNEW_MIB_COMPLIANT=1 -I/opt/csw/include/python2.7 -c psutil/_psutil_common.c -o build/temp.solaris-2.11-sun4v.32bit-2.7/psutil/_psutil_common.o
unable to execute '/opt/csw/bin/gcc-5.2': No such file or directory
error: command '/opt/csw/bin/gcc-5.2' failed with exit status 1
----------------------------------------

Command "/opt/csw/bin/python2.7 -u -c "import setuptools, tokenize;file='/tmp/pip-build-TxNAvo/psutil/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-KW04Mu-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-TxNAvo/psutil/
You are using pip version 8.1.2, however version 22.0.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@xxxxxxxx # ls -ld /opt/csw/bin/gcc-5.2
/opt/csw/bin/gcc-5.2: No such file or directory
root@xxxxxxxx # gcc -v
Using built-in specs.

Description: This package constrains system package versions to the same
build.  WARNING: Proper system update and correct package
selection depend on the presence of this incorporation.
Removing this package will result in an unsupported system.
For more information see:
https://support.oracle.com/rs?type=doc&id=2433413.1
Category: Meta Packages/Incorporations
State: Installed
Publisher: solaris
Version: 0.5.11 (Oracle Solaris <IP_ADDRESS>.0)
Build Release: 5.11
Branch: <IP_ADDRESS>.0.24.0
Packaging Date: Wed Jan 13 20:40:04 2021
Size: 5.46 kB
FMRI: pkg://solaris/entire@0.5.11,5.11-<IP_ADDRESS>.0.24.0:20210113T204004Z
root@xxxxxxxxxx #

The problem is in absent file
/opt/csw/bin/gcc-5.2
Try to create links from installed gcc-5.4 or install gcc-5.2.
Also you could open support ticket for psutil - you have problems with its installation:  https://github.com/giampaolo/psutil

Hi
update below still not success.
root@xxxxxxxxx # /opt/csw/bin/pip2.7 install psutil==5.7.0
Collecting psutil==5.7.0
Using cached https://files.pythonhosted.org/packages/c4/b8/3512f0e93e0db23a71d82485ba256071ebef99b227351f0f5540f744af41/psutil-5.7.0.tar.gz
Installing collected packages: psutil
Found existing installation: psutil 5.9.0
Uninstalling psutil-5.9.0:
Successfully uninstalled psutil-5.9.0
Running setup.py install for psutil ... done
Successfully installed psutil-5.7.0
You are using pip version 8.1.2, however version 22.0.4 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root@xxxxxxxxx # /opt/csw/bin/python2.7 solaris_exporter.py
Traceback (most recent call last):
File "solaris_exporter.py", line 133, in
from prometheus_client.core import REGISTRY, Counter, Gauge, GaugeMetricFamily, CounterMetricFamily, UntypedMetricFamily
File "/opt/csw/lib/python2.7/site-packages/prometheus_client/init.py", line 3, in
from . import (
File "/opt/csw/lib/python2.7/site-packages/prometheus_client/exposition.py", line 113
def make_wsgi_app(registry: CollectorRegistry = REGISTRY, disable_compression: bool = False) -> Callable:
^
SyntaxError: invalid syntax
root@xxxxxxxxx #
root@xxxxxxxxx # /opt/csw/bin/python3.3 solaris_exporter.py
Traceback (most recent call last):
File "solaris_exporter.py", line 130, in
import psutil
ImportError: No module named 'psutil'
root@xxxxxxxxx #
Actually I already installed 3 servers pass 2/3 not sure why this server not success.


(doc) Fix a buffer overflow bug in checkCapacity

Hello,
I noticed that there was a small bug in the checkCapacity in the TextStringBuilder class during, funnily enough, a midterm Software Quality and Testing where I had to write tests for it. It always doubles the new capacity without checking the size so it overflows when it is around half the maximum integer size.
This patch fixes the bug by switching over to just growing it to the given size for a high enough capacity, but it does lose the amortized property. Luckily, this probably won't be an issue since you will hit the maximum capacity of the VM way before that (which is also why the test catches that error). An alternative solution would be checking if it reaches a certain max capacity and throwing a exception, but I thought that was too much of a change.
If there is anything you would like me to modify, I would love to hear it. But throwing it out entirely is also fair. Here is to hoping this will give me some bonus points on my exam.
With kind regards,
Valentijn van de Beek.

Coverage increased (+0.0004%) to 97.963% when pulling f9b26132b2c7b576db76d52366943420c1dd99ce on ValentijnvdBeek:(doc)-TextStringBuilderCheckCapacity into ea03fa0a03b9b31beeac1cb12224334f93964a07 on apache:master.

I wonder if instead of inventing yet another capacity growing algorithm, we should not just imitate what java.lang.AbstractStringBuilder does in OpenJDK.

@garydgregory
That is good point, I have changed it so when it overflows it will put the value at the maximum value possible which similar to how OpenJDK does it. I also put the bound as a separate variable so we don't have to recompute it.
Is this more along the lines of what you would like?

Couple conflicts now. Could you rebase, please, @ValentijnvdBeek ?

Ping

@ValentijnvdBeek Ping on the rebase SVP

Hello @ValentijnvdBeek
Closing:

No feedback for a few months
This has been addressed differently in git master or a snapshot build from https://repository.apache.org/content/repositories/snapshots/org/apache/commons/commons-text/
Please review git master and see if your use case is fixed
TY!


Clickhouse DB Additional JDBC options

Hello,
Not exactly a bug but more of a consultation. I don't see advanced options for clickhouse on latest 42.1 anymore, as we had on 39.X. Why and when was it removed, should we create a task in metabase in order to return it back?

Same problem on our side. Is it possible to get it back? We would like to set socket_timeout=xxxx but it seems not possible without the dedicated JDBC options.

You still can edit system database, where in table metabase_database for clickhose row in details column in json add next values. "advanced-options":true,"additional-options":"socket_timeout=3000000"

@webver Thanks for the tip but in that db, the in the column defails I see only crypted characters. I guess because that column is crypted. Isn't it?

Ran into this issue as well, and like @saimon46 said the db is encrypted.
Solved this by updating the database via the API.  What I suggest is that you go to the database page on the admin, don't change anything, and hit "save'.  Capture the request in in your browser's dev tools. You'll see the body was something like:
{"engine":"clickhouse","name":"NAME","details":{"dbname":"DBNAME","host":"HOST","port":PORT,"user":"USER","password":"PASSWORD","ssl":true},"is_full_sync":true,"is_on_demand":false}

Add the "additional-options" inside the details object, so you'd end up with:
{"engine":"clickhouse","name":"NAME","details":{"dbname":"DBNAME","host":"HOST","port":PORT,"user":"USER","password":"PASSWORD","ssl":true,"additional-options":"socket_timeout=300000"},"is_full_sync":true,"is_on_demand":false}

Which you should be able to send (either via curl or by re-issuing the modified request, if your browser supports that)

Ran into this issue as well, and like @saimon46 said the db is encrypted.
Solved this by updating the database via the API. What I suggest is that you go to the database page on the admin, don't change anything, and hit "save'. Capture the request in in your browser's dev tools. You'll see the body was something like:
{"engine":"clickhouse","name":"NAME","details":{"dbname":"DBNAME","host":"HOST","port":PORT,"user":"USER","password":"PASSWORD","ssl":true},"is_full_sync":true,"is_on_demand":false}

Add the "additional-options" inside the details object, so you'd end up with:
{"engine":"clickhouse","name":"NAME","details":{"dbname":"DBNAME","host":"HOST","port":PORT,"user":"USER","password":"PASSWORD","ssl":true,"additional-options":"socket_timeout=300000"},"is_full_sync":true,"is_on_demand":false}

Which you should be able to send (either via curl or by re-issuing the modified request, if your browser supports that)

Thx, it works.


Slow framerate with new AMOLED touch 1.91 Plus

I just got the new AMOLED touch 1.91 Plus and the factory example shipped on it had very slow frame rate.  I loaded my program and the screen dragging is choppy.   I have 3 other units prior to the plus model and they work pretty smooth.  Would you have a suggestion on what I might  do to fix this?

Plus the driver type is SPI instead of QSPI so it is normal that the frame rate is not as high as before, which is caused by the limited SPI rate.

I am currently writing a rust driver for this SPI display and I experience the same slow framerate - I got much better results using DMA on the SPI bus but this produces artefacts in my case. I would be interested in a DMA example in this repository.

Plus the driver type is SPI instead of QSPI so it is normal that the frame rate is not as high as before, which is caused by the limited SPI rate.

Ah okay. Thanks


Usage Question about OpenPAI的HA机制

Organization Name: advantech

Short summary about the issue/question:我们打算将Azure上部署的cluster的openPAI部署为HA。有几个问题希望能澄清一下。
Brief what process you are following:

Etcd目前同步了哪些东西，仅包括子服务状态，还是子服务的一些数据也包含了？
OpenPAI有一个默认的存储路径，这个下面存储的东西在切换M/S node的时候会进行同步吗？
HDFS的存储需要进行同步吗？

How to reproduce it:

OpenPAI Environment:cluster

OpenPAI version:0.8.2
Cloud provider or hardware configuration:
OS (e.g. from /etc/os-release):
Kernel (e.g. uname -a):
Hardware (e.g. core number, memory size, storage size, GPU type etc.):
Others:

Anything else we need to know:

related to https://github.com/Microsoft/pai/issues/852 and https://github.com/Microsoft/pai/issues/724

Not sure I fully understand your question. Let me first explain the fault tolerance issue in an Azure deployment.
If don't care about details, here is the short conclusion: running OpenPAI on Azure can be fault tolerant. But our current released deployment may experience some temporal service availability in case of VM crashes (according to our 1-yr experience, VM crash is a very rare event. You may see at most one VM crash in a month for a cluster with 70 VMs )

We enforce etcd replicas for high availability. A single etcd instance crash will not result in any service unavailability)
For HDFS, losing single node will not result in any data lost. OpenPAI config multiple replicas for HDFS DataNode.
Sll other components are run in single instance, including key components like WebPortal, Pylon, RestServer, Launcher, Zookeeper, RM, NM, etc. If a VM that contains these components crashed, user will experience service unavailability. But after the VM heals it self (mostly through auto-heal or seldom via manual reboot), the service will resume with no information loss (since we can config the service to keep the data in persistent storage, or the service itself is stateless.)
Any single component failure will not affect existing training jobs, unless the jobs run on the VM that crashed.
we plan to improve the service availability by adding replicas for the single-instance components. It will be a gradual process and start from next month.

Hope it addresses your questions.

Hi,
I create 5 VMs for OpenPAI cluster with HA:

Master Node 0
Master Node 1
Worker Node 0
Worker Node 1
Proxy Node
And I try to stop all services on Master Node 0(using 'systemctl stop docker'), we lost Port 80 connection on Master Node 1.
Then, I repeat same steps on Master Node 1, the Master Node 0 works fine.
Is there something wrong?

@fanyangCS Could you please help me to solve this issue? Thanks.

@hellocandy, can you explain your question in details? @ydye ,could you follow up this issue?

Hi,
We expect Master Node 0 to have the same functionality as Master Node 1 to ensure HA. So we tried to shut down services on Master Node 1 (including all components of kubernetes and OpenPAI), at which point pylon (port 80) on Master Node 1 could not be accessed properly. But we tried to stop all services on Master Node 1 (including all components of kubernetes and OpenPAI), at which point Pylon (Port 80) on Master Node 0 can be accessed normally.

we will provide HA in pure k8s version.


page.window_minimized, page.window_maximized, and page.window_close_async don't work on async apps.

When using an Async Flet App none of the above mentioned in the title work.

page.window_minimized
page.window_maximized
page.window_close_async

The below code snippet works showing that there is nothing wrong with ALL Async functions.
async def exit_button_callback(self, e: flet.TapEvent):
    await self.page.window_destroy_async()

The window_destroy_async function works fine for exiting an app. However using window_close_async in the exact same manner will result in nothing happening. No errors are raised, just simply nothing happens.

Similarly the below code snippets also do nothing. No errors, nothing happens.
async def minimize_button_callback(self, e: flet.TapEvent):
    self.page.window_minimized = True
    await self.page.update_async()

async def maximize_button_callback(self, e: flet.TapEvent):
    self.page.window_maximized = True
    await self.page.update_async()

As mentioned this issue only exists inside of an Async Flet App using them in a Non-Async Flet App they all work as expected, granted for page.window_destroy_async, and page.window_close_async you wouldn't use the async functions.
Flet Version: 0.7.4
Windows 10 Version: 22H2

These async functions are now deprecated and will be soon removed.


Refactor views and add CouncilmeetingNext

Coverage remained the same at 63.495% when pulling 1fcef5d85d11748845c444019418b8057a938e72 on refactor into d484bad53b55118188cdca11e50fdf6bf3744fe6 on master.


Documentation!

Needs broad documentation plus several examples

Map + search in a button
Map + search in an expanded searchbox
Configuring a fullpage map to work at particular breakpoints for an expanded searchbox

I'd say this is more than done.
@louh?

DONE


[Docs] Auto-generate list of rules in README

This adds a script to auto-generate the list of rules and embeds special tags to README.
Usage:
$ npm run generate-list-of-rules

Check whether the auto-generated changes are committed:
$ npm run generate-list-of-rules:check

See also #1742

I've synced this PR with #1742, so closing this.

To be clear, i need both to stay open until both are merged :-)

Sorry, I see 👍


Firefox: Strange horizontal scrollbar on div#content

On Firefox there is a strange horizontal scrollbar on div#content. #content:before { display:block; } seems to help.

Can you attach a screenshot?

http://s10.postimg.org/dagc9f3cp/lychee_ff_scrollbar.jpg

http://s24.postimg.org/q0ena76ut/lychee_ff_scrollbar.jpg

Will take a look at it.
BTW: Don't use the v3.0.0 branch. It's in development and a lot of things may be broken.

thnx


[Bug] setOption not working

Version
5.5.1
Link to Minimal Reproduction
https://codepen.io/lstoryc/pen/BagEmrV
Steps to Reproduce

open demo
It will update 【 option.aria.decal.decals[0].color = "red"; 】in next 2s

Current Behavior
setOption not working when I change option.aria.decal.decals value
Expected Behavior
chart wll update after setOption
Environment
- OS: windows 11
- Browser: chrome
- Framework: js

Any additional comments?
No response

confirmed - aria cannot be updated with setOption.
Workaround is to update itemStyle.decal for individual series - Demo.


Kinetic energy consistently delayed by one step

Intro
Hi!
I am a graduate student, I use MuJoCo to practice control theory in my research.
My setup
MuJoCo - 3.2.3
Python - 3.10.14 | packaged by Anaconda, Inc. | (main, Mar 21 2024, 16:20:14) [MSC v.1916 64 bit (AMD64)]
OS - Windows-10-10.0.22631-SP0
What's happening? What did you expect?
When calculating kinetic energy in a simulation, I noticed a consistent one-step delay between the manually calculated kinetic energy (calc_kinetic_energy) and the kinetic energy stored in data.energy[1] provided by the simulation.
Specifically, the value at step N for calc_kinetic_energy matches the value at step N+1 for data.energy[1], causing a mismatch at every step.
Interestingly, while the potential energy returned by data.energy[0] remains consistent at each step, the kinetic energy seems to be delayed.
Additionally, even when accounting for the one-step delay, there is still a slight discrepancy between the manually calculated kinetic energy and the simulation-provided value. I apologize for the confusion if this is due to a mistake in my calculations or my approach.

I expected the manually calculated kinetic energy (calc_kinetic_energy) to match the simulation-provided kinetic energy from data.energy[1] at each step without any delay.

Steps for reproduction

Install packages.

pip install mujoco
pip install robot_descriptions

Run the code below.
Compare the results of the following two loops.

zip(calc_kinetic_energy, data_kinetic_energy)
zip(calc_kinetic_energy, data_kinetic_energy[1:])

Minimal model for reproduction
No response
Code required for reproduction
import sys
import platform
import numpy as np
import mujoco
from robot_descriptions import ur5e_mj_description

print('python version:', sys.version)
print('mujoco version:', mujoco.__version__)
print('OS version: ', platform.platform())

# model & data
model = mujoco.MjModel.from_xml_path(ur5e_mj_description.MJCF_PATH)
for i in range(model.nu):
    model.actuator_dynprm[i][:3] = [1,0,0]
    model.actuator_gainprm[i][:3] = [1,0,0]
    model.actuator_biasprm[i][:3] = [0,0,0]

model.opt.enableflags = mujoco.mjtEnableBit.mjENBL_ENERGY
data = mujoco.MjData(model)

calc_kinetic_energy = []
data_kinetic_energy = []

for _ in range(10):

    mujoco.mj_step(model, data)

    # calc_potential = model.body_mass @ data.xipos @ (-model.opt.gravity)

    vec = np.zeros(model.nv)
    mujoco.mj_mulM(m=model, d=data, res=vec, vec=data.qvel)
    calc_kinetic = 0.5*mujoco.mju_dot(vec, data.qvel)

    # energy = np.array([calc_potential, calc_kinetic])
    # print(data.energy - energy)

    calc_kinetic_energy.append(calc_kinetic)
    data_kinetic_energy.append(data.energy[1])

s = f"| {'calc_kinetic':^12} | {'data_kinetic':^12} | {'diff':^11} |"
# compare all steps
print('-'*len(s))
print(s)
for calc_kin, data_kin in zip(calc_kinetic_energy, data_kinetic_energy):
    print(f'| {calc_kin:^12.8f} | {data_kin:^12.8f} | {calc_kin - data_kin:^11.8f} |')
print('-'*len(s))
print()
# compare from the second step onward
print('-'*len(s))
print(s)
for calc_kin, data_kin in zip(calc_kinetic_energy, data_kinetic_energy[1:]):
    print(f'| {calc_kin:^12.8f} | {data_kin:^12.8f} | {calc_kin - data_kin:^11.8f} |')
print('-'*len(s))

Confirmations

[x] I searched the latest documentation thoroughly before posting.
[X] I searched previous Issues and Discussions, I am certain this has not been raised before.

WAI.
Carefully read the sections here for an explanation, though we should make this clearer somewhere in the docs.
Also read this and this.
TL;DR, everything in mjData is always wrt the previous state (or rather, the state transitions). This is unavoidable. If you want up to date values you need to call mj_forward.
Also, please try to do a better search next time 🙂


[web][form][SIMS #1093] UI updates for Student Application - Buttons and formatting

used a narrow layout;
moved buttons to the top;
layout issues for the form pointed out by @hellolynn-tbtb ;

how did u took the 2nd last screenshot 😉

how did u took the 2nd last screenshot 😉

I use a chrome extension: https://chrome.google.com/webstore/detail/gofullpage-full-page-scre/fdpohaocaechififmbbbbbknoalclacl


added doc demonstrating custom http routes

Summary
Issue: #514
Requirements (place an x in each [ ])

[x] I've read and understood the Contributing Guidelines and have done my best effort to follow them.
[x] I've read and agree to the Code of Conduct.

@seratch need to add to JA docs.

@stevengill thanks for the heads up. will do.

#519 for Japanese doc updates